jqgrid load large data set without pagination

Question

I was wondering whether there is a better way to load large Json data set from the server.

I am using jqgrid as loadonce:true. i need to load around 1500 records at once and also i don't use pagination options. is there any better way to achieve this? Thank you in advance.

This is my Grid code -

  $(function(){
        $("#testgrid").jqGrid({
            url:getGridUrl,
            datatype: 'json',
            mtype: 'GET',
            height: 250,
            colNames:['Inv No','Date', 'Client', 'Amount','Tax','Total','Notes'],
            colModel:[
                {name:'id',index:'id', width:60, sorttype:"int",search:false},
                {name:'invdate',index:'invdate', width:90, sorttype:"date",search:false},
                {name:'name',index:'name', width:100,search:false},
                {name:'amount',index:'amount', width:80, align:"right",sorttype:"float"},
                {name:'tax',index:'tax', width:80, align:"right",sorttype:"float",search:false},        
                {name:'total',index:'total', width:80,align:"right",sorttype:"float",search:false},     
                {name:'note',index:'note', width:150, sortable:false,search:false}      
            ],
            multiselect: true,
            multiboxonly:true,
            caption: "Manipulating Array Data",
            pager: '#testgridpager',
            //Auto load while scrolling
            //scroll: true,
            //to hide pager buttons
            pgbuttons:false,
            recordtext:'',
            pgtext:'',
            loadonce: true,
            sortname: 'id',
            sortorder: 'asc',
            viewrecords: true,
            multiselect: true,

            jsonReader : {
                root: "rows",
                //page: "page",
                //total: "total",
                records: "records",
                repeatitems: false,
                cell: "cell",
                id: "id"
            },
            loadComplete: function(data) {
            var rowId;
            //alert(data.length);
            //alert('load complete'+data.rows.length);
            //set checkboxes false if mode is set to true
            if(mode){
                for(var i=0;i<data.rows.length;i++){
                    rowId=data.rows[i].id;
                    disableRow(rowId);
                    var searchVal =  $("#gs_amount").val().trim();
                    if(searchVal ==data.rows[i].amount){
                        jQuery("#testgrid").jqGrid('setSelection',rowId);
                        //heighlightSearch();
                    }
                }
            }
        }
     });
        //toolbar search
        $("#testgrid").jqGrid('filterToolbar',{stringResult:true,searchOnEnter:false});
    });

    function disableRow(rowId){
    $("#testgrid").jqGrid('setRowData', rowId, false, {color:'gray'});
    var trElement = jQuery("#"+ rowId,$('#testgrid'));
    trElement.removeClass("ui-state-hover");
    trElement.addClass('ui-state-disabled');
    trElement.attr("disabled",true);
}

Thrombokinase

Could you include the javascript code which you currently use? Do you use local paging of the data? Do you use gridview:true parameter? – Nessus

@Nessus - Thanks oleg. i used gridView parameter. please find the updated question. – Thrombokinase

@Sam: I am very busy now, but if your problem will not be solved till Saturday I'll write you some suggestions. What do disableRow? I am not sure that I understand what you do inside of loadComplete. If in the filter bar for amount no data exist you should do nothing. If in amount the data exist the data are filtered and you should select all rows. Is it so? Moreover I don't see gridview:true parameter. Is the example which you posted the real example which you tested? How long time is the loading in your tests? Which browser is the most important for you? – Nessus

@Nessus - Thanks for your valuable time Oleg. this is the grid i used to test jqgrid. i just removed gridview param to check the consequences. inside loadComplete if the amount match the grid value amount it needs to get selected. you are right i do need to put empty check in loadComplete. I am still searching the better ways to do this(load large data). IE 8 will be my most important browser. Thanks for your response Oleg. I will looking forward to your thoughts among this.Appreciate it. – Thrombokinase

@Nessus - If you have some time can you please send me some directions about any approach for this? if you have time to spare only..Appreciate your help. – Thrombokinase

@Sam: I am preparing the demo for you at the time. By the way in the example you forget to include rowNum with some value greater or equal to 1500. After this you will see that your grid is mostly slow because of the actions which you do inside of loadComplete. If you commented th part and use gridview: true and hoverrows: false the grid construction will be acceptable. What is unclear for me is the following: Your current code disable all rows (in many ways) and then try to select all the rows. What you really want to do? – Nessus

@Sam: rows having 'ui-state-disabled' class can not be selected. – Nessus

@Nessus - Thanks for your response.according to the mode i need to disable the filtered rows.that rows cannot be selected. else i need to filter whatever value typed in the amount search box must be filtered. i assume there may be a unique amount for each and every row. so if the amount match the particular amount in the grid that row needs to be automatically checked. thats what i am trying to do. if my approach is wrong please let me know what needs to be change. Thank you. – Thrombokinase

@Sam: I don't quite understand what you mean. If you use searchOnEnter:false, then the grid start be filtered almost immediately after starting the typing in the "#gs_amount". So all rows which will be displayed in the grid are the filtered rows. So if you write "i need to disable the filtered rows" it means that all rows of grid will be disabled. I don't quite understand which matching you need. If you define searchoptions with sopt:["eq"] for 'amount' column only exact matching will be done. – Nessus

@Sam: you can use defaultSearch parameter of filterToolbar to specify the default matching operations. I use almost everywhere for the integer and 'select' columns the exact matching 'eq' and case insensitive (ignoreCase:true parameter of jqGrid) contain searching ('cn') for the columns having strings. – Nessus

@Nessus - Thank you very much oleg for the advices. i will follow those. is there any clue regarding large data set loading? – Thrombokinase

@Sam: In my answer I included an example which loads 1500 rows and the performance is not bad. I recommend only better to use more compact packing of data as with repeatitems:false in jsonReader. If one column can be interpret as the id one can use cell:"" instead (see my answer). Much more important is how to work with the data. Simple loop where you find row by rowid is too slow. Moreover would recommend you to use local data paging. 1500 rows can not be displayed at once. If you will use local paging by setting rowNum to some small value the performance will be much much better. – Nessus

Answer 1

On example of this demo you can see the time of loading 1500 rows for your grid in case of usage of gridview: true.

The most performance problem of your example are inside of loadComplete function. If you do need to make some modifications on the grid you should use jQuery to manipulate the grid contain. The best performance you can archive if you use DOM elements of the grid directly like in the example

loadComplete: function() {
    var i=0, indexes = this.p._index, localdata = this.p.data,
        rows=this.rows, rowsCount = rows.length, row, rowid, rowData, className;

    for(;i<rowsCount;i++) {
        row = rows[i];
        className = row.className;
        //if ($(row).hasClass('jqgrow')) { // test for standard row
        if (className.indexOf('jqgrow') !== -1) {
            rowid = row.id;
            rowData = localdata[indexes[rowid]];
            if (rowData.amount !== "200") {
                // if (!$(row).hasClass('ui-state-disabled')) {
                if (className.indexOf('ui-state-disabled') === -1) {
                    row.className = className + ' ui-state-disabled';
                }
                //$(row).addClass('ui-state-disabled');
            }
        }
    }
}

You can see the corresponding example live here.

In the implementation of loadComplete function I use the fact, that jqGrid having loadonce:true parameter use internal parameters _index and data which can be used to access the contain of the grid. In the example I disabled the rows which not contain "200" in the amount column.

UPDATED: The answer describes how to use rowattr callback to simplify the solution and to have the best performance (in case of gridview: true of cause).

Nessus

@Oleg- Thank you very much oleg. now I got the base line from The best performance you can archive if you use DOM elements phrase. examples are really helpful. Thanks for your knowledge sharing and the kind guidance.Greatly appreciate your time & the effort. – Thrombokinase

Answer 2

I would be tempted to look into the Autoloading on scroll feature of jqGrid. I would never load 1500 rows upfront. Any reason you cannot page?

Puzzler

thanks for the turn over. Requirement is not to use pagination. i used to look at scroll:true function and i found this article. Thats why i wonder what will be the best way to use. – Thrombokinase

@Thrombokinase - the issue maybe his js/php who knows. I would test for myself. I am sure others would have mentioned and it is upto version 4.0 now. – Puzzler