AngularJS and Javascript large data loop performance

Question

I've been battling with a large data array of varied length roughly between 100 ~ 200 items, but can be greater.

The data items themselves come from a custom reporting algorithm that produces the array of objects.

Object example:

ts:budget: "foo"
ts:category: "foo"
ts:client: "foo"
ts:entryTime: 14.5
ts:project: "foo (00000000)"
ts:task: "foo"
ts:user: "foo"
ts:userGroup: "foo"

Multiply that object example by 100 ~ 200 and that's what I'm working with. Nothing massive, so I wasn't expecting such a large performance hit. I very much suspect it's my factory code that's the issue, but I've gone through and optimised several areas (I.e. replacing angular.ForEach loops with native for loops) and have clawed back performance, but still hit a CPU usage of around 25% when the factory is called.

Factory code:

var report = '{"userID":"111868022517936189634", "jsonPayload":{"metrics":[{"ID":"ts:entryTime"}],"dimensions":[' + dimensions + '],"filters":[]}}';
var promise = $http.post("report.php?action=show&type=report&rangeStart=" + rangeStart + "&rangeEnd=" + rangeEnd, report).success(function(results) {
    var uniqueArray = [];
    for (m = array.length - 1; m >= 0; m--) {
        var subItemUniqueArray = [];
        if (uniqueArray.indexOf(array[m]) == -1) {
            uniqueArray.push(array[m]);
            for (k = uniqueRequireArray.length - 1; k >= 0; k--) {
                uniqueRequireArray[k] == 'category' ? array[m]['categories'] = [] : array[m][uniqueRequireArray[k] + 's'] = [];
            }
            for (j = results.length - 1; j >= 0; j--) {
                for (var x in results[j]) {
                    if (results[j].hasOwnProperty(x)) {
                        if (x == 'ts:' + type) {
                            var relationString = null;
                            type != 'project' ? relationString = array[m][relation] : relationString = array[m].name + ' (' + array[m].projectNumber +')';
                            if (results[j][x] == relationString) {
                                for (i = uniqueRequireArray.length - 1; i >= 0; i--) {
                                    var obj = {
                                        'item': results[j]['ts:' + uniqueRequireArray[i] + '']
                                    }
                                    if (subItemUniqueArray.indexOf(obj.item) == -1) {
                                        uniqueRequireArray[i] == 'category' ? array[m]['categories'].push(obj) : array[m][uniqueRequireArray[i] + 's'].push(obj);
                                        subItemUniqueArray.push(obj.item);
                                        obj.entryTime = results[j]['ts:entryTime'];

                                        if (uniqueRequireArray[i] == 'user') {obj.externalID = $userlookup.email(obj.item);} // add user externalID
                                        if (uniqueRequireArray[i] == 'category') {obj.colour = $colourlookup.name(obj.item);} // add category colour
                                        if (angular.isDefined(results[j]['ts:category'])) {
                                            if (uniqueRequireArray[i] == 'budget') {obj.colour = $colourlookup.name(results[j]['ts:category']);} // add budget colour
                                            if (uniqueRequireArray[i] == 'task') {obj.colour = $colourlookup.name(results[j]['ts:category']);} // add task colour
                                        }
                                    }
                                    else {
                                        obj.entryTime += results[j]['ts:entryTime'];
                                    }
                                }
                                break;
                            }
                        }
                    }
                }
            }
        }
    }
}).error(function(data) {
        $debug.admin('Service: Failed getting report results for overview', 'error', true);
}).then(function(results) {
    return results;
});
return promise;

I doubt having 5 for loops is ideal for performance, especially as the very top for loop is driven by another array fed into the service that can be anywhere from 30 to 200 items in length.

Given roughly a minute, the function successfully creates the correct data bindings and populates the front-end repeater and CPU usage drops. Its just for that time, the app is completely unusable.

Any help will be massively appreciated!

I believe, if you expect people to debug/optimize your code, you should at least supply a usable/runable example. Maybe set up a plunker. — Yoshi
– Yoshi, Commented Oct 23, 2013 at 12:44

Ganaraj · Accepted Answer · 2013-10-23 12:48:06Z

2

Your case might be a really good one for using something like http://danieltao.com/lazy.js/ or if that is not something useable for you yet, then atleast go the http://underscorejs.org/ route.

It looks like what you are trying to achieve might be better solved using a functional route than the iterative approach that you are taking. Why?

It will end up being more readable.
Those libraries are written by experts out there. Might be a good idea to utilize their expertise.
Your requirement begs for a more functional and chained approach (seeing all those for loops).

Once you do this, if you are still hitting performance issues - then I would suggest replacing the whole ng-repeat sequences with a hand written directive.

You might want to look up stuff that is there already online which helps you in this direction.

http://tech.small-improvements.com/2013/09/10/angularjs-performance-with-large-lists/

answered Oct 23, 2013 at 12:48

Ganaraj

26.8k6 gold badges68 silver badges60 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

tinyBIGideas Over a year ago

Thank you very much! Pointed me the right direction with: tech.small-improvements.com/2013/09/10/… which pointed me onto:github.com/Pasvaz/bindonce - $apply and $digest were responsible for ~90% of the problem. I've also reverted several performance enhancers to ensure the service is much more readable.

Collectives™ on Stack Overflow

AngularJS and Javascript large data loop performance

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related