why Array of Float64Array is taking so much memory in Node.js?

Question

Found a very interesting situation, node.js 6.11.0, Win 10. After running this code

function rand() {return Math.floor(Math.random()*10);}
let s = new Array(10000000).fill(0).map(a => new Float64Array([rand(), rand()]));

and calling global.gc() few times, the node.js environment was taking 1,7GB of space. I have no explanation of this - Float64Array of two numbers is taking 16bytes, times 10000000 is ~160MB. Even if you assume that each element of array s is actually a pointer to Float64Array, which is another 8 bytes, it makes 240MB, but not 1,7GB for sure.

What could be the explanation to this?

AKX · Accepted Answer · 2018-02-22 14:45:46Z

1

Looking at the node --inspect (Node 9.5.0) output for

function rand() {
    return Math.floor(Math.random()*10);
}
const arr = [];
for(var i = 0; i < 1000000; i++) {
    arr.push(new Float64Array([rand(), rand()]));
    if(i % 1000 == 0) {
        console.log(i);
    }
}

global.x = arr;

it looks like each of those Float64Arrays of 2 items requires 208 bytes of memory, so there's "simply" a significant per-object overhead there.

If you need something like this, I'd suggest allocating a single flat Float64Array of 2 * 10000000 items and indexing into it. (FWIW, I just tried that: the single 200-million-item Float64Array consumes 600 megabytes of memory and the allocation and execution near-instant.)

edited Feb 22, 2018 at 14:45

answered Feb 22, 2018 at 14:37

AKX

171k16 gold badges146 silver badges228 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

why Array of Float64Array is taking so much memory in Node.js?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related