Large 2D array in C - Stack OverFlow error

Question

I haven't used C in long time, and I'm having an issue filling a 2D array from a CSV. The file format is like this:

Node, In, Out

1,200,10393

...

This is essentially an array representation of a linked list. There are 150000 elements and whenever I try to fill the array I get an error saying "Unhandled exception at 0x000000013facb957 in main.exe: 0xC00000FD: Stack overflow." I'm on a 64-bit machine with 16GB of RAM and I'm using VS C++ 2010 Express with an x64 build configuration.

int main(int argc, char *argv[])
{

int counter = 0;
char line [ 1024 ];
int map[150000][2] = {0};
char *comma = ",";
char *token;
int index;
int in, out;
char* end;
int nodeID;
FILE *fp;

fp = fopen("mapsorted.txt","r"); // read mode

if( fp == NULL )
{
  perror("Error while opening the file.\n");
  exit(EXIT_FAILURE);
}

//Skip header line
fgets ( line, sizeof line, fp );

while ( fgets ( line, sizeof line, fp ) != NULL) /* read a line */
{
    //first part - the index for storage
    token = strtok(line,comma);
    index = strtol(token,&end,10);

    //second part
    token = strtok(NULL,comma);
    in = atoi(token);

    //third part
    token = strtok(NULL,comma);
    out = atoi(token);

    //store in array
    map[index][0] = in;
    map[index][1] = out;
}
fclose ( fp );
}

The code seems to work when I allocate a smaller array, but fails when it is this large. I think I should have enough memory to be able to handle an array of this size.

I thought Windows had increased the stack size from the ridiculously small 1MB default. — Daniel Fischer
– Daniel Fischer, Commented Dec 30, 2012 at 18:29
Does this answer your question? Getting a stack overflow exception when declaring a large array — Nate Eldredge
– Nate Eldredge, Commented Feb 9, 2021 at 22:39

score 7 · Accepted Answer · 2012-12-30 18:37:38Z

7

int map[150000][2];

seems at least 2 * 4 * 150000 bytes (assuming a modern 32-bit architecture), which is roughly 1.2MB. Knowing that modern OSes typically set up a stack size of a few megabytes, this could actually be the problem. Your computer having several gigabytes of RAM doesn't mean all of it can be consumed by your process, especially not on the stack. For large arrays, try malloc()ating some memory on the heap:

int (*map)[2] = malloc(sizeof(*map) * 150000);

or

int *map = malloc(150000 * 2 * sizeof(*map));

(pay attention to the dimensions in the second case!), or declare it as static to move it out of stack space:

static int map[150000][2];

Or simply make it a global variable to achieve a similar behavior:

int map[150000][2];

void foo()
{
    ...
}

edited Dec 30, 2012 at 18:37

answered Dec 30, 2012 at 18:29

user529758

Sign up to request clarification or add additional context in comments.

6 Comments

WhozCraig Over a year ago

Or even just let the compiler crunch the math for you: int (*ar)[2] = malloc(sizeof(int[150000][2])); =P

user529758 Over a year ago

@WhozCraig Not preferred, risky. The less risk is in the way I used.

WhozCraig Over a year ago

Genuinely curious. whats the risk? (i.e. should I start perusing code I've done this in and chain-retool)?

user529758 Over a year ago

@WhozCraig When the dimensions of the array and/or the base type changes, and one forgets to (don't say you don't forget it, we all do) change that in the malloc part as well, you get into trouble. And well, you don't have to go back and rewrite all code, but you can get used to this from now on, if you wish.

WhozCraig Over a year ago

I was about to post before your response to nm. My alloc is type-based -only, not var-type based like yours. Touche, and i concur with you this regard. btw. The behavior is fine so long as you don't change it, whereby you then incur risk, and I agree. (and +1, btw).

|

Erik Ekman · Accepted Answer · 2012-12-30 18:28:08Z

2

The array is too big to fit in the stack. Try moving it outside of the function:

static int map[150000][2] = {0};
int main(int argc, char *argv[])
{

and so on.

answered Dec 30, 2012 at 18:28

Erik Ekman

2,08612 silver badges13 bronze badges

12 Comments

K-ballo Over a year ago

...or to allocate it dynamically on the free store

K-ballo Over a year ago

It's hardly a big change, the rest of the code that deals with map won't change...

user529758 Over a year ago

@K-ballo Well, it would, since 2D access syntax would have to be changed.

K-ballo Over a year ago

@H2CO3: How would it change??

Daniel Fischer Over a year ago

@H2CO3 int (*map)[2] = malloc(150000*sizeof *map);.

|

Collectives™ on Stack Overflow

Large 2D array in C - Stack OverFlow error

2 Answers 2

6 Comments

12 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

6 Comments

12 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related