I have to create an array of char pointers each of them of size 10000000 using the best and optimized way to do this in C.
4 Answers
I think this will do (haven't checked for nulls though):
int i;
int num_arrays;
char **huge_char_array;
num_arrays = 10; //number of arrays you want.
huge_char_array = (char **)malloc(sizeof(char *) * num_arrays);
for(i = 0; i < num_arrays; i++)
{
huge_char_array[i] = (char *)malloc(sizeof(char) * 10000000);
}
9 Comments
Ed Swangren
Don't cast the return value of
malloc in C.
Gir
You could use only one malloc outside the loop of num_arrays*10000... and then assign each element in the array apointer with an offset from that allocated memory. but this makes free() a bit stranger
netcoder
This is very bad. You're allocating
num_arrays (10 bytes), instead of num_arrays * sizeof(char *) (10 pointers). You'll probably overrun huge_char_array after 2 or 3 iterations.
Abhinav
@netcoder Ohh yes, thanks for pointing that out. It's been more than 6 years when I last touched C (not an excuse, I am just sad).
Ed Swangren
@Chris: Because in C a
void* can be implicitly and safely converted to any other pointer type, so the cast is redundant. On top of that, on older compilers (<= C89) you may forget to include stdlib.h, in which case malloc will be assumed to be a function which returns int and you will get strange problems when you run the code. The cast hides the error. Also, these "older" standard compilers are not as rare as you may think. VS ships with a C89 compiler and they have no plans to meet later standards. |
I believe this is the optimal way because there is only one dynamic allocation, reducing overhead from fragmentation of the heap and the time taken to allocate. You can use the STRING_INDEX utility function to access the nth string.
Also, using calloc() instead of malloc() zeroes out buffer to ensure that all strings are NUL terminated.
#define STRING_SIZE 10000000
#define NUM_STRINGS 10
#define STRING_INDEX(array, string_idx) ((array) + (string_idx) * STRING_SIZE)
int main(int argc, char **argv) {
char *array_of_strings = calloc(NUM_STRINGS, STRING_SIZE);
// Access 8th character of 7th string
char c = STRING_INDEX(array_of_strings, 7)[8];
// Use array
// Free array when done
free(array_of_strings);
return 0;
}
Comments
I have been allocating 50K with each request I process this resulted in memory fragmentation. So I switched to allocate 50k * 100 = 5MB and reuse the pool when ever I need to allocate one more. If the requests increases beyond my pool capacity I double the pool size to avoid memory fragmentations. Based on this I would recommend allocating a huge chunk of memory maybe in your case 10 * 10000000 to reuse it. I am sorry but I cannot share the code that handle the pool here. Also use malloc and free and do not use new and delete.
malloc won't through exception it will simply return null if there are no memory available.
if you insist on using malloc for each element individually you can create a heap with maximum size to avoid fragmentations and in that case you will have to define the heap size.
please referee to MSDN for more details about how to create a heap and use it or the other allocation methods available for windows users. http://msdn.microsoft.com/en-us/library/windows/desktop/aa366533(v=vs.85).aspx
3 Comments
Ed Swangren
"as you are allocating primitive data type which has no constructor/destructor you should use malloc." - This question is tagged C, not C++, so what other choice is there? -1 for the C++ lingo that is present in this response.
Mahmoud Fayez
forgive me I am new to the StackOverFlow. I will remove that part
Ed Swangren
Haha, nothing to forgive :) It wasn't a personal attack, just have to be careful to discern between C and C++ quesitons. Once the C++ stuff is removed I'll take my downvote away
char *arr[SOME_SIZE];
for(int i = 0; i < SOME_SIZE; ++i) {
arr[i] = malloc(10000000);
if(!arr[i])
// allocation failed, do something
}
Just realize that you're allocating ~9.5MB for every element in the array (i.e., SOME_SIZE * 10000000 total bytes)
char *c[]={malloc(),malloc(),malloc()};?