I am using following code to read a file into chararcter array. Now, for small file (say for 2 MB) it is executing properly but for large file (140 MB), in my 18 GB UBUNTU server it is giving segmentation fault. Can anybody help me how to solve this ? I think 18 GB is enough to hold a 240 MB file into memory. I am using 64 bit UBUNTU and compiling using g++.
ifstream is;
char chararray [fileSize] ;
is.read(chararray, fileSize) ;
If the array is a local variable you will get a stack overflow, as it will not fit on the stack. Allocate the "array" on the heap instead, either directly using new or indirectly by using std::vector.
Or use memory mapping. See the mmap function.
std::vector is probably preferable, but since he said it was a Unix variant, ulimit -s 500000, invoked before he runs his program, should work.
ulimit run from the command line is an ugly way of non-portably solving a problem which can, in this case, be just as easily managed in the code itself.std::vector is probably preferable. But there's nothing wrong about knowing about ulimit as well.
new[] (and, of course, malloc...)mmap in a class, with munmap in the destructor. And probably give the class an interface that is as close to that of std::vector or std::array as possible, so that users feel at home with it. Because of the work involved, however, I would just read into an std::vector until I knew I needed the performance (but if I'd needed it once, I'd keep the class in my toolkit, and then use it immediately).Instead of allocating the char array on the stack, I'd try using std::vector, which will allocate dynamically on the heap:
std::vector<char> buffer(fileSize);
is.read(&buffer[0], fileSize);
The GCC compiler has the default command called size for this! Compile the program using the GCC Compiler. Then you can get the file size!
gcc -Wall test.c
size
This is for a normal C program! Since you had specified no parameter, it takes ./a.out as its default parameter!
If you have to apply some optimization, the code will become like as follows..
praveenvinny@ubuntu:~/Project/New$> gcc -Wall -o1 -fauto-inc-dec test.c -o Output
praveenvinny@ubuntu:~/Project/New$> size output
text data bss dec hex filename
1067 256 8 1331 533 output
Use text section for code size. You can use data and bss, if you want to consider global data size as well.
This will print the code size,
time -f "%e" -o Output.log ./a.out
will print the execution time to the log file called as Output.log
fileSizeis something like asize_twith value240*1024*1024...char* chararray = new char[fileSize];.delete[]instead ofdelete. Vectors are better...and as of C++11 you can even return big ones from functions without paying the cost of copying it, thanks to the copy elision that happens during return value optimization.