read entire binary file into an array in single call c++

Question

I am trying to read a binary file into an array of structure

struct FeaturePoint
{  
  FeaturePoint (const int & _cluster_id, 
            const float _x, 
            const float _y, 
            const float _a, 
            const float _b
            ) : cluster_id (_cluster_id), x(_x), y(_y), a(_a), b(_b) {}
  FeaturePoint (){}
  int cluster_id; 
  float x;
  float y;
  float a;
  float b;
};

The code below works but does this one element at a time, by pushing each new element onto an array

void LoadImageFeaturesFromBinaryFile(const char * FileName, std::vector<FeaturePoint>& features )
{
  char strInputPath[200];
  strcpy (strInputPath,"/mnt/imagesearch/tests/");
  strcat (strInputPath,FileName);
  strcat (strInputPath,".bin");
  features.clear();
  ifstream::pos_type size;
  ifstream file (strInputPath, ios::in|ios::binary|ios::ate);
  if (file.is_open())
  {
    size = file.tellg();
    cout<< "this file size is : "<<size<<" for "<<strInputPath<<" " <<sizeof( FeaturePoint )<<endl;
    file.seekg (0, ios::beg);
    while (!file.eof())
    {
      try
      { 
        FeaturePoint fp;
        file.read( reinterpret_cast<char*>(&fp), sizeof( FeaturePoint ) );  
        features.push_back(fp); 

      }
      catch (int e)
      { cout << "An exception occurred. Exception Nr. " << e << endl; }
    }

    sort (features.begin(), features.begin()+features.size(),CompareClusterIndexes);  
    file.close();
  }
}

I want to speed it up by reading the entire array in at once, which I think should look something like the following

    void LoadImageFeaturesFromBinaryFile(const char * FileName, std::vector<FeaturePoint>& features )
{
  char strInputPath[200];
  strcpy (strInputPath,"/mnt/imagesearch/tests/");
  strcat (strInputPath,FileName);
  strcat (strInputPath,".bin");
  features.clear();
  ifstream::pos_type size;
  ifstream file (strInputPath, ios::in|ios::binary|ios::ate);
  if (file.is_open())
  {
    size = file.tellg();
    file.seekg (0, ios::beg);
    features.reserve( size/sizeof( FeaturePoint ));
    try
    { 
      file.read( reinterpret_cast<char*>(&features),  size );  
    }
    catch (int e)
    { cout << "An exception occurred. Exception Nr. " << e << endl; }

    sort (features.begin(), features.begin()+features.size(),CompareClusterIndexes);  
    file.close();
  }
  else cout << strInputPath<< " Unable to open file for Binary read"<<endl;
}

But the read is causing a seg fault, how do I fix this?

I predict that once you get this working, you will be surprised at how little it improves your performance. — Nemo
– Nemo, Commented Jun 27, 2011 at 5:20
Do yourself a favor and use std::string instead of strcat. — Fred Foo
– Fred Foo, Commented Jun 27, 2011 at 5:43

Benjamin Lindley · Accepted Answer · 2011-06-27 06:35:01Z

3

This is wrong:

features.reserve( size/sizeof( FeaturePoint ));

You're about to read data into the vector, you should resize it, not just reserve, like this:

features.resize( size/sizeof( FeaturePoint ));

This also is wrong:

file.read( reinterpret_cast<char*>(&features),  size );

You're not writing over the vector's data there, you're overwriting the structure itself, along with who knows what else. It should be this:

file.read( reinterpret_cast<char*>(&features[0]),  size );

Like Nemo said though, this is unlikely to improve your performance.

edited Jun 27, 2011 at 6:35

answered Jun 27, 2011 at 5:41

Benjamin Lindley

104k11 gold badges210 silver badges285 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Nemo Over a year ago

Right, this initializes the entire (larger-than-cache) array, then fills it in. So even if the disk were not the bottleneck, this would be no faster than the original code. "reserve" plus "push_back" is usually faster... If you are not reading from the million-times-slower-than-memory disk.

cababunga · Accepted Answer · 2011-06-27 05:41:06Z

0

Your features has type is an std::vector and you casing it to char. Type vector is not an array.

answered Jun 27, 2011 at 5:41

cababunga

3,13419 silver badges23 bronze badges

Comments

andrewdski · Accepted Answer · 2011-06-27 05:41:38Z

0

I think you want

file.read( reinterpret_cast<char*>(&features[0]),  size );

You also need to make certain that size is a multiple of sizeof(FeaturePoint). Otherwise, you will read slightly too much.

answered Jun 27, 2011 at 5:41

andrewdski

5,5752 gold badges22 silver badges20 bronze badges

Collectives™ on Stack Overflow

read entire binary file into an array in single call c++

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related