Efficient way to initialize array with a byte sequence?

Question

Let's say I have a byte sequence of some size n (which could be 1..4 elements in the "real" code), with n = 3 for the sake of this example:

char source[n] = { 'a', 'b', 'c' }

And I have a memory range of sufficient size to hold m copies of this sequence:

char * dest = new char[m*n]

(And yes, I know std::vector, and yes, it's generally to be preferred over new'ing your own memory, and no, it's not an option for the code I am currently working on -- and anyway the problem would still be the same.)

Now I want to initialize dest with those m copies of source. There are various ways to do m copies of a single value, but apparently none for doing m copies of a sequence of values. Sure, I could use a nested loop:

for ( unsigned i1 = 0; i1 < m; ++i1 )
{
    for ( unsigned i2 = 0; i2 < n; ++i2 )
    {
        dest[ i1 * n + i2 ] = source[ i1 ];
    }
}

But somehow this lacks all the finesse that usually tells me that I got the "right" solution for a problem.

Does C++ offer some more efficient way for this?

@CaptainObvlious: Yes, several. Not part of the question. std::vector is not an option in the larger scheme this problem is a part of. — DevSolar
– DevSolar, Commented Jan 30, 2016 at 21:08
Write c++ that clearly expresses intent. Use standard containers and algorithms when possible (almost always). The optimiser will produce efficient code. — Richard Hodges
– Richard Hodges, Commented Jan 30, 2016 at 21:11

Lingxi · Accepted Answer · 2016-01-30 21:26:09Z

2

Would this give you a right feeling? (see it live here)

auto it = dest;
while ((it = std::copy(source, source + n, it)) 
       != dest + m * n);

edited Jan 30, 2016 at 21:26

answered Jan 30, 2016 at 21:19

Lingxi

15.1k4 gold badges44 silver badges104 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Rob11311 · Accepted Answer · 2016-01-30 23:20:37Z

zero-initialising is the most efficient. Where m is large and access sparse particularly, the OS may even use soft page faults to do COW of the same virtual zero-ed memory page for example, lazy allocating the requested memory when it's actually used.

Now, if you XOR every store & load from the byte arrary, with the appropriate byte from source, you can change the meaning of the NULL bit pattern.

dest[ i] = c1 ^ source[ i % n];  // store update
c2 = dest[ j] ^ source[ j % n];  // load, if dest[ j] is 0 it was never updated

In modern Out Of Order CPUs operations are not expensive compared to memory cache misses.

What you do need for this technique is to allocate the byte arrary in an OS specific way that guarantees it is zero-ed eg) mmap under Linux

Thomas · Accepted Answer · 2016-01-31 00:09:26Z

1

I would use std::copy or std::copy_n in a for loop:

for( int i = 0; i < m; ++i )
   std::copy_n(source, n, dest+n*i );

for( int i = 0; i < m; ++i )
   std::copy(source, source+n, dest+n*i );

edited Jan 31, 2016 at 0:09

answered Jan 31, 2016 at 0:02

Thomas

5,1682 gold badges17 silver badges31 bronze badges

Comments

TonyK · Accepted Answer · 2016-01-31 13:31:06Z

0

If m is large, and your memcpy is faster than a byte-based loop, it may be worth while doing an 'expanding binary fill' (I don't know what it's called because I just made it up). The idea is this:

Prime the destination array by copying n bytes from source to dest
Copy n bytes from dest to dest+n
Copy 2*n bytes from dest to dest+2*n
Copy 4*n bytes from dest to dest+4*n
Copy 8*n bytes from dest to dest+8*n
...

until you have filled at least half the destination array. Now you can fill dest with one more copy.

This might actually be slower than a naive version, depending on things like caching. So if you do try this, you will have to run some performance tests.

edited Jan 31, 2016 at 13:31

answered Jan 31, 2016 at 13:05

TonyK

17.2k4 gold badges41 silver badges77 bronze badges

Collectives™ on Stack Overflow

Efficient way to initialize array with a byte sequence?

4 Answers 4

Comments

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related