cpp - representing an array of variable-sized static stack based arrays

Question

I want to create a static array of instances of a struct, which has a member -- that points to a static stack-based variable-sized byte array. (can be represented as an array)

Due to the large size of the array and the fact that it holds static data -- I want to avoid heap allocations and keep the data stack-based.

A concrete example:

struct DataBlock
{
    size_t id;
    const unsigned char* data;
};

void test()
{
    // creating an array of DataBlock, with static data of different size
    DataBlock dataArray [] = {
        {
            .id = 1;
            .data = {0xFF, 0xCC, 0XDD};
        },
        {
            .id = 1;
            .data = {0xFF};
        }
    };
}

If the data was of the same size, I could've just defined data member like that –

const unsigned char data[STATIC_SIZE];

But due to the fact that the length can be different for each member, this approach isn't relevant.

An obvious solution would be to replace data with an std::vector -- but again, I want to avoid heap allocations, due to the fact that the data is 100% static and known in compile-time.

I've explored some template-based solutions -- which eventually produce a separate function for every array size, which is also not a desired results.

I'm sure C++ has a solution for that sort of scenario.

"Due to the large size of the array [...] I want to avoid heap allocations and keep the data stack-based." - this is contradicting I think. — lorro
– lorro, Commented Dec 5, 2022 at 22:35
Side note: Stack and large size of data often don't mix because the available stack storage space is typically quite small, 1-10MB on a conventional desktop PC. Large data is normally one of the times you want a dynamic allocation. — user4581301
– user4581301, Commented Dec 5, 2022 at 22:41
Stack space is a) way less than the heap b) transient. Probably you shouldn't use it here — pm100
– pm100, Commented Dec 5, 2022 at 22:49
Given that the amount of memory that is available to a program in the "stack" tends to be significantly less (sometimes orders of magnitude less) than what is available in the "heap", keeping large arrays in the stack is simply dumb. — Peter
– Peter, Commented Dec 6, 2022 at 0:43

bitmask · Accepted Answer · 2022-12-06 07:07:06Z

The impulse to try and avoid superfluous allocations in dynamic memory ("heap") is good, but in this case it will lead to problems. Firstly, C++ does not have variable sized arrays. Secondly, if you want to use it outside the function you create it in, it will no longer exist. But most importantly, third, if your data is indeed very large, putting it in automatic storage will fill up the tiny stack-space that typical processes run with.

So, if I understand you correctly you have a large array of DataBlock objects, each of which may hold a small number of data items. The compromise then is to

not have an std::vector in each individual struct, because that will create a lot of small dynamic memory allocations. Instead allocate one large std::vector and have your data items carry iterator-pairs pointing into that vector.
Furthermore don't have your primary array in automatic memory either. For the same reasons.

Maybe you would go with something like this:

using DataBlock = std::pair<std::vector<unsigned char>::const_iterator>;
template <std::size_t N>
struct DataBlocks {
  std::vector<unsigned char> data;
  DataBlock array[N];
};
auto makeDataBlocks() {
  // return std::unique_ptr<DataBlocks<N>> for some N
}

Rulle · Accepted Answer · 2022-12-06 07:25:17Z

1

You could store all the data together in a single array whose size needs to be at least the total size of all the data. For that we have a helper class:

struct Data {
  static const int MAX_SIZE = 100;
  uint8_t data[MAX_SIZE];
  int index = 0;

  const uint8_t* initBlock(const std::vector<uint8_t>& src) {
    int at = index;
    for (auto x: src) {
      data[index++] = x;
    }
    return data + at;
  }
};

Then just call the initBlock method to put some data in that array and obtain a pointer that points at the data:

void test()
{
  Data data;
  
  // creating an array of DataBlock, with static data of different size
  DataBlock dataArray [] = {
    {
      .id = 1,
      .data = data.initBlock({0xFF, 0xCC, 0XDD})
    },
    {
      .id = 1,
      .data = data.initBlock({0xFF})
    }
  };
}

answered Dec 6, 2022 at 7:25

Rulle

4,9312 gold badges19 silver badges23 bronze badges

1 Comment

golosovsky Over a year ago

That's a really cool implementation of a solution. Thx!

Collectives™ on Stack Overflow

cpp - representing an array of variable-sized static stack based arrays

2 Answers 2

Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related