4

I'm trying to figure out if the boost::multi_array constructor or resize method can throw a bad_alloc exception (or some other exception indicating the allocation or resize failed). I can't find this information in the documentation anywhere.

Clarification (added from comment):

This is a scientific algorithm that can fall back to a less memory intensive (slower) method if the allocation fails. Basically there are two dynamically allocated 3-dimensional arrays to hold "distances" (correlation) between all pairs of genes in a query and all genes in a cross-validation set for each of a large number of datasets. The slower method recalculates each distance on the fly as it is needed. This is for a C++ version of an existing Java implementation, which implemented both methods and would fall back on an out of memory exception. I don't really expect to run out of memory.

Improve this question

3 Answers 3

Reset to default
1

1st: (answering the real question): As it uses dynamically allocated memory, yes, it can throw std::bad_alloc (I have never seen boost translation std::bad_alloc exceptions; it would be crazy to do so).

2nd: (comment on your clarification): You do need the information of available physical memory to optimize the performance of your algorithm at run-time. However, you cannot rely on std::bad_alloc to determine how much memory you have available, as modern operating systems use such a thing as overcommit, meaning: they (almost) never return a failed allocation attempt, but instead just give you some "memory", which will only fail to jump into existence when you actually try to access it.

In Java this may work as the VM is doing many things for you: it tries to allocate some continuous memory chunks, and does so with respect to the available physical memory, and the available unused physical memory to decide whether it should stress the GC more or just allocate a larger junk. Also, for performance reason you need to take into account that virtual memory and physical memory are quite different concepts.

If you need to performance-optimize you algorithms for such cases (which may well be necessary, depending on your area of work), you need to inspect your platform-specific functions which can tell you how "the real world" looks like.

Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

see my comment above, the algorithm, as implemented by the original researcher in Java, supports two methods. A slower one is used if it is unable to allocate enough memory. I was looking to re-create this behavior for the C++ version, although I normally don't expect to run out of memory. We already got rid of this type of fall-back on another one of the algorithms used by the program when we found my C++ implementation was much more efficient in terms of memory than the older Java one.
0

The absence of explicit exception-specification is intentional. See this sub-section for an explanation. Further, note that an absence of an explicit specification means that there is no restriction on the type of exception a function can throw. So, at the least, the ctor and the resize function can throw exceptions in case memory is exhausted or if an item-object copy fails.

Some generic references which inspired Boost which you may be interested in are:

Improve this answer

2 Comments

@gimpf: No, but the lack of the same may have confused him.
May well be. I hope for clarification.
0

Why not test it ? It's easy to pass an absurdly high value to generate the exception.

On the other hand, what do you plan on doing if does generate this exception ? std::bad_alloc is the kind of exception that you usually cannot deal with at micro level...

For example, on a webserver, you would typically perform some cleanup (rollback on db transaction ?) and then return a 500 error to the user.

But when memory is exhausted, there is no much you can do safely, since you have to tread carefully if you don't want to once again hit the memory wall that you know is close :)

Improve this answer

11 Comments

For reference, @alfC is referring here to overcommit, which is enabled by default on most Linux distributions. Even with overcommit active, memory allocation can fail, for very large allocations. Otherwise, the system will try to find a page (4KB, 2MB, or 1GB being the typical sizes on x64) only on first access, possibly freeing up space from its caches, or roping in the OOM killer.
@alfC: Catching signals is possible on Linux, at least, though it gets finicky in multi-threaded applications AFAIK, and there's the issue of communicating the answer to the call site (a signal handler has not return) though if encapsulated in the allocation function (just writing a single byte every 4K bytes) then it's possible to (1) set a probing flag before probing and reset a segfault flag, (2) probe, (3) in the signal handler set a segfault flag if the probing flag is set, otherwise forward the signal, (4) reset the probing flag, return NULL if the segfault flag is set...
... This is still assuming that the signal handler is executed on the same thread -- otherwise you can't have thread-local flags and it's a mess. It'd be more reliable to disable overcommit at this point.
@alfC: The 4KB would be the page size, you need to write at least 1 byte per page to ensure it's committed. You can check stackoverflow.com/questions/4888067/… . 4KB will work on all existing architectures, AFAIK, though it may not be optimal (ie, it may underestimate the page size, leading to more writes than strictly necessary).
@alfC: Actually, a 4GB allocation only has 1 million 4KB pages. At this stage, the OS has hopefully switched to 2MB pages... but it's unclear how you can detect that it did...
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.