C++ Error when creating threads in loops

Question

I have a problem when running the following in Visual studio 2015.

#include <thread>
#include <vector>
#include <iostream>
using namespace std;

int main() {

    for (int i = 0; i < 10000; i++) {
        cout << "Loop: " << i << endl;

        vector<thread> threads;
        for (int j = 0; j < 8; j++) {
            try {
                threads.push_back(thread([]() {int a = 4; a++; }));
            }
            catch (const system_error &se) {
                cout << se.what() << endl;
                exit(1);
            }
        }

        for (int j = 0; j < 8; j++) {
            threads.at(j).join();
        }
    }

    return 0;
}

After a few thousand loops have been run the program catches a system_error with the output:

...
3994
3995
3996
3997
3998
resource unavailable try again: resource unavailable try again

I know that there is a maximum number of threads that can be run simultaneously but in this case there's only 8 being run simultaneously after which they're destroyed. Is there a maximum on the total number that can be created?

I have tried running the above on Linux with g++ and it runs fine its only on windows (both visual studio and mingw) that the error occurs. Any help? Thanks.

push_back copies, you probably want to try again with emplace_back — Mgetz
– Mgetz, Commented Feb 18, 2017 at 13:37
@SamVarshavchik please don't spout opinionated lies in comments. The fact of the matter is C++ is what the majority of windows is built on and the VC++ team has a very interested customer in WINDIV. — Mgetz
– Mgetz, Commented Feb 18, 2017 at 13:40
@D.Rouse I would suggest giving this article a read it may illuminate what may be limiting you I would guess you're building 32bit, if you changed your build to 64bit you'll probably achieve your goal. — Mgetz
– Mgetz, Commented Feb 18, 2017 at 13:41

Mgetz · Accepted Answer · 2017-02-18 14:30:29Z

5

What happened here is best described by understanding how threads are created on windows.

Each thread when it's created is given a default 1Mb stack size. You'll notice you're getting to approximately 3998 threads which would correspond to approximately 3.9Gb of used user address space. This is not an accident, as this corresponds to most of the full 4Gb user address space available to a 32bit process under 64bit windows (Kernel space is 64bit). The extra space is where the executable and necessary system DLLs are getting loaded into.

By changing the build to 64bit you've massively expanded that address space. Having 10000 threads with a 1Mb stack is a drop in a bucket and the limiter becomes non-pagable kernel memory although the 1Mb stacks can and will be paged out causing the system to slow to a crawl.

answered Feb 18, 2017 at 14:30

Mgetz

5,1582 gold badges36 silver badges56 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

G.M. Over a year ago

Sorry, but I'm missing something here... As stated in the original post there should never be more than 8 threads in existence (9 if you include main itself) -- all 8 threads are created and destroyed within the scope of a single outer loop iteration. So why the increasing resource usage? Again, sorry if I've missed the point.

Mgetz Over a year ago

@G.M. because join doesn't mean that the threads are deallocated from an OS perspective (although the thread HANDLE must be closed), just that they join up. Like many things on windows I would suspect that thread destruction is asynchronous, but creation is synchronous. So the OS will gladly let you create as many threads as you like under the understanding you'll manage them. Therefore in my estimation the TEBs are probably waiting for destruction on the next DPC that happens at IPC < DISPATCH.

Collectives™ on Stack Overflow

C++ Error when creating threads in loops

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related