How to parallelize for loop inside other for loop with OpenMP

Question

I want obtain best perfomance parallelizing this loops:

// EXAMPLE
for (;;) {
    // DO SOMETHING
    for(;;) {
        // DO SOMETHING
    }
}

I know three ways of parallelize about example:

// EXAMPLE - FIRST LOOP PARALLEL
#pragma omp parallel for
for (;;) {
    // DO SOMETHING
    for(;;) {
        // DO SOMETHING
    }
}

// EXAMPLE - FIRST AND SECOND LOOP PARALLEL NO NESTED
omp_set_nested(0); // default option
#pragma omp parallel for
for (;;) {
    // DO SOMETHING
    #pragma omp parallel for
    for(;;) {
        // DO SOMETHING
    }
}

// EXAMPLE - FIRST AND SECOND LOOP PARALLEL NESTED
omp_set_nested(1);
#pragma omp parallel for
for (;;) {
    // DO SOMETHING
    #pragma omp parallel for
    for(;;) {
        // DO SOMETHING
    }
}

Which is the best way of doing that? Or in which situations should I use one or other?

Thank you.

Brice · Accepted Answer · 2018-11-08 13:06:58Z

2

A single level of parallelization is almost always the best, i.e. Example 1: first loop parallel.

Example 2 will in fact do the same as example 1 (the second #pragma omp parallel for statement is ignored as nested parallelization is disabled).

Example 3 is likely a bad idea. The overhead for spawning new threads is incurred at each iteration from loop 1. Cases where it would beat parallelizing the first loop only (or second loop only) are non trivial. It is not guaranteed to be supported by all compilers (some may still ignore the nested part).

Example 4 is another option, when parallelization of the outermost loop can't be done for any reason (memory use, for instance) but we want to spawn the threads only once and re-use them at each iteration of the outermost loop. Some synchronization overhead will be incurred at each iteration, but much less than what repeatedly spawning threads would cost.

// EXAMPLE - SECOND LOOP PARALLEL
#pragma omp parallel
for (;;) {
    // DO SOMETHING EVERY THREAD SHOULD DO
    // e.g. declare local variables, increment private counter...
    #pragma omp single
    {
    // DO SOMETHING ONCE ONLY
    // e.g. read data from a file, initialize a shared variable
    }
    #pragma omp for
    for(;;) {
        // DO SOMETHING
    }
}

answered Nov 8, 2018 at 13:06

Brice

1,5808 silver badges10 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

JuMoGar Over a year ago

OK, thank you for the explaining, I'll keep it in mind. I am new in OpenMP and I was not very clear about that

Collectives™ on Stack Overflow

How to parallelize for loop inside other for loop with OpenMP

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related