Correct atomic memory order, when 2 threads update same atomic variable and both use the value in condition (C++11)

I have one atomic integer variable which has a minimum and maximum value. Two threads update the variable, one increments it and the other decrements it. If incrementing would increase the value over the maximum value, the thread blocks and waits for a condition variable. Same thing happens with decrementing when the value hits the minimum. When the value is decremented and the old value is the maximum value, decrementing thread should notify incrementing thread and same thing should happen the other way around when incrementing.

Decrementing function body:

if (atomic_var.load(std::memory_order_acquire) == minimum) {
    std::unique_lock<std::mutex> lk(mutex);
    if (atomic_var.load(std::memory_order_acquire) == minimum) {
        //we have hit the minimum so we have to wait for the other thread to increase the variable
        condition_var.wait(lk, [&]() {
            return atomic_var.load(std::memory_order_relaxed) > minimum;
        });
    }

    //do stuff

    std::atomic_fetch_sub_explicit(&atomic_var, 1u, std::memory_order_release);
    lk.unlock();
    condition_var.notify_all();
    return;
}

//do stuff

if (std::atomic_fetch_sub_explicit(&atomic_var, 1u, std::memory_order_release) == maximum) {
    //we have hit the maximum so the other thread might be waiting
    std::atomic_thread_fence(std::memory_order_acquire);
    condition_var.notify_all();
}
//adding condition_var.notify_all() here fixes the problem but I'm afraid
//that causes a bit too great performance penalty when it could be easily avoided

Which memory orderings I should use for these checks? My current implementation seems to cause dead locks...

Edit: Changing all memory orders to std::memory_order_seq_cst doesn't seem to remove the problem.