Why are log lines missing without a flush?

Question

I have a simple class in C++20 that logs data to a file line by line for Windows platform. To ensure thread safety, I use a static std::mutex to serialize the write operations.

However, I encounter an issue: if I don't explicitly call flush after each write while holding the lock, some lines are missing in the output file. When I call flush, all lines are correctly written to the file.

Here’s a simplified version of the code:

class Logger {
public:
    void Log(const std::string& message) {
        std::lock_guard lock(mtx_);
        log_file_ << std::format("{}\n", message);
        // log_file.flush(); // Uncommenting this works fine
    }

private:
    static std::mutex mtx_;
    std::ofstream log_file_{ "log.txt", std::ios::app };
};

std::mutex Logger::mtx_;

int main()
{
    std::vector<std::future<void>> futures;

    // Launch the function asynchronously
    for (int i = 0; i < 25; ++i) {
        futures.push_back(std::async(std::launch::async, []() { 
            Logger logger;
            logger.Log("Test message"); }));
    }

    // Wait for all tasks to complete
    for (auto& future : futures) {
        future.get();
    }
}

Why does this happen? Why doesn't the operating system automatically handle flushing the output when it's serialized using a mutex, without explicitly calling flush?

Is this expected behavior for file streams in C++?

If you don't flush you have no idea when the OS will actually commit the data to the backing file. That's why flush exists; to force the OS to commit buffered writes. — Jesper Juhl
– Jesper Juhl, Commented Dec 9, 2024 at 21:36
If writing to a file then (by default) buffers are not flushed. If the flushing is not serialised along with the content being written (i.e. before releasing the mutex) then the clearing of buffers and (physically) writing to the file are not serialised relative to writes on other threads. That causes undefined behaviour. — Peter
– Peter, Commented Dec 9, 2024 at 21:37
The mutex makes guarantees about memory in the same process. It says nothing about side effects beyond that; in this case, it has no effect like triggering a call to flush() or waiting for the completion of operating system calls. — Peter - Reinstate Monica
– Peter - Reinstate Monica, Commented Dec 9, 2024 at 21:38
ofstream is typically line-buffered and flushes automatically on \n. flush exists for handling streams that are not line-buffered, or when you want manual flushing. In any case, your log_file_ is not static so you are repeatedly opening and closing the file outside of your lock. — Remy Lebeau
– Remy Lebeau, Commented Dec 9, 2024 at 22:26
@zdf Singletons are just glorified global variables and have all sorts of problems. I don't think advising anyone to use singletons is a good idea. It's IMHO a rather bad idea. — Jesper Juhl
– Jesper Juhl, Commented Dec 10, 2024 at 2:13

zdf · Accepted Answer · 2024-12-10 10:08:01Z

In your example, every thread creates a new Logger instance. Each instance has its own stream, which means each instance also has its own internal buffer. You do not synchronize these buffers.

One solution is to use a single shared stream. While broken messages (e.g., "this is a te" followed by "this is a st") could occur in your example, this won't happen here because there is only one buffer. Note that depending on your platform and compiler, it might be necessary to define _DISABLE_CONSTEXPR_MUTEX_CONSTRUCTOR.

A possible implementation:

#include <fstream>
#include <mutex>
#include <vector>
#include <thread>

class logger_t
{
public:
  static logger_t& instance()
  {
    static logger_t inst; // thread safe since c++11
    return inst;
  }

  void message(const std::string m)
  {
    std::lock_guard lock(mutex_);
    os_ << m;
  }

protected:
  logger_t() = default;
  ~logger_t() = default;

  std::mutex mutex_;
  std::ofstream os_{"c:/temp/log.txt", std::ios::app};
};

void f()
{
  logger_t::instance().message( "this is a test " );
}

int main()
{
  std::vector<std::thread> threads;

  for (int i = 0; i < 123; ++i)
    threads.emplace_back(f);

  for (auto& t : threads)
    t.join();
}

Collectives™ on Stack Overflow

Why are log lines missing without a flush?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related