there was a discussion here: http://www.gamedev.net/community/forums/topic.asp?topic_id=275240 but I'd like to elaborate this topic. My logger uses TLS buffer for log-message formatting and std::ofstream for output. Buffering is disabled by std::ofstream::rdbuf()->pubsetbuf(0, 0). The question is: can I use std::ofstream::write() w/o locking? Locking is undesirable because logging is pretty intensive and this can reduce performance.

I'm pretty sure that nothing in the standard C++ library is thread-safe by design. C++ as such has no notion of threads, hence the standard can't make any guarantees regarding thread-safety. I'm afraid you'll have to lock if you want to share an ostream object between threads.

Quote:
The question is: can I use std::ofstream::write() w/o locking?

Maybe. The results may vary if you need to write more than one atomic piece of data, or if underlying implementation cannot perform fully atomic write.

Quote:Locking is undesirable because logging is pretty intensive and this can reduce performance.

The file write will lock on its own. File is a sequential device and cannot be accessed concurrently, unless implemented via multiplexing using one of async APIs.


You're solving the right problem but in the wrong way, sort of premature optimization.

1) Do you need each log to be reliably written. In that case you need to use fsync() or similar to cause OS to flush pending writes before proceeding, or fail. In this case, there is no way around blocking. Application cannot proceed if log write fails, which means concurrent logs will need to wait for each other, or each log to their own per-thread file.
2) Are you sure file writes are the problem? Do you generate more than 10MB per second? Can you log to a SSD? Can you log over gigabit network with hardware accelerate card? If not, then it is likely that logging will not be a bottleneck.

To avoid stalls or prevent increased jitter when logging, the preferred way would be to use either async file access (cannot be done via fstream, depends on OS), or an in-memory database running in a separate process.

A user-mode, cross-platform logging can be implemented like this (See services, logging). This uses serial out-of-thread file access. If application crashes, logs will be lost. There is no way around this without blocking.

For bulk logging (10MB+ per second), scatter/gather style logging is possible. Each thread has its own file, each log is timestamped. They can write without impeding each other (if each commit is atomic and completes), but logs must be merged before analyzing. SSD, RAID or network might be preferable for this type anyway.

Quote:Original post by Antheus
The file write will lock on its own. File is a sequential device and cannot be accessed concurrently [...]

Maybe the actual, physical file will but what about the ostream-object representing the file? What if internal state is modified by writing to a file via an ostream object?

fyi, Boost.Log is pending review and I think should be accepted pretty soon.

Thread Safety in (Microsoft's implementation of) the Standard C++ Library

Quote:Original post by cache_hit
fyi, Boost.Log is pending review and I think should be accepted pretty soon.

Boost log was pending review some 3 years ago when I first saw it. It has been in development since before then.

I no longer care about stuff coming "real soon". Give me something that has been in use for a year.

Seriously, logging is not some newfangled functionality, but the very core of every application for past 40 years.

Check out log4cxx, we use it in our codebase and are very pleased.

Quote:Original post by Antheus

Quote:Original post by cache_hit
fyi, Boost.Log is pending review and I think should be accepted pretty soon.

Boost log was pending review some 3 years ago when I first saw it. It has been in development since before then.

I no longer care about stuff coming "real soon". Give me something that has been in use for a year.

Seriously, logging is not some newfangled functionality, but the very core of every application for past 40 years.

This is a completely new logging library. The last one was abandoned by the author, this one is by a new author and his own implementation, written from the ground up, and the author is more committed to supporting it than the other author.