Quote:Original post by krez
This is just speculation, but I wouldn't think using more than one thread to load files would help you anyway because they are reading off the same disk. Having 2+ threads reading different files "simultaneously" would cause the drive to have to jump around, giving worse performance than just doing one after the other.

It actually does help. The reason is that blocking reads will almost never read as fast as possible, even if the file is completely defragmented.

Non SSDs have multiple platters, and data can be read from multiple platters at the same time (literally). If you're only reading one file, even completely defragmented, chances are you aren't hitting both all the platters at once, and some of the read heads of your disk will be idle. Reading from multiple threads can address this problem.

One problem that reading from multiple threads CAN'T address is that of request ordering. Only by using the native async i/o features provided by your kernel can you get the best performance. Consider the following two files:



Here, this means that first block of file 1 is at block 1 on the physical disk, and the second block of file 1 is at block 100 on the physical disk, etc. In other words, these two files are fragmented.

Now let's suppose you read just read these all from the same thread, back to back. The reads will be ordered like this:

1 -> 100 -> 5 -> 200 -> 10 -> 2

Terrible right? That's 587 blocks seeked over total.

Now let's suppose you read from multiple threads, and the reads come in like this:

1 | 200
100 | 10
5 | 2

where | means that the 2 threads issue those reads at the same time. Let's just say thread 1 always wins and is first. Then the seeking will be like this:

1 -> 200 -> 100 -> 10 -> 5 -> 2

This is 199 + 99 + 90 + 5 + 3 = 386 blocks seeked over. A little better (although change the numbers around and it could just have easily been worse).

How does async i/o come to the rescue? The reason is that you give the kernel information about ALL DESIRED BLOCKS immediately, before it issues anything.

You issue the first read, for block 1. The read queue is empty so it goes ahead and fires off the read and returns immediately. Then you immediately issue reads for 200, 100, 10, 5, and 2. Doesn't matter what order. All of these happen before 1 is finished. The kernel can re-order these for you since by definition of async you're giving it permission to return at a later date, and so the kernel can issue reads in the order 1 -> 2 -> 5 -> 10 -> 100 -> 200. This is a total of 199 blocks seeked over, which you can see is the theoretical minimum possible.


This is a contrived example, and when you take into consideration multiple heads, platters, etc it gets more complicated. But hopefully you can see that fast i/o is not as simple as it first seems :)

[Edited by - cache_hit on September 2, 2010 2:17:08 AM]

Quote:Original post by krez
Here's some tests on multithreaded file i/o (more contrived examples heh heh), I guess it depends on your usage and what disk setup you have.

Async I/O is probably your best bet, although that is a whole other animal.

Interesting article. But yea, basically same conclusion. It might hurt, it might help, depending on variables such as file fragmentation and the location of the specific files you're trying to read.

If you are looking to use multiple threads with boost, you might want to check out threadpool, a sourceforge project. As previously stated, thread creation is expensive...

GCS584

boost comes with boost::asio. It supports either native asynchronous file IO or facilities to implement such yourself.

See the logger example on asio's tutorial page.