Quote:

It seems like no matter how much data I try to read with ReadFile() it still behaves synchronously.

Quote:

Personally, I use a block size of 32Mb.

Quote:

Original post by Jan Wassenberg I bet the cause is this: your CreateFile flags do not include FILE_FLAG_NO_BUFFERING, so the IO still goes through the OS's (synchronous) file cache.

Quote:

Quote: Personally, I use a block size of 32Mb. Wow, that's a lot. Did you mean 32KB? That's what I have found to be optimal with my aio code.

Quote:

Original post by BradDaBug Or maybe it is working and the results aren't quite as obvious as they are on *nix. But if I can read in several MB without seeing any evidence that it's actually asynchronous, what's the point? I hope I'm still doing something wrong.

Quote:

While that would help increase the speed of transfers, it is not a requirement to enable asynchronous read/writes.

Quote:

I actually use a variable block transfer size depending on the total amount of data that needs to be transferred -- but this block size is capped between 64KB and 1MB. So no, I don't use 32MB, sorry.

#include <stdio.h>
#include <windows.h>

#define BLOCK_SIZE				65536
#define TEST_FILENAME			"C:\\Games\\Quake 4 Demo\\q4base\\pak001.pk4"

static HANDLE hdl;
static unsigned long offset, bytesRead, blockCount, remainder;
static void *basePtr;
static bool asyncDone;
static OVERLAPPED ov;


static void _readNextBlock(void);

static VOID CALLBACK _blockComplete(DWORD dwErrorCode, DWORD dwNumberOfBytesTransferred, LPOVERLAPPED lpOverlapped)
{
	basePtr = (void *) ((char *)basePtr + dwNumberOfBytesTransferred);
	bytesRead += dwNumberOfBytesTransferred;

	// check for remaining blocks
	if ((blockCount > 0) || (remainder > 0))
		_readNextBlock();
	else
		asyncDone = true;
}

static void _readNextBlock(void)
{
	LARGE_INTEGER li;
	unsigned long size;

	// determine block size
	if (blockCount > 0)
	{
		size = BLOCK_SIZE;
		blockCount--;
	}
	else
	{
		size = remainder;
		remainder = 0;
	}

	// setup overlapped structure
	memset(&ov, 0, sizeof(ov));
	li.QuadPart		= offset + bytesRead;
	ov.Offset		= li.LowPart;
	ov.OffsetHigh	= li.HighPart;
	ov.hEvent		= NULL;

	ReadFileEx(hdl, (LPVOID)basePtr, size, &ov, _blockComplete);
}

void asyncRead(void *ptr, unsigned long offs, unsigned long size)
{
	blockCount = size / BLOCK_SIZE;
	remainder  = size % BLOCK_SIZE;

	basePtr = ptr;
	asyncDone = false;
	bytesRead = 0;
	offset = offs;

	_readNextBlock();
}

void main(void)
{
	void *ptr;
	unsigned long size = 1024 * 1024 * 128;
	unsigned long offset = 1024 * 1024 * 0;

	printf("opening file...\n");
	hdl = CreateFile(TEST_FILENAME, GENERIC_READ, 0, NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL | FILE_FLAG_OVERLAPPED, NULL);

	printf("allocating memory...\n");
	ptr = malloc(size);

	printf("reading...\n");
	asyncRead(ptr, offset, size);

	while (!asyncDone)
	{
		printf("read %d bytes\n", bytesRead);

		// allow our thread to enter an alertable state so the async callback function will be called
		SleepEx(0, TRUE);
	}

	printf("done! total bytes read: %d (size=%d)\n", bytesRead, size);
	gets((char *)ptr);

	free(ptr);
	CloseHandle(hdl);
}

Quote:

Original post by Jan Wassenberg Can you please go into more detail as to the application? Any publication/article on the topic?

Quote:

I think you critically misunderstand how the cache works. Data from files is read INTO the cache, then copied out of it as needed. If I recall correctly, the only time this would cause async I/O to complete synchronously is if the entire amount of data for the read is in the cache, and so just needs to be copied from memory.

Quote:

Also it seems like the HasOverlappedIoCompleted() macro call isn't doing its job

Quote:

I'll send you a PM about it.

Quote:

Original post by Jan Wassenberg Quote: Also it seems like the HasOverlappedIoCompleted() macro call isn't doing its job Aha. I believe you need to zero out the overlapped structure. This macro is checking the OVERLAPPED.Internal(or InternalHigh?) field, which is set when transfer is complete.

Quote:

Original post by Jan Wassenberg Maybe, but I'm still not seeing how to "read INTO the cache" without making IO synchronous (given the Windows APIs). The HD DMAs into the *application's* buffer, which may change the data when it *believes* the IO has completed (either by seeing its data, or polling via macro, or actual GetOverlappedResult). If the OS copies data into its file cache after that, we have screwed the pooch.

Quote:

Aha. I believe you need to zero out the overlapped structure. This macro is checking the OVERLAPPED.Internal(or InternalHigh?) field, which is set when transfer is complete.

Quote:

(there are lots of other reasons it's impossible to read directly into the application buffer, but I think this one should be sufficient to convince you).

Quote:

Original post by Jan Wassenberg Namely? As stated it works great in my user-mode library (which uses FILE_FLAG_NO_BUFFER; the only copying going on is for scatter-gather list, if at all).

Quote:

I have done some digging and the NT file cache works as follows: 1) grab memory in form of VM sections 2) upon cache miss, map (not copy!) file into one of these sections. Since Mm doesn't support async page faults, we have explained why synchronous operation is creeping in. => use FILE_FLAG_NO_BUFFER.

Quote:

Lots of requirements have to be met to do DMA (though not all are necessary in all cases).

Quote:

You could have just read Inside Windows 2000, like I have.

Quote:

UPDATE: After doing some digging in MSDN and the IFS kit, and then having that information confirmed by two friends (more knowledgeable than I in driver matters), I've determined that async I/O reads that involve the cache do NOT block when a page-in is necessary. It's possible to detect when a page-in would occur, and offload the operation to a worker thread.

Quote:

The only time that page-in blocking is possible (apart from crappy drivers that don't check for that) is using the fast I/O path, which is only used for synchronous reads in which all of the data is in the cache (in an earlier post I mistakenly thought this path was used for async I/O, as well).

Quote:

Original post by BradDaBug Is there any significance performance overhead with calling HasOverlappedIoCompleted() once per frame per outstanding IO request?

#define HasOverlappedIoCompleted(lpOverlapped) (((DWORD)(lpOverlapped)->Internal) != STATUS_PENDING)