I've been working on this Asynchronous file I/O class for a while now. It compiles fine on it's own, but generates strange linker errors when I try to test the class to verify it works properly. I have no idea why it is generating linker errors, and the functions within the class are defined in it's own .cpp file. I've been at this for a little while now and I'm kinda lost. Hopefully I didn't overcomplicate this class definition. It uses a template and it's defined within it's own namespace. I'll post the code below; and also, if anyone finds this code useful in some way, feel free to use it for whatever you want.

The header:

#pragma once
//-----------------------------------------------------------------------------
// Asynchronous File I/O class
//-----------------------------------------------------------------------------
#define ASYNC_IO_SEARCH_COMPLETE 0x00000001
#define ASYNC_IO_SEARCH_INCOMPLETE 0x00000002
#define ASYNC_IO_SEARCH_FAILED 0x00000003
#define ASYNC_IO_READWRITE_SUCCEEDED 0x00000010
#define ASYNC_IO_READWRITE_FAILED 0x00000020
#define ASYNC_IO_READWRITE_PENDING 0x00000030
#define ASYNC_IO_READWRITE_INCOMPLETE 0x00000040
namespace async_io
{
template <class ptrType>
class CAsyncIO
{
public:
CAsyncIO(): Status(0), /*FileName(NULL),*/ SearchThreadHandle(NULL), SearchEventHandle(NULL), DataPtr(NULL),
FileSize(0)
{
// Initialize the critical section for the file search
InitializeCriticalSection( &CriticalSection );
// Clear the overlapped struct
::ZeroMemory( &Overlapped, sizeof( OVERLAPPED ) );
//
::ZeroMemory( &FileName, 128 );
}
virtual ~CAsyncIO()
{
// Delete the critical section
DeleteCriticalSection( &CriticalSection );
}
public:
DWORD LocateFile( CHAR* szFileName, bool bFullPathIncluded = false );
DWORD Open( DWORD dwDesiredAccess, DWORD dwShareMode, DWORD dwCreationDisposition, bool bNoAllocPointer = false );
DWORD Read( DWORD dwBytesToRead, PDWORD pdwBytesRead, ptrType Ptr = NULL );
DWORD Write( DWORD dwBytesToWrite, PDWORD pdwBytesWritten, ptrType Ptr = NULL );
void Cancel();
void Close();
DWORD GetStatus() { return Status; } // Only returns the immediate status last set!
DWORD GetAsyncStatus();
HANDLE GetFileHandle() { return FileHandle; }
ptrType GetDataPtr() { return DataPtr; }
DWORD GetSize() { return FileSize; }
protected:
unsigned int __stdcall FileSearchThread( void* Params )
{
// Set the file search status to incomplete
Status = ASYNC_IO_SEARCH_INCOMPLETE;
// Enter the critical section. STL is not thread safe.
EnterCriticalSection( &CriticalSection );
// Search for the file requested (this may take a while)
strcpy( FileName, effect_api::GetFilePath( (char*) Params ).c_str() );
// Leave the critical section; all the non-thread safe code is finished.
LeaveCriticalSection( &CriticalSection );
// Signal the event of this thread finishing...
SetEvent( SearchEventHandle );
// Did we find the file at all?
if( !strcmp( FileName, "" ) )
Status = ASYNC_IO_SEARCH_FAILED;
else
Status = ASYNC_IO_SEARCH_COMPLETE;
}
protected:
DWORD Status;
DWORD FileSize;
CHAR FileName[128];
OVERLAPPED Overlapped;
HANDLE SearchThreadHandle;
HANDLE SearchEventHandle;
HANDLE FileHandle;
CRITICAL_SECTION CriticalSection;
ptrType DataPtr;
};
};


The source file:


#include "Platform.h"
#include "AsyncIO.h"




//-----------------------------------------------------------------------------
// Name: async_io::CAsyncIO<ptrType>::LocateFile
// Desc: Searches for the full path of a given file name. You can specify the
// entire file path if you need to, but remember to specify that you'll be
// doing so by setting the second parameter to true. The return value is
// a WinNT error message from GetLastError().
//-----------------------------------------------------------------------------
template <class ptrType>
DWORD async_io::CAsyncIO<ptrType>::LocateFile( CHAR *szFileName, bool bFullPathIncluded )
{
// Is the full path included?
if( bFullPathIncluded )
{
// If so, then just copy the file path over...
strcpy( FileName, szFileName );
// ... and state that the file was found (sorta)
// TODO: Verify that the file exists before continuing?
Status = ASYNC_IO_SEARCH_COMPLETE;

return ERROR_SUCCESS;
}
else
{
// If not, then create a new thread and an event that is signaled when
// the thread is finished. This thread will search for the file without
// stalling the current thread.
SearchThreadHandle = (HANDLE) _beginthreadex( NULL, 0, this->FileSearchThread, szFileName, CREATE_SUSPENDED, NULL );
if( !SearchThreadHandle )
return GetLastError();

// Create the event used to signal the end of this thread's existence.
SearchEventHandle = CreateEvent( NULL, FALSE, FALSE, NULL );
if( !SearchEventHandle )
{
CloseHandle( SearchThreadHandle );
return GetLastError();
}

// Start the thread
ResumeThread( SearchThreadHandle );

// The class instance's status should be set to incomplete by now, go ahead
// and exit with a success...
}

return ERROR_SUCCESS;
}


//-----------------------------------------------------------------------------
// Name: async_io::CAsyncIO<ptrType>::Open
// Desc: Attempts to open the file that was previously searched for. If the file
// exists, then a handle to the file will be created to read/write to/from
// the file asynchronously when ready. If the function returns E_PENDING,
// then the file search thread has not finished.
//-----------------------------------------------------------------------------
template <class ptrType>
DWORD async_io::CAsyncIO<ptrType>::Open( DWORD dwDesiredAccess, DWORD dwShareMode, DWORD dwCreationDisposition, bool bNoAllocPointer )
{
// Is the file is already open?
if( FileHandle )
return E_FAIL;

// Has the file search thread finished searching for the file?
if( SearchThreadHandle )
{
// Test the status of this thread
DWORD dwResult = WaitForSingleObject( SearchThreadHandle, WAIT_OBJECT_0 );

// The thread is still searching? Return with a pending error message.
if( dwResult == WAIT_TIMEOUT )
{
return E_PENDING;
}
// If the thread is finished, go ahead and close it now.
else if( dwResult == 0 )
{
CloseHandle( SearchThreadHandle );
SearchThreadHandle = NULL;
}
// If the thread failed somehow, return the detailed error code...
else
{
return GetLastError();
}
}

// Attempt to open the actual file.
FileHandle = CreateFile( FileName, dwDesiredAccess, dwShareMode, NULL, dwCreationDisposition,
FILE_FLAG_RANDOM_ACCESS|FILE_FLAG_OVERLAPPED, 0 );
if( !FileHandle )
{
return GetLastError();
}

// Get the file size and allocate a pointer large enough to hold the entire file (if desired)
FileSize = GetFileSize( FileHandle, NULL );

if( !bNoAllocPointer )
{
DataPtr = new BYTE[FileSize];
if( !DataPtr )
{
return E_OUTOFMEMORY;
}
}

// Now we can call read/write when necessary.

return ERROR_SUCCESS;
}


//-----------------------------------------------------------------------------
// Name: async_io::CAsyncIO<ptrType>::Read
// Desc: Reads in the file's data asynchronously. The first parameter can be
// either the desired number of bytes to read or the size of the entire file.
// If 0 is specified, then this method will default to the file's total
// size. The second parameter returns the number of bytes actually read.
// The third parameter is an optional pointer to read data into. If this
// parameter is NULL, then data will be read into the default data pointer.
//-----------------------------------------------------------------------------
template <class ptrType>
DWORD async_io::CAsyncIO<ptrType>::Read( DWORD dwBytesToRead, PDWORD pdwBytesRead, ptrType Ptr )
{
BOOL bRet = FALSE;

// Has the file been opened?
if( !FileHandle )
return E_FAIL;

// How much data are we reading in?
if( dwBytesToRead == 0 )
{
dwBytesToRead = FileSize;
}

// Are we using the third parameter?
if( Ptr != NULL )
{
// If so, read data into this pointer. Assume the pointer is unallocated
// and that the caller is responsible for deallocating it.
Ptr = new BYTE[dwBytesToRead];

if( !Ptr )
return E_OUTOFMEMORY;

// Now let's read in the data
bRet = ReadFile( FileHandle, Ptr, dwBytesToRead, pdwBytesRead, &Overlapped );
}
else
{
// If not, use the previously allocated pointer. If it does not exist, then
// the user didn't want it allocated when ::Open was called. In my honest
// opinion, I wouldn't recommend allocating it at the specific size set because
// what if we have subsequent calls to this method where a larger pointer is
// needed later? Sure I could use a variable to keep track of the current
// allocation size, but I don't feel like going through that hassle, and I won't.
// So if you didn't allocate it before and all of a sudden you want it now? Guess
// what?? Tough luck pal! Too bad, so sad, cry me a river! I hope you like getting
// a big fat E_INVALIDARG as your return value!!!

if( !DataPtr )
return E_INVALIDARG;

// Read in the desired amount of data
bRet = ReadFile( Filehandle, DataPtr, dwBytesToRead, pdwBytesRead, &Overlapped );
}

// Check for errors.
if( !bRet )
{
DWORD dwError = GetLastError();

// If it's pending or still in the process of reading, we're okay.
if( dwError == ERROR_IO_PENDING || dwError == ERROR_IO_INCOMPLETE )
{
Status = ASYNC_IO_READWRITE_PENDING;
}
else // Uh oh, something else happened...
{
Status = ASYNC_IO_READWRITE_FAILED;

return dwError;
}
}

return ERROR_SUCCESS;
}

//-----------------------------------------------------------------------------
// Name: async_io::CAsyncIO<ptrType>::Write
// Desc: Writes in the file's data asynchronously. The parameters are pretty
// much the same as the above (with a hint of common sense), only the
// 3rd parameter is required to be valid, not optional. Just don't bite off
// more than you can chew; be sure that the allocation size and the pointer
// itself match or the allocation size isn't greater than the pointer itself.
//-----------------------------------------------------------------------------
template <class ptrType>
DWORD async_io::CAsyncIO<ptrType>::Write( DWORD dwBytesToWrite, PDWORD pdwBytesWritten, ptrType Ptr )
{
BOOL bRet = FALSE;

// Has the file been opened?
if( !FileHandle )
return E_FAIL;

// Do we have a valid pointer passed in?
if( !Ptr )
return E_INVALIDARG;

// How much data are we reading in?
if( dwBytesToRead == 0 )
{
dwBytesToRead = FileSize;
}

// Start writing...
bRet = WriteFile( FileHandle, Ptr, dwBytesToWrite, pdwBytesWritten, &Overlapped );

// Check for errors.
if( !bRet )
{
DWORD dwError = GetLastError();

// If it's pending or still in the process of reading, we're okay.
if( dwError == ERROR_IO_PENDING || dwError == ERROR_IO_INCOMPLETE )
{
Status = ASYNC_IO_READWRITE_PENDING;
}
else // Uh oh, something else happened...
{
Status = ASYNC_IO_READWRITE_FAILED;

return dwError;
}
}

return ERROR_SUCCESS;
}


//-----------------------------------------------------------------------------
// Name: async_io::CAsyncIO<ptrType>::Cancel
// Desc: Cancels any asynchronous file read/write operations going on (if any).
//-----------------------------------------------------------------------------
template <class ptrType>
void async_io::CAsyncIO<ptrType>::Cancel()
{
// Verify we have a valid file handle first.
if( !FileHandle )
return;

// Cancel any asynchronous file read/write operations now.
::CancelIoEx( FileHandle, &Overlapped );
}

//-----------------------------------------------------------------------------
// Name: async_io::CAsyncIO<ptrType>::Close
// Desc: Cancels any asyncronous operations, closes all handles, and deallocates
// any pointers. Note that it doesn't delete the critical section. The
// critical section is deleted in the deconstructor. That is by design
// in case the class instance is used more than once.
//-----------------------------------------------------------------------------
template <class ptrType>
void async_io::CAsyncIO<ptrType>::Close()
{
// Be sure that there are no asynchronous reads/writes being performed now.
Cancel();

// Kill the thread if it exists
if( SearchThreadHandle )
{
// Wait for the thread to finish or abort
WaitForSingleObject( SearchThreadHandle, INFINITE );

// Now kill the thread
CloseHandle( SearchThreadHandle );
SearchThreadHandle = NULL;
}

// Kill the event if it exists
if( SearchEventHandle )
{
CloseHandle( SearchEventHandle );
SearchEventHandle = NULL;
}

// Deallocate the default pointer if it exists
SAFE_DELETE_ARRAY( DataPtr );

// Now close the file
if( FileHandle )
{
CloseHandle( FileHandle );
FileHandle = NULL;
}

return 0;
}

//-----------------------------------------------------------------------------
// Name: async_io::CAsyncIO<ptrType>::GetAsyncStatus
// Desc: Returns the status of the current asynchronous operation. This method
// only returns a WinNT error code. To get the simplified error code ass-
// ociated with this class, call CAsyncIO::GetStatus() afterwards.
//-----------------------------------------------------------------------------
template <class ptrType>
DWORD async_io::CAsyncIO<ptrType>::GetAsyncStatus()
{
DWORD dwBytesTransferred;

// Get the overlap status (don't wait for it to finish).
BOOL bRet = GetOverlappedResult( FileHandle, &Overlapped, &dwBytesTransferred, FALSE );

// If it returns false, either it's not done yet or it failed...
if( !bRet )
{
// Check the error code
DWORD dwError = GetLastError();

// Has the operation even started?
if( dwError == ERROR_IO_PENDING )
{
Status = ASYNC_IO_READWRITE_PENDING;
return dwError;
}
// Is the operation started but unfinished?
else if( dwError == ERROR_IO_INCOMPLETE )
{
Status = ASYNC_IO_READWRITE_INCOMPLETE;
return dwError;
}
// Anything else is considered a major error...
else
{
Status = ASYNC_IO_READWRITE_FAILED;
return dwError;
}
}

// If we get here, then the data is finished being read or written.

return ERROR_SUCCESS;
}


As you can see, the class's function methods are defined, but I still get these darn linker errors! And if this matters, this is the test program I've written to test the class:


#include "AsyncIO.h"

using namespace async_io;

async_io::CAsyncIO<void*>* FileIO = NULL;

int main()
{
// Create a new instance of the class
FileIO = new async_io::CAsyncIO<void*>();

// Lets try searching for a file
printf( "Beginning search for the file...\n" );
if( FAILED( FileIO->LocateFile( "eb_property.cpp", false ) ) )
{
delete FileIO;
return -1;
}

// Wait for it to finish searching for the file...
printf( "Searching" );

while( true )
{
DWORD dwStatus = FileIO->GetStatus();

if( dwStatus == ASYNC_IO_SEARCH_INCOMPLETE )
{
printf( "." );
continue;
}
else if( dwStatus == ASYNC_IO_SEARCH_COMPLETE )
{
printf( "Done\n" );
break;
}
else
{
printf( "An error has occured...\n" );
delete FileIO;
return -1;
}
}

// We're finished
FileIO->Close();

delete FileIO;

return 0;
}


This is the linker error it gives me:

asyncio2.obj : error LNK2001: unresolved external symbol "public: void __thiscall async_io::CAsyncIO<void *>::Close(void)" (?Close@?$CAsyncIO@PAX@async_io@@QAEXXZ)
asyncio2.obj : error LNK2001: unresolved external symbol "public: unsigned long __thiscall async_io::CAsyncIO<void *>::LocateFile(char *,bool)" (?LocateFile@?$CAsyncIO@PAX@async_io@@QAEKPAD_N@Z)
Debug/asyncio2.exe : fatal error LNK1120: 2 unresolved externals

It doesn't really make any sense to me. Does it have to do with the template? This sucks. Any ideas? Thanks.

EDIT: It doesn't matter what version of Visual Studio C++ I use, I still get the same linker errors. From Visual Studio 6.0 to .net 2008 and beyond...

Yes, it is because of the template; you can not declare a templated class or function in a header and then define it in a .cpp file.

The whole templated class, declarations and definitions, must be avalible when being used in a translation unit.

Typically this means that you'll either write the whole thing in the header file, or declare in a header file, define in another file (typically with a .inl extension) and then include that file at the end of the header file.

In your project itself you'll include the header file and that is all. You do not have a .cpp file for the definition of the functions.

Why thank you all for your replies. Now that makes perfect sense now that I think about it. It looks like that's how the std::vector classes do it. I'm surprised I didn't put 2 and 2 together, especially after working on a project that actually used .inl files for template classes.

Phantom's nailed your immediate problem, but there are a few other things with the code // Attempt to open the actual file. FileHandle = CreateFile( FileName, dwDesiredAccess, dwShareMode, NULL, dwCreationDisposition, FILE_FLAG_RANDOM_ACCESS|FILE_FLAG_OVERLAPPED, 0 ); if( !FileHandle ) { return GetLastError(); } CreateFile doesn't return NULL on failure. DataPtr = new BYTE[FileSize]; if( !DataPtr ) { return E_OUTOFMEMORY; } You may have overloaded new so this doesn't apply, but new doesn't return NULL on failure, it throws std::bad_alloc // The thread is still searching? Return with a pending error message. if( dwResult == WAIT_TIMEOUT ) { return E_PENDING; } ... // If the thread failed somehow, return the detailed error code... else { return GetLastError(); } You're mixing error code genuses (Win errors and COM HRESULT errors). Mixing these only works if you explicitly text against 0 as success/failure conditions, you're using FAILED() which checks for negative values. This will catch E_PENDING and suchlike (since those are negative), but will not catch any error returned from GetLastError() (which aren't). You can transform Windows errors to COM errors with HRESULT_FROM_WIN32(). If you do this, change the return type of your functions to HRESULT for clarity. DWORD async_io::CAsyncIO::LocateFile( CHAR *szFileName, bool bFullPathIncluded ) Since you're already tied to Windows, consider using something like PathIsRelative or checking whether the second character of szFileName is a colon instead of asking the user to tell you whether it is a full path or not. I'd also explicitly document the lifetime requirements of the szFileName parameter. As it's passed to a different thread, if you do something like this: std::ostringstream fileNameStream; fileNameStream << someNonRootSelectedFolder << "myFile.cpp"; FileIO->LocateFile(fileNameStream.str().c_str(), false); The thread could potentially end up working on deleted data. SearchThreadHandle = (HANDLE) _beginthreadex( NULL, 0, this->FileSearchThread, szFileName, CREATE_SUSPENDED, NULL ); if( !SearchThreadHandle ) return GetLastError(); // Create the event used to signal the end of this thread's existence. SearchEventHandle = CreateEvent( NULL, FALSE, FALSE, NULL ); if( !SearchEventHandle ) { CloseHandle( SearchThreadHandle ); return GetLastError(); } CancelIoEx ties you to Vista+ so it's worth cleaning up the allocated thread stack and removing the thread from your process with TerminateThread() before closing the handle. // Test the status of this thread DWORD dwResult = WaitForSingleObject( SearchThreadHandle, WAIT_OBJECT_0 ); This is semantic but WAIT_OBJECT_0 is generally used as to check against the return value to see if the handle is signalled. It works as intended here since it has a value of 0, but it's not terribly usual. Cancel(); ... SAFE_DELETE_ARRAY(DataPtr); After cancelling the IO, you have to wait for the IO operation to complete before you start deallocating all the buffers used in the operation.

Looks like great advice, but I do have one question. Is CancelIoEx really for Vista and later? If so, then why can I compile that function without errors in Visual Studio 6.0? I'll take your word for it though, since I'm still a bit new to async I/O.

Okay, one more problem. Remember that function that I'm passing in _beginthreadex? Now VC++ is complaining about it. This is the error I get after moving it into the .inl file.


h:\cybercrash\source\asyncio.inl(33) : error C3867: 'async_io::CAsyncIO<ptrType>::FileSearchThread': function call missing argument list; use '&async_io::CAsyncIO<ptrType>::FileSearchThread' to create a pointer to member
1> with
1> [
1> ptrType=void *
1> ]
1> h:\cybercrash\source\asyncio.inl(16) : while compiling class template member function 'DWORD async_io::CAsyncIO<ptrType>::LocateFile(CHAR *,bool)'
1> with
1> [
1> ptrType=void *
1> ]
1> h:\cybercrash\source\game.cpp(153) : see reference to class template instantiation 'async_io::CAsyncIO<ptrType>' being compiled
1> with
1> [
1> ptrType=void *
1> ]

I followed the compiler's instruction and now it gives me this instead:


1>h:\cybercrash\source\asyncio.inl(33) : error C2664: '_beginthreadex' : cannot convert parameter 3 from 'unsigned int (__stdcall async_io::CAsyncIO<ptrType>::* )(void *)' to 'unsigned int (__stdcall *)(void *)'
1> with
1> [
1> ptrType=void *
1> ]
1> There is no context in which this conversion is possible
1> h:\cybercrash\source\asyncio.inl(16) : while compiling class template member function 'DWORD async_io::CAsyncIO<ptrType>::LocateFile(CHAR *,bool)'
1> with
1> [
1> ptrType=void *
1> ]
1> h:\cybercrash\source\game.cpp(153) : see reference to class template instantiation 'async_io::CAsyncIO<ptrType>' being compiled
1> with
1> [
1> ptrType=void *
1> ]

I tried casting, but that didn't work. Now what?

Using #define on keywords is a good way to break existing code. For example, pretty much any STL header is going to directly or indirectly include a file that uses placement new. If that happens after that kind of #define then lots of code is going to blow up unhappily.