Questions regarding WSASend using IOCP
Author
1/
Quote from MSDN:
"If an overlapped operation completes immediately, WSASend returns a value of zero and the lpNumberOfBytesSent parameter is updated with the number of bytes sent."
Is lpNumberOfBytesSent always equal to the total length of WSABUF passed to WSASend?
2/
If WSASend returns SUCCESS or WSA_IO_PENDING, a completion packet will be queued to the IOCP.
However, in my program's logic, I don't need completion packet when WSASend returns SUCCESS immediately.
Is it possible to configure IOCP so that completion packets will be posted only when WSASend returns WSA_IO_PENDING?
Quote:Is lpNumberOfBytesSent always equal to the total length of WSABUF passed to WSASend?
Short answer yes, long answer probably. The underlying protocol can have limitations, but I think this is not the case with TCP/IP. Nevertheless, you should always check the bytes sent versus the buffer size to ensure that it was sent properly.
Quote:If WSASend returns SUCCESS or WSA_IO_PENDING, a completion packet will be queued to the IOCP.
However, in my program's logic, I don't need completion packet when WSASend returns SUCCESS immediately.
Is it possible to configure IOCP so that completion packets will be posted only when WSASend returns WSA_IO_PENDING?
Your write completion routine gets called when the write completes. That's the logic, and that's how the API works. I write a ton of Win32 code, but it always gets wrapped up into appropriate classes. If I don't like how the API behaves, it's usually possible to provide a better interface this way.
If sending on TCP, partial sends are possible.
If sending on UDP, the entire buffer is sent, or nothing.
If sending on UDP, the entire buffer is sent, or nothing.
At the socket API level? I suppose that would indicate that some packets were sent successfully, then one failed? Certainly possible, though I'd also expect to receive an error code in that case. Is there another, non-error situation that could cause it? Obviously the TCP stack is free to split things up however it wants, but that should be abstracted by the API.
I don't *think* there's a situation that would return success without sending the entire buffer, but I could be wrong, and as I said before, it never hurts to add some sanity checks.
I don't *think* there's a situation that would return success without sending the entire buffer, but I could be wrong, and as I said before, it never hurts to add some sanity checks.
This whole discussion is kind of pointless, because WSASend is well documented on MSDN. Specifically, the following section:
So, for overlapped requests, all data will be sent, if possible by the underlying protocol (if you queue more than 64K for a UDP socket, you will get an error, for example). For non-blocking sockets, whatever can be sent, will be sent, and the rest not, like send() will behave on any socket. For blocking sockets, the function will block until everything is sent or there is an error, like write() will behave on a socket under UNIX.
For non-overlapped sockets, the last two parameters (lpOverlapped, lpCompletionRoutine) are ignored and WSASend adopts the same blocking semantics as send. Data is copied from the buffer(s) into the transport's buffer. If the socket is non-blocking and stream-oriented, and there is not sufficient space in the transport's buffer, WSASend will return with only part of the application's buffers having been consumed. Given the same buffer situation and a blocking socket, WSASend will block until all of the application buffer contents have been consumed.
So, for overlapped requests, all data will be sent, if possible by the underlying protocol (if you queue more than 64K for a UDP socket, you will get an error, for example). For non-blocking sockets, whatever can be sent, will be sent, and the rest not, like send() will behave on any socket. For blocking sockets, the function will block until everything is sent or there is an error, like write() will behave on a socket under UNIX.
Author
Okay, one more question (I am new to IOCP).
Suppose I have a linked list of data blocks to send via WSASend.
Since I have timeout associated with each block, I cannot use one WSASend to send the whole list.
Now, I wonder what is the best way of using WSASend:
(1) Issue one WSASend each time. Always wait in GetQueuedCompletionStatus before issue another WSASend.
(2) Issue WSASend in a loop until WSA_IO_PENDING is returned, then wait in GetQueuedCompletionStatus.
Since whenever WSASend returns success an completion packet will be queued to the completion port, issue WSASend in a loop will cause multiple dummy packets to be queued.
(3) Issue ::send (non-overlapped) in a loop until WSAEWOULDBLOCK, then issue WSASend and wait in GetQueuedCompletionStatus.
What is your opinion?
Suppose I have a linked list of data blocks to send via WSASend.
Since I have timeout associated with each block, I cannot use one WSASend to send the whole list.
Now, I wonder what is the best way of using WSASend:
(1) Issue one WSASend each time. Always wait in GetQueuedCompletionStatus before issue another WSASend.
(2) Issue WSASend in a loop until WSA_IO_PENDING is returned, then wait in GetQueuedCompletionStatus.
Since whenever WSASend returns success an completion packet will be queued to the completion port, issue WSASend in a loop will cause multiple dummy packets to be queued.
(3) Issue ::send (non-overlapped) in a loop until WSAEWOULDBLOCK, then issue WSASend and wait in GetQueuedCompletionStatus.
What is your opinion?
The very point of IOCP is that there is no "waiting". You're dealing with events.
Typical logic is something like this:
But this isn't enough, since it still requires you to still explicitly trigger the first send. This may become a problem if you have multiple threads, which may start multiple sends, causing stream contents to be interleaved.
Fully async approach has a queue. When application wants to send data, it puts all the blocks into this queue, something like this:
This allows you to have arbitrary number of producers queueing data to send, but each connection having only one send request active at any given time.
For UDP, if you need to maintain a on-the-wire buffer, you simply issue n send requests up until n packets are outstanding. Then you don't send anything anymore and exit the send chain. When your receive handler gets an ack, you then start resending. Optionally, you many need either a global (for all connections), or perhaps even local timer, to keep track of timeouts.
Actual implementation is somewhat more verbose simply due to IOCP API. Make sure to read the documentation carefully, especially about the threading requirements. Personally, I prefer boost ASIO's front-end, since it hides all the annoying details. YMMV.
The biggest advantage of such approach is that it allows for no waiting and no blocking, everything is handled in callbacks whenever data is ready, or something arrives via network.
Typical logic is something like this:
void sendFirst(Connection * c) { send(c->getFirst(), OnSend)};void onSend(Connection * c, Error e) { if (e) { if (e == EWOULDBLOCK) {} if (e == ....) ... } else { sendFirst(c); } }But this isn't enough, since it still requires you to still explicitly trigger the first send. This may become a problem if you have multiple threads, which may start multiple sends, causing stream contents to be interleaved.
Fully async approach has a queue. When application wants to send data, it puts all the blocks into this queue, something like this:
bool sending = false;void addBlocks(Connection * c, std::list<Block> & blocks) { lock(mutex); queue->add(blocks); if (!sending) sending = true; unlock(mutex); sendFirst();}void onSend(Connection * c, Error e) { if (e) { ... } else { lock(mutex); if (queue.empty()) { sending = false; unlock(mutex); return; } unlock(mutex); sendFirst(c); } }This allows you to have arbitrary number of producers queueing data to send, but each connection having only one send request active at any given time.
For UDP, if you need to maintain a on-the-wire buffer, you simply issue n send requests up until n packets are outstanding. Then you don't send anything anymore and exit the send chain. When your receive handler gets an ack, you then start resending. Optionally, you many need either a global (for all connections), or perhaps even local timer, to keep track of timeouts.
Actual implementation is somewhat more verbose simply due to IOCP API. Make sure to read the documentation carefully, especially about the threading requirements. Personally, I prefer boost ASIO's front-end, since it hides all the annoying details. YMMV.
The biggest advantage of such approach is that it allows for no waiting and no blocking, everything is handled in callbacks whenever data is ready, or something arrives via network.
This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement
