Archived Forum Post

Index of archived forum posts

Question:

Socket timeout with large file

Apr 11 '13 at 16:51

I am having an issue with messages overwriting each other.

My client apps send large messages because they have images in them. When my server application has heavy load and cannot respond to the client quickly the client times out and tries to send the next item without closing/reestablishing the connection. The server uses the ReceiveUntilMatch function to get messages from the client.

The server will not get the first message, and when ReceiveUntilMatch returns a string, it has the first 131,071 characters (same number of characters every time) of the first message that timed out and all of the second message.

Has anyone seen behavior like this?


Answer

What values do you have for MaxReadIdleMs, and MaxSendIdleMs properties on both the client and server apps? I had to play with those to match my data size. For example when I had small data transfers in testing I originally had them set low, but when I started working with larger sets of data I had to increase them to 1000. Also, if your using asynchronous sockets, you might even set it to 0 (which waits indefinitely). I guess if you did that you would use some other mechanism to determine failure, and stop the transfer... Not really sure about all that, but like I said, adjusting those properties helped me when I had that issue.


Answer

Also, if you are using ReceiveUntilMatch you would need to make absolute certain that your image data didn't by some cosmic coincidence have that exact byte sequence, because that could cause a stoppage... That's why I use a fairly random sequence for ReceiveUntilMatch, but when moving byte data I use ReceiveBytesN(byteCount), because I have no way of guaranteeing that.


Answer

I have played the timeout values and I am adjusting that, but the fact that it is timing out is not my real issue. My concern is that when there is a timeout like this, it merges two messages. To me, even if i do send two messages back to back, the ReceiveUntilMatch method should not be combining the message.

Also, i have confirmed that the image does not have the byte sequence i am giving ReceiveUntilMatch.


Answer

Here is the main parts of the client/server code.

//Server while (socket.IsConnected) { message = socket.ReceiveUntilMatch("-EOT-");

if (!String.IsNullOrEmpty(message))
{
    failedCount = 0;

    message = message.Substring(0, message.Length - 5);

    XmlDocument xmlDoc = new XmlDocument();

    try
    {
        xmlDoc.LoadXml(message);
    }
    catch (Exception)
    {
        //Invalid XML due to overlapping messages
        WriteToFile(message, MyThreadID);
    }

    DoWork(); //Simulates work load seen in production.

    socket.SendString(responseString); //Send response string.
}
else
{
    failedCount++;
}

if (failedCount > 3)
    break;

}

//Client for (int a = 0; a < numMessagesToSend; a++) { if (!clientSocket.IsConnected) { clientSocket.Connect(ServerAddress, PortNumber, false, 20000); WriteToFile(MyThreadID.ToString() + " Connected"); Console.WriteLine(MyThreadID.ToString() + " Connected"); }

bool processSuccess = false;

WriteToFile(a.ToString() + ": Start Send");
bool bSent = clientSocket.SendString(sendMessgeStart + MyThreadID.ToString() + "." + a.ToString() + sendMessageEnd);

if (bSent)
{
    Console.WriteLine(MyThreadID.ToString() + " Sent item " + a.ToString());
    String incommingMessage = clientSocket.ReceiveUntilMatch("-EOT-");
    if(!String.IsNullOrEmpty(incommingMessage))
    {
        processSuccess = true;
    }
}

WriteToFile(a.ToString() + ": " + processSuccess.ToString());//Log if each message was successful

}


Answer

well, just to be sure, I would close the client on success

if(!String.IsNullOrEmpty(incommingMessage))
{
    clientSocket.Close(100)
    processSuccess = true;
}

and if breaking from the server's while loop, I would also close on that end (if you have that immediately after the while loop, then it's probably not necessary)

if (failedCount > 3)        //also, it seems if you are sending images, this fail count should be higher, maybe try ten?
{
    socket.Close(100)
    break;
}

That is what I would try first.

In the documentation the ReceiveUntilMatch method stipulates that it

Reads text from the connected

So I might venture to guess that it is not meant for image data that is not formatted as a string... If it hits that and fails, that might be why it jumps to the next input. Have you been able to send small images? Or is it possible that this problem first occurred when you tried to pass your first image?