TCP Packets Out of Order

[ecasey]

I am trying to extract data from a website, much like the Weather Demo on the website. In fact, I started with that program and modified from there. When I set it up for the site I wanted, it would connect quickly, but then time-out in the read loop on most attempts after reading some of the data. Only when I expanded the time-out for each read to over 5 minutes would the data complete in about 50% of the attempts, but would take half an hour to complete.

Wireshark shows a very large number of packets out of order. I saw a function in tcp.h, SetOutOfOrderbuffers(int fd,BYTE max) and tried that. It worked great on max settings of 5 through 255. It took 2-3 seconds to complete (50K of data) and would only time-out about 5% of the time at the lower settings.

Now to the issue. The program is set up to loop and get the data after a period of time. After a few runs, the connect() function returns with a -5 (no sockets available). After that, there is no TCP. The program closes the fd after each run. The fd starts out as 35 and sometimes, after a time-out, it re-starts as 34 the next time through. This never happens unless the SetOutOfOrderbuffers(int fd,BYTE max) is used.

Is there a way to use SetOutOfOrderbuffers(int fd,BYTE max) so that it doesn't lock up the sockets? Or is there a preferred way to deal with a large number of out of order packets?

Configuration:
MOD54415 (tried it on MOD5282 with same results)
Mod-Dev-70 R1.93
NBEclipse 2.80 (tried it on 2.75 with same results)

Here is the HTTP request set-up:

Code: Select all

siprintf(DestGet,
			"GET /eng/station/month/?type=1&sid=11375&tz=EST&pres=2 HTTP/1.1\r\n"
					"Accept: text/html\r\n"
					"User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko\r\n"
					"Host: www.tides.gc.ca\r\n"
					"\r\n");

Thanks
Ed

[pbreed]

That seems wrong, looking at the code I don't see how it would keep the socket from closing.....

Submit a support request...

Ask whomever you are connecting to to update thir linux kernal version, about 4 years ago there was a miinline linux kernal that did not do tcp retranmits properly
if you did not negotiate SACK and window scaling...

[ecasey]

I will submit a support request.

The code loads quickly on all browsers I have tried. I don't think its the Linux Sack bug. Here is the response header:

Code: Select all

Response Header (2680 bytes):
HTTP/1.1 200 OK
Cache-Control: private
Content-Type: text/html; charset=utf-8
Server: Microsoft-IIS/7.5
X-AspNetMvc-Version: 3.0
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Date: Thu, 12 May 2016 18:31:03 GMT
Content-Length: 53861

Thanks
Ed

[pbreed]

By default windows and linux negotiate window scaling....

So the umbutu version that had the problem worked fine with devices that negotiated both, but not
if you did not negotiate the options....

IE it was tested with win and linux boxes, but was not compliant with TCP, its retranmit times were off by a factor of 1000,
ie it caluclated the RTO in msec and implemented re transmit in seconds....

[ecasey]

Sounds like that could be it. The website is Government of Canada, so I am not feeling hopeful about finding the webmaster.

The SetOutOfOrderbuffers function works because it minimizes the SACKs and the need for a retransmit.

It is also possible that my Cisco router is messing it up. I will try with a different router before submitting a ticket. I want to make sure that support can reproduce the problem.

[ecasey]

I did extensive testing on the MOD54415/MOD-DEV-70 using three release versions: 2.73, 2.75 and 2.80.

The problem of the connect() function returning with a -5 (no sockets available) only shows up with V2.80. On the earlier versions, it times out a lot, but never lock up. The time-outs look like a problem with the website - I tested two different routers with the same results.

I have attached the program I used to do the testing. It will run through for a while but eventually it gets stuck in the connect loop returning -5, and stops responding. It has a #define to switch between 2.80 and earlier versions.

Should I still raise a support ticket, or do you want to treat this as a bug report on version 2.80?

[pbreed]

Support ticket please...