TCP Packets Out of Order
Posted: Thu May 12, 2016 10:28 am
I am trying to extract data from a website, much like the Weather Demo on the website. In fact, I started with that program and modified from there. When I set it up for the site I wanted, it would connect quickly, but then time-out in the read loop on most attempts after reading some of the data. Only when I expanded the time-out for each read to over 5 minutes would the data complete in about 50% of the attempts, but would take half an hour to complete.
Wireshark shows a very large number of packets out of order. I saw a function in tcp.h, SetOutOfOrderbuffers(int fd,BYTE max) and tried that. It worked great on max settings of 5 through 255. It took 2-3 seconds to complete (50K of data) and would only time-out about 5% of the time at the lower settings.
Now to the issue. The program is set up to loop and get the data after a period of time. After a few runs, the connect() function returns with a -5 (no sockets available). After that, there is no TCP. The program closes the fd after each run. The fd starts out as 35 and sometimes, after a time-out, it re-starts as 34 the next time through. This never happens unless the SetOutOfOrderbuffers(int fd,BYTE max) is used.
Is there a way to use SetOutOfOrderbuffers(int fd,BYTE max) so that it doesn't lock up the sockets? Or is there a preferred way to deal with a large number of out of order packets?
Configuration:
MOD54415 (tried it on MOD5282 with same results)
Mod-Dev-70 R1.93
NBEclipse 2.80 (tried it on 2.75 with same results)
Here is the HTTP request set-up:
Thanks
Ed
Wireshark shows a very large number of packets out of order. I saw a function in tcp.h, SetOutOfOrderbuffers(int fd,BYTE max) and tried that. It worked great on max settings of 5 through 255. It took 2-3 seconds to complete (50K of data) and would only time-out about 5% of the time at the lower settings.
Now to the issue. The program is set up to loop and get the data after a period of time. After a few runs, the connect() function returns with a -5 (no sockets available). After that, there is no TCP. The program closes the fd after each run. The fd starts out as 35 and sometimes, after a time-out, it re-starts as 34 the next time through. This never happens unless the SetOutOfOrderbuffers(int fd,BYTE max) is used.
Is there a way to use SetOutOfOrderbuffers(int fd,BYTE max) so that it doesn't lock up the sockets? Or is there a preferred way to deal with a large number of out of order packets?
Configuration:
MOD54415 (tried it on MOD5282 with same results)
Mod-Dev-70 R1.93
NBEclipse 2.80 (tried it on 2.75 with same results)
Here is the HTTP request set-up:
Code: Select all
siprintf(DestGet,
"GET /eng/station/month/?type=1&sid=11375&tz=EST&pres=2 HTTP/1.1\r\n"
"Accept: text/html\r\n"
"User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko\r\n"
"Host: www.tides.gc.ca\r\n"
"\r\n");Ed