TCP/IP Timing over satellite

[SLOweather]

New member here, but happy NB SB72 EX user for a few years.

My app gathers data from a local serial device, and then on a schedule uploads it to my server via HTTP GET. The server can respond with one of several commands back to the unit, but usually just sends OK.

It works great on cable modems, DSL fiber, and cellular routers.

Then we tested it on a geostationary satellite Internet connection on WildBlue. It took me a while of troubleshooting and thinking to determine what I think was happening.

We were getting 3x or so as many data uploads as we should have been and there was a lot of duplication. I believe that somewhere there is an ACK timer in the program or TCP/IP stack that says, in essence, if I don't get something back from the server in X amount of time, resend the data.

So it works OK on terrestrial low latency links, but resends 3x or so on a satellite link with long latency, before the first ACK manages to traverse space and get back to the NB and it stops resending.

Is this a fair assessment of what's happening?

Because of satellite bandwidth requirements, I'd rather not just throw out the dupes on the server, but modify the code to not send dupes in the first place.

I did a little research. Is TCP TIME_WAIT the correct place to address this? If not, what is?

This brings up a related question.

No Internet link is 100% reliable. (Duh!) Upon lack-of-ack, I would like to buffer the to-send data to RAM and continue retrying on the regular upload schedule until the packet goes through. At that time, then send all of the stored data as well.

What flag should I look at to see if the ack has been received, or should I just look for the OK or whatever from the server? We already do that to parse out for commands.

I'm not the programmer, but I'm the system architect that designed this thing, helping him with research.

Thanks!

Chris

[lgitlitz]

Hi,
I believe you are correct on the assessment of this problem. The high latency connections, such as a satellite link, can trigger premature retransmits.

Because of satellite bandwidth requirements, I'd rather not just throw out the dupes on the server, but modify the code to not send dupes in the first place.

Modifying the low level TCP stack is probably the wrong approach. Without retransmits on TCP you might as well just send the data via a UDP socket.
The first thing I would try is just increasing the minimum time before a retransmit on TCP. There are TCP timing constants defined in the following file:
C:\nburn\include\constants.h
First one I would try is TCP_MIN_RTO. This is the minimum time that will occur before a retransmit is sent. By default this value is set to be (TICKS_PER_SECOND/2). I would increase this to be at least 3 times your average latency. After changing this make sure to rebuild the system library.

-Larry

[pbreed]

If you are making a TCP connection from the device to your server and sending a "GET"
the server should be throwing out any duplicate retransmitted packets as each TCP connection has a sequence number
that identifies retransmitted packets.

If this is actually the symptom you have, my guess is that the satellite service is trying to cache the GET's and messing with
your system.

So please be clear on EXACTLY where you are seeing the duplicates, and the protocols and software that is running on each end of the connection.
Is it possible that your own software is timing out and trying again?

[sehsamudra]

We were getting 3x or so as many data uploads as we should have been and there was a lot of duplication. I believe that somewhere there is an ACK timer in the program or TCP/IP stack that says, in essence, if I don't get something back from the server in X amount of time, resend the data.

VSAT Satellite Internet service providers commonly employ TCP/UDP accelerators to streamline/shape bandwidth for the smoothest possible and most efficient packet traffic flow. They are concerned about presenting rapid response (an illusion of course) of traffic, across a GEO satellite connection which is at the very least 250 mS one way delay.

The method should work like:

User PC - Satellite Terminal Ethernet Interface - Accelerator || Modem || Antenna || Free Space || Satellite Transponder || Free Space || Antenna || Modem || Accelerator + VSAT Hub

Essentially, the TCP/IP Slow-Start algorithms are faked into working faster and faster by the Accelerators (hardware! software!) sending out packets to the limit of its elastic buffers and aggregating those responses into sub-frames.

Here is a very simplified description: http://www.highspeedsat.com/tcp-ip-over ... ration.htm