Software Architecture Advice

[jwormsley]

I've got an app I need to build on an SB70LC. This app will use both serial ports running at 115200 with no handshaking. The Ethernet interface will have upwards of three sockets, one outbound (connection triggered by the state of an I/O line), and two inbound, one of which will only be listening when the other is connected (complicated reasoning behind that). This application is being built to emulate a competitors product (Lantronix) that I can no longer use, with the goal to not have to change the code in the device it is connected to.

I've built a first pass at this using a single task with nothing but select(). The timeout for select() is fairly short, and when it times out, I check the I/O line to see if I need to make or drop the outbound connection. The application works fine unless there is a large amount of data sent over the serial port in a single burst. When this happens, I miss some of the serial data. The connected device will send about 4K of data in one burst. At best, I usually only get about 3.8K of it, missing some of the data in the middle. My code double buffers the data, in that I read() into one buffer, and transfer that data to another buffer, which is then used by the write() to the appropriate socket. This is a circular buffer that has head and tail pointers and I am careful to handle the wrap appropriately, and is larger than this burst of data.

My question is, is using select() this way the most efficient way to handle reading large amounts of serial data over two ports? Or should I break this application down into multiple tasks? Perhaps one task per serial port, or even one task per file descriptor? Would doing this enable me to process data faster and not miss any like my first pass attempt?

[Chris Ruff]

In my experience, you must implement some kind of flow control to do what you are trying to do. In my opinion, without hardware flow control, you will not succeed in this. Adding tasks only makes things worse. To actually do this with two 115200 serial ports-you need to do a lot in the receive interrupts and you need to hold off the source while you deal with the sockets. Without flow control- I don't believe you will succeed.

I have been fatally bit before trying to perform 115200 burst on an older netburner card that didn't have hardware flow control at the serial port. I believe all of the recent designs support RTS-CTS all the way from the hardware into the driver.

Chris

[jwormsley]

Unfortunately, it is very difficult to change the software on the other side to add flow control. The Lantronix device I am replacing was able to manage this, and from what I gather the Netburner is supposed to be much faster. The reason I'm dropping the Lantronix is because it could not make an SSL connection with a 2048 bit key in less than 30 seconds, whereas the Netburner does it in about 4 seconds, so I would have hoped it would be just as fast handling the serial data.

I read in another thread where the serial code buffers about 2 packets worth of data, which is close to what I'm seeing as the limit to the size of burst I can handle. I'm going to look to see if I can expand this so that I can handle a larger packet.

[Chris Ruff]

You might have a chance if there is some sort of fixed delay between bursts of data. You can make the RX buffer very large to gather all of the data. The problem would show up if the serial data shows up while you are writing to the sockets. If the system is deterministic in the way data moves through it, you may have time to send out a burst before another comes in. You have a few different possibilities to 'tune' the netburner environment- you can change the default 50ms task period to something smaller, you can make the buffers larger, you can lock the task switching so that when something starts coming in you can stop uCOS from switching to a different task. If you can get it to work at all-it will be difficult. In this kind of application the RTOS actually gets in your way.

Chris

[jwormsley]

Fortunately, I only ever get one burst that big, and it is requested, so I don't have to worry about them coming at random or back to back. While I do use both serial ports, fortunately I don't use them both at the same time, and each port uses a command/response type of communication, so I shouldn't be getting network traffic at the same time I'm getting serial data. So I think as long as I can read the long burst of data reliable, then the rest falls into place.

I've been working with the serial receive routine, based on another message thread that has had activity today, and believe I have the receive working solidly now. My application level receive buffer is larger than the maximum packet, plus I'm limiting my serial receive based on my buffer remaining. Presumably, any data I don't read() this time will still trigger the select() on the read file descriptor array next time, so I can let the Netburner buffers hold it a little while until I've written some of the data out the network port and have more room to read more. So this in effect gives me a triple buffer, 1) the internal Netburner buffer of 3K, 2) my temporary read buffer of 1.5K, and 3) my application buffer of 5K (needed as the source of the subsequent write, because I was afraid writing immediately after read as the examples do might overwhelm the write() mechanism).

Even with all this, I may still miss a byte once in a blue moon, but this can be handled at a higher layer by requesting the data again.

It's not quite working yet, but I expect it to by the end of the day. The light at the end of the tunnel no longer looks like a train, anyway.