Is eTPU function calls thread safe?

[v8dave]

Hi all,

Are the eTPU function calls thread safe????

I seem to have a crash that has started since I added an additional task that hands serial data on the eTPU.

I have 1 task that uses the eTPU serial port to listen for Modbus data. As I currently have nothing attached to the COM port, the task simply calls to check if there is any valid data waiting with a call to dataavail(mb_param->fd);

The Modbus functions expect you to pass in a struct pointer with the COM port configuration which has the file descriptor for the eTPU port.

I have a second task I now added which is a Modbus master. This uses the same modbus library to make the calls to send and listen for Modbus data.

This all seems to work but after anything from about 10 mins to hours, it crashes. The debug window shows all the tasks in an idle state except for the new Modbus master task indicating that the failure is within.

All I get is a lot of <symbol is not available> in the list.

I suspect this is because 2 tasks are using the Modbus library but as both are not using the same functions, I don't see any issues with thread safe access. The Modbus master and slave calls are all indepenant of each other.

Just curious if anyone has any thoughts on where to start looking (and yes Tod, I am about to order PC Lint shortly as I think this would be a wise investment)

Cheers
Dave...

[Ridgeglider]

V8:

You mentioned that both task use a struct holding the port configuration. Although it sounds like, once configured, this struct's content remains the same, is there any chance that the two tasks instead alter non-atomic elements of the struct without some kind of locking mechanism that therefore allow the struct to get corrupted?

Also, if you have two tasks using the same serial port, is there any chance that a priority inversion or a deadlock is occuring to lock out the higher priority task tht should be running? If two tasks use the same serial port, how do you decide who gets it and when? Do two semaphores compete?

[v8dave]

Hi RG,

The two tasks use different serial ports. The struct is local to each task and holds information about the serial port and Modbus state. Each function in the Modbus library uses this to handle the data.

Each tasks opens it's own serial port and sets the fd for this within the struct. As you make each call to the Modbus library, you pass in this struct as a pointer.

I am currently running the application without the slave task actually being started. All I have is the master task running to determine if the fault is related to threading. If this is the case, then I may need to wrap up the Modbus library with semaphores to protect from access at the same time.

Dave...

[Ridgeglider]

V8
I haven't fooled with it much, but when you init the eTPU uarts, one of the parameters is the eTPU priority. I run 4 eTPU serial ports (but at <9600 baud) and all use the default FS_ETPU_PRIORITY_MIDDLE, but perhaps your master might need the higher FS_ETPU_PRIORITY_HIGH???

int eTPUFDUartOpen(int RXchannel, int TXchannel, int baud_rate = 115200,
int bits_per_data_word = 8, int parity = FS_ETPU_UART_NO_PARITY,
int priority = FS_ETPU_PRIORITY_MIDDLE);

[v8dave]

Even with the priority set to middle, it works just fine until I hit this error and it crashes. Master is at 9600 and slave is at 19200.

I am doing about 10 Modbus register requests on a round robin with the task idling for 4 ticks between. I am not seeing any Modbus failures unless I have the graphics task (which is higher priority at present) doing a large refresh otherwise it can run for as long as it does without any comms errors.

Dave...

[tod]

Dave,
While you're waiting for Lint to arrive I would offer that corruption of the passed struct might be a likely culprit (assuming you can get the problem to happen when running just the one task). Wrapping that struct in validation code (before and after) might yield dividends.

If it is a re-entrancy problem or thread-like shared resource problem in a lib, lint probably won't help you much. The system libs aren't very lint friendly so even though I promote lint's use, I don't have it check the libs, that's too much "noise" even for me.

Tod

[v8dave]

Thanks for the pointers Tod.

Sadly my testing last night was interrupted overnight by Windows deciding it should reboot because of an update. This feature is now disabled. I want to control when to reboot.

I am re-running it today and see what happens. I have added additional code but I have a feeling this is in the Modbus source (which is not mine but I ported it to the Netburner from source for a Linux system)

Dave..

[v8dave]

OK. I think I found it.

In the return from the crc16 check in the Modbus library, the function would return the CRC value it failed on instead of CRC_FAILURE flag. This was OK some of the time as the value would end up being negative and this indicated an error but if the CRC was positive, it would cause the return to be valid, ei, it assumed the returned value was the number of registers read out. In my case I caught it with a return of 124 which is way more than the 8 uint_16 array that was declared therefore overwriting past the end of the array.

I had noticed the debug output was showing the occasional CRC failure (the Modbus library has strict inter character timing and some task of a higher priority seems to be causing the slight delays, which are not really an issue as I have retries to recover from failures) and this lead me to breakpoint at the return from the CRC and there I think I have found the error.

Now off to run it with a fix to return the correct state.

Dave...

[v8dave]

So far so good. Everything is running fine all day.

Cheers for all the pointers.

Dave...