Page 1 of 1

Odd behavior after opening and closing files

Posted: Tue Jun 22, 2010 7:30 pm
by Ridgeglider
Maybe some of you could shed light on something odd... This will take some explanation, so please bear with me.

I have the following tasks (among others) :
ControlTask: highest priority (lowest PRIO value, lower than user main)
SensorTask: (prio less important than user main, more important than Controltask)
UserMain: medium (prio=50)
ProcessCmdInput (55)

ControlTask has three open files on the SD card.
SensorTask has another open file on the SD card.

ControlTask and SensorTask each log data to each file about once a second. On the hour, all the files are automatically closed, and new files are opened to limit the amount of data logged to any one file.

A PIT itimer ISR runs 100X/sec. At varying intervals, and when the ControlTask is enabled, the ISR posts a semaphore to trigger the Control task.

Each time the ControlTask runs, it queries two serial ports for command strings which are parsed into variables used to do some calcs, producing a command ouput sent to a third serial port. The various data streams are logged to SD card in the three SD card files.

The SensorTask is doing a similar job reading a 4th serial port, parsing the data, and logging to a 4th SD card file. The resulting data is used by the Controltask.

All these files (hundreds of them) created over a week or so (>500Mb) can be moved off the SD card by accessing them via the EFFS FTP server. However, with all the task activity, and the likelyhood of at least 4 open files we've found it best to disable the ControlTask and close all the files before attempting to drag them off the SD card. We do this by entering a PAUSE command over another serial port or telnet port that runs a command proccessor (the ProcessCmdTask) to accept some user input. When the PAUSE command is issued, the log files are closed and a flag is set in the PIT ISR preventing the posting of the semaphore triggering the control task. After the files are removed from SD, a RESUME command is given reopening a new (different) set of files & also clearing the flag to once again allow posting the semaphore to the ControlTask. All of this works. However, the ControlTask never receives the posted semaphores. You can see the OSsemCnt for the semaphore increase, but the ControlTask never gets it even though it has the highest task PRIO. All other tasks seem to run. Changing the prio of ControlTask lower or higher does not help. If the PAUSE command just stops the Semaphore to the ControlTask, (but avoids closing the files) things work as expected, so the issue is with either closing or re-opening the files.

One thought is that the SensorTask originally opens one file, while ControlTask opens 3 others. Can these files be closed and opened from other tasks (eg ProcessCmdTask ), or do they need to be opened and closed from that tasks that normally create and use them?

Any ideas? Thanks for reading this far!

Re: CLARIFICATION _ Odd behavior after opening and closing files

Posted: Tue Jun 22, 2010 7:43 pm
by Ridgeglider
I should say that initially, the ControlTask runs as expected upon receiving a semaphore. However, if the PAUSE command (handled within the ProcessCmdTask) closes the files, the files get succesfully closed. Next, after the RESUME command is rec'd, the PIT ISR starts posting semaphores to the ControlTask, but they are never rec'd. If the file close is avoided, ControlTask gets the semaphore and runs as expected.

The PAUSE function that closes the files basically calls f_enterFS, then f_close for each file, but it does this from a different task than the task in which the files were originally opened.

Re: Odd behavior after opening and closing files

Posted: Tue Jun 22, 2010 11:36 pm
by v8dave
Hi Rg,

Interesting reading as I am about to find out if I can access the same file from another task, albeit not at the same time. I plan to use a Semaphore to make sure only one task at a time accesses the file.

It would be interesting to write a simple test programme to open some files in 2 tasks and try to close them in another and see if that basic idea works and not use semaphores, just delay for a set time long enough to open them.

I am assuming your file descriptor is global?

Dave...

Re: Odd behavior after opening and closing files

Posted: Wed Jun 23, 2010 12:07 am
by Ridgeglider
hi Dave:
Yes the fds are all global.

I may be running into misconceptions about how to use f_enterFS too? Not sure if the ProcessCMD task has properly called it? Will check in AM. Also, wondering if instead of having ProcessCmdTask try to close all the open files itself, whether instead it should send a message to all the individual tasks to have them each close their own files?

will post what I learn, but interested in what others may know already.

Re: Odd behavior after opening and closing files

Posted: Wed Jun 23, 2010 12:42 am
by v8dave
Looking at the example in EFFS]EFFS-AppUpdate it says that f_enterFS() is called only once by each task before making any calls to the file system.

I also found with my design that I had to call f_chdrive() before I could access the SD card from each task. Are you doing this in your control task?

Setting a flag so that each task closes and waits might be a more elegant solution as somehow you are going to have to wait anyway if the Control task wants to use the files. Your control task can then wait until a series of flags are set indicating that the files are closed. Once you are done with the files, you can set the resume flag and away you go.

Dave...

Re: Odd behavior after opening and closing files

Posted: Wed Jun 23, 2010 12:53 am
by v8dave
Something interesting in the sample for Email\SSL_pop3

They call f_enterFS 3 times from the main task! I have been calling this from each task itself when it starts.

f_enterFS();

InitExtFlash(); // Initialize the CFC or SD/MMC external flash drive

OSChangePrio( HTTP_PRIO );
f_enterFS();
OSChangePrio( FTP_PRIO );
f_enterFS();
OSChangePrio( MAIN_PRIO );

It looks like a call to f_enterFS gets the current priority from these calls and uses this informaition during calls the file system. No source so not able to confirm this.

Dave...

Re: Odd behavior after opening and closing files

Posted: Wed Jun 23, 2010 9:56 am
by lgitlitz
Calling f_enterFS(); is ok to be done from each task when it starts. You should just make sure that you call it once from main and then InitExtFlash(); before you start those tasks. The examples with the file system make all the f_enterFS(); calls from user main. It does this by changing the priority of UserMain to the task priorities using the file system before making the calls. This is essentially the same thing as calling f_enterFS(); from the tasks themselves. You should only call f_enterFS(); once for each task. What this does is makes a current working directory for each task, I think up to 10 or so are supported. This will keep track of the current directory or opened file for each task.

I am still a bit confused how closing a file could be holding up the highest priority task. If this is the task with an open file do you still have the same problem?

Re: Odd behavior after opening and closing files

Posted: Wed Jun 23, 2010 11:10 am
by tod
Is it possible that the PAUSE command closed a file that the ControlTask was just about to use and is therefore now waiting for? In other words is it possible it's not pending on the semaphore?

Re: Odd behavior after opening and closing files

Posted: Thu Jun 24, 2010 8:59 am
by Ridgeglider
Thanks guys for the suggestions, however I think I was doing what folks have
suggested. For some reason, my highest PRIO ControlTask runs fine, but still
hangs after the following sequence :

1) Another task (intentionally) closes all the app's open SD card files. Notice
that these files are typically used by their own individual tasks, not by
the task that closes them all at once. Yes, there is a global object that
keeps track of all the open files, names, fps, etc. And yes Tod, we wait
until all the open files report a succesful closure from their respective
f_close() calls before continuing to the next step below.

2) Disable the PIT-instigated ControlTask semaphore trigger so the ControlTask
stops (because the semaphore is never posted).

3) Wait for user FTP access activity to complete. When that't done,

4) Re-open a new, (different) set of SD files with appropriately time-stamed
file names. Once again, the set of multiple files that's opened will
henceforth be used (and eventually closed) by the appropriate
individual tasks. Once again, wait for the appropriate f_open() return
values indicating success.

5) Finally, re-enable the PIT variable so the ISR issues the semaphore to the
ControlTask. This also works and new semaphores begin to be posted.

For some reason, it looks like the ControlTask gets the first semahpore, but
something hangs and it never gets back to the pend() to receive more.

With regard to getting SD access setup for the tasks, I handle it this way in a
clall issued from UserMain:


void ConfigFileSysAccessForTasks (void) {
//Call from UserMain
f_enterFS();

OSChangePrio(AltimeterPendFromFifoTaskPriority);
f_enterFS();

OSChangePrio(BatteryPendFromFifoTaskPriority);
f_enterFS();

OSChangePrio(CONTROL_TASK_PRIO);
f_enterFS();

OSChangePrio( MISSION_TASK_PRIO );
f_enterFS();

OSChangePrio( FTP_PRIO );
f_enterFS();

OSChangePrio( HTTP_PRIO );
f_enterFS();

OSChangePrio( MAIN_PRIO );

InitExtFlash(); // Initialize the CFC or SD/MMC external flash drive
}

The instrument is away right now, but I will continue hunting ASAP.
Thanks for input. Will keep you posted...

Re: Odd behavior after opening and closing files

Posted: Thu Jun 24, 2010 8:04 pm
by v8dave
If you pause the running system and look at the debug window (top left) and find your task, where does it show it is currently executing? It should be sitting at the OSSemPend but if something is wrong, it could be sitting somewhere else?

Dave...