CW MobileGT - Recommended Serial Tx/Rx Scheme

[Tim562]

Hi Everyone,

    I've created a task that's main responsibility is to receive data from a serial port and write it into a buffer for later processing by another task. This serial task is always running but blocks so that other tasks can run when the read() function call determines that no serial data is waiting to be received (serial device opened in blocking, interrupt mode).

    My question is, what would be the best way to service the transmission of data on this same serial port? Would I open the same serial device in another task and call write(), then terminate that task when the transmit data buffer has been emptied? Is it allowed to have two separate tasks open the same serial device at the same time?

    Maybe the task that opened the serial device and services the receipt of data needs to do the transmission as well? But to do that this task would need to open the serial device in a non-blocking fashion (so calls to read() don't block) and then the task would never block and allow other tasks to run. I suppose I could include a _time_delay(xx) if read() returns no data waiting. During the _time_delay(xx) the serial task would block and allow other tasks to run but this doesn't seem like the way this should be done.

Any ideas on how others have accomplished this would be really helpful. Thanks!

~Tim

[MarkP_ · Accepted solution]

Hi Tim,

Probably your task is receiving some other messages/events from other tasks too.

Write to uart can be done directry by _io_write().

I've made the receive like this:

#define BUF_SIZE 256
uint_8 buffer[BUF_SIZE];
uint_16 charsInBuffer=0;

/************* Task main loop **************/   
while (TRUE)
{
    waitTimeout = 10; // Event wait 10 ms
    readCount = _io_read(uart_device, buffer+charsInBuffer, BUF_SIZE-charsInBuffer);
    if (readCount > 0)
    {
        charsInBuffer += readCount;
        <handle data in buffer>
        waitTimeout = 1; // Message(s) coming, shorter (1 ms) wait time
    }
    eventType = TaskEvtMsgWait(&CommTaskEventDataBox, &eventMsg, waitTimeout);
    ...
    <handle messages>
}

Event/message wait principle:

TaskEventType_t TaskEvtMsgWait(TaskEventDataBox_t *evtDataBox, TaskEventMsg_t *msg, uint_16 timeoutMS)

...

evtStatus = _lwevent_wait_ticks(&evtDataBox->taskMQXEvt, TaskEventAll, FALSE, timeoutTicks);

if (evtStatus == MQX_OK)
{
   retval = _lwevent_get_signalled(); // This may have multiple bits on
}
else if (evtStatus == LWEVENT_WAIT_TIMEOUT)
{
   //retval = _lwevent_get_signalled(); // This doesn't work, replaced by code below
   _int_disable();
   retval = evtDataBox->taskMQXEvt.VALUE; // Get bits on from event struct
   evtDataBox->taskMQXEvt.VALUE = 0;      // And reset bits
   _int_enable();

...           

[Tim562]

Hi All,

    I've tried a concept that (with minimal testing) seems to work pretty well for both transmitting and receiving data on a serial port. This way uses two seperate tasks, one for transmit and one for receive, with both tasks blocking (and not using processor cycles) when not actually servicing data.

First, create a globally accessable serial device descriptor and one task for serialTX and one task for serialRX. The main task inits the serial device (interrupt mode, blocking) and creates the two serial tasks.

TASK_TEMPLATE_STRUCT MQX_template_list[] = { /* Task number,   Entry point,       Stack, Pri, String, Auto? */  {MAIN_TASK,     Main_task,         3000,  11,  "main", MQX_AUTO_START_TASK},  {SERIALTX_TASK, MAIN_SerialTxTask, 3000,  10,  "setx", 0},  {SERIALRX_TASK, MAIN_SerialRxTask, 3000,   9,   "serx", 0},  {0,             0,                 0,      0,   0,     0}};//Declare a global FileDescriptor for access to the serial portFILE_PTR serial_fd;//Declare some global vars to config portuint_32 baud=57600;uint_32 flags=IO_SERIAL_RAW_IO;void Main_task(uint_32 initial_data)    {    _task_id tiSerialTxTaskID;    _task_id tiSerialRxTaskID;    //Init file descriptor to access serial port(9)    serial_fd =  fopen("ittyj:", IO_SERIAL_RAW_IO);    //Open in blocking mode    ioctl(serial_fd, IO_IOCTL_SERIAL_SET_BAUD, &baud); //Set baud rate    //Create the task to transmit serial data    tiSerialTxTaskID = _task_create_blocked(0, SERIALTX_TASK, 0);    if(tiSerialTxTaskID == MQX_NULL_TASK_ID)        _time_delay(1);     //Create the task to receive serial data    tiSerialRxTaskID = _task_create(0, SERIALRX_TASK, 0);    if(tiSerialRxTaskID == MQX_NULL_TASK_ID)        _time_delay(1); //While serial task is waiting for data we can run this task and do whatever//----------------------------------------------------------------------------    while(1)        {        //Do whatever needs doing...        }    fclose(serial_fd);    _mqx_exit(0);    }

The serial receive task calls read() (blocking) in an endless loop like this:

void MAIN_SerialRxTask(uint_32 initial_data)    {    uint_32 iCount = 0;    unsigned char data_buffer[2048];    while(1)        {        //Function blocks so MQX can allow other tasks to run        iCount = read(serial_fd, data_buffer, 1);        if(iCount == IO_ERROR)            break;        //Write the recvd data back out the same port        write(serial_fd, data_buffer, iCount);        }      return;    }

The transmit task calls write() to transmit the data in the TxBuffer then blocks itself again in an endless loop:

void MAIN_SerialTxTask(uint_32 initial_data)    {    while(1)        {        //Write contents of TxBuffer        write(serial_fd, TxBuffer, iTxBufferCount);        //Return to blocking state again when finished transmitting TxBuffer        _task_block();        }    return;    }

Now in order to transmit, any function can load the TxBuffer with data and unblock the transmit task:

//Load TxBuffer....//Unblock serialTx task_task_ready( _task_get_td(tiSerialTxTaskID) );

Obviously, care has to be taken to avoid conflicts when accessing the transmit and receive buffers from multiple locations (that code not shown here). But it seems like the concept of a single common serial device descriptor, that is accessed by both a transmit and receive tasks could work? The advantage of this method is that neither task is unblocking and using processor cycles unless it actually has something to do.

I suppose a variation on this theme would be to have the Txtask block when finished transmitting and unblock on a lwevent. Then, whoever wants to send data could load the TxBuffer and trigger the lwevent that wakes up the serial transmit task.

Any thoughts, cautions, observations, etc on this concept would be greatly appreciated as I'm very new to the whole MQX / RTOS thing. Thanks!!

~Tim

[DavidS]

Hi Tim,

Cool coding...good job.

Just like most stuff in life there are usually a number of ways to achieve the results you want.

The method I think you have choosen where the UART is in interrupt mode with non-blocking set means you are creating the code to decide to block your task if no charater is present. 

If you leave the UART in interrupt mode and have it configured for blocking, then the RTOS handles the blocking for you and when a charater is received then it unblocks the task and it continues.

Since you are looking to have two task access the same UART you need to use semaphore or mutext so only one task can access the UART at a time.  To your question of using the same file descriptor...yes.

Here is an email I sent a customer back in July:

I think the IO_SERIAL_NON_BLOCKING parameter should or could be setup to be independently configure what happens with the Transmit and Receive capability of a UART or more correctly stated allow the non-blocking for the Receive side only….But currently it is either set or not as in the customer code below.  If customer wants non-blocking then best to switch to using interrupt mode that has buffers/queues to receive/transmit and let code continue running without issue.

Here is summary of what is happening with customer code.

For fprintf/write’s:

With IO_SERIAL_NON_BLOCKING parameter set, the code checks to see if the UART transmit buffer is free to accept another character.  If it is OK to accept, then the code will transmit a character (reference _io_serial_polled_write() in serl_pol.c).

If it is not OK to accept, then the code will not transmit the character and return to higher level function that will try to send the next character (i.e. higher level function doesn’t process the error code in the _io_doprint() in io_dopr.c ).

Summary:  With the IO_SERIAL_NON_BLOCKING parameter set, the code will execute full speed and not block when transmitter is FULL thereby affectively demonstrating a software overrun of the fprintf/write (i.e. characters will not be seen on the terminal).

For the fgetc/read’s:

With the IO_SERIAL_NON_BLOCKING parameter set, the code checks to see if a character is present to receive from UART, if not then it continues on with code (e.g. with this code base it loops and tries again).  If character is present then the code grabs the character and writes it back to the terminal (echoes it back).

With the IO_SERIAL_NON_BLOCKING parameter off, the fgetc() will block waiting for a character to arrive and no other code will run. 

So to have the best of both worlds turn off the IO_SERIAL_NON_BLOCKING parameter and to implement non-blocking reads just IOCTL the UART to see if character is present and if not, move on, if present, issue the fgetc().

Bonus notes:  I never use IO_SERIAL_NON_BLOCKING with polling UART mode. 

Code Referenced is attached.

Hope this helps.

Regards,

David

[Tim562]

Hi David,

I've been thinking about your comment regarding using a semaphore or mutext to protect access to the UART and I realize that I just don't understand how that could be done, or even why it's really necessary.

First, I can see how a semaphore could be used in the serialTX task since my application is in control of when the UART is to be used to transmit. The serialTX task could easily check for the semaphore before calling write(). If it's not available, the serialTX task could just block until it is.

What I don't yet understand is how that would apply to the serialRX task. With the serialRX task making a "Blocking" call to the read() function, my application doesn't know when a serial reception might actually occur. This means it can't know when the UART is actually in use receiving something, right? Maybe there is some way to configure MQX to use a semaphore when it's Interrupt Service Routines are called into use by the hardware notice of a received data byte?

If my application can't know when the MQX ISR is actually using the UART to receive data, how / when would my application set a semaphore? Perhaps the use of semaphores isn't practical with interrupt driven / blocking  serial ports? Maybe it's mandatory to prevent other code from using the UART until read() has returned? If that's true then using the UART in "Blocking" mode isn't really possible since the UART would always be unavailable except when my application is processing the data returned by a read() call.

Also, I'm not entirely sure why I would need to prevent a call to write() (for example) if the UART is in the process of receiving data (even if I could know when that is). All the UART's that I'm familiar with have separate hardware Tx / Rx FIFO's and are capable of simultaneous transmit and receive. Admittedly the Interrupt Service Routines that I've written over the years can only service one interrupt "event" at a time but if both Tx and Rx chores need to be performed they are performed one after the other. I.e., load the UART TxFIFO if data is waiting to go, then read the UART RxFIFO into a receive buffer if data is waiting in the UART. Continue this until nothing further to do.

Actually, on further thought, even if I verified that the UART wasn't in use and then called write() to transmit a large amount of data there is no way to prevent incoming data from arriving during my transmission right?  I would like to think that data received while my application is busy transmitting wouldn't be lost.

I'm very grateful for all the help and replies that I've received here and I absolutely do not  want to start an argument, but I really need to understand how this stuff works with MQX in order to write a solid application. Since MQX makes the source code available, maybe I'm being a little bit lazy here by not reading through the MQX code and figuring out for myself how each task is being accomplished . Naah, couldn't be!

Thanks again for any and all replies, have a great day!

Regards,

Tim

[DavidS]

Hi Tim,

Will do same as last reply by adding my answers to your questions with a "//DES""

What I don't yet understand is how that would apply to the serialRX task. With the serialRX task making a "Blocking" call to the read() function, my application doesn't know when a serial reception might actually occur. This means it can't know when the UART is actually in use receiving something, right?

//DES This is getting into system software architecture.  I view it as there must be a decision making process to determine which task is to receive data.  That can be done using one receive task that then directs the data to other tasks, or a receiving task might determine it is done receiving the payload it wanted so it gives up the UART/SCI to another tasks.  Or any other countless ways to make that decision.  Now if the serial driver is in interrupt mode with "Blocking", the serialRX can ask to get data,  If no data is present in the buffer/queue the driver will suspend/Block until it has received all the characters you have requested.

Maybe there is some way to configure MQX to use a semaphore when it's Interrupt Service Routines are called into use by the hardware notice of a received data byte?

//DES I'm not totally understanding this question.  The whole point of the receive ISR is to not let you waste your time waiting for charaters to arrive at the UART/SCI.  When a character arrives, the ISR grabs the byte, stuffs it in the buffer/queue, sets a flag to wake up a waiting task if there is a waiting task, and exits the ISR to wait to be awaken by the next character. 

If my application can't know when the MQX ISR is actually using the UART to receive data, how / when would my application set a semaphore? Perhaps the use of semaphores isn't practical with interrupt driven / blocking  serial ports?

//DES the semaphore comment was intented to indicate there is some decision making process going on to determine who has access to the UART/SCI.  So whether it is time based, or priority based, or event driven, a decision process must occur.

Maybe it's mandatory to prevent other code from using the UART until read() has returned?

//DES In general whoever is doing the read() can post the semaphore (or most likely a mutex) when it has gotten all the data it wants so the next task can use it.

If that's true then using the UART in "Blocking" mode isn't really possible since the UART would always be unavailable except when my application is processing the data returned by a read() call.

Also, I'm not entirely sure why I would need to prevent a call to write() (for example) if the UART is in the process of receiving data (even if I could know when that is). All the UART's that I'm familiar with have separate hardware Tx / Rx FIFO's and are capable of simultaneous transmit and receive.

//DES Yes our UARTs/SCIs are full duplex with independent receive and transmit paths.

Admittedly the Interrupt Service Routines that I've written over the years can only service one interrupt "event" at a time but if both Tx and Rx chores need to be performed they are performed one after the other. I.e., load the UART TxFIFO if data is waiting to go, then read the UART RxFIFO into a receive buffer if data is waiting in the UART. Continue this until nothing further to do.

Actually, on further thought, even if I verified that the UART wasn't in use and then called write() to transmit a large amount of data there is no way to prevent incoming data from arriving during my transmission right?

//DES Correct.  The receive side will do it job to place characters into the buffer/queue while the transimtter is operating.

  I would like to think that data received while my application is busy transmitting wouldn't be lost.

//DES It would not be lost.

Hope this helps.

Regards,

David

[Tim562]

My head is starting to hurt just a little but I think I understand what your telling me.

I think your saying that as long as I use just a single serialRX task to take data from the uart by calling read() (in blocking mode) there's really no need for semaphore or mutex protection, right?

Allowing a single serialTX task to call write() while the serialRX task is blocked waiting for data isn't a problem, right?

Best Regards,

Tim

[DavidS]

Your head my fingers ;-)  Just kidding....it's Friday!!!

I think your saying that as long as I use just a single serialRX task to take data from the uart by calling read() (in blocking mode) there's really no need for semaphore or mutex protection, right?

//DES yes...correct.

Allowing a single serialTX task to call write() while the serialRX task is blocked waiting for data isn't a problem, right?

//DES correct...not a problem.

Regards,

David

[MarkP_]

Hi,
Didn't like the lost characters generated by buffer overrun when printing to non-blocking uart.
I replaced the IO_SERIAL_NON_BLOCKING by:
#define IO_SERIAL_NON_BLOCKING_RD    (0x10)
#define IO_SERIAL_NON_BLOCKING_WR    (0x40)

Read functions uses .._RD and write/put uses .._WR flag bit.
Changed also buffer size definition:
#define BSPCFG_SCI5_QUEUE_SIZE 256 /* sci5=ittyf */

Seems to be working OK.

When will be the next MQX version 3.8 available?

~Mark

[DavidS]

Hi Mark,

ETA is beginning of November, 2011.

Regards,

David

[iafg]

Ais MQX 3.8 still on track to be released early November?

[Tim562]

Hi David,

    Thanks very much for your reply and the tip about using a semaphore or mutext to guard against both tasks accessing the uart at the same time. That should save me some troubleshooting time (and gives me another new thing to learn)!

    Also, I am advocating using the uart in "Blocking" mode so that MQX will block the serialRX task for me when read() has nothing waiting to read. The serialTX task is a little different in that I only want it active when I have placed some data in the TxBuffer. I think that "Blocking" mode for write() works here as well to avoid overruns right? If I place 200K bytes in a TxBuffer and activate the serialTX task by making it active,  write() (blocking mode) is not going to return until it has successfully output all the bytes in the TxBuffer I pass right? And once the write() function returns I can let the serialTX task "Block" itself until I need it again, right?

    I do have a question though. Is there a way to set the size of the serial receive buffer that the MQX Interrupt Service Routine stores incoming data in? Lets say my application is delayed in calling read() while data is being received. The MQX interrupt routines are going to receive the data for me in the ISR and store it in a buffer somewhere until I call read() right? How big is that buffer and can I change it's size?

    The same question could be asked of the transmit function. In interrupt mode, when I call write() does the MQX ISR take the bytes from the buffer I pass and load them into an MQX maintained transmit buffer or does the ISR just feed bytes from the buffer I pass directly to the UART FIFO. If MQX uses a buffer can I set the size of it? How would I do that? If I could adjust the size of the buffer the MQX ISR uses to be large enough to hold my entire largest message then write() would return nearly immediately. This would allow my application to block the serialTX task quickly and let the MQX ISR do the work of transmitting the data in the background.

    Thanks again for your reply David. These forums are a lifesaver for developers like me who are new to MQX and trying to get an application working on a reasonable schedule. Have a great day.

Regards,

Tim

[DavidS]

Hi Tim,

Answers to your questions embedded in your text below with a "//DES" marker:

Also, I am advocating using the uart in "Blocking" mode so that MQX will block the serialRX task for me when read() has nothing waiting to read. The serialTX task is a little different in that I only want it active when I have placed some data in the TxBuffer. I think that "Blocking" mode for write() works here as well to avoid overruns right?

//DES With serial driver will always wait for a character transmit buffer to free up before sending the next character so you don't have to worry about overrun.

If I place 200K bytes in a TxBuffer and activate the serialTX task by making it active,  write() (blocking mode) is not going to return until it has successfully output all the bytes in the TxBuffer I pass right?

//DES 200K is a big buffer!  Most likely not needed in this context.  The buffer is used to decouple the software routine from the hardware actions.  With the serial driver in the interrupt mode, it has a buffer that gets filled and the transmit interrupt routine will grab character by character off that buffer.  If the buffer is 64 byte (default in MQX) and you want to write 128 bytes, the buffer gets filled with 64 bytes, the serialTX task will block until space is freed up in the buffer and more bytes can fill in until it has written all 128 bytes.  At that point the 64byte buffer is "full" and your serialTX task has finsihed but the serial transmit interrupt will continue to send the characters from the buffer until it is empty.  Note this is for interrupt serial drivers...not polled.  A polled driver will literally hand each byte to the UART/SCI module to be sent and a lot of time is wasted.  But if you have time who cares.

 And once the write() function returns I can let the serialTX task "Block" itself until I need it again, right?

 //DES Sure.

    I do have a question though. Is there a way to set the size of the serial receive buffer that the MQX Interrupt Service Routine stores incoming data in?

//DES Size is limited to how much RAM you have.  Buffer is define in the BSP Files folder....Ex: twrk60n512.h for Tower K60.  The buffer is defined with "#define BSPCFG_SCI0_QUEUE_SIZE             64".

Lets say my application is delayed in calling read() while data is being received. The MQX interrupt routines are going to receive the data for me in the ISR and store it in a buffer somewhere until I call read() right?

//DES Correct...64 by default and you can change them per above instructions.

How big is that buffer and can I change it's size?

//DES As big as you can fit into your RAM usage.

    The same question could be asked of the transmit function. In interrupt mode, when I call write() does the MQX ISR take the bytes from the buffer I pass and load them into an MQX maintained transmit buffer or does the ISR just feed bytes from the buffer I pass directly to the UART FIFO?

//DES The MQX ISR will take copy the bytes to its buffer/queue.  If buffer/queue if full, then task will block until room is available in the buffer/queue.

 If MQX uses a buffer can I set the size of it?

//DES Yes...answered up above by changing the #define BSPCFG_SCI0_QUEUE_SIZE.  This value is used for a Tx and Rx buffer/queue.

How would I do that?

//DES Change the BSPCFG_SCI0_QUEUE_SIZE in the twrk60n512.h header.  Then re-compiler the RTOS.

 If I could adjust the size of the buffer the MQX ISR uses to be large enough to hold my entire largest message then write() would return nearly immediately. This would allow my application to block the serialTX task quickly and let the MQX ISR do the work of transmitting the data in the background.

Regards,

David

[Tim562]

Thank you for the info about the buffer sizes David, that's really great info!

I found the "#define BSPCFG_SCI0_QUEUE_SIZE             64" entry in the twrk60n512.h file where you said it was but this define isn't in the twrmpc5125.h file that I'm using. I did find a similar entry :

#define BSPCFG_PSC_DEFAULT_QUEUE_SIZE 256"

in twrmpc5125.h that looks like it sets the buffer size for all serial ports to 256 bytes. I looks like I could change this as required but it would set the buffer size for all 10 serial ports, right? As you point out that's a lot of RAM if I need 200K buffers (x 10 serial ports).

As it so happens I do have need to send / receive serial data messages that are in the 200K byte range so increasing the default buffer sizes would be a definite help. Especially in the transmit side so I can call write() and get it to swallow an entire message in one shot! On the receive side it's only important that the buffer is large enough to cover me if my application doesn't get around to servicing a data reception event in a timely fashion. This shouldn't happen since it's probably going to be my highest priority.

Thank you for your help!

Best Regards,

Tim 

[Tim562]

Hi Mark,

I think I understand what you're saying in your post.

1. One task to service both read and write chores for the uart.

2. Serial device opened in non-blocking, interrupt mode.

3. Call read() from endless loop and store received data in RxBuffer.

4. Pause / Block that loop to wait for an event msg (TxData waiting for example).

5. Event wait has max time to Wait / Block then back around to read() again.

6. To transmit, I load a TxBuffer and send an event that is detected by this serial task.

Do I have that right? If I do, that means the serial service task is being unblocked and allowed to run periodically just to check and see if there is any data waiting to be received right? Is there no way to keep the task Blocked until there is something to be done (Tx or Rx)? The Blocking read() function call is perfectly efficient but if you do that I guess there's just no way to unblock the task to make use of the serial device for a transmission.

Do you know if it's possible to use a second task to transmit data? This second task would have to be able to open the same serial device that the serial receive task already has open. This second task would block indefinitely until it received an event notification that there's data to be transmitted. When it receives that event, it wakes up, loads the outbound data to the serial device and then returns to the Blocking state. Maybe it could use the same FileDescriptor as the serial receive task?

Thanks very much for taking the time to reply to my question, I really appreciate it. Now I can see that I need to think about the solution to this for a bit. Have a great day.

Best Regards,

Tim