USB stack and real time requirement

[lpcware]

Content originally posted in LPCWare by fjarnjak on Mon Dec 09 04:38:28 MST 2013
Hello,

I use LPCOpen with USB stack for virtual COM port on my LPC11U35 board.
There I also have an I2C sensor which I want to access at 100 Hz (every 10ms).

Idea is to send sensor data every 10ms to the PC.

To obtain actual reading from the sensor it takes 2us (before reading I put one GPIO pin high and after reading I set it to low and I look at logic analyzer for the pulse width).

To get 10ms intervals, I use Systick timer that sets the flag (volatile uint8_t) in its ISR which is then handled in the main() having a loop to service the USB stack and checking for flag:

Pseudo code:

while(1)
{
      Call_To_Service_USB_Stack();
      if( 1 == flag)
      {
           flag = 0;
           GPIO_High();
           // acquire and send data over USB - ~ 2uS.
           GPIO_Low();
      }
}

I have noticed that my readings are not spaced at 10ms intervals (again looking at the GPIO pin low to high transition spacings).

Delays may come in USB stack servicing and who knows what Windows wants there; delay in setting the flag and going into the if(), etc. However, intervals I see on analyzer are sometimes 10ms, sometimes 13ms, sometimes 26ms.

Regarding data, every 10ms, I send about 28 bytes (2800 bytes/sec). Meaning if 28 bytes = 280 bits (10 bits per byte) at 115,200 bps that takes 2.43 ms for sending it to the PC.


Did anyone face something similar?

[lpcware]

Content originally posted in LPCWare by fjarnjak on Wed Dec 11 01:17:01 MST 2013

---

Quote: Tsuneo
Sound like you are working on ROM driver.

LPCOpen's UsbdCdc_SendData() places a cyclic buffer between the bulk endpoint, which runs under "latency" timer. This implementation is useful just for byte-stream input, like USB-UART bridge or printf-console applications. For most of applications which exchange packets, this implementation kills performance.

Instead, call WriteEP() API directly, to pass the packet to the endpoint, directly.

...

Tsuneo

---

Yes, I am using ROM driver.

I will check your suggestions and see what happens....

[lpcware]

Content originally posted in LPCWare by fjarnjak on Wed Dec 11 01:13:36 MST 2013

---

Quote: capiman
Tsuneo:
This is from USB 2.0 spec. chapter "5.8 Bulk transfers" (page 52):
"Guaranteed delivery of data but no guarantee of bandwidth or latency"

Even in USB 3.1 spec. chapter "4.4.6 Bulk transfers" (page 79) there is still this remark:
"Guaranteed delivery of data, but no guarantee of bandwidth or latency"

Ok, 28 bytes per 10 ms is a small amount of data.
But I would not guarantee that this works 100%, under all circumstances, on every system.

---

USBs are used to transfer large amount of data and are used in DAQ cards, streaming audio/video, HDD, USB sticks, etc. Compared to that, my transfer rate is few KB per second....
And I can live with packets not arriving in regular intervals, or joining them together since in PC software I know they are 10ms apart. But seems to me due to USB stack handling I can't acquire at regular intervals. That is the real issue.

[lpcware]

Content originally posted in LPCWare by capiman on Tue Dec 10 23:13:31 MST 2013
Tsuneo:
This is from USB 2.0 spec. chapter "5.8 Bulk transfers" (page 52):
"Guaranteed delivery of data but no guarantee of bandwidth or latency"

Even in USB 3.1 spec. chapter "4.4.6 Bulk transfers" (page 79) there is still this remark:
"Guaranteed delivery of data, but no guarantee of bandwidth or latency"

Ok, 28 bytes per 10 ms is a small amount of data.
But I would not guarantee that this works 100%, under all circumstances, on every system.

[lpcware]

Content originally posted in LPCWare by Tsuneo on Tue Dec 10 13:55:19 MST 2013
Sound like you are working on ROM driver.

LPCOpen's UsbdCdc_SendData() places a cyclic buffer between the bulk endpoint, which runs under "latency" timer. This implementation is useful just for byte-stream input, like USB-UART bridge or printf-console applications. For most of applications which exchange packets, this implementation kills performance.

Instead, call WriteEP() API directly, to pass the packet to the endpoint, directly.

#include "usbd_rom_api.h"

    uint8_t EpAdd = USB_ENDPOINT_IN(CALLBACK_UsbdCdc_Register_DataInEndpointNumber());
    USBD_API->hw->WriteEP(UsbHandle, EpAdd, your_buffer_pointer, its_length );

The CDC stack code also hits the bulk IN endpoint.
Comment the code, so that it doesn't disturb us.

\lpcopen\software\LPCUSBLib\Drivers\USB\Core\DCD\USBRom\usbd_cdc.c

ErrorCode_t UsbdCdc_Bulk_IN_Hdlr(USBD_HANDLE_T hUsb, void* data, uint32_t event)
{
//    uint8_t EpAdd = USB_ENDPOINT_IN(CALLBACK_UsbdCdc_Register_DataInEndpointNumber());

//    if (event == USB_EVT_IN)
//    {
//        USBD_API->hw->WriteEP(UsbHandle, EpAdd, UsbdCdc_EPIN_buffer,UsbdCdc_EPIN_buffer_count);
        /* Clear EP buffer */
//        UsbdCdc_EPIN_buffer_count = 0;
//    }

    return LPC_OK;
}

---

Quote:
capiman:
Have you seen the above statement on bulk endpoints:
> "No guarantee of bandwidth or minimum latency."
You wrote:
> "Does anyone have any experience in using LPCOpen USB stack in hard real time tasks?"
According to Wikipedia:
> "Missing a deadline is a total system failure."

---

capiman,
Do you really have practical USB experience?
Or just talking about ambiguous Wiki description?

Either UHCI, OHCI or EHCI, PC host controller gives equal chance to each active bulk endpoint.
Even on crowded bus, one bulk transaction (28 byte) should occur within 10 frames (10 ms).

Tsuneo

[lpcware]

Content originally posted in LPCWare by fjarnjak on Tue Dec 10 11:39:02 MST 2013

---

Quote: rocketdawg
I used FreeRTOS and LPCOpen on a LPC1769.
It provided a custom HID report.
the only issue I had was that I had to do the USB enumeration before starting up the RTOS because I could not service the USB fast enough during enumeration.
I found that there are different timing requirements during enumeration.
A USB thread was set to poll the USB function every 10ms

---

Yes. Enumeration on the USB bus is critical and should be done without additional delays. In my code I handle this by (a) init all hardware first, setup registers, etc as required and only then (b) start USB stack which would trigger enumeration.

Maintaining USB stack can go with up to 30ms intervals.

[lpcware]

Content originally posted in LPCWare by rocketdawg on Tue Dec 10 09:12:39 MST 2013
I used FreeRTOS and LPCOpen on a LPC1769.
It provided a custom HID report.
the only issue I had was that I had to do the USB enumeration before starting up the RTOS because I could not service the USB fast enough during enumeration.
I found that there are different timing requirements during enumeration.
A USB thread was set to poll the USB function every 10ms

[lpcware]

Content originally posted in LPCWare by fjarnjak on Tue Dec 10 02:01:22 MST 2013

---

Quote: capiman

When the (systick) timer fires, you set a flag and wait, wait, wait till someone works on your flag,
why do you use a interrupt at all? Just query the timer in your main loop. But for your problem, it is useless...

---

In this case either way can work. But as the system grows, it would be difficult to maintain and improve the code, as you add more features to it.
Setting the flag approach is usually used, even in RTOS kernel implementations.

But if I count the timer in main() or use the flag, UsbdCdc_IO_Buffer_Sync_Task(); needs to be called regularly to maintain the USB stack and its connectivity to PC.
I think this function call sometimes ends quicker and sometimes takes longer to finish. Meaning I do not reach if() with fixed or somewhat fixed delay (depending on compiler optimization used that delay would vary but would essentially be fixed). And due to this, I handle the flag if it is set at varying intervals - or I would check the current timer value also at varying intervals.

So if I count delays: I would have a fixed delay (calling ISR, exiting from ISR, for(;;) loop, if statement) then I would have varying delay of USB stack function, lastly there is sensor data acquisition (2us) and sending the data to PC ~2.4ms. Calling ISR, few if statements and increments and all this fixed would be measured in us....but let's take it to be 1ms. So 1ms + 2.4ms = 3.4ms. I still have 6.6ms before next sensor acquisition. Seems to me UsbdCdc_IO_Buffer_Sync_Task(); sometimes needs even more than 6.6ms to finish.

Perhaps  I can call it only if flag is not set that should also help a bit....

---

Quote:

Why don't you start getting sensor data in systick timer?
Why don't you set the flag when sensor data is available? (You can also use an interrupt again, even multiple time for I2C or whatever you use...)

---

In case my I2C sensor is broken or disconnected (wire is cut), my ISR would halt for a while...disrupting other services (again when the code would grow and more features are added - LCD, keypad, encoders, etc.).

[lpcware]

Content originally posted in LPCWare by capiman on Tue Dec 10 01:47:30 MST 2013
Have you seen the above statement on bulk endpoints:
> "No guarantee of bandwidth or minimum latency."
You wrote:
> "Does anyone have any experience in using LPCOpen USB stack in hard real time tasks?"
According to Wikipedia:
> "Missing a deadline is a total system failure."

You can change it how you want, you won't get hard real time with an USB bulk channnel...

To your interrupt topic:
When the (systick) timer fires, you set a flag and wait, wait, wait till someone works on your flag,
why do you use a interrupt at all? Just query the timer in your main loop. But for your problem, it is useless...

What can you do better:
Why don't you start getting sensor data in systick timer?
Why don't you set the flag when sensor data is available? (You can also use an interrupt again, even multiple time for I2C or whatever you use...)

But:
Be prepared, that your sensor has finished and sensor data is available and flag that sensor data is available is still set...

[lpcware]

Content originally posted in LPCWare by fjarnjak on Tue Dec 10 01:28:10 MST 2013

---

Quote: stalisman
Delays may come in USB stack servicing and who knows what Windows wants there; delay in setting the flag and going into the if(), etc. However, intervals I see on analyzer are sometimes 10ms, sometimes 13ms, sometimes 26ms.

maybe you should consider using interrupts instead of relying upon luck?

---

It seems I am constantly repeating myself...of course I am using interrupts to do the timing. How else would you do such task on the MCU?
Please look at my posts before and the pseudo code I have posted. I am using systick interrupt as my interval timer.
I am also using all the best practices of interrupt handling in embedded programming which advertise making ISR as short as possible by usually asserting a flag which is then handled in the regular code.

I am not sure where does luck come from...

Does anyone have any experience in using LPCOpen USB stack in hard real time tasks?

To stop further advices about using the interrupts which are being used following is the full code. As you can see it is based on the USB virtual serial demo coming with LPCOpen:

static int16_t v1;
staticint16_t v2;
staticint16_t v3;

volatile uint32_t SysTickCnt = 0;
volatile uint32_t SysTickCnt_Old = 0;
volatile uint8_t SecondFlag = 0;
volatile uint8_t Mili10Flag = 0;
volatile uint8_t SendData = 0;


#define MAX_BUFF 40
uint8_t buff[MAX_BUFF];

void ClearBuff()
{
memset(buff, '\0', MAX_BUFF);
}

void SysTick_Handler(void)
{
     ++SysTickCnt_Old;
     ++SysTickCnt;
     if(SysTickCnt_Old >= 1000)
     {
     SecondFlag = 1;
     SysTickCnt_Old = 0;
     }

     if(SysTickCnt_Old >= 10)
     {
     Mili10Flag = 1;
     }
}

int main(void)
{
uint8_t dummy = 0;
uint32_t len = 0;


SetupHardware();

if (SysTick_Config(SystemCoreClock / 1000)) { /* Setup SysTick Timer for 1 msec interrupts */
while (1); /* Capture error */
}

for (;;)
{
UsbdCdc_IO_Buffer_Sync_Task();

EchoCharater();

if(SecondFlag == 1)
{
SecondFlag = 0;
if(0 == dummy)
{
LED_ON;
dummy = 1;
}
else if(1 == dummy)
{
LED_OFF;
dummy = 0;
}
}

if(1 == SendData && 1 == Mili10Flag)
{
Mili10Flag = 0;

GPIO_SetValue(PORT0, (1<<17));

ClearBuff();
sensor_read(&v1,&v2,&v3);
snprintf((char *)buff, MAX_BUFF, "DAT:$%d,%d,%d$\n\r",v1,v2,v3);
len = strlen((const char*)buff);
UsbdCdc_SendData(buff, len);
UsbdCdc_IO_Buffer_Sync_Task();

GPIO_ClearValue(PORT0, (1<<17));
}
}
}


void EchoCharater(void)
{
uint16_t len = 0;
uint8_t recv_byte[CDC_TXRX_EPSIZE];
uint32_t recv_count;

recv_count = UsbdCdc_RecvData(recv_byte, CDC_TXRX_EPSIZE);
if(recv_count)
{
if(recv_byte[0] == 's' || recv_byte[0] == 'S')
{
SendData = !SendData;
}
}
}

[lpcware]

Content originally posted in LPCWare by stalisman on Tue Dec 10 01:01:29 MST 2013
Delays may come in USB stack servicing and who knows what Windows wants there; delay in setting the flag and going into the if(), etc. However, intervals I see on analyzer are sometimes 10ms, sometimes 13ms, sometimes 26ms.

maybe you should consider using interrupts instead of relying upon luck?

[lpcware]

Content originally posted in LPCWare by fjarnjak on Mon Dec 09 13:18:51 MST 2013

---

Quote: rocketdawg

---

Quote: fjarnjak

For CDC specification, Bulk and Isochronous endpoints are necessary for data transfer + endpoint 0.

---

I would switch to Bulk for all endpoints other than endpoint 0 which is a control endpoint.
You receive no benefit using Isochronous for CDC.  Bulk is guaranteed (in hardware) and your packets are small.

---

I am seeing in the CDC class code it uses Bulk:

if (EndpointNum == CDCInterfaceInfo->Config.DataINEndpointNumber)
{
Size         = CDCInterfaceInfo->Config.DataINEndpointSize;
Direction    = ENDPOINT_DIR_IN;
Type         = EP_TYPE_BULK;
DoubleBanked = CDCInterfaceInfo->Config.DataINEndpointDoubleBank;
}
else if (EndpointNum == CDCInterfaceInfo->Config.DataOUTEndpointNumber)
{
Size         = CDCInterfaceInfo->Config.DataOUTEndpointSize;
Direction    = ENDPOINT_DIR_OUT;
Type         = EP_TYPE_BULK;
DoubleBanked = CDCInterfaceInfo->Config.DataOUTEndpointDoubleBank;
}
else if (EndpointNum == CDCInterfaceInfo->Config.NotificationEndpointNumber)
{
Size         = CDCInterfaceInfo->Config.NotificationEndpointSize;
Direction    = ENDPOINT_DIR_IN;
Type         = EP_TYPE_INTERRUPT;
DoubleBanked = CDCInterfaceInfo->Config.NotificationEndpointDoubleBank;
}

[lpcware]

Content originally posted in LPCWare by fjarnjak on Mon Dec 09 13:17:28 MST 2013

---

Quote: Pacman
Just a note...

I think your SysTick interrupt may have a too low priority and may be delayed by other interrupts or DMA being busy (eg. UART / USB interrupts / DMA?)
Try changing the interrupt priority for the SysTick interrupt, otherwise change to use one of the high resolution timers.
Does your microcontroller have an ADC interrupt (I believe it would). If so, it might be a much better solution to use that.

---

There is only 1 interrupt being used. Systick. No other interrupts are active in the system.

The whole setup is very simple:

MCU services the USB stack (using LPCOpen) in the while loop. When systick interrupt sets the flag, in main() in while loop it resets the flag back to 0 and acquires data using I2C bus from the sensor (sensor is not analog - no ADC). Then data is sent to PC immediately. Again loop is only doing USB stack servicing until next flag set and so on.

[lpcware]

Content originally posted in LPCWare by Pacman on Mon Dec 09 10:19:35 MST 2013
Just a note...

I think your SysTick interrupt may have a too low priority and may be delayed by other interrupts or DMA being busy (eg. UART / USB interrupts / DMA?)
Try changing the interrupt priority for the SysTick interrupt, otherwise change to use one of the high resolution timers.
Does your microcontroller have an ADC interrupt (I believe it would). If so, it might be a much better solution to use that.

[lpcware]

Content originally posted in LPCWare by rocketdawg on Mon Dec 09 09:11:07 MST 2013

---

Quote: fjarnjak

For CDC specification, Bulk and Isochronous endpoints are necessary for data transfer + endpoint 0.

---

I would switch to Bulk for all endpoints other than endpoint 0 which is a control endpoint.
You receive no benefit using Isochronous for CDC.  Bulk is guaranteed (in hardware) and your packets are small.

[lpcware]

Content originally posted in LPCWare by fjarnjak on Mon Dec 09 07:01:58 MST 2013
I do use that - systick timer interrupt.

But I do not handle the sensor acquisition over I2C bus in the ISR since it is lengthy. 
I set the flag and then handle I2C in the main loop.

[lpcware]

Content originally posted in LPCWare by capiman on Mon Dec 09 06:48:57 MST 2013
You can use some timer interrupt service routine, if you want your sensor to get values periodically?

[lpcware]

Content originally posted in LPCWare by fjarnjak on Mon Dec 09 06:32:49 MST 2013
Yes.

However, I have noticed 2 things though:

(a) if I concentrate only on GPIO low to high transitions (as per my pseudo code above) I notice that this transition on the pin happens at irregular intervals.
Which means that some random time is spend in the USB stack service routine (which needs to be called regularly to make sure USB device runs smoothly towards the host side).
Meaning the time my Systick interrupt sets the flag to 1 by the time I enter (1== flag) if statement, that time is sometimes longer, sometimes shorter.
(my systick ticks at 1ms).
There is ample time between two (1==flag) entries to send required amount of data (2us to acquire + 2.4ms to send; 10ms time available).

(b) to have my USB device running as a virtual COM port it has to use USB CDC drivers. Endpoint types to be used are specified by the USB standard.

Point a) I do not know how to solve unless I use and external USB chip like FT232 or similar, which would then handle USB related stuff while my MCU does real time acquisitions in precise time frame.
Point b) and partially a) I can solve by making my own PC driver that uses interrupt transfer end point and code on MCU would use that directly without some class driver.

Whar irks me though is that I have irregular time intervals. I would be more happy if at least my GPIO low to high transitions happen at some regular intervals be it 10ms or 15ms or even 20ms. Then what happens on the USB side when sending to PC I could "blame" on the USB specification. But randomness in servicing my flag happens due to USB stack handling which sometimes finished sooner sometimes later; and that makes intervals not equally spread out - since I need to correlate sensor data with time; thus I need equidistant time points.

[lpcware]

Content originally posted in LPCWare by capiman on Mon Dec 09 06:05:34 MST 2013
Not related to LPC uC, but properties of USB in general:

http://www.beyondlogic.org/usbnutshell/usb4.shtml

Bulk Transfers
   Used to transfer large bursty data.
   Error detection via CRC, with guarantee of delivery.
   No guarantee of bandwidth or minimum latency. ***!!!***
   Stream Pipe - Unidirectional
   Full & high speed modes only.

[lpcware]

Content originally posted in LPCWare by fjarnjak on Mon Dec 09 05:31:43 MST 2013
I have used USB CDC class driver - that is to make a virtual COM port on the PC with my device. As such, this is all specified as the USB standard.
I did not create my own implementation on the device side and a matching driver for the PC (even though I could do that if it is necessary).
For CDC specification, Bulk and Isochronous endpoints are necessary for data transfer + endpoint 0.

[lpcware]

Content originally posted in LPCWare by capiman on Mon Dec 09 04:46:11 MST 2013
Please have a look which endpoint types (bulk, isochronous, interrupt) you used,
and which (general) property each endpoint type has.