serial port timeout before expected # of bytes arrives‏

[synthnassizer]

Hello everyone,
  I am trying to port an application that is receiving/sending data through the serial port.
  On the sabrelite board that is the /dev/ttymxc0 port, ie the 2ndary serial port (the primary being /dev/ttymxc1).
 
  I am using the functions select() and read() to read from the serial port , in order to implement a timeout mechanism that is waiting for a few seconds for bytes to arrive and if should they not arrive, then the timeout is reached.
 
  The receiving loop works like this:
 

1. I wait for 2 bytes - the opcode of the command.
2. Once I resolve a code, I wait for 4 bytes designating the length of data that will follow (depending on the command)
3. I create a buffer in my application of size equal to the length I received above.
4. I start receiving the data.

 
  Note that I have implemented reception of the data in 2 ways (since the 1st way used to work, it was tried as is. Then, I tried the next as well):
    a)receive my data using a for loop, reading() one byte at time. until all the expected bytes arrive , or a timeout is reached.
    b)receive my data using a while loop where I ask for all the remaining bytes to be read(). Read() returns how many bytes have actually arrived, so I update the remaining Bytes and re-read(). Again if a select() times-out without any data arriving, the loop will exit.

  Now, my problem is that while applications seem to be using this port without any problem (f.e. minicom, rx, cat /dev/ttymxc0, echo "something" > /dev/ttymxc0 ) , through the application I get timeouts for any file that is bigger than 3-4kBytes.

  Say I am trying to send a file 138kB. I may fail at byte ~40k, ~56k, or even ~12k. This is very weird. And I don't know what to assume is going wrong. Slow processing on the local end seems to not be a problem, as I can read (in (b) above) approx 58 bytes in the beginning and then 1 byte at a time. Could it be the memory buffer? If yes, how can I increase its size? Could another application be interfering with /dev/ttymxc0?

  I am running yocto dylan  fsl-image-test with all -dev, -sdk & -dbg options enabled.

  Thank you for your help.

[wpflueger]

Maybe this reply is a little bit silly, but...

Did you try to deactivate DEBUG_SERIAL? If the baud rate of your debug output is slow and the baud rate of the data port is fast it is no wonder that you don't get all data.

Just an idea.

[synthnassizer]

Hi Willheim,

thank you for the input. Theoretically - even if it was as you say- data would get buffered (up to the memory buffer of the driver I believe) and I would get to read a bunch of them. This does happen (sometimes between consecutive reads I'll read >1 bytes, more often up to 50 bytes but rarely even up to ~4000). So most probably this was never the problem.

In fact in a desperate attempt at my office, we modified the host application (running on windows, build on MS Visual Studio,  as a .net application). So we created a small C++ serial port control application to just bypass the .net and voila! Without altering the code on the embedded system side, I can now read the full data I expect!

I am not going to blame .net since the problem started showing up after I ported the embedded sytem's code from an older device ARM9TDMI-ARMv4T (running linux with kernel 2.4), to the newer freescale cortex A9 imx6q sabrelite.

I wonder if there is something with the serial port driver of the boundary-devices linux kernel 3.0.35 that is used by the yocto project (and which I am running on the board right now). If somebody knows anything about serial port problems with the sabrelite, pleasee share them, :smileyhappy: .

However, I will note here that moving away from .net code, the C++ serial port control that we wrote worked both with the old device and the new one.

So the code above should work for reading from the serial port in linux..

[timothybean]

Could you post your code? I have code that I use, although poll() instead of select. It is for a terminal server portion of code I have...

Maybe there is something wrong with the logic?

Tim

[synthnassizer]

Hi there. The code of interest is here:

http://pastebin.com/rpU01MwB

Basically, the readComData(const int toReadBytes, char *serDataBuf) fn is run with a pointer to a buffer that is of size "toReadBytes". The buffer has already malloc'ed enough space for the read to commence.

So initially the debug output looks like:

    SERIAL: DATA will read DATA bytes 138669

    SERIAL: DATA read 59 bytes and a total of 0 .

    SERIAL: DATA read 1 bytes and a total of 59 .

    SERIAL: DATA read 1 bytes and a total of 60 .

    SERIAL: DATA read 2 bytes and a total of 61 .

    SERIAL: DATA read 1 bytes and a total of 63 .

    SERIAL: DATA read 1 bytes and a total of 64 .

    SERIAL: DATA read 1 bytes and a total of 65 .

    SERIAL: DATA read 1 bytes and a total of 66 .

    SERIAL: DATA read 1 bytes and a total of 67 .

then towards byte 12278

    SERIAL: DATA read 1 bytes and a total of 12273 .

    SERIAL: DATA read 1 bytes and a total of 12274 .

    SERIAL: DATA read 1 bytes and a total of 12275 .

    SERIAL: DATA read 1 bytes and a total of 12276 .

    SERIAL: DATA read 1 bytes and a total of 12277 .

    SERIAL: No DATA have been read. Prob Timeout @ byte 12278. exiting fn

for baudrate 115200, the timeout occurs at this specific byte.

[timothybean]

Hi Nass,

First thing I would do for testing purposes is take out the timeout portion of your code and test to see if you are actually getting all the data and it really is a timeout, or did the data actually stop coming.

If you can verify that all the data does get through, eventually, then I would probably put a sleep() in there to slow it down a bit. I am a little suspicious that you are reading the data too quick and your timeout might be to short. What are you setting your timeout to? Are you sure the timeout is getting set correctly?

Reading 0 bytes doesn't necessarily mean that the connection is broken though and that no more data is coming.

Are you able to use glib? My code uses poll method for reading from network socket, and I am using GIO for reading from serial port, although, I am just passing through and am not looking for a specific size of data like you are.

Tim

[synthnassizer]

Hi Timothy,

VTIME seems to work correctly:

while opening the serial port, i set

serial_opts.c_cc[VTIME]=60; //6 seconds 

then the code is in : http://pastebin.com/W0vPGDBm

I have implemented time measurement for the timeout, and an attempt to reread() from the serial port until MAX_RETRIES (=5) is met.

The timeout seems to work right, and the debug output is:

SERIAL: DATA read 1 bytes and a total of 12275 .
SERIAL: DATA read 1 bytes and a total of 12276 .
SERIAL: DATA read 1 bytes and a total of 12277 .
SERIAL: time diff is tv_sec=5 , tv_usec=995891
SERIAL: No DATA have been read. Timeout @ byte 12278, timeout counter 0.
SERIAL: time diff is tv_sec=5 , tv_usec=999827
SERIAL: No DATA have been read. Timeout @ byte 12278, timeout counter 1.
SERIAL: time diff is tv_sec=5 , tv_usec=999812
SERIAL: No DATA have been read. Timeout @ byte 12278, timeout counter 2.
SERIAL: time diff is tv_sec=5 , tv_usec=999819
SERIAL: No DATA have been read. Timeout @ byte 12278, timeout counter 3.
SERIAL: time diff is tv_sec=5 , tv_usec=999821
SERIAL: No DATA have been read. Timeout @ byte 12278, timeout counter 4.
SERIAL: time diff is tv_sec=5 , tv_usec=999833
SERIAL: No DATA have been read. Timeout @ byte 12278, timeout counter 5.
SERIAL: time diff is tv_sec=5 , tv_usec=999832
SERIAL: No DATA have been read. Timeout @ byte 12278, timeout counter 6.

it also looks like after 6sec/retry*6 retries = 36 secs , I still haven't received any data.

I would start looking into the sender side code, but the code I am porting here (from a kernel 2.4 embedded system, to kernel 3.0.35 emb. system) used to work..... so it must be something on the receiving side.

just to not leave room for wonder, I am running an openembedded distro in the imx6q sabrelite board. It was generated through the yocto project tools (version is 1.4.1 - dylan). So there is a full linux distro running with libc. Why do you ask about glib though?

[timothybean]

I ask about Glib because my code uses it...

Are you using flow control? You might want to check there... also, maybe try your code on a Linux computer with a serial port connected to the sender and see if you still have problems. At least narrow down to the receiver and not the sender...

Tim

[synthnassizer]

Hi Timothy,

flow control can not be used with raw i/p - o/p , as I a using now.

However, attempting to run the code on the pc instead of the embedded device was a very interesting idea.

Indeed running the code and doing a read() times out at the very same byte!

This is odd. I can't make out what is going on wrong yet, but this is "good".

I still dont think I can blame the sender. Mind you I was just porting the code from an old embedded device to a new one, and this happened.

I am appending the last 2 read() operations and the bytes that were received:

SERIAL: DATA read 11 bytes and a total of 12262 .

SERIAL: serDataBuf[12262]=      0x32

SERIAL: serDataBuf[12263]=      0x30

SERIAL: serDataBuf[12264]=      0x32

SERIAL: serDataBuf[12265]=      0x30

SERIAL: serDataBuf[12266]=      0x32

SERIAL: serDataBuf[12267]=      0x30

SERIAL: serDataBuf[12268]=      0x32

SERIAL: serDataBuf[12269]=      0x30

SERIAL: serDataBuf[12270]=      0x32

SERIAL: serDataBuf[12271]=      0x30

SERIAL: serDataBuf[12272]=      0x32

SERIAL: DATA read 5 bytes and a total of 12273 .

SERIAL: serDataBuf[12273]=      0x30

SERIAL: serDataBuf[12274]=      0x32

SERIAL: serDataBuf[12275]=      0x30

SERIAL: serDataBuf[12276]=      0x32

SERIAL: serDataBuf[12277]=      0x30

SERIAL: time diff is tv_sec=5 , tv_usec=996447

SERIAL: No DATA have been read. Timeout @ byte 12278, timeout counter 0.

SERIAL: time diff is tv_sec=5 , tv_usec=999983

SERIAL: No DATA have been read. Timeout @ byte 12278, timeout counter 1.

SERIAL: time diff is tv_sec=5 , tv_usec=999973

SERIAL: No DATA have been read. Timeout @ byte 12278, timeout counter 2.

SERIAL: time diff is tv_sec=5 , tv_usec=999961

SERIAL: No DATA have been read. Timeout @ byte 12278, timeout counter 3.

SERIAL: time diff is tv_sec=5 , tv_usec=999974

SERIAL: No DATA have been read. Timeout @ byte 12278, timeout counter 4.

SERIAL: time diff is tv_sec=5 , tv_usec=999960

SERIAL: No DATA have been read. Timeout @ byte 12278, timeout counter 5.

SERIAL: time diff is tv_sec=5 , tv_usec=999982

SERIAL: No DATA have been read. Timeout @ byte 12278, timeout counter 6.

So the last byte received is a valid 0x30...