topic Re: Video delay problem in Android in i.MX Processors

Video delay problem in Android

sergeikotlyachk — Tue, 11 Jun 2013 14:52:21 GMT

Hi,

My company is trying to make video playing in Android with i.MX6 board to work with live video sources. Unfortunately there is significant delay of 12-14 seconds between changes in the live video and showing these changes on the screen. I was trying various options and came to understanding that root cause of the problem is somewhere in Freescale code in OMX / VPU / GMPlayer, but can't figure out what should I change in firmware in order to reduce this delay. Please help, otherwise we will be unable to use Freescale for our commercial product.

Thanks.

Re: Video delay problem in Android

LeonardoSandova — Tue, 11 Jun 2013 15:30:10 GMT

You can reduce it here:

external/fsl_imx_omx/OpenMAXIL/src/component/streaming_parser/StreamingParser.cpp,

#define PACKET_CACHE_SIZE (4*OMX_TICKS_PER_SECOND)

Leo

Re: Video delay problem in Android

sergeikotlyachk — Tue, 11 Jun 2013 17:08:03 GMT

Thanks. Using 65536 for PACKET_CACHE_SIZE reduced it to 4.5 seconds. What else can be done? We need this to be 1 sec or less. We connect to live stream directly over local ethernet. VideoLan (VLC) application running on the same network has an negligible delay less than 0.5 second.

Re: Video delay problem in Android

sergeikotlyachk — Tue, 11 Jun 2013 17:14:58 GMT

Note: I figured that reducing fps reduces the delay. Reducing resolution also seems to help a little bit. However, even with SD resolution and 5 fps, we still get 2.5 seconds.

There is also another problem, after a little bit of time delay increases to 7 seconds. If streaming application is restarted, delay becomes shorter again.

Re: Video delay problem in Android

LeonardoSandova — Wed, 12 Jun 2013 15:45:13 GMT

Not sure if it will help but can you add 'enable_wait_mode=off' to your bootargs?

Leo

Re: Video delay problem in Android

sergeikotlyachk — Wed, 12 Jun 2013 16:23:24 GMT

Unfortunately this did not help (no changes, still high delay after 2-3 minutes of running video).

Re: Video delay problem in Android

karina_valencia — Thu, 13 Jun 2013 15:41:05 GMT

jackmao Jun 12, 2013 6:42 PM (in response to imxcommunityscout)

Hi,

what's your meaning "there is significant delay of 12-14 seconds between changes in the live video and showing these changes on the screen"? please give a detail description, please give a full log when the issue happen,by the way, what's your software version?

Re: Video delay problem in Android

sergeikotlyachk — Thu, 13 Jun 2013 17:17:17 GMT

1. Setup is simple: ethernet camera that transmits live rtsp stream in H264 and Android application that uses MediaPlayer class shows this video on HDMI HD monitor connected to i.MX6 Quad core board.

2. Delay: when I wave my hand over the camera, it takes 12-14 seconds for it to show in Android application.

(or whatever number of seconds in other situations that I explained in other posts of this thread).

3. What kind of log do you need and how do I get it?

4. Software version:

PLATFORM_VERSION_CODENAME=REL

PLATFORM_VERSION=4.0.4

TARGET_PRODUCT=sabresd_6dq

TARGET_BUILD_VARIANT=user

TARGET_BUILD_TYPE=release

TARGET_BUILD_APPS=

TARGET_ARCH=arm

TARGET_ARCH_VARIANT=armv7-a-neon

HOST_ARCH=x86

HOST_OS=linux

HOST_BUILD_TYPE=release

Linux version 3.0.35-05524-g8513494 (kotl@android1004) (gcc version 4.4.3 (GCC) ) #1 SMP PREEMPT Thu May 2 17:48:46 EDT 2013

Re: Video delay problem in Android

jack_mao — Mon, 17 Jun 2013 02:25:01 GMT

It seems that the delay time is increasing during the play, we suspect the sync issue, so could you dump the rtsp stream to us for analysis. By the way, what's the encode format? how many reference frame will it support?

Re: Video delay problem in Android

sergeikotlyachk — Mon, 17 Jun 2013 21:48:43 GMT

I have posted this file for you to try.

http://www.aimetis.com/public_downloads/FreescaleBoard/sample.mp4

You can see my hands in front of the camera at these times:

0:00 - 0:01

0:30 - 0:41

1:00 - 1:10

- When played through VLC 2.0.7 it works fine and shows my hands at proper times.

- I streamed it using VOD mode ( Multiple streaming and Video on demand ) to check if the problem can be reproduced through this method instead of using direct connection to camera. I also ran VLC client alongside connecting to the same instance of streaming VLC. The following happened for Freescale board:

- delay at 0:30 was 4 seconds

- delay at 0:40 was ~5.5 seconds

- at 55 seconds video on Freescale board stopped working while VLC client continued playing at proper times and till the end.

I hope this is good enough for you to fix it. If you don't have Android app that can play url given to it, you can use MX player if you use Network Stream feature in it and HW acceleration enabled. I tried to use MX player and it seem to work the same way as our test app, except that MX player stopped playing video at 1:02 (Total length of the video is 1:31), may be this stopping of the video is random, I did not do too many tests.

Thanks for your help.

Re: Video delay problem in Android

jack_mao — Wed, 19 Jun 2013 10:10:49 GMT

Hi,

I just setup my android to play the mp4 file local, seems no issue, then what's you android release version? By the way how do you setup the vod system , use what's the software to play the video on android? if it's easy to setup, could you give some detail. I still guess there are network issue such as package lose cause the A/V sync issue, when you give me the version number, I will added some debug code in the multimedia lib for you test to get the A/V sync log

Re: Video delay problem in Android

sergeikotlyachk — Wed, 19 Jun 2013 14:39:44 GMT

Playing network streams is very different from playing local files. Please follow these exact steps to reproduce:

1. Install VideoLan (sometimes named VLC) ver 2.0.7 on Windows or Linux (I used Windows 7 64bit). You can download it from http://www.videolan.org

2. copy sample.mp4 to installation directory of VLC

3. Start command line window and go to installation directory of VLC

4. Use this guide for reference: http://www.videolan.org/doc/streaming-howto/en/ch05.html specifically this section: Video On Demand

I used this command vlc --ttl 12 -vvv --color -I telnet --rtsp-host IP_ADDRESS:554

Where instead of IP_ADDRESS you use actual (not loopback) IP address of the desktop. (you can find it in the output of ipconfig)

5. Use telnet to connect to vlc (it uses port 4212). Don't worry that what you type is not displayed - it is expected. If you don't have telnet on Windows box, you may have to install it using Server Manager or Programs and Features. Or just use PuTTy:

PuTTY Download Page

telnet localhost 4212

6. Issue these commands in telnet which will setup vlc to stream sample.mp4 in VOD mode:

new Test vod enabled

setup Test input sample.mp4

7. Test that vlc running in streaming mode is working by running second instance of vlc with these arguments (replace IP_ADDRESS):

vlc rtsp://IP_ADDRESS:554/Test

8. Install MX player on Freescale board: https://sites.google.com/site/mxvpen/

MX Player - Android Apps on Google Play

9. In MX Player select context menu and choose 'Network Stream'

10. Enter rtsp://IP_ADDRESS:554/Test and start timer when video starts showing. Observe delays at 0:30 and 0:40 and stream stopping playing around 55/60 seconds. (entire stream is 1:31)

Re: Video delay problem in Android

sergeikotlyachk — Wed, 19 Jun 2013 14:43:18 GMT

I am not sure what you mean by android release version. I have given environment settings that are output by the build before. Is this what you are looking for? PLATFORM_VERSION=4.0.4 If not, please tell me how to determine this version.

Re: Video delay problem in Android

jack_mao — Thu, 20 Jun 2013 01:39:21 GMT

Hi, I think the PLATFORM_VERSION number is your side, I want to know the fsl android release version. Anyway, I will try to setup such a network environment first.

Re: Video delay problem in Android

sergeikotlyachk — Wed, 26 Jun 2013 15:11:35 GMT

Is it r13.4.1 number you are looking for?

Re: Video delay problem in Android

jack_mao — Thu, 27 Jun 2013 09:41:50 GMT

Hi,

Sorry for late, I have setup the environment and reproduce this issue , currently I 'm try do some modification and try on the board, if there is still a gap to the target I will look for help internal.

Re: Video delay problem in Android

jack_mao — Fri, 28 Jun 2013 02:51:25 GMT

Hi,

I tried to reduce the size of the buffer data to 30*1000, the first frame come out more quickly, by the way I compare the mx player and native video player of android, the video player start very quickly, so it seems some delay is caused by the application, I read the log, due to no audio , the mx player will wait for timeout , so it will take longer.

what I did test is install the vlc player apk, and in the application select more-> manual rtsp -> select the player, different player has different result.

So I think you need a realtime player.

best regards

Jack

Re: Video delay problem in Android

sergeikotlyachk — Fri, 28 Jun 2013 14:01:25 GMT

Hi Junping,

Unfortunately, I will still need your help:

1. When you say 'native video player of android', what do you mean? We simply used MediaPlayer class and gave it rtsp url. Are you saying that MediaPlayer class is not supposed to play rtsp urls? Then why does it support it?

2. When you said ' the video player start very quickly', which player you were referring to?

3. What application you are referring to when you said 'delay is caused by the application'?

4. VLC Player can not be used for comparison because it uses software mode which pegs cpu ( check cpu usage when using MX player in HW accelerated mode and VLC player, you should see big difference). I have tried other software mode players and while they have smaller delay, they all peg cpu. We are not interested to use software mode because it will only play one stream and not more. We are interested in using Freescale VPU as explained in https://community.freescale.com/docs/DOC-93611

And since delay and de-syncronization is occurring in Freescale code/hardware, I am expecting your help on this matter. What is the reason of having VPU when you can not use it?

Thank you,

Sergei.

Re: Video delay problem in Android

jack_mao — Mon, 01 Jul 2013 01:49:21 GMT

Hi Sergei,

1.The native video player means gallery which is pre-installed in android system. It will use the hardware VPU, all my test is based on the hardware VPU, I just want you to compare the different on your side, I need make sure that we meet the same result.

2. Gallery start very quickly.

3. mx player, I notice it spend some time on audio play by the log. please confirm this.

4. I use VLC Direct Pro apk to use it RTSP streaming function, it will auto popup the window to let the user select the Gallery or mxplayer.

All I did is base on the VPU, VLC is using as the streaming protocol, mxplayer or gallery as the video player.

Please try those to see if we meet the same result.

best regards

Jack

Re: Video delay problem in Android

sergeikotlyachk — Tue, 02 Jul 2013 17:54:09 GMT

I tried what you asked for.

3. Mx player takes longer time to start, but video delay is the same as through Gallery.

4. VLC Direct Pro + Gallery: initial delay for 1080p resolution is 4.5 seconds. Initial delay for 800x600 resolution is ~2 seconds. After couple of minutes, delay for SD resolution becomes 5 seconds.

Problem is still there.

What did you do in firmware to ' reduce the size of the buffer data to 30*1000' ?