BSP 11.09: video decoding issues

[WolfgangW]

Hi,

after having sorted out some of my own problems I am still having various issues with video decoding using the MX53QSB. I am using the basic ext2 rootfs and playing video using gst-launch playbin2 from the command line.

1) With some [1080p] videos, playing them immediately after boot crashes the video decoder hard (kernel oops with NULL pointer dereference), only chance for recovery is board reboot. However, playing another [720p] video before "fixes" this, so I can also play these problematic videos without a crash after having first played any of these [720p] movies.

On of these candidates is hd_other_samsung_led_motion.m2ts from demo world. Can anybody else reproduce this and/or comment on this?

2) Video is always a bit jerky - sometimes more, sometimes less. This is especially influenced by audio, as I saw with the original hd_other_samsung_led_motion.m2ts audio can not be decoded and is discarded, playback is almost good. Recoded with AAC audio (thus audio is played back, too), the video is always very jerky. However, even the "good" clips (without audio or those [720p] clips that work good with audio) are jerky on some playbacks while playing really smooth on another occasion.

So both things seem to point to some synchronisation problems?

Unfortunately, I was not able to disable overlay and use synchronous decode/output mode, which should be available, too, if I understand correctly?

3) With some recoded videos, decoding is not possible at all due to dma buffer allocation failure. I get messages like these:

allocated software buffer
>>V4L_SINK: Actually buffer status:
hardware buffer : 11
software buffer : 3
[ERR] mem allocation failed!
ERROR: from element /GstPlayBin2:playbin20/GstURIDecodeBin:uridecodebin0/GstDecodeBin2:decodebin20/MfwGstVPU_Dec:mfwgstvpu_dec0: fatal error
Additional debug info:
Allocation of the Frame Buffers Failed

(leaving out kernel messages here, which basically tell me dma_alloc_coherent with order 10 failed)

This will likely be an encoding error [although these videos still play on other decoders...], but it would be interesting to get more information which buffer is too large here. Or is there any chance to reserve more memory for this purpose via kernel command line?

Does anybody have experience what could be done to avoid these problems?

Thank you for reading,

Wolfgang

[WolfgangW]

Indeed, there is a difference between NFS and other sources - but only during the first pass. After this, the file is completely in vfs cache, and it does not matter any more. I did not really care about this because I used many passes for all tests anyways as there are slight differences in every pass.

However, this does of course not apply if you have a playlist exceeding available system memory... ;-)

[LVJACKEY]

Once it finished playing the HD vdieo file, the gplay program fails to play the same video file, even I do ''echo 1 > /proc/sys/vm/lowmem_reserve_ratio". Below is the output log from the gplay.

root@freescale ~$ gplay /mnt/pinoeer.mpg                    
playbin2 is employed!
fsl_player_init(): Successfully initialize!
fsl_player_set_media_location(): filename=/mnt/pinoeer.mpg
[Stopped  ][Vol=01][00:00:00/00:00:00][fps:0]Aiur: 2.0.3
Core: BLN_MAD-MMLAYER_MPG2PARSER_ARM11_04.00.05  build on Sep 22 2011 11:18:22
  mime: video/mpeg, mpegversion=(int)[1,2]; video/mpegts, systemstream=true; video/x-cdxa
  file: lib_mpg2_parser_arm11_elinux.3.0.so.4.00.04
Content Info:
        URI:
              file:///mnt/pinoeer.mpg
        Idx File:
              /root/.aiur/.mnt.pinoeer.mpg.aidx
        Seekable  : Yes
        Size(byte): 1435914108

Mpeg2CreateParser:parser created successfully
Movie Info:
        Seekable  : Yes
        Live      : No
        Duration  : 0:09:16.096000000
        ReadMode  : File
        Track     : 2

Track 00[Video]: Enabled
        Duration: 0:00:00.000000000
        Language:
        Mime:
              video/mpeg, systemstream=(boolean)false, parsed=(boolean)true, width=(int)1920,
              height=(int)1080, framerate=(fraction)30000/1001
[INFO]  Product Info: i.MX53
VPU Version: firmware 13.4.41; libvpu: 5.3.2
MFW_GST_VPU_DECODER_PLUGIN 2.0.3 build on Oct 24 2011 17:40:06.
Track 01[Audio]: Enabled
        Duration: 0:09:16.096000000
        Language:
        Mime:
              audio/x-ac3, channels=(int)2, rate=(int)48000, bitrate=(int)192000
get GST_MESSAGE_ELEMENT missing-plugin, type=(string)decoder, detail=(GstCaps)audio/x-ac3, bitrate=(int)192000, name=(string)"AC-3\ \(ATSC\ A/52\)\ decoder";
FOUND GST_MESSAGE_TAG!
          codec: AC3
        bitrate: 192000
FOUND GST_MESSAGE_TAG!
container format: MPEG
MFW_GST_V4LSINK_PLUGIN 2.0.3 build on Oct 24 2011 17:40:32.
FOUND GST_MESSAGE_TAG!
          codec: MPEG2
        bitrate: 20000000
gplay: page allocation failure. order:10, mode:0xd1
[<80038538>] (unwind_backtrace+0x0/0xf0) from [<8009c4c8>] (__alloc_pages_nodemask+0x500/0x568)
[<8009c4c8>] (__alloc_pages_nodemask+0x500/0x568) from [<80038f70>] (__dma_alloc+0xec/0x2b8)
[<80038f70>] (__dma_alloc+0xec/0x2b8) from [<800391b4>] (dma_alloc_coherent+0x54/0x60)
[<800391b4>] (dma_alloc_coherent+0x54/0x60) from [<802e40b8>] (mxc_allocate_buffers+0x24/0x7c)
[<802e40b8>] (mxc_allocate_buffers+0x24/0x7c) from [<802e6098>] (mxc_v4l2out_do_ioctl+0x488/0xd9c)
[<802e6098>] (mxc_v4l2out_do_ioctl+0x488/0xd9c) from [<802d945c>] (video_usercopy+0x1e4/0x310)
[<802d945c>] (video_usercopy+0x1e4/0x310) from [<802d889c>] (v4l2_ioctl+0x44/0x4c)
[<802d889c>] (v4l2_ioctl+0x44/0x4c) from [<800d0bb4>] (vfs_ioctl+0x7c/0xac)
[<800d0bb4>] (vfs_ioctl+0x7c/0xac) from [<800d1204>] (do_vfs_ioctl+0x52c/0x57c)
[<800d1204>] (do_vfs_ioctl+0x52c/0x57c) from [<800d1288>] (sys_ioctl+0x34/0x54)
[<800d1288>] (sys_ioctl+0x34/0x54) from [<80033f80>] (ret_fast_syscall+0x0/0x30)
Mem-info:
DMA per-cpu:
CPU    0: hi:   18, btch:   3 usd:   0
Normal per-cpu:
CPU    0: hi:  186, btch:  31 usd:   2
active_anon:332 inactive_anon:438 isolated_anon:0
 active_file:1832 inactive_file:224507 isolated_file:1024
 unevictable:0 dirty:0 writeback:0 unstable:0
 free:14911 slab_reclaimable:707 slab_unreclaimable:786
 mapped:1034 shmem:142 pagetables:28 bounce:0
DMA free:43808kB min:388kB low:484kB high:580kB active_anon:0kB inactive_anon:0kB active_file:24kB inactive_file:17068kB unevictable:0kB isolated(anon):0kB isolated(file):0kB po
lowmem_reserve[]: 0 226496 226496 226496
Normal free:15836kB min:3612kB low:4512kB high:5416kB active_anon:1328kB inactive_anon:1752kB active_file:7304kB inactive_file:880960kB unevictable:0kB isolated(anon):0kB isolao
lowmem_reserve[]: 0 0 0 0
DMA: 18*4kB 5*8kB 17*16kB 15*32kB 15*64kB 10*128kB 11*256kB 14*512kB 6*1024kB 12*2048kB 0*4096kB 0*8192kB 0*16384kB = 43808kB
Normal: 37*4kB 31*8kB 13*16kB 8*32kB 6*64kB 4*128kB 3*256kB 2*512kB 0*1024kB 2*2048kB 2*4096kB 0*8192kB 0*16384kB = 15836kB
226749 total pagecache pages
0 pages in swap cache
Swap cache stats: add 0, delete 0, find 0/0
Free swap  = 0kB
Total swap = 0kB
253952 pages of RAM
14973 free pages
4354 reserved pages
1372 slab pages
2295 pages shared
0 pages swap cached
dma_alloc_coherent failed.
allocated software buffer
>>V4L_SINK: Actually buffer status:
        hardware buffer : 9
        software buffer : 3
gplay: page allocation failure. order:10, mode:0xd1
[<80038538>] (unwind_backtrace+0x0/0xf0) from [<8009c4c8>] (__alloc_pages_nodemask+0x500/0x568)
[<8009c4c8>] (__alloc_pages_nodemask+0x500/0x568) from [<80038f70>] (__dma_alloc+0xec/0x2b8)
[<80038f70>] (__dma_alloc+0xec/0x2b8) from [<800391b4>] (dma_alloc_coherent+0x54/0x60)
[<800391b4>] (dma_alloc_coherent+0x54/0x60) from [<803085dc>] (vpu_alloc_dma_buffer+0x2c/0x50)
[<803085dc>] (vpu_alloc_dma_buffer+0x2c/0x50) from [<803086c4>] (vpu_ioctl+0xc4/0x72c)
[<803086c4>] (vpu_ioctl+0xc4/0x72c) from [<800d0bb4>] (vfs_ioctl+0x7c/0xac)
[<800d0bb4>] (vfs_ioctl+0x7c/0xac) from [<800d1204>] (do_vfs_ioctl+0x52c/0x57c)
[<800d1204>] (do_vfs_ioctl+0x52c/0x57c) from [<800d1288>] (sys_ioctl+0x34/0x54)
[<800d1288>] (sys_ioctl+0x34/0x54) from [<80033f80>] (ret_fast_syscall+0x0/0x30)
Mem-info:
DMA per-cpu:
CPU    0: hi:   18, btch:   3 usd:   0
Normal per-cpu:
CPU    0: hi:  186, btch:  31 usd:   0
active_anon:332 inactive_anon:438 isolated_anon:0
 active_file:1784 inactive_file:219163 isolated_file:0
 unevictable:0 dirty:0 writeback:0 unstable:0
 free:12749 slab_reclaimable:484 slab_unreclaimable:786
 mapped:1035 shmem:142 pagetables:41 bounce:0
DMA free:28760kB min:388kB low:484kB high:580kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB preseno
lowmem_reserve[]: 0 226496 226496 226496
Normal free:22236kB min:3612kB low:4512kB high:5416kB active_anon:1328kB inactive_anon:1752kB active_file:7136kB inactive_file:876652kB unevictable:0kB isolated(anon):0kB isolao
lowmem_reserve[]: 0 0 0 0
DMA: 20*4kB 19*8kB 9*16kB 9*32kB 9*64kB 9*128kB 5*256kB 3*512kB 13*1024kB 5*2048kB 0*4096kB 0*8192kB 0*16384kB = 28760kB
Normal: 67*4kB 34*8kB 18*16kB 13*32kB 8*64kB 4*128kB 4*256kB 3*512kB 1*1024kB 2*2048kB 3*4096kB 0*8192kB 0*16384kB = 22236kB
221089 total pagecache pages
0 pages in swap cache
Swap cache stats: add 0, delete 0, find 0/0
Free swap  = 0kB
Total swap = 0kB
253952 pages of RAM
12806 free pages
4354 reserved pages
1155 slab pages
2020 pages shared
0 pages swap cached
Physical memory allocation error!
Physical memory allocation error!

[LVJACKEY]

I got the exactly same troubles with the video playback issue under 11.09 SDK,  The video jerky happens when I played HD video files from NFS path. But the video goes smooth when I play the same HD video from a SSD card through the SATA interface. My display is HDMI on 720p resolution.

[WolfgangW]

One more reply to straighten things up:

- decoder crash with 1080p as the first video to play happens only if the output (HDMI in my case) is set to 1080p (1920x1080), too.

All other effects/problems are independent of output resolution, though a bit worse with 1080p output.

- video is always jerky if audio is played alongside. More or less depending on other things, but always a bit. When using gplay as a player the framerate displayed is always less than what the video file is coded in (29 instead of 30, 23 instead of 24 and so on). I could not see a dependency on CPU load, it happens with videos consuming 80% CPU time to play as well as those using only 10-15%.

- from time to time, the decoder gets in a strange state where video decoding is crippled completely. When playing video+audio, the audio is played correctly but the video is not. There are messages like these:

mxc_ipu mxc_ipu: IPU Error - IPU_INT_STAT_5 = 0x08000000
mxc_sdc_fb mxc_sdc_fb.2: MXCFB_WAIT_FOR_VSYNC: timeout 0
mxc_sdc_fb mxc_sdc_fb.2: MXCFB_WAIT_FOR_VSYNC: timeout 0
mxc_sdc_fb mxc_sdc_fb.2: MXCFB_WAIT_FOR_VSYNC: timeout 0
WARNING: from element /GstPlayBin2:playbin20/GstPlaySink:playsink0/GstBin:vbin/GstAutoVideoSink:videosink/MFW_GST_V4LSINK_INFO_T
:videosink-actual-sink-mfw_v4l: A lot of buffers are being dropped.
Additional debug info:
gstbasesink.c(2597): gst_base_sink_is_too_late (): /GstPlayBin2:playbin20/GstPlaySink:playsink0/GstBin:vbin/GstAutoVideoSink:vid
eosink/MFW_GST_V4LSINK_INFO_T:videosink-actual-sink-mfw_v4l:
There may be a timestamping problem, or this computer is too slow.

When video-only is played, the playback finishes in around 1/6 of the time it should take. Seems to me like a time base error?

So all in all for me there seem to be severe synchronisation issues. I am not familiar with gstreamer enough yet to really try other synchronisation scheme (different clock source?), or is there maybe some other more minimalistic player to test such a thing?

Regards, Wolfgang

[WolfgangW]

Hi,

thanks for the reply!

I tried media files located on USB, SDCard and NFS - without any noticeable difference.

5.1 audio support is not needed - it was just a coincidence that this special video stream has a 5.1 audio track. This caused video-only playback which plays quite smooth in contrast to any video + audio playback I tried.

I could not find any reference to memory allocation. Can you remember in which document you found it? I tried IMX_MMCODECS_11.09/Documentation/Linux_Multimedia_Framework_Docs_MX53Ubuntu_2.0.3/docs/Linux_Multimedia_Framework_User_Guide.pdf and also the accompanying html documents, but did not find anything except something specific to encoding on i.MX31.

The biggest problem I see is jerky video, especially when a stream with audio is played but also without audio in some playback passes. Don't get me wrong, it is not really bad, but it is clearly visible if you know how playback should be, and as it is not present in most passes when playing without audio, I am quite convinced it is simply some synchronisation issue.

Regards,

Wolfgang

[daiane_angolini]

Are your media file located on sdcard? Try to avoid huge memory access delay

You may try to configure a piece of ram to be exclusive to VPU. Take a look on multimedia user guide, I don´t know it by heart, but I remember it´s a command line like: $ echo 1,1,1, > /lalalalala

I´m not sure fsl plugin has 5.1 audio support. Please, double check it for your files.