GStreamer crashing on i.MX6 (Boundary Devices Nitrogen6x)

jimheath · ‎02-20-2013

Hi Everyone,

I've been developing an application for the new Boundary Devices Nitrogen6x board running an i.MX6Q processor. I'm using the Timesys BSP, kernel version: 3.0.35-ts-armv7l-2026-geaaf30e (LinuxLink Development Center for Boundary Devices Nitrogen6X | Timesys Embedded Linux). The application relies on GStreamer 0.10.36 and gst-fsl-plugin for playing MPEG-2 transport stream files that are H.264 encoded.

I ran into a problem playing back files that are around 2GB or larger. If I use playbin2 for the GStreamer pipeline and specify a large video file, the pipeline transitions to the preroll state and appears to identify the elements needed to play the video, but then it just sits there. I don't see any video output, not even the first frame of the video and within 30 seconds the program crashes stating it ran out of memory. This happens regardless of whether I use my program or use the gst-launch command.

On the other hand, if I construct the following pipeline, the video plays fine:

gst-launch filesrc location=video.ts ! tsdemux name=demux demux. ! queue max-size-buffers=0 max-size-time=0 ! vpudec ! mfw_v4lsink demux. ! queue max-size-buffers=0 max-size-time=0 ! mfw_mp3decoder ! audioconvert ! 'audio/x-raw-int,channels=2' ! alsasink

The problem is, I would rather use playbin2 because it offers more flexibility. I have encountered certain video files that will not play back using the pipeline above, but will with playbin2. I have compared a video that does playback and one that does not using gst-discoverer and see no difference; both are transport streams and both are H.264 encoded.

Can anyone offer any suggestions on how to get playbin2 to play these large video files?

Thanks,

Jim

jack_mao · ‎04-06-2013

hi Jim& Lonsn,

To DMA allocation issue, I have a patch, please try it, please remember , it's not the the formal patch from freescale.

Jack

View solution in original post

lonsn · ‎04-03-2013

I also meet with the same memory allocation failed issue in i.mx6 solo wandboard.

>>V4L_SINK: Actually buffer status:

hardware buffer : 9

software buffer : 0

[ 687.629140] multiqueue0:src: page allocation failure: order:10, mode:0x1

[ 687.641748] Backtrace:

[ 687.644256] [<800372d0>] (dump_backtrace+0x0/0x10c) from [<80390e2c>] (dump_stack+0x18/0x1c)

[ 687.652997] r6:00000000 r5:00000001 r4:00000000 r3:00000002

[ 687.658761] [<80390e14>] (dump_stack+0x0/0x1c) from [<8009be40>] (warn_alloc_failed+0xec/0x10c)

[ 687.667523] [<8009bd54>] (warn_alloc_failed+0x0/0x10c) from [<8009e150>] (__alloc_pages_nodemask+0x5e8/0x640)

[ 687.677501] r3:00000000 r2:00000000

[ 687.681131] r7:0000000a r6:804dd18c r5:98ffe000 r4:00000001

[ 687.686883] [<8009db68>] (__alloc_pages_nodemask+0x0/0x640) from [<8003a718>] (__dma_alloc+0xcc/0x2bc)

[ 687.696241] [<8003a64c>] (__dma_alloc+0x0/0x2bc) from [<8003aef0>] (dma_alloc_coherent+0x5c/0x68)

[ 687.705150] [<8003ae94>] (dma_alloc_coherent+0x0/0x68) from [<80251bf0>] (__videobuf_mmap_mapper+0x98/0x170)

[ 687.715025] r7:9c1af540 r6:9c116c48 r5:9c0d35ec r4:81fff1b8

[ 687.720771] [<80251b58>] (__videobuf_mmap_mapper+0x0/0x170) from [<80250888>] (videobuf_mmap_mapper+0xc0/0x110)

[ 687.730909] r8:9cfabba0 r7:000000ff r6:81fff1b8 r5:81fff1b8 r4:9c116c48

[ 687.737728] [<802507c8>] (videobuf_mmap_mapper+0x0/0x110) from [<8024d7f4>] (mxc_vout_mmap+0x28/0x58)

[ 687.747013] r5:9c116c00 r4:9c7ed200

[ 687.750631] [<8024d7cc>] (mxc_vout_mmap+0x0/0x58) from [<802418e0>] (v4l2_mmap+0x70/0x90)

[ 687.758856] r6:81fff1b8 r5:9cfe3f00 r4:9c7ed200 r3:8024d7cc

[ 687.764584] [<80241870>] (v4l2_mmap+0x0/0x90) from [<800b0bc8>] (mmap_region+0x208/0x410)

[ 687.772808] r6:30c3d000 r5:9cfe3f00 r4:81fff1b8 r3:80241870

[ 687.778555] [<800b09c0>] (mmap_region+0x0/0x410) from [<800b1088>] (do_mmap_pgoff+0x2b8/0x314)

[ 687.787222] [<800b0dd0>] (do_mmap_pgoff+0x0/0x314) from [<800b117c>] (sys_mmap_pgoff+0x98/0xd0)

[ 687.795949] [<800b10e4>] (sys_mmap_pgoff+0x0/0xd0) from [<80033b80>] (ret_fast_syscall+0x0/0x30)

[ 687.804777] Mem-info:

[ 687.807072] Normal per-cpu:

[ 687.809870] CPU 0: hi: 90, btch: 15 usd: 75

[ 687.814676] active_anon:1201 inactive_anon:0 isolated_anon:0

[ 687.814682] active_file:12205 inactive_file:34915 isolated_file:0

[ 687.814687] unevictable:8 dirty:0 writeback:0 unstable:0

[ 687.814692] free:27285 slab_reclaimable:542 slab_unreclaimable:947

[ 687.814697] mapped:1732 shmem:0 pagetables:73 bounce:0

[ 687.843497] Normal free:109140kB min:2332kB low:2912kB high:3496kB active_anon:4804kB inactive_anon:0kB active_file:48820kB inactive_file:139660kB unevictable:32kB isolated(anon):0kB isolated(file):0kB present:339968kB mlocked:0kB dirty:0kB writeback:0kB mapped:6928kB shmem:0kB slab_reclaimable:2168kB slab_unreclaimable:3788kB kernel_stack:416kB pagetables:292kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no

[ 687.882555] lowmem_reserve[]: 0 0 0

[ 687.886113] Normal: 1083*4kB 925*8kB 642*16kB 61*32kB 33*64kB 23*128kB 21*256kB 26*512kB 8*1024kB 20*2048kB 1*4096kB 1*8192kB 0*16384kB 0*32768kB = 109140kB

[ 687.900345] 47137 total pagecache pages

[ 687.916497] 131072 pages of RAM

[ 687.919658] 27457 free pages

[ 687.922542] 47430 reserved pages

[ 687.925835] 901 slab pages

[ 687.928545] 3296 pages shared

[ 687.931513] 0 pages swap cached

[ 687.934663] mxc_v4l2_output mxc_v4l2_output.0: dma_alloc_coherent size 3133440 failed

[ 687.942541] mxc_v4l2_output mxc_v4l2_output.0: offset invalid [offset=0x2fd000]

jimheath · ‎03-18-2013

Junping,

Any update on when Freescale will assist me with this problem?

Thanks,

Jim

jimheath · ‎03-11-2013

Right, the tsdemux from the "bad" plugins isn't ideal but the aiurdemux from Freescale won't play >=2gb files, plus it causes random segmentation faults when i perform seek operations. playbin2 selects aiurdemux since it has a higher rank than tsdemux.

LeonardoSandova · ‎02-28-2013

Hi jimheath

I understand your situation. Your problem has been posted in an internal Freescale group, meaning that it will end up in a specialist which can provide the proper solution. In the other hand, I will try to reproduce the issue you are facing but this can take me some time to have the setup ready.

Leo

jimheath · ‎03-04-2013

Will I be hearing from someone at Freescale soon?

Thanks,

Jim

LeonardoSandova · ‎03-04-2013

Hi Jim. I am not sure how long it will delay to get you an answer but I am sure your question is now on someone's queue. BTW, this document had some comments from EricNelson, where did they go? EricNelson, do you delete them?

Leo

EricNelson · ‎03-04-2013

Hi Leo,

I didn't even know I __could__ delete a comment, so I didn't do this intentionally.

My comment had to do with a result that I couldn't seem to replicate:

In my first attempt, it seemed that I was able to see a memory leak w/gplay even flushing the disk cache.
Later attempts to isolate it to aiurdemux failed, then
Repeated attempt using gplay succeeded.

If I'm not mistaken, Jim's not seeing memory leaks either though. What he was seeing seemed related to the disk cache using available RAM and a sequence like this cleaned things up:

for n in 0 1 2 3 ; do echo $n > /proc/sys/vm/drop_caches ; done

Jim, please confirm.

jimheath · ‎03-04-2013

I'm not sure what happened to this thread, I didn't modify anything.

I do see a memory leak still when my application is using GStreamer. If I continue to let it run it eventually crashes because it runs out of memory. Eric did clear up the other issue I reported that turned out to be the disk cache.

jimheath · ‎02-28-2013

Hi Leo,

Thanks for your response, it's good to hear a specialist will be helping me out.

I think I can offer some more information on this problem. I've been trying everything I can think of to determine the root cause of the problem. From some tests I've been running yesterday and today, I believe the segmentation faults are linked to me performing a seek operation on the transport stream video when using the aiurdemux element. I'm using the GStreamer C function: gst_element_seek_simple. This is an intermittent problem so it sometimes happens upon playing the first video or 1 to 10+ minutes of looping through the various videos. If I disable my application from performing a seek and always start the video from the beginning, I'm not seeing it crash.

If I use the tsdemux element from the gst-bad-plugins, I can perform seeks on the same set of videos without any segmentation faults. I've run this test for over an hour without any problems. I'm using a little test application written using Qt 4.8.3 that just continuously cycles through a playlist of transport stream videos. It selects a random position in the video to begin playing and plays it for 20 seconds then moves on to the next video.

I would use the tsdemux element because it handles the large videos (+2GB files) but for some videos I get a generic "stream error" message and it won't play it. But these videos that won't play with tsdemux WILL play with aiurdemux as long as they are under 2GB. All videos have the same encoding (h.264) so I'm not sure why tsdemux can't deal with them.

With both transport stream demuxers, I still see kernel error messages every once in awhile which I attached in case these were lost from my original posts. These don't stop the execution of the program, in fact, once the program moves on to the next video, video playback continues and everything appears normal.

Here's a backtrace on the core dump when using aiurdemux:

#0 0x2bbd17f4 in memcpy () from /home/jim/timesys/nitrogen6x/toolchain/lib/libc.so.6

#1 0x2c26d340 in MPEG2FindH264Frames ()