GStreamer crashing on i.MX6 (Boundary Devices Nitrogen6x)

[jimheath]

Hi Everyone,

I've been developing an application for the new Boundary Devices Nitrogen6x board running an i.MX6Q processor. I'm using the Timesys BSP, kernel version: 3.0.35-ts-armv7l-2026-geaaf30e (LinuxLink Development Center for Boundary Devices Nitrogen6X | Timesys Embedded Linux). The application relies on GStreamer 0.10.36 and gst-fsl-plugin for playing MPEG-2 transport stream files that are H.264 encoded.

I ran into a problem playing back files that are around 2GB or larger. If I use playbin2 for the GStreamer pipeline and specify a large video file, the pipeline transitions to the preroll state and appears to identify the elements needed to play the video, but then it just sits there. I don't see any video output, not even the first frame of the video and within 30 seconds the program crashes stating it ran out of memory. This happens regardless of whether I use my program or use the gst-launch command.

On the other hand, if I construct the following pipeline, the video plays fine:

gst-launch filesrc location=video.ts ! tsdemux name=demux demux. ! queue max-size-buffers=0 max-size-time=0 ! vpudec ! mfw_v4lsink demux. ! queue max-size-buffers=0 max-size-time=0 ! mfw_mp3decoder ! audioconvert ! 'audio/x-raw-int,channels=2' ! alsasink

The problem is, I would rather use playbin2 because it offers more flexibility. I have encountered certain video files that will not play back using the pipeline above, but will with playbin2. I have compared a video that does playback and one that does not using gst-discoverer and see no difference; both are transport streams and both are H.264 encoded.

Can anyone offer any suggestions on how to get playbin2 to play these large video files?

Thanks,

Jim

[bjurr]

Sorry to revive an old post, but does anyone know if a similar patch has been made or integrated for newer kernels? I'm experiencing these issues with linux-imx-3.10.53 and linux-imx-3.14.28?

Any pointers would be appreciated. Thanks!

[jack_mao]

Hi,

    I think you could use kmplayer to get the properties of the stream, and paste here to let me see if it's ok. By the way, could you describe the current status of your issue here?

[jimheath]

This thread is showing "Assumed Answered" but it is not. I'm still needing help with this.

[jack_mao]

Jim,

    I'm sorry, something make me miss your reply, could you use the transweb? if Yes, could you send the file to me, my email is b39360@freescale.com, you have two question now, right?one for playbin2(2G), the other is stable. I  try to reproduce it on my site and consult it deeply.

Jack

[jimheath]

I'm not sure what the "transweb" is. What files do you want me to send you? I've posted the files in this thread. Yes the two issues still remain, the aiurdemux will not play videos that are >= 2GB and I'm seeing kernel errors about memory allocation.

Eric Nelson did suggest a workaround that seems to be working for stopping the memory allocation errors. If I tell the OS to drop clean cashes before playing a video, I don't ever see the allocation errors:

echo 3 > /proc/sys/vm/drop_caches

According to (drop_caches | LinuxInsight) this frees pagecache, dentries and inodes.

This seems to confirm what Eric believes is the problem; the IPU/VPU drivers are not able to allocate a contiguous block of memory. When this happens I see the kernel error message about memory allocation. So by forcing the OS to drop caches, the IPU/VPU drivers are able to allocate the required block of contiguous memory.

[jack_mao]

Hi Jim,

   I can't access the torrent file from office network, prohibited by policy. I'm looking for such file as you mentioned >2GB, H264, ts stream. Do you have any channel to send file to freescale or later I download from private network?

Jack

[jimheath]

Jack,

I'm uploading a public domain file that I converted to an H.264 encoded transport stream. As soon as it finishes I'll post the link for you.

Jim

[jack_mao]

Hi Jim,

      I tried your file on our latest code by following command , all it's ok. It seems  some change did, our multimedia release will be 3.0.7 and release date should be the end of this month or early next month, could you wait for it ?

gst-launch playbin2 uri=file:///public-domain-test.ts

gst-launch filesrc location=public-domain-test.ts typefind=tru

e ! aiurdemux name=demux demux. ! queue max-size-buffers=0 max-size-time=0 ! vpu

dec ! mfw_v4lsink device="/dev/video17" disp-width=1024 disp-height=768 demux. !

queue max-size-buffers=0 max-size-time=0 ! beepdec ! audioconvert ! 'audio/x-ra

w-int, channels=2' ! alsasink

gplay  public-domain-test.ts

Jack

[jimheath]

Jack,

That's great news, thanks for taking the time to perform the test. We're still maybe a month away from being ready to release our product so hopefully that will be enough time for 3.0.7 to be released.

Thanks again!

Jim

[gavinbao]

can you slove this issue,can you help me ?Thanks!

[jack_mao]

Hi Jim,

      I finished download, you could delete it. I need sometime to look into it.

Jack

[lonsn]

Junping,

For the mxc_v4l2_output memory allocation failed issue, there is no need to test the >2GB file. Just use a file which size is bigger than i.mx6 device physical memory. After the page cache is very big which you can see from linux top command output, then kernel will print mxc_v4l2_output dma memory allocation failed.

[jack_mao · Accepted solution]

hi Jim& Lonsn,

     To DMA allocation issue, I have a patch, please try it, please remember ,  it's not the the formal patch from freescale.

Jack

[hyunjaejun]

hi Junping Mao

look at this page https://community.freescale.com/thread/315282

thanks.

[Dennis_Han]

Hello all

I have a very similar problem on this.

cron rule for dropping caches, plus DMA size is increased  to 500MB, but without patch (patch-v4l.zzip.zip)

Anway I'm using 3.0.36-4.1.0 kernel (sabrelite), and fsl-codec and others are 3.0.8 by yocto

with these environment,  random running of more than 800 files in 12 hours showed the memory error.

I read all about this issue and found that it 'caused from defragmentation of DMA memory and it's handling.

no memory leak ?  just defragmentation ?

even though i flush the caches, i see memory leak too.

i tested with very simple gst base player.

am i supposed to patch the kernel with patch-v4l.zzip.zip ?

anyone got new patch for 4.1.0 ?

this is my log

[<800c3f1c>] (__alloc_pages_nodemask+0x4e8/0x728) from [<8004d8a8>] (__dma_alloc+0x9c/0x2f4)

[<8004d8a8>] (__dma_alloc+0x9c/0x2f4) from [<8004df18>] (dma_alloc_coherent+0x5c/0x64)

[<8004df18>] (dma_alloc_coherent+0x5c/0x64) from [<803b1ffc>] (vpu_alloc_dma_buffer+0x2c/0x54)

[<803b1ffc>] (vpu_alloc_dma_buffer+0x2c/0x54) from [<803b23b0>] (vpu_ioctl+0x38c/0x854)

[<803b23b0>] (vpu_ioctl+0x38c/0x854) from [<80102380>] (do_vfs_ioctl+0x80/0x538)

[<80102380>] (do_vfs_ioctl+0x80/0x538) from [<8010286c>] (sys_ioctl+0x34/0x60)

[<8010286c>] (sys_ioctl+0x34/0x60) from [<80044140>] (ret_fast_syscall+0x0/0x30)

DMA per-cpu:

CPU    0: hi:  186, btch:  31 usd: 167

CPU    1: hi:  186, btch:  31 usd:  30

CPU    2: hi:  186, btch:  31 usd:   0

CPU    3: hi:  186, btch:  31 usd:   0

CPU    0: hi:  186, btch:  31 usd: 167

CPU    1: hi:  186, btch:  31 usd:  19

CPU    2: hi:  186, btch:  31 usd:   0

CPU    3: hi:  186, btch:  31 usd:  13

active_anon:10566 inactive_anon:148 isolated_anon:0

active_file:78575 inactive_file:15416 isolated_file:32

unevictable:0 dirty:0 writeback:0 unstable:0

free:96020 slab_reclaimable:508 slab_unreclaimable:1705

mapped:1245 shmem:151 pagetables:222 bounce:0

[INFO]  bitstreamModeDMA free:352428kB min:2128kB low:2660kB high:3192kB active_anon:26504kB inactive_anon:12kB active_file:46392kB inactive_file:64kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:508000kB mlocked:0kB dirty:0kB writeback:0kB mapped:4kB shmem:24kB slab_reclaimable:184kB slab_unreclaimable:44kB kernel_stack:136kB pagetables:196kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:16 all_unreclaimable? no

1, chromaInterleave 1, mapType 0lowmem_reserve[]:, tiled2LinearEnable 0

0 391 391 391

Normal free:39836kB min:1680kB low:2100kB high:2520kB active_anon:15760kB inactive_anon:580kB active_file:236060kB inactive_file:84760kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:401312kB mlocked:0kB dirty:0kB writeback:0kB mapped:4976kB shmem:580kB slab_reclaimable:1848kB slab_unreclaimable:6776kB kernel_stack:640kB pagetables:692kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:19 all_unreclaimable? no

lowmem_reserve[]: 0 0 0 0

DMA: 892*4kB 334*8kB 227*16kB 155*32kB 157*64kB 88*128kB 82*256kB 65*512kB 58*1024kB 43*2048kB 28*4096kB 0*8192kB 0*16384kB 0*32768kB = 352560kB

Normal: 1493*4kB 913*8kB 394*16kB 130*32kB 92*64kB 19*128kB 16*256kB 10*512kB 5*1024kB 2*2048kB 0*4096kB 0*8192kB 0*16384kB 0*32768kB = 50492kB

89303 total pagecache pages

0 pages in swap cache

Swap cache stats: add 0, delete 0, find 0/0

Free swap  = 0kB

Total swap = 0kB

262144 pages of RAM

102712 free pages

37023 reserved pages

1141 slab pages

1712 pages shared

0 pages swap cached

Physical memory allocation error!

Physical memory allocation error!

[jimheath]

I'm not able to apply the patch:

patching file arch/arm/mach-mx6/board-mx6q_sabresd.c

Hunk #1 succeeded at 1385 (offset -248 lines).

Hunk #2 FAILED at 2029.

Hunk #3 succeeded at 1965 (offset -298 lines).

1 out of 3 hunks FAILED -- saving rejects to file arch/arm/mach-mx6/board-mx6q_sabresd.c.rej

patching file arch/arm/mach-mx6/devices-imx6q.h

Hunk #1 succeeded at 131 (offset 8 lines).

patching file drivers/media/video/videobuf-dma-contig.c

patching file drivers/media/video/mxc/output/mxc_vout.c

Hunk #1 succeeded at 1985 (offset 33 lines).

patching file drivers/mxc/vpu/mxc_vpu.c

Hunk #1 succeeded at 126 (offset 3 lines).

I'm using the latest Timesys Nitrogen6x BSP and they apply patches to these same files so they must be conflicting.

Jim

[EricNelson]

Hi Jim,

I just remembered that Jack's patch is for sabresd, not nitrogen6x. We'll try to re-work it to apply there.

[troy_kisky]

Here's a cleaned up version of Jack's patch for nitrogen6x.

It could be broken up into 5 patches, as each change to each file is independent, but I left it as 1.

[tengri]

Dear Troy,

Is this needed for Nitrogen6_max board as well ? I am running to following error while running gstreamer for tw6865 chip :

0:00:01.362053912  2902   0x7a6060 WARN          v4l2bufferpool gstv4l2bufferpool.c:748:gst_v4l2_buffer_pool_start:<v4l2src0:pool:src> Uncertain or not enough buffers, enabling copy threshold

Thanks in Advance

[jimheath]

So far this patch that Jack provided and Troy modified to work with the Nitrogen6x board, seems to be working. But I see this error message from the kernel when first playing a video after booting the system. I don't see it again until I reboot the system and play another video...

------------[ cut here ]------------

WARNING: at mm/page_alloc.c:1334 get_page_from_freelist+0x500/0x518()

Modules linked in:

[<800a2e08>] (unwind_backtrace+0x0/0xf8) from [<800cc9dc>] (warn_slowpath_common+0x4c/0x64)

[<800cc9dc>] (warn_slowpath_common+0x4c/0x64) from [<800cca10>] (warn_slowpath_null+0x1c/0x24)

[<800cca10>] (warn_slowpath_null+0x1c/0x24) from [<80118ec8>] (get_page_from_freelist+0x500/0x518)

[<80118ec8>] (get_page_from_freelist+0x500/0x518) from [<80119584>] (__alloc_pages_nodemask+0xf0/0x6cc)

[<80119584>] (__alloc_pages_nodemask+0xf0/0x6cc) from [<800a5d80>] (__dma_alloc+0xa4/0x300)

[<800a5d80>] (__dma_alloc+0xa4/0x300) from [<800a65b8>] (dma_alloc_coherent+0x54/0x60)

[<800a65b8>] (dma_alloc_coherent+0x54/0x60) from [<80416710>] (vpu_alloc_dma_buffer+0x2c/0x54)

[<80416710>] (vpu_alloc_dma_buffer+0x2c/0x54) from [<80416ccc>] (vpu_ioctl+0x594/0x864)

[<80416ccc>] (vpu_ioctl+0x594/0x864) from [<80157f80>] (do_vfs_ioctl+0x80/0x54c)

[<80157f80>] (do_vfs_ioctl+0x80/0x54c) from [<80158484>] (sys_ioctl+0x38/0x5c)

[<80158484>] (sys_ioctl+0x38/0x5c) from [<8009bf40>] (ret_fast_syscall+0x0/0x30)

---[ end trace 860c1a3c578513a8 ]---

Jim