topic Re: Memory Reserved on i.MX6 for VPU in i.MX Processors

Memory Reserved on i.MX6 for VPU

Tarek — Tue, 07 May 2013 13:38:47 GMT

Hi All,

I need to play 4 x 1080p streams concurrently but the VPU fails to allocate memory. I'm getting the following kernel error message:

Physical memory allocation error!

The "gpumem=" command line argument explained in this post does not affect the behaviour. I tried 64M,128M and 256M without any success.

Also this patch does not solve the problem.

Is there any more suggestions?

Thanks,

Tarek

Re: Memory Reserved on i.MX6 for VPU

FranciscoCarril — Tue, 07 May 2013 20:11:01 GMT

There are several limitations when trying to analyse a problem like this.

One is VPU troughput

Other is DDR troughput.

VPU Max decoding trougput is

99,532,800 pixels/s

So for example with Dual 1080p playback @30fps:

Dual 1080p	X	Y	Rate	Pixel Rate
1080p Raw	1080p@30	1920	1080	30	62,208,000
1080p Raw	1080p@30	1920	1080	30	62,208,000
124,416,000

Exceeds supported throughput at default VPU Frequency (266MHz)

Can be handled if VPU Frequency is 352MHz but is still under test.

So for example at 24fps

Dual 1080p	X	Y	Rate	Pixel Rate
1080p Raw	1080p@30	1920	1080	24	49,766,400
1080p Raw	1080p@30	1920	1080	24	49,766,400
99,532,800

Is right on the limit.

DDR use

Theoretical Peak DDR bandwith of a DDR3 32Bits is 3200 MB/s

Estimated bus utilization is about 50% so 1600 MB/s available

Decoding one 1080p video at 30fps uses:

DDR load (VPU Decode)
H.264 1080p 30fps		383.0	MB/s
DDR load (VDOA )
1080p YUV420 tiled -> YUV422 raster				210.0	MB/s
Display refresh
display DDR load
BW = FW x FH x fps x data format (2 bytes/pixel)
For 1080p60, YUV422 output this would be
FW	FH	fps	bytes/pix
1920	1080	60	2	248.8	MB/s	background - combining done in the DP
1920	1080	60	2	248.8	MB/s	overlay - combining done in the DP

With a total of 1090.7 MB/s of DDR, so even dual playback at 30fps is not possible.

The max 1080p playback possible is dual playback, and the videos need to be at <=24fps.

Re: Memory Reserved on i.MX6 for VPU

Tarek — Tue, 07 May 2013 20:55:10 GMT

Hi Francisco,

Thank you very much for the detailed explanation.

I understand that the system is only capable of handling up to 2 full rate HD streams. However I'm working on surveillance application so the frame rate can be reduced even to 6 fps and this is what my camera is sending.

The error message that I'm getting doesn't imply that it's a throughput problem. The VPU doesn't even start decoding. The gstreamer pipeline fails at start up due to memory allocation problem.

I believe the solution to this problem is to preserve enough contiguous memory for the VPU at boot time. The question is how to achieve this and how much memory is needed?

Thanks,

Tarek

Re: Memory Reserved on i.MX6 for VPU

Tarek — Thu, 09 May 2013 17:07:46 GMT

The patch provided solves the VPU allocation problem but I think there is still another element that needs a similar fix. Perhaps it's the IPU!!?

The VPU itself is allocating 10MB for each pipeline but the total memory allocated by a single pipeline is about 90MB?

Where is this memory going? is it the IPU?

Re: Memory Reserved on i.MX6 for VPU

FranciscoCarril — Thu, 09 May 2013 19:05:57 GMT

Could you post your pipeline? Maybe LeonardoSandovalGonzalez Can take a look.

Re: Memory Reserved on i.MX6 for VPU

Tarek — Thu, 09 May 2013 19:45:00 GMT

The problem can be reproduced if you run 3 pipelines using

gst-launch playbin2 uri=file:///1080p-file video-sink=mfw_isink.

My exact pipeline is:

appsrc ! typefinder ! vpudec ! mfw_isink

But I don't think this problem is specific to my pipeline

Re: Memory Reserved on i.MX6 for VPU

Tarek — Thu, 23 May 2013 13:51:38 GMT

You can reduce the amount of memory the VPU is allocating to minimum by setting the frame-plus property.

For example:

gst-launch filesrc location=wheel.mp4 typefind=true ! aiurdemux ! vpudec frame-plus=1 ! mfw_isink axis-left=0 axis-top=0 disp-width=960 disp-height=540 & gst-launch filesrc location=wheel.mp4 typefind=true ! aiurdemux ! vpudec frame-plus=1 ! mfw_isink axis-left=960 axis-top=0 disp-width=960 disp-height=540 & gst-launch filesrc location=wheel.mp4 typefind=true ! aiurdemux ! vpudec frame-plus=1 ! mfw_isink axis-left=0 axis-top=540 disp-width=960 disp-height=540 & gst-launch filesrc location=wheel.mp4 typefind=true ! aiurdemux ! vpudec frame-plus=1 ! mfw_isink axis-left=960 axis-top=540 disp-width=960 disp-height=540

This command can play 4 x 1080p streams

Re: Memory Reserved on i.MX6 for VPU

senykthomas — Tue, 23 Jul 2013 14:39:28 GMT

I'm using wandsolo.

I do get this error when I combine 1080p video with an active opengl context.

e.g.:

gst-launch filesrc location=sintel_trailer-1080p.mp4 typefind=true ! aiurdemux ! vpudec ! glimagesink

=> " Physical memory allocation error! "

(note that it would work with mfw_v4lsink, but I'm after the combined-with-opengl-use-case)

It's working with:

gst-launch filesrc location=sintel_trailer-1080p.mp4 typefind=true ! aiurdemux ! vpudec frame-plus=1 ! glimagesink

fyi: I still get:

[WARN] VPU iram is less than needed, some parts don't use iram

... but it's rendering properly anyway!

The problem is that I want to use this within Qt. (glimage sink is just a example so simple test that frame-plus=1 is working)

Qt doesn't build the whole pipeline manually, rather lets gstreamer define most of it and just adds the 'end pieces'.

Therefor I think it's practically impossible to add frame-plus=1 to vpudec ... unless one restructures QtMultimedia significantly.

Is there a way to set that as environment variable or something like that?

So something like:

export GST_VPU_FRAME_PLUS=1; gst-launch ...

would be awesome! :smileyhappy:

Alternate question: can I somehow adjust the size of the memory in question? (gpu mem? vpu mem? ipu mem?)

I've seen the Android mem-alloc page, gut gpumem=... seems to not work on linux.

I've tried fbmem=100M, but it's not working either.

I also tried added the patch (an adjusted version) from here: Re: GStreamer crashing on i.MX6 (Boundary Devices Nitrogen6x)

.. to board-wand.c

Nothing changes.

Re: Memory Reserved on i.MX6 for VPU

Tarek — Tue, 23 Jul 2013 15:12:51 GMT

Hi Thomas,

1. For your first suggestion you can modify gst-fsl-plugins to set the default value to 1. I think if you changed the number "6" to "1" in vpudec.c line 182 that will do.

2. For your alternative solution you can modify the kernel memory map to give more space for DMA zone and reduce the VMALLOC zone. The VPU is allocating memory from DMA and at some point there is it enough for it.

If you look at kernel messages at boot time you will see the memory map. Something like this:

Memory: 640MB 256MB = 896MB total

Memory: 896808k/896808k available, 151768k reserved, 0K highmem

Virtual kernel memory layout:

vector : 0xffff0000 - 0xffff1000 ( 4 kB)

fixmap : 0xfff00000 - 0xfffe0000 ( 896 kB)

DMA : 0xe0a00000 - 0xffe00000 ( 500 MB)

vmalloc : 0xc0800000 - 0xde400000 ( 476 MB)

lowmem : 0x80000000 - 0xc0000000 (1024 MB)

pkmap : 0x7fe00000 - 0x80000000 ( 2 MB)

modules : 0x7f000000 - 0x7fe00000 ( 14 MB)

.init : 0x80008000 - 0x8003c000 ( 208 kB)

.text : 0x8003c000 - 0x80a95b64 (10599 kB)

.data : 0x80a96000 - 0x80af9ca0 ( 400 kB)

.bss : 0x80af9cc4 - 0x80b47eac ( 313 kB)

Hope that helps

Re: Memory Reserved on i.MX6 for VPU

FranciscoCarril — Tue, 23 Jul 2013 15:52:52 GMT

This is the patch created by LeonardoSandovalGonzalez for the change suggested by Thomas.

Signed-off-by: Leonardo Sandoval <leo.san.gon@gmail.com>

---

src/video/vpu/src/vpudec.c | 2 +-

1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/video/vpu/src/vpudec.c b/src/video/vpu/src/vpudec.c index 6066ef8..2f0af18 100644

--- a/src/video/vpu/src/vpudec.c

+++ b/src/video/vpu/src/vpudec.c

@@ -179,7 +179,7 @@ static GstsutilsOptionEntry g_vpudec_option_table[] = {

G_STRUCT_OFFSET (VpuDecOption, adaptive_drop), "true"},

{PROP_FRAMES_PLUS, "frame-plus", "addtionlal frames",

"set number of addtional frames for smoothly playback", G_TYPE_INT,

- G_STRUCT_OFFSET (VpuDecOption, bufferplus), "6", "1", STR_MAX_INT},

+ G_STRUCT_OFFSET (VpuDecOption, bufferplus), "1", "1",

+ STR_MAX_INT},

{PROP_OUTPUT_FORMAT, "output-format", "output format",

"set raw format for output",

G_TYPE_ENUM,

1.7.9.5

Re: Memory Reserved on i.MX6 for VPU

senykthomas — Wed, 24 Jul 2013 13:13:33 GMT

Thanks for the pointer!

That's working ... apparently still not enough memory saved for QtMultimedia .. :smileywink:

Re: Memory Reserved on i.MX6 for VPU

senykthomas — Wed, 24 Jul 2013 14:13:43 GMT

hmm I tried to adjust the memory ... this is roughly what I've tried:

http://pastebin.com/SMRHpVu4

.. this version doesn't boot .. without the modifications on the DMA size, it's booting.

the kernel error:

[	0.423988] ------------[ cut here ]------------
[	0.428696] WARNING: at arch/arm/mm/dma-mapping.c:174 consistent_init+0x60/0xd0()
[	0.436201] Modules linked in:
[	0.439353] [<c0043f3c>] (unwind_backtrace+0x0/0xf4) from [<c006ec60>] (warn_slowpath_common+0x54/0x64)
[	0.448787] [<c006ec60>] (warn_slowpath_common+0x54/0x64) from [<c006ed0c>] (warn_slowpath_null+0x1c/0x24)
[	0.458506] [<c006ed0c>] (warn_slowpath_null+0x1c/0x24) from [<c000bab0>] (consistent_init+0x60/0xd0)
[	0.467772] [<c000bab0>] (consistent_init+0x60/0xd0) from [<c00375c0>] (do_one_initcall+0x11c/0x174)
[	0.476963] [<c00375c0>] (do_one_initcall+0x11c/0x174) from [<c00089ec>] (kernel_init+0xc0/0x144)
[	0.485870] [<c00089ec>] (kernel_init+0xc0/0x144) from [<c003da7c>] (kernel_thread_exit+0x0/0x8)
[	0.494712] ---[ end trace 1b75b31a2719ed1c ]---
[	0.499427] ------------[ cut here ]------------
[	0.504068] WARNING: at arch/arm/mm/dma-mapping.c:174 consistent_init+0x60/0xd0()
[	0.511589] Modules linked in:
[	0.514694] [<c0043f3c>] (unwind_backtrace+0x0/0xf4) from [<c006ec60>] (warn_slowpath_common+0x54/0x64)
[	0.524141] [<c006ec60>] (warn_slowpath_common+0x54/0x64) from [<c006ed0c>] (warn_slowpath_null+0x1c/0x24)
[	0.533834] [<c006ed0c>] (warn_slowpath_null+0x1c/0x24) from [<c000bab0>] (consistent_init+0x60/0xd0)
[	0.543104] [<c000bab0>] (consistent_init+0x60/0xd0) from [<c00375c0>] (do_one_initcall+0x11c/0x174)
[	0.552272] [<c00375c0>] (do_one_initcall+0x11c/0x174) from [<c00089ec>] (kernel_init+0xc0/0x144)
[	0.561189] [<c00089ec>] (kernel_init+0xc0/0x144) from [<c003da7c>] (kernel_thread_exit+0x0/0x8)
[	0.570000] ---[ end trace 1b75b31a2719ed1d ]---

Are you actually sure it's the kernel-DMA size?... I've checked on my other boards (nitrogen and sabreauto) none of them has a bigger DMA size, but the problem doesn't occur on them ...?

nitrogen:

[	0.000000]	vector : 0xffff0000 - 0xffff1000 ( 4 kB)
[	0.000000]	fixmap : 0xfff00000 - 0xfffe0000 ( 896 kB)
[	0.000000]	DMA : 0xf4600000 - 0xffe00000 ( 184 MB)
[	0.000000]	vmalloc : 0xc0800000 - 0xf2000000 ( 792 MB)
[	0.000000]	lowmem : 0x80000000 - 0xc0000000 (1024 MB)
[	0.000000]	pkmap : 0x7fe00000 - 0x80000000 ( 2 MB)
[	0.000000]	modules : 0x7f000000 - 0x7fe00000 ( 14 MB)
[	0.000000]	.init : 0x80008000 - 0x80035000 ( 180 kB)
[	0.000000]	.text : 0x80035000 - 0x8067e550 (6438 kB)
[	0.000000]	.data : 0x80680000 - 0x806d7160 ( 349 kB)
[	0.000000]	.bss : 0x806d7184 - 0x80721818 ( 298 kB)

AI:

	vector : 0xffff0000 - 0xffff1000 ( 4 kB)
	fixmap : 0xfff00000 - 0xfffe0000 ( 896 kB)
	DMA : 0xf4600000 - 0xffe00000 ( 184 MB)
	vmalloc : 0xea800000 - 0xf2000000 ( 120 MB)
	lowmem : 0x80000000 - 0xea000000 (1696 MB)
	pkmap : 0x7fe00000 - 0x80000000 ( 2 MB)
	modules : 0x7f000000 - 0x7fe00000 ( 14 MB)
	.init : 0x80008000 - 0x8003e000 ( 216 kB)
	.text : 0x8003e000 - 0x80b0d54c (11070 kB)
	.data : 0x80b0e000 - 0x80b7a070 ( 433 kB)
	.bss : 0x80b7a094 - 0x80bee4d0 ( 466 kB)

Re: Memory Reserved on i.MX6 for VPU

senykthomas — Wed, 24 Jul 2013 14:37:37 GMT

actually .. it was saving enough memory .. but the pipeline seems to break down (for the QtMultimedia) case .. the lower the value the 'easier'/faster it breaks down.

6 seems to be a good value after all :smileywink:

Re: Memory Reserved on i.MX6 for VPU

ChucoChe — Mon, 29 Jul 2013 23:07:37 GMT

Hi,

You can try to increase the GPU memory size in the boardfile. Currently it's 128M.

static struct viv_gpu_platform_data imx6q_gpu_pdata __initdata = {

.reserved_mem_size = SZ_128M,

};

For the dma zone size:

The change you showed in Kconfig, plus setting the proper size in the kernel config menu should do the trick.

diff --git a/arch/arm/plat-mxc/Kconfig b/arch/arm/plat-mxc/Kconfig
index 32408ed..39e9cde 100755
--- a/arch/arm/plat-mxc/Kconfig
+++ b/arch/arm/plat-mxc/Kconfig
@@ -164,7 +164,7 @@ config CLK_DEBUG
config DMA_ZONE_SIZE
int "DMA memory zone size"
- range 0 184
+ range 0 220
default 24
help
This is the size in MB for the DMA zone. The DMA zone is used for

Michel

Re: Re: Memory Reserved on i.MX6 for VPU

Tarek — Tue, 30 Jul 2013 07:43:22 GMT

Hi Michel,

Unfortunately the menu config doesn't work. The values are hard coded in the drivers and any menu config value has no effect.

You need to modify the Linux source code.

Thanks

Re: Memory Reserved on i.MX6 for VPU

ChucoChe — Wed, 07 Aug 2013 15:30:28 GMT

Did changing any of the memory settings helped?

Michel

Re: Memory Reserved on i.MX6 for VPU

Tarek — Wed, 07 Aug 2013 20:39:36 GMT

In short, with frame-plus=1 you can play 4x1080p streams without problems. With changing the memory map you can play 8x1080p but the system is unstable.

Re: Memory Reserved on i.MX6 for VPU

senykthomas — Tue, 13 Aug 2013 14:35:17 GMT

Sorry for the late response. I was out of office for a week.

This patch looks good, unfortunately it's not for the kernel and board I was looking for.

I was looking for linux-wandboard (git://github.com/johnweber/linux.git) and board is wandboard-solo

Also 500mb-DMA-size is not applicable (the wandboard-solo only has 512mb ram) .. it's a 'enough for 1x1080p when someone else is consuming GPU as well' (within a Qt/OpenGL application)

Re: Memory Reserved on i.MX6 for VPU

ChucoChe — Fri, 16 Aug 2013 17:22:56 GMT

Hi Thomas,

You could ping timesys on the issue. They may provide the proper patch.

Michel

Re: Memory Reserved on i.MX6 for VPU

Jessie_Lee — Fri, 03 Jan 2014 03:16:21 GMT

Dear Michael.

I have met vpu memory allocation failure too.:smileycry:

Test environment

- LTIB, version : 3.0.35-4.1.0

- mxc_vpu_test.out (jpeg decoding test)

I applied "dma memory size" patch (increase from 184M to 500M).

But I have memory issue still.

As I heard, the vpu memory size is limited to 32M now .

So, I'm trying to test about increasing vpu memory size from 32M to 35M.

Could I increase? If yes, How could I increase this vpu memory size to 35M?

Thank you.

BRs,

Jessie.