gst1.0-fsl-plugin 4.0.2 VPU memory leak?

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

gst1.0-fsl-plugin 4.0.2 VPU memory leak?

Jump to solution
2,436 Views
andrewmurray
Contributor I

I believe I have observed a memory leak with the VPU object (as interfaced from GStreamer through vpu_wrapper.h). I've attached source code for the simplest pipeline that I could create which reproduces this issue.

 

The pipeline consists of an instance of vpuenc, and a playbin which transitions state at a regular interval. After no more than 20 transitions the following output is observed from the vpudec decoder:

 

[ERR]   mem allocation failed!

 

At the same time the kernel outputs:

 

mxc_vpu 2040000.vpu: Physical memory allocation error!

mxc_vpu 2040000.vpu: Physical memory allocation error!

 

Adding debug to the mxc_vpu.c driver and the gstvpuallocator.c file in gst1.0-fsl-plugins results in the observation that calls to VPU_DecFreeMem do not always translate to calls to vpu_free_dma_buffer in the kernel driver - leading to the suspicion that there is an issue with the binary VPU object. Furthermore when exiting the attached test application, due to VPU object's file handle being release (vpu_release) all allocated memory is freed. Thus restarting the application allows it work for another 20 or so iterations.

 

Interestingly this only occurs if vpuenc is included in the GStreamer bin. I.e. removing the line 'gst_bin_add_many' from the test application makes this issue go away.

 

This is a sample of the instrumented kernel driver whilst running the test application - notice the absence of free's:

 

[ 5198.386704] mxc_vpu: Attempt to alloc 5238784

[ 5198.416878] mxc_vpu: Attempt to alloc 606208

[ 5198.467019] mxc_vpu: Attempt to alloc 106496

[ 5198.480263] mxc_vpu: Attempt to alloc 106496

[ 5198.494836] mxc_vpu: Attempt to alloc 106496

[ 5198.509385] mxc_vpu: Attempt to alloc 106496

[ 5198.521062] mxc_vpu: Attempt to alloc 106496

[ 5198.532712] mxc_vpu: Attempt to alloc 106496

[ 5198.544384] mxc_vpu: Attempt to alloc 106496

[ 5198.555995] mxc_vpu: Attempt to alloc 106496

[ 5198.567652] mxc_vpu: Attempt to alloc 106496

[ 5206.368775] mxc_vpu: Attempt to alloc 5238784

[ 5206.399722] mxc_vpu: Attempt to alloc 606208

[ 5206.449683] mxc_vpu: Attempt to alloc 106496

[ 5206.463097] mxc_vpu: Attempt to alloc 106496

[ 5206.474578] mxc_vpu: Attempt to alloc 106496

[ 5206.486237] mxc_vpu: Attempt to alloc 106496

[ 5206.497889] mxc_vpu: Attempt to alloc 106496

[ 5206.509543] mxc_vpu: Attempt to alloc 106496

[ 5206.521244] mxc_vpu: Attempt to alloc 106496

[ 5206.532883] mxc_vpu: Attempt to alloc 106496

[ 5206.544542] mxc_vpu: Attempt to alloc 106496

[ 5214.378583] mxc_vpu: Attempt to alloc 5238784

[ 5214.408730] mxc_vpu: Attempt to alloc 606208

[ 5214.461846] mxc_vpu: Attempt to alloc 106496

[ 5214.477157] mxc_vpu: Attempt to alloc 106496

[ 5214.489876] mxc_vpu: Attempt to alloc 106496

[ 5214.504329] mxc_vpu: Attempt to alloc 106496

[ 5214.515929] mxc_vpu: Attempt to alloc 106496

[ 5214.527587] mxc_vpu: Attempt to alloc 106496

[ 5214.539245] mxc_vpu: Attempt to alloc 106496

[ 5214.550935] mxc_vpu: Attempt to alloc 106496

[ 5214.562609] mxc_vpu: Attempt to alloc 106496

[ 5222.378546] mxc_vpu: Attempt to alloc 5238784

[ 5222.387345] mxc_vpu 2040000.vpu: mxc_vpu: Physical memory allocation error!

[ 5222.395904] mxc_vpu 2040000.vpu: Physical memory allocation error!

 

Upon terminating the test application process, the previously unfree'd buffers are freed (this is a sample - there is much more output):

 

[ 6171.345389] mxc_vpu: Attempt to free 5238784

[ 6171.349142] mxc_vpu: Attempt to free 106496

[ 6171.352093] mxc_vpu: Attempt to free 106496

[ 6171.355018] mxc_vpu: Attempt to free 106496

[ 6171.357942] mxc_vpu: Attempt to free 106496

[ 6171.360884] mxc_vpu: Attempt to free 106496

[ 6171.363822] mxc_vpu: Attempt to free 106496

[ 6171.366748] mxc_vpu: Attempt to free 106496

[ 6171.369694] mxc_vpu: Attempt to free 106496

[ 6171.372641] mxc_vpu: Attempt to free 106496

[ 6171.375563] mxc_vpu: Attempt to free 5238784

[ 6171.379322] mxc_vpu: Attempt to free 106496

[ 6171.382274] mxc_vpu: Attempt to free 106496

[ 6171.385197] mxc_vpu: Attempt to free 106496

[ 6171.388120] mxc_vpu: Attempt to free 106496

[ 6171.391064] mxc_vpu: Attempt to free 106496

[ 6171.393993] mxc_vpu: Attempt to free 106496

[ 6171.396915] mxc_vpu: Attempt to free 106496

[ 6171.399835] mxc_vpu: Attempt to free 106496

[ 6171.402782] mxc_vpu: Attempt to free 106496

[ 6171.405722] mxc_vpu: Attempt to free 5238784

 

 

Notice how the pattern of alloc/free change when removing vpuenc from the bin:

 

[ 6381.859723] mxc_v4l2_output v4l2_out.33: Bypass IC.

[ 6381.863845] mxc_v4l2_output v4l2_out.33: Bypass IC.

[ 6381.900043] mxc_vpu: Attempt to alloc 106496

[ 6381.904814] mxc_vpu: Attempt to alloc 106496

[ 6381.910441] mxc_vpu: Attempt to alloc 106496

[ 6381.916423] mxc_vpu: Attempt to alloc 106496

[ 6381.922205] mxc_vpu: Attempt to alloc 106496

[ 6381.927937] mxc_vpu: Attempt to alloc 106496

[ 6381.933804] mxc_vpu: Attempt to alloc 106496

[ 6381.939588] mxc_vpu: Attempt to alloc 106496

[ 6381.945433] mxc_vpu: Attempt to alloc 106496

[ 6384.868637] mxc_vpu: Attempt to free 606208

[ 6384.878662] mxc_vpu: Attempt to free 106496

[ 6384.883118] mxc_vpu: Attempt to free 106496

[ 6384.888839] mxc_vpu: Attempt to free 106496

[ 6384.894651] mxc_vpu: Attempt to free 106496

[ 6384.900389] mxc_vpu: Attempt to free 106496

[ 6384.906170] mxc_vpu: Attempt to free 106496

[ 6384.911962] mxc_vpu: Attempt to free 106496

[ 6384.917698] mxc_vpu: Attempt to free 106496

[ 6384.923579] mxc_vpu: Attempt to free 106496

[ 6384.929301] mxc_vpu: Attempt to free 5238784

[ 6384.935113] mxc_vpu: Attempt to free 0

[ 6389.853914] mxc_vpu: Attempt to alloc 5238784

[ 6389.858814] mxc_vpu: Attempt to alloc 606208

[ 6389.875994] mxc_v4l2_output v4l2_out.33: Bypass IC.

[ 6389.881220] mxc_v4l2_output v4l2_out.33: Bypass IC.

[ 6389.916418] mxc_vpu: Attempt to alloc 106496

[ 6389.919658] mxc_vpu: Attempt to alloc 106496

[ 6389.930917] mxc_vpu: Attempt to alloc 106496

 

I've overcome this implementing a memory pool in GStreamer's gstvpuallocator.c file - though it's not an ideal fix.

 

Is this a known issue? Does it still exist in your current development? Are there any workarounds?

 

Thanks,

 

Andrew Murray

Original Attachment has been moved to: vpu-leak.cpp.zip

Labels (4)
0 Kudos
1 Solution
1,336 Views
KevinSong
NXP Employee
NXP Employee

Thanks. We already fixed this issue. Please try latest Yocto release.

View solution in original post

0 Kudos
6 Replies
1,336 Views
andrewmurray
Contributor I

I've since found a fix for this.

The gst_vpu_free_phys_mem and gst_vpu_alloc_phys_mem functions of gst1.0-fsl-plugins-4.0.2/plugins/vpu/gstvpuallocator.c need to ensure that the VpuMemDesc passed to VPU_DecFreeMem and VPU_DecGetMem has a sensible value for nCpuAddr. In this version of the plugin this field is not passed.

I've fixed this (please see attached patch) by adding a new member to PhyMemBlock that will store the value of nCpuAddr as populated by VPU_DecGetMem, such that when calling VPU_DecFreeMem it can be used to repopulate nCpuAddr.

I've not investigated why it worked sometimes without this patch. Any thoughts?

0 Kudos
1,337 Views
KevinSong
NXP Employee
NXP Employee

Thanks. We already fixed this issue. Please try latest Yocto release.

0 Kudos
1,336 Views
liqun_wei
Contributor III

Hi, which version of Yocto do you fix this problem?

How about Yocto-4.14.78 ?

0 Kudos
1,336 Views
jiaxingliang
Contributor I

Hi, Song:

     which version you used?  

Command:

gst-launch-1.0 playbin uri=rtsp://192.168.1.88:556/vga

 

The memory of gst-launch-1.0 will increase more than 30M every hour, then the video stop, but audio soud ok。

 

 Delete the libgstvpu.so , use the same commad. memor will not increase,  but play the high resolution (1920x1080) not smooth.

 

Anybody  has faced the same problem and resolved? Please help

 

CPU: IMX6DL

Kernel: 4.1.15

BSP: 2.0.1 GA

GST: imx-gst1.0-plugin-4.1.4

0 Kudos
1,336 Views
rajeshsatya
Contributor III


Please provide the extact details of the patch which has fixed the VPU memory leak, we are facing the same issues when playing the video, would like to apply the patch and verify if it fixes our problem.

0 Kudos
1,336 Views
rajeshsatya
Contributor III

Can you please provide me the excat patch details where the VPU memory leak issue is fixed. We are facing similar kind of

issue, when playing the Video using gstreamer.

0 Kudos