g2d_alloc failed after a few thousands iterations on i.MX8MP

[stelljes]

Hi,

our applications throws an error after a few thousands reruns.

g2d_alloc: alloc memory fail with size 24385536!

This can be reproduced with this minimal example:

#include <iostream>
#include <cassert>
#include "g2d.h"

int main(int /*argc*/, char * /*argv*/[]) {
  int counter = 0;
  while(true){
    counter++;
    std::cout << counter << " " << std::flush;
    int is_cachable = 1;
    g2d_buf* m_dstBuffer = 0;
    void* m_g2dHandle = 0;
    ssize_t m_dstBufferSize = 24385536;

    if (g2d_open(&m_g2dHandle)){
      std::cout << std::endl << "Opening G2D failed." << std::endl;
      break;
    }     
    m_dstBuffer = g2d_alloc(m_dstBufferSize, is_cachable);
    if (!m_dstBuffer) {
      std::cout << std::endl << "Allocating G2D buffer failed." << std::endl;
      break;
    }    
    if (g2d_free(m_dstBuffer)) {
      std::cout << std::endl << "Free G2D buffer failed." << std::endl;
      break;
    }
    if (g2d_close(m_g2dHandle)) {
      std::cout << std::endl << "G2D close failed." << std::endl;
      break;
    }
  }
  return 0;
}

I'm using an i:MX8MP with yocto kirkstone and a g2d lib version 2.1.0

After ca 57000 iterations and 80 minutes this error occurs and is 100% reproducible on our system.

Do I miss some mandatory function call?

Kind regards.

[NBulaev]

Actually, this is a very old issue; we discovered it about a year and a half ago. With each call to g2d_open()/g2d_close(), 256 bytes leak. g2d_close() does not free all previously allocated memory.

 816 (256 direct, 560 indirect) bytes in 2 blocks are definitely lost in loss record 248 of 257
    at 0x48850C8: malloc (vg_replace_malloc.c:381)
    by 0x6420607: gcoOS_AllocateMemory (in /imx/usr/lib/aarch64-linux-gnu/libGAL.so)
    by 0x64206D7: gcoOS_Allocate (in /imx/usr/lib/aarch64-linux-gnu/libGAL.so)
    by 0xAD95287: vscCreatePrivateData (in /imx/usr/lib/aarch64-linux-gnu/libVSC.so)
    by 0x6228397: ??? (in /imx/usr/lib/aarch64-linux-gnu/libOpenCL.so.3.0.0)
    by 0x4B98413: g2d_init_opencl (in /imx/usr/lib/aarch64-linux-gnu/libg2d-viv.so.2.1.0)
    by 0x4B950A7: g2d_open (in /imx/usr/lib/aarch64-linux-gnu/libg2d-viv.so.2.1.0)
    by 0x48CCF43: ImgInitGraphics (imx_g2d.c:34)
    by 0x48D2C97: ImxInitGraphics (main_lib.c:1078)
    by 0x48D0657: ImxVpuCaptureSetup (main_lib.c:100)
    by 0x109ADB: main (test_lib.c:178)
 
 LEAK SUMMARY:
    definitely lost: 256 bytes in 2 blocks
    indirectly lost: 560 bytes in 1 blocks

To start, you should move g2d_open()/g2d_close() outside of the while{} loop.

But there's a question: is the G2D library open source or not? Where can the source code be found?

[stelljes]

I tried the same example on an i.MX8MMini on nanbield. The result is even worse, g2d can not allocate the memory after 1900 iterations or 30 seconds.

I just wanted to know if it is a i.MX8MP problem or it is on other boards too, so I tested the i.MX8MMini, our target device is still the i.MX8MP.

[stelljes]

I investigated further into this problem, it seems like you fixed a bug in mickeldore, which leads to a memory leak in the video memory, this is great!

As a result here we have the video usage in kirkstone:

VIDEO MEMORY:
POOL SYSTEM:
Free : 33181134 B
Used : 235254322 B
MinFree : 33181134 B
MaxUsed : 235254322 B
Total : 268435456 B
POOL VIRTUAL:
Used : 16789504 B
MaxUsed : 16789504 B

and nanbield:

VIDEO MEMORY:
POOL SYSTEM:
Free : 263872390 B
Used : 4563066 B
MinFree : 263872390 B
MaxUsed : 4563066 B
Total : 268435456 B
POOL VIRTUAL:
Used : 17354752 B
MaxUsed : 17354752 B

but as already said in a previous post, it still fails, but take much longer. I guess there is still a different leak. The file /sys/kernel/debug/gc/database growth very large, like a few hundred megabytes and is full of the following lines:

Signal           0          379fff0                0                0
Signal           0          379fa80                0                0
Signal           0          379f510                0                0
Signal           0          379efa0                0                0
Signal           0          379ea30                0                0
Signal           0          379e4c0                0                0
Signal           0          379df50                0                0
Signal           0          379d9e0                0                0
Signal           0          379d470                0                0
Signal           0          379cf00                0                0
Signal           0          379c990                0                0
Signal           0          379c420                0                0

I don't know if this causes the new problem in allocating the desired memory, but this looks like something isn't cleaned up properly.

[stelljes]

I tried the newest yocto version nanbield imx-6.6.3-1.0.0.xml. It takes much longer until this error occours, but it is still present. Now it takes 5700000 iterations and 710 minutes.