How to get started with coding an imx6 encoder?

[moire59]

I've never done anything like this before. It took me a while to setup a working cross compiler, I now boot my imx6 from NFS and save my files there. And that's great. But there is a lot I still don't understand.

Where should I start from? Is there some working reference encoder out there? Because I've found some: https://github.com/ahmedammar/imx-testing-apps-misc/ and https://github.com/Freescale/libimxvpuapi. I've tried with the first, but I have no idea of where to find some of the included headers (like drm/imx-ipu-v3-ioctls.h). Even by removing the .c/.h files related to it I still can't compile the encoder after having placed vpu_x.y in my includes directory. I've yet to try with libimxvpuapi.

Is the correct way of compiling those projects to simply add the vpu_x files in my eclipse include directory? They are gonna exist in the host once the executable is ran there, right?

Anyway, if someone had some tutorial or documentation I missed, please let me know. Sorry for my dumbness.

[chuck1 · Accepted solution]

Giovanni,

I come from the TI DaVinci digital media processor SoC world, i.e., the DM 36x and DM81xx platforms, so I too am new to the i.MX6 SoC as well as the Freescale application development infrastructure.  Clearly, the two companies have very different developer support models: TI delivers a complete SDK with each of its platforms while Freescale seems to expect the developer to fend for his/herself and/or contract with third party companies for product development assistance, at least that is my impression.

In addition to the two documents recommended by Guanqiong suggested, I recommend that—if you intend to develop A/V applications that interface directly to the i.MX6 hardware—you will want to take a look at the VPU API documentation (which can be found on the Freescale Website).  Next, you can find the VPU libraries—for Yocto Project Open Embedded i.MX6 BSP builds—at .../oe-core/out-eglibc/work/<your_bsp>/imx-vpu/..., e.g., vpu_lib.c/h, vpu_io.c/h, etc.—this library can be used to build your application with Eclipse then copy it to your target for execution.  Next, with its baseline BSP for Linux/Android, Freescale provides applications that test nearly every aspect of the i.MX6 SoC hardware—these test applications can be found at the Freescale Git repository as well as other locations. You will find applications that encode, decode, transcode, for example, MPEG-4, H.264, and MPEG-2 video as well as audio using various audio codecs (e.g., AAC).  Finally, I suggest that you continue to post your questions/concerns to this community forum—hopefully, you will get the attention of experienced community members who are willing to share some helpful tips and tricks.

Regards,

-Chuck

P.S. Although I'm struggling with some of the same "getting started" concerns that you are, I've posted some H.262 encode application code here i.MX6 VPU H.264 Encoder Quality Issues that may help you get started.

[JohnCannizzo]

Where is this sample code? I've followed the link and see no attached files or links to the source content. If you're willing, I'd love to take a look at a working example...

Thanks,

John

[chuck1]

John,

   For H.264 video encoder code using the i.MX6 VPU—please take a look at Freescale's test code which should have been included with your BSP.  If you need additional information, feel free to post back.

Good luck!

-Chuck

[JohnCannizzo]

Hi Chuck,

Thanks for the reply.  I'll download the latest 3.10 Yocto.  Is there a recipe you would recommend?

I'm waiting on parts to enable MIPI on the Sabre board, so I won't be able to try it for a few days.

Thanks,

John

[chuck1]

John,

If the i.MX test applications are already installed with your BSP, you should search for the "imx-test" directory then find the "mxc_vpu_test" directory wherein you will find "enc.c"—this test application illustrates how to implement many of the VPU encoding modes and capabilities.

If not installed, use BitBake to run the recipe called "imx-test" (or something similar) then search as above.

Have fun...

-Chuck

[JohnCannizzo]

Thanks Chuck, I'll get the latest imx-test Yocto build...

John

[moire59]

Well – I'm doing the opposite. I suspended my imx6 project and tried with a dm368 chinese dev board, and I was up and running in a few hours. I just wanted to say thank you for your link, I hope to be able to contribute to that soon.

[chuck1]

Giovanni,

That's interesting—I'm very familiar with the TMS320DM368.  It is able to encode/decode 1080i at 30fps with reasonable margin for the CPU to encode/decode audio, for example, AAC-LC, at reasonable bit rates.  I should warn you that, using the current DVSDK, the PCM driver has a bug that makes audio encoding somewhat erratic—the audio gets out of sync with video—but there are several patches that fix this issue.

Good luck,

-Chuck

[moire59]

So, for example, where do I find the library for vpu_Init? There is one on github: imx-lib/vpu_lib.c at master · genesi/imx-lib · GitHub and it says that it's from the BSP, that I think to have downloaded, but without finding the libraries. Are they in the .sdcard files? (that I can't burn to sdcards and I don't know how to read)?.

All of this is really confusing to me.

[joanxie]

what bsp did you use? yocto or ltib?

if yocto, pls refer to the  Chapter 19 Video Processing Unit (VPU) Driver of  i.MX_6_Linux_Reference_Manual.pdf and chapter 7 Multimedia of i.MX_Linux_User's_Guide.pdf

I attached the two files.

[chuck1]

Guanqiong,

It appears that Freescale has done a good job with GStreamer—still there are many developers who want to develop "bare metal" multimedia applications for the i.MX6.  For those of us in this category who are also new to the i.MX6, the chapters you cite in both the i.MX6 Linux Reference Manual and i.MX6 Linux Users Guide, while helpful, really do not provide the proper level of instruction that a few, straightforward sample A/V encode, decode, and transcode applications would do.

And while Freescale has included the VPU test application (located in the "mxc_vpu_test" folder) with its BSP, this application is not at all straightforward since it has to cover so many use cases in a single application plus it is clear that the application is not written with the new developer in mind.

So to my question: is it possible for Freescale to provide the developer community a few, straightforward applications that illustrate how to create A/V applications using the VPU directly for video encoding, decoding, and transcoding from/to various inputs/outputs (without involving GStreamer)?  As I see it, these sample applications would illustrate how:

  1. to set up and manage the input/output video frame buffers for both multiple buffer and ring buffer instances;

  2. to set up, initialize, and then manage the VPU codecs for encoding, decoding, and transcoding of H.264, MPEG-4,

      MPEG-2, etc. video from a file, a stream, or other interface then output the result to the display or other interface;

  3. to set up then manage multiplexing/demultiplexing/synchronization for various video and audio types, e.g., H.264

      video and AAC audio; and

  4. specific VPU codec initialize parameters effect encoding/decoding/transcoding performance (currently, this essential

      information is missing—if it's available—I have not been able to find it).

If Freescale cannot make the described sample A/V described available to the i.MX6 developer community, perhaps it can enlist the community to do so.

Regards,

-Chuck