IMX RT1060 How to make a GUI application overlayed on a live video with graphics libraries included in SDK v2.7.0

[garpaci]

Hi,

I have MIMXRT1060-EVK and I am trying to build a GUI application overlayed on a video. Video is coming from live camera input as done in the SDK csi_rgb565 example. By the way I am using SDK_2_7_0_evkmimxrt1060. I know that I need to use PXP. Process surface buffer should contain camera display and alpha surface buffer should contain GUI drawings and widgets. I managed to do it with basic bitmaps but I want to use a enhanced GUI libs such as crank-storyboard, emwin or embedded-wizard as they are included in SDK. But when I work with these 3 GUI lib examples in the SDK(like brick game, coffe app etc.), I can not understand how to use PXP with their API and manipulate LCD buffers. Can you help me how to manage GUI overlayed on video application with graphics lib included in SDK? or can you tell me do these graphic libraries support overlayed GUIs on a live video? If so, how?

Thank you so much

Best regards

gorkem

[kerryzhou]

Hi Görkem Arpacı .

  Do you want to take the camera inputs to the LCD, and the LCD contains the GUI, eg, emwin?

  If yes, please check my colleagues post:

Design an IoT edge node for CV application base on the i.MXRT1050 

  This post is using the emwin to draw a GUI frame, and take the camera live image in one of the GUI block.

   Please check the document attached code, it will make you more clear.

Wish it helps you!

If you still have questions about it, please kindly let me know.

Best Regards,

Kerry

-------------------------------------------------------------------------------
Note:
- If this post answers your question, please click the "Mark Correct" button. Thank you!

- We are following threads for 7 weeks after the last post, later replies are ignored
Please open a new thread and refer to the closed one, if you have a related question at a later point in time.
-------------------------------------------------------------------------------

[garpaci]

Thank you Kerry Zhou,

This is a very good template for my application. But I have MIMXRT1060 EVK board not 1050. Could you tell me the steps how can I port this project to 1060?

Thank you so much!

Best regards,

gorkem

[kerryzhou]

Hi Görkem Arpacı,

   R1060 can compatible with the RT1050, so you can refer to the RT1050 project directly, but the RT1050-EVKB is using the hyper flash, not the qspi flash. So you need to modify the XIP code to the qspi flash, you can refer to the RT1060 SDK,  evkmimxrt1060_flexspi_nor_config.c:

const flexspi_nor_config_t qspiflash_config = {
.memConfig =
{
.tag = FLEXSPI_CFG_BLK_TAG,
.version = FLEXSPI_CFG_BLK_VERSION,
.readSampleClkSrc = kFlexSPIReadSampleClk_LoopbackFromDqsPad,
.csHoldTime = 3u,
.csSetupTime = 3u,
.sflashPadType = kSerialFlash_4Pads,
.serialClkFreq = kFlexSpiSerialClk_100MHz,
.sflashA1Size = 8u * 1024u * 1024u,
.lookupTable =
{
// Read LUTs
FLEXSPI_LUT_SEQ(CMD_SDR, FLEXSPI_1PAD, 0xEB, RADDR_SDR, FLEXSPI_4PAD, 0x18),
FLEXSPI_LUT_SEQ(DUMMY_SDR, FLEXSPI_4PAD, 0x06, READ_SDR, FLEXSPI_4PAD, 0x04),
},
},
.pageSize = 256u,
.sectorSize = 4u * 1024u,
.blockSize = 64u * 1024u,
.isUniformBlockSize = false,
};

The other code, you can use it directly.

You can try it on your side after modify the XIP, whether you can download it to your MIMXRT1060-EVK board directly .

Wish it helps you!

If you still have questions about it, please kindly let me know.

Best Regards,

Kerry

-------------------------------------------------------------------------------
Note:
- If this post answers your question, please click the "Mark Correct" button. Thank you!

- We are following threads for 7 weeks after the last post, later replies are ignored
Please open a new thread and refer to the closed one, if you have a related question at a later point in time.
-------------------------------------------------------------------------------

[garpaci]

Hello again Kerry,

sorry to interrupting you again but when I examined the app, I noticed that it is not and actual overlay example. I think Camera view uses some part of screen and emwin GUI uses other parts of screen, right? Also, when I looked at the PXP functions, AS buffer is disabled. In my opinion, for an exact overlay app with imx rt, one should use 2 layers(camera view in the PS and GUI in AS) of PXP. So I am guessing emwin lib does not support overlay, am I right?

Please inform me when your available. 

Best regards,

Gorkem

[kerryzhou · Accepted solution]

Hi Görkem Arpacı，

     Yes, the camera view uses some part of the screen.

     Do you want the emwin use the whole screen, then the camera overlay it in the top screen?

     More emwin details, you also can check with segger side, as the emwin is the Segger product, you also can check with the emwin side, whether the emwin lib can support the overlay or not.

Wish it helps you!

If you still have questions about it, please kindly let me know.

Best Regards,

Kerry

-------------------------------------------------------------------------------
Note:
- If this post answers your question, please click the "Mark Correct" button. Thank you!

- We are following threads for 7 weeks after the last post, later replies are ignored
Please open a new thread and refer to the closed one, if you have a related question at a later point in time.
------------------------------------------------------------------------------

[garpaci]

Thank you Kerry,

Actually, in the bottom layer I want my camera view and GUI on top of it with some alpha value. Yes, both camera and GUI will use whole screen. I will check segger forums.

Thanks for your help

Best regards,

gorkem

[kerryzhou]

Hi Görkem Arpacı

  Thanks for your understanding.

  If you get any valuable information from Segger side, welcome to post it here!

Best Regards,

Kerry

[garpaci]

Hi Kerry Zhou,

Thank you for your help. I did some other changes in both project settings and project folders, files and here is rt1060 suitable version of your project for those who are interested.