VPU hangs in iMX6 Platform SDK

[endl]

Hi,

I'm using iMX6DL board with "Platform SDK v1.1.0" for encoding raw h264 stream.

When encoding a single frame, the BIT_BUSY_FLAG is set to 1 and not reset, VPU hangs. What could be the reason for this issue? Thanks in advance.

[endl]

Has anyone an idea ?

[chuck1]

Vadim,

I assume that you tried setting GOP = 0 (I believe that that is 'single picture mode')?

If you are trying to capture a single video frame, you would probably be better off using the services of V4L2 to directly capture YUV, YUYV, or MJPEG video frames.  Then you could use the VPU to encode/transcode a slice (or frame) at a time.

-Chuck

[endl]

Hello

I set GOP = 3, IPPIPP... ans so on. GOP = 0 means that only first picture is I, i.e. IPPPPPP... and so on.

I'm not using linux and V4L2. At present I don't need to capture the image. I'm trying to encode a video in any format (H264, MPEG4 or MJPEG), but VPU unit hangs and I don't understand why.

How are you using VPU ? What cross-compiler are you use ? and how are you build your application and execute it ?

[chuck1]

Vadim,

I think I remember reading somewhere that the VPU has a "single picture" mode but perhaps not.

Currently, I'm using the VPU to encode 1920x1080 or 1280x720 30fps YUV files into H.264 files but my future work will involve real-time capture of local audio/video, encoding to H.264 / AAC, encapsulation of both into a MPEG-2 TS, and, finally, the application will stream them via RTP/UDP/IP across a broadband network.

As for my development environment: I'm using Eclipse with the Linaro GNU tools for Linux along with the VPU libraries all under Yocto 1.6 (Open Embedded Angstrom) as well as the BSP from my board manufacturer (Toradex).  In the 'imx-test' folder there is a VPU test application named 'imx_vpu_test' which has an 'enc.c' part—I used this application component as a pattern for my video encoder application.  As a result, I'm able to encode 1920x1080 30fps YUV video at 45+ fps with full control over the quality, I/P frame pattern, etc.  Also, CPU utilization peeks at around 15 - 20% during an encoding session.

I hope that this is helpful,

-Chuck

[Yuri]

  I tested the vpu encode application from the Platform SDK – it works.
Nevertheless, in case if the VPU is always busy, you can issue
vpu_SWReset() of VPU API to recover the VPU HW.

Have a great day,
Yuri

-----------------------------------------------------------------------------------------------------------------------
Note: If this post answers your question, please click the Correct Answer button. Thank you!
-----------------------------------------------------------------------------------------------------------------------

[endl]

Hello

VPU_SWReset() doesn't help. After VPU_EncStartOneFrame() VPU is always busy.

I wrote a very simple code for testing (see below). But I have got the same result.

When I run vpu_test(), encoder hangs (when BIT processor runs PIC_RUN command in VPU_EncStartOneFrame ).

Bitstream buffer is empty. BIT Current PC in a loop 0x0CB2 - 0x0CB4.

Execution of VPU Init(), VPU_GetVersionInfo(), VPU_EncOpen(), VPU_EncRegisterFrameBuffer() is successful.

Execution of ENC_PUT_AVC_HEADER command is successful too.

What i'm doing wrong ?

If needed I can provide more information.

Thanks in advance.

#define VPU_WORK_BUFFERS        (0x24100000) 
#define VIDEO_BUFFERS_START     (0x28000000) 
#define VIDEO_BUFFERS_END       (0x2FFFFFFF) 

#define BIT_WORK_SIZE            (300*1024) 
#define CODE_BUF_SIZE           (240 * 1024)  
#define TEMP_BUF_SIZE           (300 * 1024)     
#define PARA_BUF2_SIZE          (2 * 1024) 
#define PARA_BUF_SIZE           (10 * 1024) 


#define BUFF_FILL_SIZE          (512 * 1024) 
#define DEFAULT_WIDTH            176 
#define DEFAULT_HEIGHT            144 
#define DEFAULT_FRAME_RATE      5 
#define DEFAULT_GOP_SIZE        3 
#define VPU_DEFAULT_H264_QP     10 



void frame_fill (void *addr) 
{ 
    uint8_t *data = addr; 


    for (int i = 0; i < DEFAULT_WIDTH * DEFAULT_HEIGHT * 3/2; i++) 
        data[i] = i%256; 
} 



int vpu_test (void) 
{ 
    RetCode ret = 0; 
    int i; 
    vpu_versioninfo ver;            // vpu version information 
    vpu_mem_desc bit_stream_buf;    // input bit stream allocated memory 
    vpu_mem_desc source_buf;        // source buffer allocated memory 
    FrameBuffer source_frame;       // source frame buffer 


    // VPU specific members defined in vpu_lib.h 
    EncHandle *handle = malloc(sizeof(EncHandle)); 
    EncOpenParam *encOP = malloc(sizeof(EncOpenParam)); 
    EncInitialInfo *initialInfo = malloc(sizeof(EncInitialInfo)); 
    EncOutputInfo *outputInfo = malloc(sizeof(EncOutputInfo)); 
    EncParam *encParam = malloc(sizeof(EncParam)); 

    // Set allocated memory to zero 
    memset (initialInfo, 0, sizeof (EncInitialInfo)); 
    memset (encParam, 0, sizeof (EncParam)); 
    memset (encOP, 0, sizeof (EncOpenParam)); 
    memset (outputInfo, 0, sizeof (EncOutputInfo)); 
    memset (&bit_stream_buf, 0, sizeof (vpu_mem_desc)); 


    /*instance attached to display interface */ 
    config_system_parameters(); 


    /* initialize VPU */ 

    ret = VPU_Init(); 
    if (ret) {        // VPU Init Failure 
        ret = -1; 
        goto free; 
    } 

    ret = VPU_GetVersionInfo(&ver); 
    if (ret) {        // Cannot get version info 
        ret = -1; 
        goto uninit; 
    } 


    bit_stream_buf.size = BUFF_FILL_SIZE; 
    ret = vpu_malloc(&bit_stream_buf); 
    if (ret) { 
        ret = -1; 
        goto free; 
    } 

    // Set up encoder operation parameters 
    encOP->bitstreamBuffer = bit_stream_buf.phy_addr; 
    encOP->bitstreamBufferSize = BUFF_FILL_SIZE; 
    encOP->bitstreamFormat = STD_AVC; 
    encOP->picWidth = DEFAULT_WIDTH; 
    encOP->picHeight = DEFAULT_HEIGHT; 
    encOP->frameRateInfo = DEFAULT_FRAME_RATE; 
    encOP->gopSize = DEFAULT_GOP_SIZE; 
    encOP->rcIntraQp = -1; 
    encOP->ringBufferEnable = 0; 
    encOP->userGamma = (uint32_t) (0.75 * 32768); 
    encOP->avcIntra16x16OnlyModeEnable = 1; 

    encOP->EncStdParam.avcParam.avc_disableDeblk = 1; 


    ret = VPU_EncOpen (handle, encOP); 
    if (ret) { 
        ret = -1; 
        goto uninit; 
    } 


    ret = VPU_EncGetInitialInfo (*handle, initialInfo); 

    // Disable rotation/mirroring 
    ret = VPU_EncGiveCommand (*handle, DISABLE_ROTATION, 0); 
    ret = VPU_EncGiveCommand (*handle, DISABLE_MIRRORING, 0); 


    // Get number of destination frame buffers and allocate 
    int num = initialInfo->minFrameBufferCount; 

    vpu_mem_desc *framedesc = malloc(sizeof(vpu_mem_desc) * num); 
    FrameBuffer *frame = malloc(sizeof(FrameBuffer) * num); 
    memset (framedesc, 0, (sizeof (vpu_mem_desc) * num)); 
    memset (frame, 0, (sizeof (FrameBuffer) * num)); 


    // Allocate each destination frame buffer 
    for (i = 0; i < num; i++) { 
        framedesc[i].size = DEFAULT_WIDTH * DEFAULT_HEIGHT * 3/2; 
        ret = vpu_malloc(&(framedesc[i])); 
        if (ret) { 
            ret = -1; 
            goto uninit; 
        } 
        frame[i].myIndex = i; 
        frame[i].strideY = DEFAULT_WIDTH; 
        frame[i].strideC = DEFAULT_WIDTH >> 1; 
        frame[i].bufY = framedesc[i].phy_addr; 
        frame[i].bufCb = frame[i].bufY + (DEFAULT_WIDTH * DEFAULT_HEIGHT); 
        frame[i].bufCr = frame[i].bufCb + ((DEFAULT_WIDTH * DEFAULT_HEIGHT) >> 2); 
    } 


    // Allocate memory for source frame buffer 
    source_buf.size = DEFAULT_WIDTH * DEFAULT_HEIGHT * 3/2; 
    ret = vpu_malloc(&source_buf); 
    if (ret) { 
        ret = -1; 
        goto uninit; 
    } 

    source_frame.myIndex = num; 
    source_frame.strideY = DEFAULT_WIDTH; 
    source_frame.strideC = DEFAULT_WIDTH >> 1; 
    source_frame.bufY = source_buf.phy_addr; 
    source_frame.bufCb = source_frame.bufY + DEFAULT_WIDTH * DEFAULT_HEIGHT; 
    source_frame.bufCr = source_frame.bufCb + (DEFAULT_WIDTH * DEFAULT_HEIGHT >> 2); 


    ret = VPU_EncRegisterFrameBuffer (*handle, frame, num, DEFAULT_WIDTH, DEFAULT_WIDTH, 0, 0, NULL); 


    encParam->quantParam = VPU_DEFAULT_H264_QP; 
    encParam->picStreamBufferAddr = bit_stream_buf.phy_addr; 
    encParam->picStreamBufferSize = BUFF_FILL_SIZE; 

    // Copy and Encode 
    frame_fill((void *)source_buf.phy_addr); 
    encParam->sourceFrame = &source_frame; 


    /* Put encode header */ 
    EncHeaderParam enchdr_param = { 0 }; 

    enchdr_param.headerType = SPS_RBSP; 
    ret = VPU_EncGiveCommand(*handle, ENC_PUT_AVC_HEADER, &enchdr_param); 

    enchdr_param.headerType = PPS_RBSP; 
    ret = VPU_EncGiveCommand(*handle, ENC_PUT_AVC_HEADER, &enchdr_param); 


    // Encoding single frame 
    ret = VPU_EncStartOneFrame (*handle, encParam); 
    while (VPU_IsBusy()) { 
        hal_delay_us(100); 
    } 


    ret = VPU_EncGetOutputInfo (*handle, outputInfo); 

    ret = VPU_EncClose (*handle); 




uninit: 
    VPU_UnInit(); 

free: 
    free(encOP); 
    free(initialInfo); 
    free(outputInfo); 
    free(encParam); 
    free(handle); 
    return (int)ret; 
}

[chuck1]

Vadim,

I took a quick look at your code and, although I may have missed it, I don't see a place where you are filling/refilling your source video frame buffers—my understanding is that the VPU will always return "IsBusy" under these conditions.  Also, I where you are allocating VPU working memory so your encoded video quality—once you get the encoder running—may suffer as well.

For your encoding "while loops", you should do something like this:

--- snip ---

    /* Open the video input file for reading. */

    FILE *inFile = fopen("BigBuckBunny_640x360.yuv", "r");

    while (fread((void*) sourceBuffer.virt_uaddr, 1, WIDTH * HEIGHT * 3/2, inFile)) {

        /* Get the YUV source video frame buffer. */

        encoderParam->sourceFrame = &sourceFrame;

        /* Encode a single frame then wait. */

        vpu_EncStartOneFrame (*encoderHandle, encoderParam);

        while (vpu_IsBusy()) {

                /* Wait for VPU. */

                vpu_WaitForInt (200);

        }

        /* Get the H.264 encoded video frame. */

        vpu_EncGetOutputInfo (*encoderHandle, encoderOI);

        /* Write the H.264 encoded video frame to outFile. */

        fwrite((void*) bitStreamBuffer.virt_uaddr, 1, encoderOI->bitstreamSize, outFile);

        frameCount++;

    }

--- snip ---

I hope this is helpful,

-Chuck

[endl]

Hello, Charles

Thank you for the advice.

I can explain:

1. In my program input and output stream are stored in memory and don't need to call fread/fwrite.

I'm not using SD card.

And I tried to encode a single frame, therefore refilling is not used.

Frame filling is located at line 163.

frame_fill(..) is located at line 21.

    // Copy and Encode   
    frame_fill((void *)source_buf.phy_addr);  // <----< source frame filling
    encParam->sourceFrame = &source_frame; 

But ... I think that, even if we don't fill source frame, encoder will not notice the difference because the encoder works with data located at the encParam->sourceFrame (or CMD_ENC_PIC_SRC_ADDR_Y register).

CMD_ENC_PIC_SRC_ADDR_Y must not be equal to NULL.

2. Working buffer, code buffer, frame buffer etc are located at start address 0x28000000.

Default (from Platform SDK) VIDEO_BUFFERS_START is 0x48000000 but my demoboard has 1GB memory.

Could you tell me,

1. Are you using "iMX6 Platform SDK v1.1.0" for your board ? Which processor are you using ?

2. Are you using CODA960 firmware ? Do you have CODA960 datasheet ? Registration on chipsnmedia.com is not valid and support does not answer the questions.

3. Could you do memory dump from 0x02040000 to 0x02040200 ?

4. What is your buffer settings ? (BIT_WORK_SIZE, CODE_BUF_SIZE, TEMP_BUF_SIZE, PARA_BUF_SIZE, BIT_STREAM_BUF_SIZE and video buffer start address)

5. Why the BIT processor may hang ? What's the problem ?

Sorry for many questions, I don't know what to try. I can't solve this problem for a very long time.

Thanks in advance.

[chuck1]

Vadim,

Thanks for your explanation—now I see what you what you are trying to do and I agree—the VPU should process just one frame for you.  However, in order to make progress, I suggest that you should put multiple video frames in memory then create a loop that will have the VPU process one video frame then read the VPU output result and so on—this may give you a clue as to what the VPU is actually doing.  Alternatively, you can try the process of: read a YUV video frame from a file —> send the frame to the frame buffer —> tell the VPU to encode the frame —> write the H.264 encoded frame to a file —> play the file using VLC.  I can post working code for this approach if you would like (however; I'm having problems with the quality of the encoded video but that's another issue).

My development system is a Toradex Apalis i.MX6Q running with Yocto 1.6 Open Embedded Angstrom Linux, 1 GBytes SDRAM, 1 GHz, etc.

I hope this is helpful,

-Chuck

[endl]

Hello, Charles

The loop is not necessary because it ends on the first iteration.

I would be grateful if you post working code for your approach. (email ignat26i@mail.ru if needed).

In your application you perform some platform-specific code ? For example enable/disable signal, enable clock, set hw registers and etc.

Thanks in advance.

[chuck1]

Vadim,

Here is a link to the posted code:

i.MX6 VPU H.264 Encoder Quality Issues

As I say in the post, the H.264 video quality is excellent if the GOP size is 1 or 2 but > 2, the quality is unacceptable—please let me know if you have any suggestions regarding how to fix the quality problem.

Thanks,

-Chuck