How do I do use the VPU on OS other than Linux or Android?

[忍松本]

We are running on i.MX6Q the T-Kernel.
Order to use the VPU, we are porting the driver from Linux, but the problem has occurred.
This timing, is CPU crashes and no response.

--- vpu_lib.c L.443 ---
BitIssueCommand(NULL, FIRMWARE_GET);
-----------------------

Additional Information (Nov.22.2013):

  - i.MX6Q have four cores, but we are using only a single core.

  - Our environment

      imx-lib version : imx-lib-12.09.01

      board : Collage-MX6Q (Sohwa & Sophia Technologies)

Thanks in Advance.

Shinobu MATSUMOTO

[HongzhangYang]

Daisuke,

As I know, register read or write does not need delay. I still can't find any clue from your test result.

Your code involved 2 tasks: VPU init and get version. We have some abnormal behaviours as below.

1) PC after VPU init is not B9 or BB

Where did you check PC(check2)? Please make sure it was in VPU init task and right after VPU init was done (right after BASE_0000=1 and BASE_0160=0). Please keep this printing in all your tests to make sure this task was completed successfully.

2) FW version is 0

We already print PC in GetVersion. It must be B9 or BB. In your case 1, PC is correct but Version is 0. Please print 920C8140 before reading BASE_0160 and before reading BASE_01C0.

3) CPU hang

It was your case 2. PC=0 which means isVpuInitialized()=0. GetVersion should return immediately. Why could CPU hang? Please insert some exit() to identify which exact command caused CPU hang.

Your case 1 and 3 didn't cause CPU hang, right?

I suspect this issue is caused by Power or Clock. Can you keep them always on?

Regards,

Hongzhang

[daisuketsuchiya]

Hello Hongzhang Yang,

>1) PC after VPU init is not B9 or BB
>Where did you check PC(check2)?

     After writing 1 to base_0000, I have done the check 2.  And it is the same context (as a GStreamer's thread on Linux) which confirming BB.

>2) FW version is 0
>Please print 920C8140 before reading BASE_0160 and before reading BASE_01C0.

     I got a log that VPU clock is disabled before writing to 0C10.  So, I will investigate the clock and power, as you said.  Could you tell me the meaning of the 0160 register and  0x20 value of 0018?

      VWRITE: addr=92040000 data=00000001
      VREAD: addr=920C8140 data=00483E12
      VREAD: addr=92040160 data=00000000
      VREAD: addr=92040018 data=000000BB
      VREAD: addr=920C8140 data=00483E12
      VREAD: addr=920C8140 data=00483E12
      VREAD: addr=92040160 data=00000000
      VREAD: addr=92040018 data=00000020

->   VREAD: addr=920C8140 data=00480012
      VWRITE: addr=920401C0 data=00000000

      VWRITE: addr=92040160 data=00000001
      VWRITE: addr=92040168 data=00000020
      VWRITE: addr=9204016C data=00000000
      VWRITE: addr=92040178 data=00000000
      VWRITE: addr=92040164 data=0000000F
      VREAD: addr=920C8140 data=00480012
      VREAD: addr=92040160 data=00000000
      VREAD: addr=920401C0 data=00000000
      VREAD: addr=920401C4 data=00000000

Thank you for cooperation,
Daisuke TSUCHIYA

[HongzhangYang]

Daisuke,

0160 is BUSY_FLAG. It will be 0 after VPU has completed a command (Init, GetVersion, etc).

0018 is CUR_PC, current PC of the processor inside VPU. When VPU completes a command, PC is in a loop B9<->BB. It requires a new command to get out of the loop. PC=20 points to an instruction during Init, but it's strange to transit from B9/BB to 20 without new command. Maybe VPU voltage is not stable.

920C8140 is voltage register. If VPU voltage is 0 (as you found), write operation will be ignored, read operation will get 0.

920C4080 is clock register. If VPU clock is off, write/read operation will hang up bus.

Regards,
Hongzhang

[daisuketsuchiya]

Hello Hongzhang Yang,

Thank you for reply.

I understood the register which you taught me.  It's strange why clock control does not work correctly because it's done by GStreamer.  I will investigate a little more where the clock goes to OFF.

By the way, I'm proceeding our development with change the clock to be always ON for the time being.  Then, I can get the version info by GetVersion command.  However, in a subsequent command, BUSY flag is set and not fall.

Q1:
Do you have any information why the BUSY flag does not fall?  Is there any problem if the VPU clock is not turned OFF regularly?

Q2:
I have tried to change from user mode to kernel mode access to the VPU.  Then, GetVersion is successful.  However, when I execute "vpu_DecGetInitialInfo()", to get the "RETCODE_FAILURE" and fail.  And, get other state,
     VREAD: addr=920401C0 data=00000000
     VREAD: addr=920401E0 data=00000016

Could you tell me the meaning of the above values?

Best Regards,
Daisuke TSUCHIYA

[HongzhangYang]

Daisuke,

Q1: BUSY register is cleared by FW. If VPU was blocked in HW or FW, BUSY is always 1.

We have verified there's no problem if VPU voltage and clock are always on.

Q2: Looks like syntax parsing error. What video standard are you using? Please analyze the clip or try another one.

Regards,
Hongzhang

[daisuketsuchiya]

Hello Hongzhang Yang,

About Q1:
I have always ON the VPU clock now, and I have done access to the VPU in kernel mode.  Then VPU process looks proceeding.

About Q2:
As you say, there was a problem with the input data.  I corrected the problem.  Thank you.

Now, I have a question about the output of vpu_DecGetOutputInfo().  I was confirmed DecOutputInfo, which is the output of the vpu_DecGetOutputInfo.  This function is called repeatedly from the application, but there is no change in the content of DecOutputInfo acquired.

    indexFrameDecoded :   -2
    indexFrameDisplay :   -3  // There is no need to display the image
    picType           :    3
    decodingSuccess   : 0x10
    numOfErrMBs       :    0
    consumedByte      :    0

Could you tell me the following?

1) The reason why indexFrameDisplay does not change from -3.
2) The meaning of 0x10 of decodingSuccess.
3) In spite of the first frame of the video, is it possible that picType is not 0?
4) That you will be worried about the other.

Thank you for cooperation,
Daisuke TSUCHIYA

[HongzhangYang]

Daisuke,

2) You are using roll back mode but there is not enough bitstream to decode a picture. Please fill more bitstream to bs buffer, otherwise decoder can not proceed (output of vpu_DecGetOutputInfo will not change (1)).

3) A clip normally begins with an I picture, so picType is 0. If (decodingSuccess & 1) == 0, picType, indexFrame..., etc can be ignored.

Regards,

Hongzhang

[daisuketsuchiya]

Hello Hongzhang Yang,

Thank you for reply.

Please let me confirm the meaning of the "not enough".

Do you mean that filled bitstream buffer is not in time?  Or do you mean that the amount of bitstream data is insufficient quantitatively to decode?

And could you tell me the meaning of 0x10 of decodingSuccess?

Best Regards,

Daisuke TSUCHIYA

[HongzhangYang]

Daisuke,

In rollback mode, if bitstream buffer doesn't have enough bits for decoding a picture, VPU returns from decoding and rolls back its read pointer to the beginning of that picture. In this case, bit 4 of decodingSuccess is 1.

VPU requires at least the complete picture in bs buffer and VPU may still need more due to some alignment requirement. So fill more bitstreams when you get 0x10 and continue decoding.

BR

Hongzhang

[daisuketsuchiya]

Hello Hongzhang Yang,

How does the rollback mode work?
If there is no key frame, does VPU operate back to the previous frame?

Is there a mode in addition to the rollback?
If so, does the mode set in the register or BIT processor firmware?

Thank you for cooperation.

Daisuke TSUCHIYA

[HongzhangYang]

Daisuke,

DecOpenParam oparam;

oparam.bitstreamMode = 1 to select roll back mode. When bitstream is not enough, VPU will not be blocked and roll back as if the picture has not been decoded.

oparam.bitstreamMode = 0 to select interrupt mode. When bitstream is not enough, VPU will be blocked (always busy) until you fill more bistream.

Key frame or not doesn't affect above behaviour.

BR

Hongzhang

[daisuketsuchiya]

Hello Hongzhang Yang,

I have not been able to decode one frame yet.
Please let me ask the one question.

In the T-Kernel environment that I have ported, it takes time to BUSY state release after accessing the VPU register in the call of the VPU API.
eg. vpu_DecGetInitialInfo() and vpu_DecRegisterFrameBuffer().

In the Linux environment, it does not take time to the BUSY state release.

"while (VpuReadReg(BIT_BUSY_FLAG));" statement ends at once.

Could you tell me the reason why it takes time to BUSY state release?

Thank you,
Daisuke TSUCHIYA

[daisuketsuchiya]

Hello Hongzhang Yang,

I have additional question.

In our T-Kernel case, when the first execution of vpu_DecGetInitialInfo(), it becomes an error that "bitstreamMode = 1" and "RET_DEC_SEQ_SUCCESS(BASE+0x1C0) is 0x10".  (line 2910 in attached vpu_lib.c)

In the linux case, "BASE+0x1C0 = 0x01" and it continues the decoding process correctly.

At this point, the data transfer of one frame is not executing yet.  So, I guess it does not relate whether the first frame is keyframe or not.

Could you answer me the following questions?

  1) The reason to become "BASE+0x1C0=0x10" when executing vpu_DecGetInitialInfo() for the first time.

  2) The above workaround.

Thank you.
Daisuke TSUCHIYA

[HongzhangYang · Accepted solution]

Daisuke,

Simple task like vpu_DecRegisterFrameBuffer() ends up very quickly, because VPU doesn't have much to do. I have no idea why it's slow in your env. You can keep printing PC(0x18) in busy loop to track where it takes VPU time.

0x10 means roll back happened. You need to call vpu_DecGetInitialInfo() again after you fill more bitstream. You'd better fill as much bistream as possible (or even complete clip) to simplify your bringing up.

BR

Hongzhang

[daisuketsuchiya]

Hello Hongzhang Yang,

In the vpu_DecUpdateBitstreamBuffer() function call of the first two after the start of decoding, both Linux and T-Kernel also not write bitstream data to VPU.
I mean that the statement "line 3520:VpuWriteReg(BIT_BIT_STREAM_PARAM, val)"  and  "line 3556:VpuWriteReg(BIT_WR_PTR, wrPtr)" in the vpu_lib.c which attached above is not executed.
Therefore, I think that the value of BASE+0x1C0 is 0x10 is strange.

Do you have any idea about this?

Thank you.

Daisuke TSUCHIYA

[HongzhangYang]

Daisuke,

It's a good idea to compare T-kernel version with Linux version. Tracking vpu_DecUpdateBitstreamBuffer() and BIT_WR_PTR is helpful for this issue.

I decoded an H.264 clip, and dumped the API call sequence FYI.

[DEBUG] vpu_lib.c:266 enter vpu_Init()

[DEBUG] vpu_lib.c:495 enter vpu_GetVersionInfo()

[DEBUG] vpu_lib.c:2563 enter vpu_DecOpen()

[DEBUG] vpu_lib.c:3538 enter vpu_DecGetBitstreamBuffer()

[DEBUG] vpu_lib.c:3622 enter vpu_DecUpdateBitstreamBuffer()

[DEBUG] vpu_lib.c:3623 Update bitstream buffer size 2094592

[DEBUG] vpu_lib.c:2835 enter vpu_DecSetEscSeqInit()

[DEBUG] vpu_lib.c:2883 enter vpu_DecGetInitialInfo()

[DEBUG] vpu_lib.c:2835 enter vpu_DecSetEscSeqInit()

[DEBUG] vpu_lib.c:4760 enter vpu_DecGiveCommand()

[DEBUG] vpu_lib.c:3269 enter vpu_DecRegisterFrameBuffer()

[DEBUG] vpu_lib.c:3749 enter vpu_DecStartOneFrame()

[DEBUG] vpu_lib.c:93 enter vpu_IsBusy()

[DEBUG] vpu_lib.c:3538 enter vpu_DecGetBitstreamBuffer()

[DEBUG] vpu_lib.c:130 enter vpu_WaitForInt()

[DEBUG] vpu_lib.c:93 enter vpu_IsBusy()

[DEBUG] vpu_lib.c:4185 enter vpu_DecGetOutputInfo()

[DEBUG] vpu_lib.c:2787 enter vpu_DecClose()

We filled as much bs as possible before vpu_DecGetInitialInfo().

According to your log, I wonder why

- VpuWriteReg(BIT_BIT_STREAM_PARAM, val); is not executed after entering vpu_DecUpdateBitstreamBuffer()?

- vpu_DecGetInitialInfo() is successful on Linux without filling any bs

Please compare wrPtr before vpu_DecGetInitialInfo() / vpu_DecStartOneFrame()

BR
Hongzhang

[daisuketsuchiya]

Hello Hongzhang Yang,

Thank you for reply.

Please let me ask about vpu_DecSetEscSeqInit.

In the manual(i.MX 6Dual/6Quad VPU Application Programming Interface Linux Reference Manual Rev. 12.09.01 (GA), 10/2012  P.61 3.3.4.3 vpu_DecGetInitialInfo()), it describes that "If the application cannot ensure to feed enough data for the stream, the application can use the forced escape option using vpu_DecSetEscSeqInit()."

So, I think that there is no problem even if the call of the vpu_DecGetInitialInfo without filling the bitstream buffer in Linux case.
Does my understanding correct?  If so, the case of T-Kernel is strange.

Best Regards,

Daisuke TSUCHIYA

[HongzhangYang]

Daisuke,

vpu_DecSetEscSeqInit() is used for interrupt mode. It prevents VPU from blocking in vpu_DecGetInitialInfo() if there's not enough data. App needs to call vpu_DecGetInitialInfo() again to retry.

In rollback mode, VPU returns RET_DEC_SEQ_SUCCESS=0x10. App also needs to try vpu_DecGetInitialInfo() again.

BR

Hongzhang

[daisuketsuchiya]

Hello Hongzhang Yang,

I am confirming by making a program which access directly the VPU without using the GStreamer.  Could you tell me about Data Buffer of BIT Processor?

1. About the macro defined in vpu_reg.h

        #define CODE_BUF_SIZE    (280 * 1024)
        #define TEMP_BUF_SIZE    (204 * 1024)
        #define PARA_BUF2_SIZE   (2 * 1024)
        #define PARA_BUF_SIZE    (10 * 1024)

Are these macros the same as macros that are described in the specifications?  (i.MX 6Dual/6Quad VPU Application Programming Interface Linux Reference Manual / Document Number:VPU_API Rev. 12.09.01 (GA), 10/2012  /  P.69  4.1.3 BIT Processor Data Buffer Management)
If it is different, please tell me what these macros are.

2. About the address of FrameBuffer

Please tell me the address to write the address of FrameBuffer?  I want to know the relative address in the case of the base address codeBuffer.

Thank you.

Daisuke TSUCHIYA

[HongzhangYang]

1. These are buffers assigned to VPU. Please allocate physically continuous memory for them with the sizes specified in source code. API doc describes the usage of them but their sizes may not be exact.

2. FrameBuffers are independant from codeBuffer. You should allocate physically continuous memory for them and pass them to VPU by vpu_DecRegisterFrameBuffer().

BR

Hongzhang

[daisuketsuchiya]

Hello Hongzhang Yang,

I will reply the following questions.

1) VPU clock relative setting (Can follow up Linux setting)
    I have set in the same way as Linux.

2) IOMUXC_GPR4 VPU relative setting (Can follow up Linux setting)
    There is a difference.
        Linux: bit2, 3, 6, 7 of IOMUXC_GPR4 is ON.
        Ours : all off

    When should I set this bit ON?  I tried in the same value as Linux, but it becomes an error again in vpu_GetInitialInfo().

I report to you.
I have been able to get the YUV data that seems with the first frame decoded by VPU.  vpu_GetInitialInfo() becomes an error several times initially, then it succeeds by repeating the retry.  Though I can not be confirmed display of the image since IPU does not work yet.
Is there any problem in this behavior?

Thank a lot for your cooperation.
Daisuke TSUCHIYA