CSI-2 JPEG Stream

[johndusing]

The camera HAL appears to be expecting a JPEG stream from the CSI-2 interface on the i.MX6.  If the data through the CSI-2 is raw YUV422 pixel format and not run through a converter, how can a raw video stream be specified? I'm not sure how to bypass this.

In an earlier post the following question was also asked:

Using UV5640_mipi.c with modifications to support video streaming from a test pattern generator into i.MX6 CSI-2 camera input (virtual channel 1, YUV422).  When code is set to 1360x768 image 1 is ok.  When code is changed to 1366x768 and the input pattern is changed to be 1366x768, picture 2 is incorrect.  Does something else need to change in the driver, or does another driver need modified?  Scrolling occurs, how is HSYNC determined?

[johndusing · Accepted solution]

If removing the -fg as indicated above causes the CSI-2 video stream to change from displaying correctly to generating a distorted image under Linux, where in the Android code can I force the use of "fg" or make a similar change to see if this fixes our video problem?

[johndusing]

Kevin,

Have you forwarded this request to BCM?

Sent from my iPhone

[KevinWong]

Hi John,

No, this was only sent to PDI. I have posted it back on the private space.

Thanks,

Kevin

[johndusing]

Thanks Kevin, I am out of the office today, but I will be back into work tomorrow.

Sent from my iPhone

[johndusing]

This observation was also made by the board vendor while running in Linux,

Just FYI. If you remove the parameter “-fg” below, you’ll see the similar distorted display like Android.

/unit_tests/mxc_v4l2_overlay.out -iw 1360 -ih 768 -it 0 -il 0 -ow 1360 -oh 768 -ot 0 -ol 0 -r 0 -t 50 -d 0 -fg -fr 30

What does using the ipu foreground overlay have to do with whether the display is distorted or not?

[johndusing]

Thank you for the reply.

Here is an update on the configuration and results.  Any additional help is appreciated.

The input into the CSI-2 interface on the i.MX6 is a raw LVDS CSI-2 video stream in a YUV422 format (set for virtual channel 1).  There is no camera hardware on the input.  The video stream is in 1366x768 format (from a display panel).  The driver has been modified to expect 1368x768 due to hardware restrictions inside the CPU.  It has been verified by our board manufacturer that under a Linux OS test environment, the video displays as expected.  They expect that it is a formatting issue between UYVY and NV12.  Using Android and the HAL, the distortion appears.  There could be issues inside the CameraHAL, but I'm not sure where to look.  Below is the capture they made.

it’s basically a format issue (UYVY vs. NV12). You’ll need find someone to bridge the gap . . . 

Linux preview:

root@ar6mxq:~# /unit_tests/mxc_v4l2_overlay.out -iw 1368 -ih 768 -it 0 -il 0 -ow 1368 -oh 768 -ot 0 -ol 0 -r 0 -t 50 -d 1 -di /dev/video0 -fg -fr 30

g_display_width = 1368, g_display[261694.464782]

>>>> ov5640_init_mode: build date = Sep 16 2016 13:07:36_height = 768

g_display_top = 0,

g_display_left = 0

[261694.474035]

[261694.474035] >>>> ov5640_init_mode: frame rate=1, mode=255, orig_mode=255[261694.487500] retval 2: 0

[261694.490477] retval 2: 0

[261694.493623] retval 2: 0

[261694.496593] retval 2: 0

[261694.499563] retval 2: 0

[261694.811804] >>>> (null): Debug ov5640 init mode: Line: -1120240688

[261694.818078] >>>> (null): 0 try: 1008 csi2 dphy status: 0x0

[261694.823807] >>>> (null): 1 try:768 csi2 dphy status: 0x0

[261694.841798] >>>> (null): 1 try:1008 csi2 dphy status: 0x0

[261694.861797] >>>> (null): 1 try:768 csi2 dphy status: 0x0

[261694.881796] >>>> (null): 1 try:1008 csi2 dphy status: 0x0

[261694.901798] >>>> (null): 1 try:768 csi2 dphy status: 0x0

[261694.921798] >>>> (null): 1 try:768 csi2 dphy status: 0x0

[261694.941797] >>>> (null): 1 try:1008 csi2 dphy status: 0x0

[261694.961797] >>>> (null): 1 try:768 csi2 dphy status: 0x0

[261694.981796] >>>> (null): 1 try:1008 csi2 dphy status: 0x0

[261695.001798] >>>> (null): 1 try:1008 csi2 dphy status: 0x0

[261695.021796] >>>> (null): mipi csi2 can not receive sensor clk!

sensor chip is ov5640_mipi_camera

sensor supported frame size:

1368x768

1368x768

1368x768

1368x768

1368x768

1368x768

1368x768

sensor frame format: UYVY

sensor frame format: UYVY

sensor frame format: UYVY

sensor frame format: UYVY

sensor frame format: UYVY

sensor frame format: UYVY

sensor frame format: UYVY

frame_rate is 30

frame buffer width 0, height 0, bytesperline 0

Android logcat:

I/CameraManager( 1609): Using legacy camera HAL.

I/Camera2ClientBase(  184): Camer

a 1: Opened. Client: com.android.[  155.855011]

[  155.855011] >>>> ov5640_init_mode: build date = Sep 12 2016 08:14:08camera2 (PID 1609, UID 10021)

I/

>>>> ov5640_init_mode: frame rate=1, mode=255, orig_mode=255FslCameraHAL(  184): camera_devic

[  155.877326]

[  155.877326] >>>> mipi_csi2_set_lanes: lanes=3  (0 based)e open: 1

I/FslCameraHAL(  184): DeviceAdapter: Create ov5640 mipi device

I/FslCameraHAL(  184): SupportedPictureSizes: 1368 x 768

I/FslCameraHAL(  184): SupportedPictureSizes: 1368 x 768

I/FslCameraHAL(  184): SupportedPictureSizes: 1368 x 768

I/FslCameraHAL(  184): SupportedPictureSizes: 1368 x 768

I/FslCameraHAL(  184): SupportedPictureSizes: 1368 x 768

I/FslCameraHAL(  184): SupportedPictureSizes: 1368 x 768

I/FslCameraHAL(  184): SupportedPictureSizes: 1368 x 768

I/FslCameraHAL(  184): SupportedPreviewSizes: 1368 x 768

I/FslCameraHAL(  184): SupportedPreviewSizes: 1368 x 768

I/FslCameraHAL(  184): SupportedPreviewSizes: 1368 x 768

I/FslCameraHAL(  184): SupportedPreviewSizes: 1368 x 768

I/FslCameraHAL(  184): SupportedPreviewSizes: 1368 x 768

I/FslCameraHAL(  184): SupportedPreviewSizes: 1368 x 768

I/FslCameraHAL(  184): SupportedPreviewSizes: 1368 x 768

I/FslCameraHAL(  184): FrameDuration is 33331760, 30000000000

I/FslCameraHAL(  184): mMaxWidth:1368, mMaxHeight:768

I/FslCameraHAL(  184): ov5640Mipi, mFocalLength:3.370000, mPhysicalWidth:3.628800, mPhysicalHeight 2.721600

E/FslCameraHAL(  184): cpu num 4

W/Camera2-Parameters(  184): buildFastInfo: Camera 1: activeArraySize is missing xmin/ymin!

W/Camera2-Parameters(  184): Unable to find the size to match the given aspect ratio 1.781250.Fall back to 96 x 96

I/Camera2-Parameters(  184): Camera 1: Disabling ZSL mode

I/CAM_PhotoModule( 1609): onCameraAvailable

I/CameraDeviceState( 1609): Legacy camera service transitioning to state CONFIGURING

D/Camera  ( 1609): app passed NULL surface

I/FslCameraHAL(  184): DEBUG(CreateDefaultRequest): making template (1)

I/FslCameraHAL(  184): DEBUG(CreateDefaultRequest): making template (3)

I/CameraDeviceState( 1609): Legacy camera service transitioning to state IDLE

I/FslCameraHAL(  184): RequestManager allocateStream...

I/FslCameraHAL(  184): allocateStream preview stream, w:1368, h:768, fmt:0x22

I/FslCameraHAL(  184): actual format 0x103

I/FslCameraHAL(  184): >>>> Line 114 - StreamAdapter::initialize()

I/FslCameraHAL(  184): RequestManager allocateStream end...

I/FslCameraHAL(  184): RequestManager registerStreamBuffers stream id:0

I/FslCameraHAL(  184): registerBuffers total 5 buffer

I/FslCameraHAL(  184): CameraFrame::initialize, i 0, phyAddr 0x3d1ec000, mBufHandle 0x7372c240

I/FslCameraHAL(  184): CameraFrame::initialize, i 1, phyAddr 0x3e2b9000, mBufHandle 0x74708300

I/FslCameraHAL(  184): CameraFrame::initialize, i 2, phyAddr 0x41974000, mBufHandle 0x74708180

I/FslCameraHAL(  184): CameraFrame::initialize, i 3, phyAddr 0x41af7000, mBufHandle 0x75f73cc0

I/FslCameraHAL(  184): CameraFrame::initialize, i 4, phyAddr 0x41c7a000, mBufHandle 0x75f73d80

I/FslCameraHAL(  184): RequestManager registerStreamBuffers end...

I/FslCameraHAL(  184): RequestManager allocateStream...

I/FslCameraHAL(  184): allocateStream jpeg stream, w:1368, h:768, fmt:0x21

I/FslCameraHAL(  184): actual format 0x103

I/FslCameraHAL(  184): >>>> Line 114 - StreamAdapter::initialize()

I/FslCameraHAL(  184): RequestManager allocateStream end...

I/FslCameraHAL(  184): RequestManager registerStreamBuffers stream id:3

I/FslCameraHAL(  184): RequestManager registerStreamBuffers end...

I/FslCameraHAL(  184): notify_request_queue_not_empty running

I/FslCameraHAL(  184): dispatchRequest running

I/FslCameraHAL(  184): handleRequest running

I/FslCameraHAL(  184): PreviewStream configure running

I/FslCameraHAL(  184): pixel format: 0x3231564e

I/FslCameraHAL(  184): Width * Height 1368 x 768 format 0x3231564e, fps: 30

I/FslCameraHAL(  184): PreviewStream start running

I/FslCameraHAL(  184): StreamAdapter start running

I/FslCameraHAL(  184): stream thread received STREAM_START command

I/FslCameraHAL(  184): Created device thread

I/CameraDeviceState( 1609): Legacy camera service transitioning to state CAPTURING

D/CameraPerformanceTracker( 1609): First preview frame received

[jimmychan]

In the camera driver (e.g. OV5640), there are different set of parameters (register values) for different capture resolution. if you change the resolution a little bit but not modify the corresponding camera's register values, the scrolling may occurs. You need to read the camera datasheet to modify those register values.