Can I show the camera preview and one video playback on one screen?

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Can I show the camera preview and one video playback on one screen?

2,224 Views
lijian
Contributor I

On the android platform, when I write one activity that show the camera preview and video playback on one screen, I got some errors:

10-19 09:44:48.179 I/FslOverlay(14861): Mixer thread refill the origin area to black

10-19 09:44:48.219 I/ipulib  (14861): external/linux-lib/ipu/mxc_ipu_hl_lib.c:956 ipu is busy

10-19 09:44:48.219 I/ipulib  (14861): external/linux-lib/ipu/mxc_ipu_hl_lib.c:958  for hope task mode 0x4!

10-19 09:44:48.219 E/FslOverlay(14861): Error!Obj0 mxc_ipu_lib_task_init failed mIPURet -1!

10-19 09:44:48.219 I/ipulib  (14861): external/linux-lib/ipu/mxc_ipu_hl_lib.c:956 ipu is busy

The fully log looks like the following:


10-19 09:44:46.179 I/ActivityManager(14861): Starting: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] flg=0x10000000 cmp=com.ceict.videoclientdemo/.VideoChat } from pid 15407

10-19 09:44:46.189 I/SurfaceFlinger(14861): mHw->getFormat() 4

10-19 09:44:46.219 D/AndroidRuntime(15407): Shutting down VM

10-19 09:44:46.219 I/ActivityManager(14861): Start proc com.ceict.videoclientdemo for activity com.ceict.videoclientdemo/.VideoChat: pid=15422 uid=10049 gids={1006, 3003, 1015}

10-19 09:44:46.229 D/dalvikvm(15407): GC_CONCURRENT freed 102K, 69% free 320K/1024K, external 0K/0K, paused 1ms+0ms

10-19 09:44:46.229 D/jdwp    (15407): Got wake-up signal, bailing out of select

10-19 09:44:46.229 D/dalvikvm(15407): Debugger has detached; object registry had 1 entries

10-19 09:44:46.289 I/WindowManager(14861): Setting rotation to 1, animFlags=1

10-19 09:44:46.309 I/ActivityManager(14861): Config changed: { scale=1.0 imsi=0/0 loc=zh_CN touch=3 keys=1/1/2 nav=1/1 orien=2 layout=268435490 uiMode=17 seq=3}

10-19 09:44:46.339 W/com.skype.rover.MainApp(15144): onConfigurationChanged changed:{ scale=1.0 imsi=0/0 loc=zh_CN touch=3 keys=1/1/2 nav=1/1 orien=2 layout=268435490 uiMode=17 seq=3}

10-19 09:44:46.359 V/KeyEvent(14953): Reset: android.view.KeyEvent$DispatcherState@2aff1d20

10-19 09:44:46.619 I/System.out(15422): CameraPreview

10-19 09:44:46.619 I/System.out(15422): Player

10-19 09:44:46.689 I/ActivityManager(14861): Displayed com.ceict.videoclientdemo/.VideoChat: +470ms

10-19 09:44:46.769 I/OMXPlayer(14850): Loading content: /mnt/sdcard/pic/camerarecorder.mp4

10-19 09:44:46.779 I/FslCameraHAL(14850): Face Back Camera is ov564, orient is 90

10-19 09:44:46.779 I/FslCameraHAL(14850): Face Front Camera is saf7113, orient is 270

10-19 09:44:46.779 I/FslCameraHAL(14850): It is the CSI device

10-19 09:44:46.779 I/FslCameraHAL(14850): V4l2CsiDevice: input=1

10-19 09:44:46.779 I/FslCameraHAL(14850): Create the software encoder

10-19 09:44:46.779 I/FslCameraHAL(14850): v4l2 input is 0x1

10-19 09:44:46.959 D/dalvikvm(14953): GC_EXTERNAL_ALLOC freed 193K, 49% free 2998K/5831K, external 4444K/4448K, paused 50ms

10-19 09:44:47.129 I/OMXPlayer(14850): mSurface changed: left: 165, right: 480, top: 0, bottom: 420, rot: 4

10-19 09:44:47.129 I/OMXPlayer(14850): Rotate to 4

10-19 09:44:47.139 D/szipinf (15324): Initializing inflate state

10-19 09:44:47.539 I/FslCameraHAL(14850): device name is /dev/video0

10-19 09:44:47.539 I/FslCameraHAL(14850): sensor name is ov5642_camera

10-19 09:44:47.539 I/FslCameraHAL(14850): mPreviewFormat :NV12

10-19 09:44:47.539 I/FslCameraHAL(14850): mUvcSpecialCaptureFormat :YUYV

10-19 09:44:47.539 I/FslCameraHAL(14850): mPreviewCapturedFormat :NV12

10-19 09:44:47.539 I/FslCameraHAL(14850): mPreviewCapturedFormat is 3231564e

10-19 09:44:47.539 I/FslCameraHAL(14850): the size is 640x480 , the framerate is 30

10-19 09:44:47.539 I/FslCameraHAL(14850): the size is 320x240 , the framerate is 30

10-19 09:44:47.539 I/FslCameraHAL(14850): the size is 720x480 , the framerate is 30

10-19 09:44:47.539 I/FslCameraHAL(14850): the size is 720x576 , the framerate is 30

10-19 09:44:47.539 I/FslCameraHAL(14850): the size is 1280x720 , the framerate is 30

10-19 09:44:47.539 I/FslCameraHAL(14850): the size is 1920x1080 , the framerate is 15

10-19 09:44:47.539 I/FslCameraHAL(14850): the size is 2592x1944 , the framerate is 15

10-19 09:44:47.539 I/FslCameraHAL(14850): ##The supportedPictureSizes is 640x480,320x240,720x480,720x576,1280x720,1920x1080,2592x1944##

10-19 09:44:47.539 I/FslCameraHAL(14850): ##the supportedPreviewSizes is 640x480,320x240,720x480,720x576,1280x720##

10-19 09:44:47.539 I/FslCameraHAL(14850): ##the supportedFPS is 15,30##

10-19 09:44:47.539 I/FslCameraHAL(14850): created the fsl Camera hal

10-19 09:44:47.539 I/FslCameraHAL(14850): ###the mesg enabled is d###

10-19 09:44:47.539 I/FslCameraHAL(14850): Trying to set overlay, but overlay is null!

10-19 09:44:47.539 I/FslOverlay(14861): overlay_createOverlay w 640,h 480,format 18, pid 14861,,gettid() 14868

10-19 09:44:47.539 I/FslOverlay(14861): num_buffer 3 width 640,height 480,format 18,buf_size 460800

10-19 09:44:47.539 I/FslOverlay(14861): V4L setting: format 0x59565955

10-19 09:44:47.549 I/FslOverlay(14861): v4l buf[0] 0x0xede20: vir 0x0x35ac8000,phy 0x95100000, size 770048

10-19 09:44:47.559 I/FslOverlay(14861): v4l buf[1] 0x0xede64: vir 0x0x35b84000,phy 0x94300000, size 770048

10-19 09:44:47.559 I/FslOverlay(14861): v4l buf[2] 0x0xedea8: vir 0x0x35c40000,phy 0x95300000, size 770048

10-19 09:44:47.559 I/FslOverlay(14861): Create overlay instance 0x0x2efdd0 id 0 total 1

10-19 09:44:47.559 I/FslOverlay(14861): overlay_setParameter param 3 value 1

10-19 09:44:47.559 I/FslOverlay(14861): getHandleRef return overlay_handle_t 0x0x2efe10

10-19 09:44:47.559 I/FslOverlay(14861): Overlay pos set: x 317,y 509,w 149,h 201

10-19 09:44:47.559 I/FslOverlay(14861): overlay_setParameter param 4 value 4

10-19 09:44:47.559 I/FslOverlay(14861): overlay_setParameter OVERLAY_TRANSFORM 4

10-19 09:44:47.559 I/FslOverlay(14861): overlay_setParameter param 5 value 21010

10-19 09:44:47.559 I/FslOverlay(14861): overlay_setParameter OVERLAY_ZORDER 21010

10-19 09:44:47.569 I/FslOverlay(14850): overlay_device_open data pid 14850,tid 14859

10-19 09:44:47.569 I/FslOverlay(14850): overlay_initialize overlay_handle 0x0x2c6b8 overlay_data_context_t 0x0x2c728 pid 14850 tid 14859

10-19 09:44:47.569 I/FslOverlay(14850): num_buffer 3 width 640,height 480,format 18, buf_size 460800

10-19 09:44:47.569 I/FslOverlay(14850): Info!get pmem total size 33554432

10-19 09:44:47.579 I/FslOverlay(14850): Allocator total size 1388544, vir addr 0x2c5df000, phy addr 0xae200000

10-19 09:44:47.579 I/FslOverlay(14850): overlay_buffer 0x0x2c830

10-19 09:44:47.579 I/FslOverlay(14850): buffer 0: vir 0x0x2c5df000, phy 0xae200000

10-19 09:44:47.579 I/FslOverlay(14850): buffer 1: vir 0x0x2c650000, phy 0xae271000

10-19 09:44:47.579 I/FslOverlay(14850): buffer 2: vir 0x0x2c6c1000, phy 0xae2e2000

10-19 09:44:47.579 I/FslOverlay(14850): Overlay init success for Id 0

10-19 09:44:47.579 I/FslCameraHAL(14850): Get the overlay to display

10-19 09:44:47.579 I/FslOverlay(14850): Id 0:overlay_data_setParameter push mode 1

10-19 09:44:47.579 I/FslCameraHAL(14850): ###start the capture the fps is 30000###

10-19 09:44:47.579 I/FslCameraHAL(14850): The capture device already open

10-19 09:44:47.599 I/FslCameraHAL(14850): the mode is 0

10-19 09:44:48.099 V/KeyEvent(15422): Reset: android.view.KeyEvent$DispatcherState@2afcf7e0

10-19 09:44:48.109 W/IInputConnectionWrapper(14953): showStatusIcon on inactive InputConnection

10-19 09:44:48.119 I/FslOverlay(14861): Overlay pos set: x 317,y 509,w 149,h 201

10-19 09:44:48.119 I/FslOverlay(14861): overlay_setParameter param 4 value 4

10-19 09:44:48.119 I/FslOverlay(14861): overlay_setParameter OVERLAY_TRANSFORM 4

10-19 09:44:48.119 I/FslOverlay(14861): overlay_setParameter param 5 value 21010

10-19 09:44:48.119 I/FslOverlay(14861): overlay_setParameter OVERLAY_ZORDER 21010

10-19 09:44:48.159 I/OMXPlayer(14850): start player.

10-19 09:44:48.159 I/OMXPlayer(14850): IpulibRender sInParam width 320, height 240,crop x 0, y 0, w 320, h 240, color 842094158

10-19 09:44:48.159 I/OMXPlayer(14850): IpulibRender sOutParam width 315, height 420,crop x 165, y 0, rot: 4, color 1498831189

10-19 09:44:48.179 I/FslOverlay(14861): Mixer thread refill the origin area to black

10-19 09:44:48.219 I/ipulib  (14861): external/linux-lib/ipu/mxc_ipu_hl_lib.c:956 ipu is busy

10-19 09:44:48.219 I/ipulib  (14861): external/linux-lib/ipu/mxc_ipu_hl_lib.c:958  for hope task mode 0x4!

10-19 09:44:48.219 E/FslOverlay(14861): Error!Obj0 mxc_ipu_lib_task_init failed mIPURet -1!

10-19 09:44:48.219 I/ipulib  (14861): external/linux-lib/ipu/mxc_ipu_hl_lib.c:956 ipu is busy

10-19 09:44:48.219 I/ipulib  (14861): external/linux-lib/ipu/mxc_ipu_hl_lib.c:958  for hope task mode 0x4!

10-19 09:44:48.219 E/FslOverlay(14861): Error!Obj0 mxc_ipu_lib_task_init failed mIPURet -1!

10-19 09:44:48.299 I/FslOverlay(14861): V4L STREAMON NOW

10-19 09:44:48.299 I/ipulib  (14861): external/linux-lib/ipu/mxc_ipu_hl_lib.c:956 ipu is busy

10-19 09:44:48.299 I/ipulib  (14861): external/linux-lib/ipu/mxc_ipu_hl_lib.c:958  for hope task mode 0x4!

10-19 09:44:48.299 E/FslOverlay(14861): Error!Obj0 mxc_ipu_lib_task_init failed mIPURet -1!

10-19 09:44:48.329 I/ipulib  (14861): external/linux-lib/ipu/mxc_ipu_hl_lib.c:956 ipu is busy

10-19 09:44:48.329 I/ipulib  (14861): external/linux-lib/ipu/mxc_ipu_hl_lib.c:958  for hope task mode 0x4!

10-19 09:44:48.329 E/FslOverlay(14861): Error!Obj0 mxc_ipu_lib_task_init failed mIPURet -1!

10-19 09:44:48.369 I/ipulib  (14861): external/linux-lib/ipu/mxc_ipu_hl_lib.c:956 ipu is busy

10-19 09:44:48.369 I/ipulib  (14861): external/linux-lib/ipu/mxc_ipu_hl_lib.c:958  for hope task mode 0x4!

10-19 09:44:48.369 E/FslOverlay(14861): Error!Obj0 mxc_ipu_lib_task_init failed mIPURet -1!


9 Replies

1,127 Views
qiang_li-mpu_se
NXP Employee
NXP Employee

    As I know, for video play back and camera preview, the default BSP will set both of them to overlay surface for rendering, but iMX5x has only one hardware overlay, so it can't be used to show two difference images at the same time.

    To support your use case, you need let one of them to use the main frame buffer to render.

    The attached is an example for such use case, but it is not based on Android framework, it calls kernel driver directly.

1,127 Views
ff
Contributor II

hi Qiang LI

when i use your test program,i use the loopback test ,i find out that the camera preview data mix with the android GUI

can you tell me ,the overlay can directly use the /dev/graphics/fb0  and the main frame buffer also can directly use /dev/graphics/fb0。i am not sure the relationship between  overlay and the framebuffer 。

thanks in advance

best regards

Forion

0 Kudos

1,127 Views
qiang_li-mpu_se
NXP Employee
NXP Employee

On iMX5 BSP, the "/dev/graphics/fb2" was used as overlay, and the "/dev/graphics/fb0" was used as main framebuffer.

If you use mxc_vpu_test to draw fb0, and Android also draws fb0, there will be problem, Android used its framework to merge all UI before show to fb0, but the mxc_vpu_test is not maintained by Android framework.

1,127 Views
ff
Contributor II

hi QiangLI

if i want to full accomplish my desire

when i show the camera preview data,i dont what to show the gui,

can i just change the mxc_vpu_test  display code to open the "/dev/graphics/fb2", or  i should first of all stop android framework  draw fb0?

thank you very much for giving such important advice,

best best best regards

0 Kudos

1,127 Views
qiang_li-mpu_se
NXP Employee
NXP Employee

I'm not sure if the mxc_vpu_test had enabled alpha or not, you can disable alpha and draw camera preview data to overlay, then overlay will cover the background Android GUI, no modification in Android framework was needed.

1,127 Views
ff
Contributor II

hi QiangLI

     does it matter if i use v4l2 display(/dev/video16) or ipu display?

would u bother tell me  the difference?

angain thank you very much

it is very difficult for me full understand the source cord wiht out any instruction。

best best regards

0 Kudos

1,127 Views
qiang_li-mpu_se
NXP Employee
NXP Employee

I think both V4l2 display and ipu display can work. V4l2 output follows Linux V4l2 architecture, it only focus on display.
IPU display is based on ipulib, the display is only one feature of ipulib.

1,127 Views
ff
Contributor II

hi Qiang Li

i will give it a try,thanks for your help:smileyhappy:

0 Kudos

1,127 Views
lijian
Contributor I

谢谢!

有没有人做过android上的方案?

0 Kudos