On the android platform, when I write one activity that show the camera preview and video playback on one screen, I got some errors:
10-19 09:44:48.179 I/FslOverlay(14861): Mixer thread refill the origin area to black
10-19 09:44:48.219 I/ipulib (14861): external/linux-lib/ipu/mxc_ipu_hl_lib.c:956 ipu is busy
10-19 09:44:48.219 I/ipulib (14861): external/linux-lib/ipu/mxc_ipu_hl_lib.c:958 for hope task mode 0x4!
10-19 09:44:48.219 E/FslOverlay(14861): Error!Obj0 mxc_ipu_lib_task_init failed mIPURet -1!
10-19 09:44:48.219 I/ipulib (14861): external/linux-lib/ipu/mxc_ipu_hl_lib.c:956 ipu is busy
The fully log looks like the following:
10-19 09:44:46.179 I/ActivityManager(14861): Starting: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] flg=0x10000000 cmp=com.ceict.videoclientdemo/.VideoChat } from pid 15407
10-19 09:44:46.189 I/SurfaceFlinger(14861): mHw->getFormat() 4
10-19 09:44:46.219 D/AndroidRuntime(15407): Shutting down VM
10-19 09:44:46.219 I/ActivityManager(14861): Start proc com.ceict.videoclientdemo for activity com.ceict.videoclientdemo/.VideoChat: pid=15422 uid=10049 gids={1006, 3003, 1015}
10-19 09:44:46.229 D/dalvikvm(15407): GC_CONCURRENT freed 102K, 69% free 320K/1024K, external 0K/0K, paused 1ms+0ms
10-19 09:44:46.229 D/jdwp (15407): Got wake-up signal, bailing out of select
10-19 09:44:46.229 D/dalvikvm(15407): Debugger has detached; object registry had 1 entries
10-19 09:44:46.289 I/WindowManager(14861): Setting rotation to 1, animFlags=1
10-19 09:44:46.309 I/ActivityManager(14861): Config changed: { scale=1.0 imsi=0/0 loc=zh_CN touch=3 keys=1/1/2 nav=1/1 orien=2 layout=268435490 uiMode=17 seq=3}
10-19 09:44:46.339 W/com.skype.rover.MainApp(15144): onConfigurationChanged changed:{ scale=1.0 imsi=0/0 loc=zh_CN touch=3 keys=1/1/2 nav=1/1 orien=2 layout=268435490 uiMode=17 seq=3}
10-19 09:44:46.359 V/KeyEvent(14953): Reset: android.view.KeyEvent$DispatcherState@2aff1d20
10-19 09:44:46.619 I/System.out(15422): CameraPreview
10-19 09:44:46.619 I/System.out(15422): Player
10-19 09:44:46.689 I/ActivityManager(14861): Displayed com.ceict.videoclientdemo/.VideoChat: +470ms
10-19 09:44:46.769 I/OMXPlayer(14850): Loading content: /mnt/sdcard/pic/camerarecorder.mp4
10-19 09:44:46.779 I/FslCameraHAL(14850): Face Back Camera is ov564, orient is 90
10-19 09:44:46.779 I/FslCameraHAL(14850): Face Front Camera is saf7113, orient is 270
10-19 09:44:46.779 I/FslCameraHAL(14850): It is the CSI device
10-19 09:44:46.779 I/FslCameraHAL(14850): V4l2CsiDevice: input=1
10-19 09:44:46.779 I/FslCameraHAL(14850): Create the software encoder
10-19 09:44:46.779 I/FslCameraHAL(14850): v4l2 input is 0x1
10-19 09:44:46.959 D/dalvikvm(14953): GC_EXTERNAL_ALLOC freed 193K, 49% free 2998K/5831K, external 4444K/4448K, paused 50ms
10-19 09:44:47.129 I/OMXPlayer(14850): mSurface changed: left: 165, right: 480, top: 0, bottom: 420, rot: 4
10-19 09:44:47.129 I/OMXPlayer(14850): Rotate to 4
10-19 09:44:47.139 D/szipinf (15324): Initializing inflate state
10-19 09:44:47.539 I/FslCameraHAL(14850): device name is /dev/video0
10-19 09:44:47.539 I/FslCameraHAL(14850): sensor name is ov5642_camera
10-19 09:44:47.539 I/FslCameraHAL(14850): mPreviewFormat :NV12
10-19 09:44:47.539 I/FslCameraHAL(14850): mUvcSpecialCaptureFormat :YUYV
10-19 09:44:47.539 I/FslCameraHAL(14850): mPreviewCapturedFormat :NV12
10-19 09:44:47.539 I/FslCameraHAL(14850): mPreviewCapturedFormat is 3231564e
10-19 09:44:47.539 I/FslCameraHAL(14850): the size is 640x480 , the framerate is 30
10-19 09:44:47.539 I/FslCameraHAL(14850): the size is 320x240 , the framerate is 30
10-19 09:44:47.539 I/FslCameraHAL(14850): the size is 720x480 , the framerate is 30
10-19 09:44:47.539 I/FslCameraHAL(14850): the size is 720x576 , the framerate is 30
10-19 09:44:47.539 I/FslCameraHAL(14850): the size is 1280x720 , the framerate is 30
10-19 09:44:47.539 I/FslCameraHAL(14850): the size is 1920x1080 , the framerate is 15
10-19 09:44:47.539 I/FslCameraHAL(14850): the size is 2592x1944 , the framerate is 15
10-19 09:44:47.539 I/FslCameraHAL(14850): ##The supportedPictureSizes is 640x480,320x240,720x480,720x576,1280x720,1920x1080,2592x1944##
10-19 09:44:47.539 I/FslCameraHAL(14850): ##the supportedPreviewSizes is 640x480,320x240,720x480,720x576,1280x720##
10-19 09:44:47.539 I/FslCameraHAL(14850): ##the supportedFPS is 15,30##
10-19 09:44:47.539 I/FslCameraHAL(14850): created the fsl Camera hal
10-19 09:44:47.539 I/FslCameraHAL(14850): ###the mesg enabled is d###
10-19 09:44:47.539 I/FslCameraHAL(14850): Trying to set overlay, but overlay is null!
10-19 09:44:47.539 I/FslOverlay(14861): overlay_createOverlay w 640,h 480,format 18, pid 14861,,gettid() 14868
10-19 09:44:47.539 I/FslOverlay(14861): num_buffer 3 width 640,height 480,format 18,buf_size 460800
10-19 09:44:47.539 I/FslOverlay(14861): V4L setting: format 0x59565955
10-19 09:44:47.549 I/FslOverlay(14861): v4l buf[0] 0x0xede20: vir 0x0x35ac8000,phy 0x95100000, size 770048
10-19 09:44:47.559 I/FslOverlay(14861): v4l buf[1] 0x0xede64: vir 0x0x35b84000,phy 0x94300000, size 770048
10-19 09:44:47.559 I/FslOverlay(14861): v4l buf[2] 0x0xedea8: vir 0x0x35c40000,phy 0x95300000, size 770048
10-19 09:44:47.559 I/FslOverlay(14861): Create overlay instance 0x0x2efdd0 id 0 total 1
10-19 09:44:47.559 I/FslOverlay(14861): overlay_setParameter param 3 value 1
10-19 09:44:47.559 I/FslOverlay(14861): getHandleRef return overlay_handle_t 0x0x2efe10
10-19 09:44:47.559 I/FslOverlay(14861): Overlay pos set: x 317,y 509,w 149,h 201
10-19 09:44:47.559 I/FslOverlay(14861): overlay_setParameter param 4 value 4
10-19 09:44:47.559 I/FslOverlay(14861): overlay_setParameter OVERLAY_TRANSFORM 4
10-19 09:44:47.559 I/FslOverlay(14861): overlay_setParameter param 5 value 21010
10-19 09:44:47.559 I/FslOverlay(14861): overlay_setParameter OVERLAY_ZORDER 21010
10-19 09:44:47.569 I/FslOverlay(14850): overlay_device_open data pid 14850,tid 14859
10-19 09:44:47.569 I/FslOverlay(14850): overlay_initialize overlay_handle 0x0x2c6b8 overlay_data_context_t 0x0x2c728 pid 14850 tid 14859
10-19 09:44:47.569 I/FslOverlay(14850): num_buffer 3 width 640,height 480,format 18, buf_size 460800
10-19 09:44:47.569 I/FslOverlay(14850): Info!get pmem total size 33554432
10-19 09:44:47.579 I/FslOverlay(14850): Allocator total size 1388544, vir addr 0x2c5df000, phy addr 0xae200000
10-19 09:44:47.579 I/FslOverlay(14850): overlay_buffer 0x0x2c830
10-19 09:44:47.579 I/FslOverlay(14850): buffer 0: vir 0x0x2c5df000, phy 0xae200000
10-19 09:44:47.579 I/FslOverlay(14850): buffer 1: vir 0x0x2c650000, phy 0xae271000
10-19 09:44:47.579 I/FslOverlay(14850): buffer 2: vir 0x0x2c6c1000, phy 0xae2e2000
10-19 09:44:47.579 I/FslOverlay(14850): Overlay init success for Id 0
10-19 09:44:47.579 I/FslCameraHAL(14850): Get the overlay to display
10-19 09:44:47.579 I/FslOverlay(14850): Id 0:overlay_data_setParameter push mode 1
10-19 09:44:47.579 I/FslCameraHAL(14850): ###start the capture the fps is 30000###
10-19 09:44:47.579 I/FslCameraHAL(14850): The capture device already open
10-19 09:44:47.599 I/FslCameraHAL(14850): the mode is 0
10-19 09:44:48.099 V/KeyEvent(15422): Reset: android.view.KeyEvent$DispatcherState@2afcf7e0
10-19 09:44:48.109 W/IInputConnectionWrapper(14953): showStatusIcon on inactive InputConnection
10-19 09:44:48.119 I/FslOverlay(14861): Overlay pos set: x 317,y 509,w 149,h 201
10-19 09:44:48.119 I/FslOverlay(14861): overlay_setParameter param 4 value 4
10-19 09:44:48.119 I/FslOverlay(14861): overlay_setParameter OVERLAY_TRANSFORM 4
10-19 09:44:48.119 I/FslOverlay(14861): overlay_setParameter param 5 value 21010
10-19 09:44:48.119 I/FslOverlay(14861): overlay_setParameter OVERLAY_ZORDER 21010
10-19 09:44:48.159 I/OMXPlayer(14850): start player.
10-19 09:44:48.159 I/OMXPlayer(14850): IpulibRender sInParam width 320, height 240,crop x 0, y 0, w 320, h 240, color 842094158
10-19 09:44:48.159 I/OMXPlayer(14850): IpulibRender sOutParam width 315, height 420,crop x 165, y 0, rot: 4, color 1498831189
10-19 09:44:48.179 I/FslOverlay(14861): Mixer thread refill the origin area to black
10-19 09:44:48.219 I/ipulib (14861): external/linux-lib/ipu/mxc_ipu_hl_lib.c:956 ipu is busy
10-19 09:44:48.219 I/ipulib (14861): external/linux-lib/ipu/mxc_ipu_hl_lib.c:958 for hope task mode 0x4!
10-19 09:44:48.219 E/FslOverlay(14861): Error!Obj0 mxc_ipu_lib_task_init failed mIPURet -1!
10-19 09:44:48.219 I/ipulib (14861): external/linux-lib/ipu/mxc_ipu_hl_lib.c:956 ipu is busy
10-19 09:44:48.219 I/ipulib (14861): external/linux-lib/ipu/mxc_ipu_hl_lib.c:958 for hope task mode 0x4!
10-19 09:44:48.219 E/FslOverlay(14861): Error!Obj0 mxc_ipu_lib_task_init failed mIPURet -1!
10-19 09:44:48.299 I/FslOverlay(14861): V4L STREAMON NOW
10-19 09:44:48.299 I/ipulib (14861): external/linux-lib/ipu/mxc_ipu_hl_lib.c:956 ipu is busy
10-19 09:44:48.299 I/ipulib (14861): external/linux-lib/ipu/mxc_ipu_hl_lib.c:958 for hope task mode 0x4!
10-19 09:44:48.299 E/FslOverlay(14861): Error!Obj0 mxc_ipu_lib_task_init failed mIPURet -1!
10-19 09:44:48.329 I/ipulib (14861): external/linux-lib/ipu/mxc_ipu_hl_lib.c:956 ipu is busy
10-19 09:44:48.329 I/ipulib (14861): external/linux-lib/ipu/mxc_ipu_hl_lib.c:958 for hope task mode 0x4!
10-19 09:44:48.329 E/FslOverlay(14861): Error!Obj0 mxc_ipu_lib_task_init failed mIPURet -1!
10-19 09:44:48.369 I/ipulib (14861): external/linux-lib/ipu/mxc_ipu_hl_lib.c:956 ipu is busy
10-19 09:44:48.369 I/ipulib (14861): external/linux-lib/ipu/mxc_ipu_hl_lib.c:958 for hope task mode 0x4!
10-19 09:44:48.369 E/FslOverlay(14861): Error!Obj0 mxc_ipu_lib_task_init failed mIPURet -1!
As I know, for video play back and camera preview, the default BSP will set both of them to overlay surface for rendering, but iMX5x has only one hardware overlay, so it can't be used to show two difference images at the same time.
To support your use case, you need let one of them to use the main frame buffer to render.
The attached is an example for such use case, but it is not based on Android framework, it calls kernel driver directly.
hi Qiang LI
when i use your test program,i use the loopback test ,i find out that the camera preview data mix with the android GUI
can you tell me ,the overlay can directly use the /dev/graphics/fb0 and the main frame buffer also can directly use /dev/graphics/fb0。i am not sure the relationship between overlay and the framebuffer 。
thanks in advance
best regards
Forion
On iMX5 BSP, the "/dev/graphics/fb2" was used as overlay, and the "/dev/graphics/fb0" was used as main framebuffer.
If you use mxc_vpu_test to draw fb0, and Android also draws fb0, there will be problem, Android used its framework to merge all UI before show to fb0, but the mxc_vpu_test is not maintained by Android framework.
hi QiangLI
if i want to full accomplish my desire
when i show the camera preview data,i dont what to show the gui,
can i just change the mxc_vpu_test display code to open the "/dev/graphics/fb2", or i should first of all stop android framework draw fb0?
thank you very much for giving such important advice,
best best best regards
I'm not sure if the mxc_vpu_test had enabled alpha or not, you can disable alpha and draw camera preview data to overlay, then overlay will cover the background Android GUI, no modification in Android framework was needed.
hi QiangLI
does it matter if i use v4l2 display(/dev/video16) or ipu display?
would u bother tell me the difference?
angain thank you very much
it is very difficult for me full understand the source cord wiht out any instruction。
best best regards
I think both V4l2 display and ipu display can work. V4l2 output follows Linux V4l2 architecture, it only focus on display.
IPU display is based on ipulib, the display is only one feature of ipulib.
hi Qiang Li
i will give it a try,thanks for your help:smileyhappy:
谢谢!
有没有人做过android上的方案?