AnsweredAssumed Answered

Frame Rate drops during heavy system Load in i.MX6 MIPI-CSI2 camera

Question asked by Rashmi Ghoael Muruganandam on Mar 21, 2018
Latest reply on Mar 24, 2018 by Rashmi Ghoael Muruganandam

Hi,

 

We are using i.Mx6 QuadPlus Nitrogen Board with Kernel Version 3.14.52.
We are using proprietary Camera with our customized driver through MIPI-CSI2 interface.
A small region of interset is set (120 x 120) with minimum exposure and RGB888 format, then the camera product produces 120fps.
For testing we have used a simple command line application and do a continuous streaming.

 

Initially we are getting the frame rate as expected 120fps.
We tried increasing the CPU load during streaming, so we are compiling a piece of code with Makefile (Make -j4) which utilizes all CPU cores.
At that time, the frame rate dropped down to 38fps and it is not recovered back even after the stop of compilation.

 

We observed few things in the mxc v4l2 driver - (mxc_v4l2_capture.c - camera_callback) camera callback implementation.
    For testing, we have added a timer which calculates the number of times the interrupt hits in camera_callback function, which is the frame rate count
and the timer will restart after every 5 seconds.
    
    In camera_callback function, till this point the frame rate remains as 120fps.

 

    spin_lock(&cam->queue_int_lock);
    spin_lock(&cam->dqueue_int_lock);
    fps++; //variable to check fps count
    
    During the execution of below code, the fps rate dropped to 38.
    
    if (!list_empty(&cam->working_q)) {
        do_gettimeofday(&cur_time);

 

        done_frame = list_entry(cam->working_q.next,
                    struct mxc_v4l_frame,
                    queue);

 

        if (done_frame->ipu_buf_num != cam->local_buf_num)
            goto next;

 

        /*
         * Set the current time to done frame buffer's
         * timestamp. Users can use this information to judge
         * the frame's usage.
         */
        done_frame->buffer.timestamp = cur_time;

 

        if (done_frame->buffer.flags & V4L2_BUF_FLAG_QUEUED) {
            done_frame->buffer.flags |= V4L2_BUF_FLAG_DONE;
            done_frame->buffer.flags &= ~V4L2_BUF_FLAG_QUEUED;

 

            /* Added to the done queue */
            list_del(cam->working_q.next);
            list_add_tail(&done_frame->queue, &cam->done_q);

 

            /* Wake up the queue */
            cam->enc_counter++;
            wake_up_interruptible(&cam->enc_queue);
        } else
            pr_err("ERROR: v4l2 capture: camera_callback: "
                "buffer not queued\n");
    }

 

next:
    if (!list_empty(&cam->ready_q)) {
        ready_frame = list_entry(cam->ready_q.next,
                     struct mxc_v4l_frame,
                     queue);
        if (cam->enc_update_eba)
            if (cam->enc_update_eba(
                cam,
                ready_frame->buffer.m.offset) == 0) {
                list_del(cam->ready_q.next);
                list_add_tail(&ready_frame->queue,
                          &cam->working_q);
                ready_frame->ipu_buf_num = cam->local_buf_num;
            }
    } else {
        if (cam->enc_update_eba)
            cam->enc_update_eba(
                cam, cam->dummy_frame.buffer.m.offset);
    }

 

    cam->local_buf_num = (cam->local_buf_num == 0) ? 1 : 0;
    spin_unlock(&cam->dqueue_int_lock);
    spin_unlock(&cam->queue_int_lock);

 

    We need a code patch, to fix this issue.
    Kindly clarify and get back with us ASAP. Thanks in advance.

 

Please refer the below link:

https://github.com/boundarydevices/linux-imx6/blob/boundary-imx_3.14.52_1.1.0_ga/drivers/media/platform/mxc/capture/mxc_v4l2_capture.c
 

Best Regards,

Rashmi

Outcomes