imx8mp isp configuration delay

[malik_cisse]

Hello,

we dynamically program the imx8mp ISP after setting up a Gstreamer pipeline. After starting the pipeline, we need to introduce a latency of over 500 milliseconds, as otherwise the ISP configurations are ignored as can be seen in attached code.

bool GstVideoPipeline::start() {
    qDebug() << "GstVideoPipeline::start()";
    GstStateChangeReturn ret = gst_element_set_state(m_pimpl->pipeline, GST_STATE_PLAYING);
    if (ret == GST_STATE_CHANGE_SUCCESS || ret == GST_STATE_CHANGE_ASYNC) {
        qDebug() << "GstVideoPipeline has been started.";

        /**
         * this delay is needed to give the pipeline some time to startup. Otherwise it will not properly
         * handle dynamic reconfiguration.
         */
        QThread::msleep(1000);
        //now ISP can be configured properly
        return true;
    } else {
        qCritical() << "failed to start GstVideoPipeline";
        return false;
    }
}

Are you familiar with this issue?

Ideally, we would like to query a "Ready to be configured" signal from the ISP. However, such a signal does not exist. Do you have any ideas on how we can proceed in this case?

Thank you.

[brian14]

Hi @malik_cisse, 

Thank you for contacting NXP Support.

Could you please help me with the steps to reproduce your problem?
I'm not familiar with this issue, and it could help me to better understand.

Have a great day!

[malik_cisse]

Hi Brian,

Unfortunatelly I cannot provide you easilly with the steps to reproduce this behaviour since this will imply to send you proprietary code. This said, any gstreamer aplication with a-posteriori ISP configuration should showcase the issue.

I am sharing a feedback that has been given to me which could explain the behavior. What is your thoughts on this?

The feedback:

Indeed, the behavior is essentially known, and we can explain it. Upon each start of the pipeline, the ISP userspace driver reads the configuration from the XML file and configures the ISP with these default values. We suspect that new dynamic settings are only considered once this process is completed. This is an assumption as there is limited visibility into the ISP userspace driver. Unfortunately, we cannot provide insight into why this process takes as long as it does.

A possible solution approach could be: The ISP's userspace driver uses V4L2 events to be informed about, for example, the stream start request on the video device and then starts the entire pipeline. It might be possible for the application to attach itself to these events and check if there is an event that indicates that the ISP is now ready for dynamic configurations. Unfortunately, we don't have any other ideas.

BR, 

[brian14]

Hi @malik_cisse, 

Based on your reply, I think it could explain the behavior of your configuration process.
To obtain more information about the relation of the ISP and the V4L2 driver I recommend you have a look on the following documentation:
i.MX 8M Plus Camera and Display Guide (nxp.com)

I hope this information will be helpful.

Have a great day!