how to understand surround view application algorithm

[双朋赵]

Hi,

I am porting the surround view application to our project based on i.MQ8QM MEK, we found that the demo must be optimised a lot to meet the requirement of surround view function in car. Regarding the algorithm provided by NXP, we can get all the source code, but the principle behind those code is hard to understand, can you provide some more reference to understand it quickly, espcially the mask generation part, we still can not understand how to use the 3D coordinate and the 2D coordinate. we need the detailed explanation about this part, can you give us some more advice ?

BTW, I am still not understand the following two functions, we need to understand every line code and it's principle.

void Masks::getSeamPoints(vector<Point3f> &polygon1, vector<Point3f> &polygon2, Seam &seam, int rotation)

void Masks::createMasks(vector<Camera*> &cameras, vector< vector<Point3f> > &seam_points, float smothing)

Best Regards

Shuangpeng

[jimmychan]

We have the Reference Manual (SVARM) and the User's Guide (SVAUG) talk about the Surround View Application.

Surround View Application | NXP 

If you want to have the details explanation of it, you may better to apply the NXP Professional Service.

NXP Professional Services | NXP 

Have a nice day.

[jcaballeros]

Hi everyone,

Adding an updated reference for anyone who comes across this older discussion while working on multi-camera surround-view or panoramic video stitching on embedded platforms.

The original questions about coordinate transformations, seam points, and mask generation are important parts of any stitching pipeline. In general, the process involves calibrating the cameras, mapping their images into a common coordinate space, identifying overlapping regions, and using masks to blend those regions into a continuous output.

For developers implementing a similar system today, it may also be worth looking into GPU-accelerated stitching integrated directly into a GStreamer pipeline, rather than building and optimizing every processing stage from scratch.

At RidgeRun, we've been working on this through our Video Stitcher 2.0 solution, which supports multi-camera panoramic stitching on embedded platforms including NXP i.MX95.

The implementation uses predefined camera calibration, geometric remapping, and mask-based blending, with GStreamer integration through rrstitcher and rrglstitcher. OpenGL acceleration is available on i.MX95.

We've documented the architecture, calibration workflow, GStreamer integration, examples, and performance measurements here:

https://developer.ridgerun.com/wiki/index.php/Video_Stitching_for_Embedded_Systems

While this doesn't explain the exact implementation of NXP's original getSeamPoints() and createMasks() functions, it may be a useful reference for developers exploring modern embedded surround-view and panoramic stitching implementations.

Hope this helps anyone working on a similar project.

Best regards