In my experience, I haven't found a way to change the video rotation in a hardware register. In software it can be done using gstreamer and the videoflip pipeline element. If you would like to render the video stream in an app (in my case, a Qt app) you can create a gstreamer pipeline and poll it for an image, store the image buffer into a QImage, and then use the QImage::mirrored or QImage::transformed functions. This is useful if, for instance, you want to do some sort of compute on the un-flipped video image but want to render the flipped one for users. Doing this on the i.MX8QXP, we ran into performance issues if the frame rate was too high, but I'm pretty sure the i.MX8QM has better video capabilities.
Hope that helps!
Nolan Z