Check it out!
Use the instructions from:
https://android-build.linaro.org/builds/~linaro-android/staging-imx53-12.01-release/#build=2
but this build:
https://android-build.linaro.org/builds/~pfefferz/bisect-imx53-mouse/
then the mouse will work. Enjoy!