Hello Sadatan Vashistha,
You may use a heatsink on the i.MX6UL but it’s not strictly necessary.
I would recommend looking at the i.MX 6 Series Thermal Management Guidelines (Application Note 4579, link below) for more information on thermal management. This document is for the i.MX6 in general and focuses on the i.MX6D/Q, but you would have a much lower power consumption in the i.MX6UL.
I hope this information helps!