Apple M1Max compiles i.MX Linux Kernel faster than i9 desktop

[OliverB]

Setup

Virtual Machine

To build on the M1 MBP I used the VMWare technical preview which can be found here:

https://blogs.vmware.com/teamfusion/2021/09/fusion-for-m1-public-tech-preview-now-available.html

Be sure to read the TP-Testing-Guide, https://communities.vmware.com/t5/Fusion-for-Apple-Silicon-Tech/Fusion-Tech-Preview-Testing-Guide/ta..., as this explains the quirks of the M1 technical preview. The main issue is that the VMWare tools is not supported in the default kernel. This limited the virtual machine to 1024x768 resolution. The guide recommends upgrading the kernel to 5.14 to fix this issue. I did go down that rabbit hole but gave up. The Technical preview is a bit out of date and the kernel upgrade steps ended up bricking my VM. So, I setup a VNC server on the VM and I’ll be using that for development.

I used an Ubuntu 20.04 image for the virtual machine, https://cdimage.ubuntu.com/focal/daily-live/current/focal-desktop-arm64.iso. Since this is a daily build your mileage may vary.

Toolchain

I built a toolchain with Yocto for both the Apple M1 and Intel hosts. But on Apple, Ubuntu VM the kernel can build with the native gcc. 

$ repo init -u https://source.codeaurora.org/external/imx/imx-manifest -b imx-linux-hardknott -m imx-5.10.72-2.2.0.xml
$ repo sync

$ DISTRO=fsl-imx-xwayland MACHINE=imx8qmmek source imx-setup-release.sh -b build-imx8qmmek
$ bitbake core-image-minimal -c populate_sdk

Kernel

I used the i.MX Linux kernel from https://source.codeaurora.org/external/imx/linux-imx. The tag I used was lf-5.10.72-2.2.0. I configured the kernel with “make imx_v8_defconfig”.

Testing

Before each build I flushed the system and file cache as superuser:
$ sync; echo 3 > /proc/sys/vm/drop_caches

For each run I ran

$ time make -jX

Where X was the number of threads that was fastest for a given host. Another interesting observation is that for the Apple silicon the number of make threads seems to be optimal at the number of cores plus 2. For Intel hosts, the optimal number of threads should be set to the number of threads the CPU supports.

Results

Host	CPU	cores	threads (-j)	compile time (s)
Apple 16" MBP M1Max	Apple M1 Max	6	8	393.89
Apple 16" MBP M1Max	Apple M1 Max	8	10	302.17
Dell Precision 3650 Desktop	Intel i9-11900 @ 2.50GHz	8	16	357.33
Dell Precision M4700 Laptop	Intel i7-3740QM CPU @ 2.70GHz	4	8	1205.65
HP z420 Desktop	Intel Xeon E5-2690 @ 2.90GHz	8	16	625.88

The MBP performed well with both 6 and 8 core virtual machines. The MBP did not have any noticeable fan noise or get hot during the builds. Also, I was able to use the browser and surf the web while building. I was not expecting the MBP to beat the i9 Dell Precision. It seems that even a 4-5 core virtual machine may be usable for kernel build flows especially if you don’t need a clean build every time. 

With any benchmark like this, next week or next month a newer, faster, result will be inevitable. I‘m also sure there are some monster machines that can get under 5 minutes for this kernel build. My main goal here was to get my new machine and a new architecture up and running for my daily workflow.

Some open questions I still have:

What is Impact on Battery life?
Is compiling natively less work inherently?
How does the compile time vary with power mode (low versus high versus auto)?