topic Re: imx8mp: NPU / tensorflow-lite in new BSP (6.1) slower 3.8 vs 2.5ms in i.MX Processors

imx8mp: NPU / tensorflow-lite in new BSP (6.1) slower 3.8 vs 2.5ms

arno_0 — Thu, 20 Apr 2023 04:51:48 GMT

For me it seems that the tensorflow lite example delegated to the NPU is getting slower.

With latest BSP (Kernel 6.1) it takes around 50% longer. Any reason?

Re: imx8mp: NPU / tensorflow-lite in new BSP (6.1) slower 3.8 vs 2.5ms

arno_0 — Fri, 21 Apr 2023 05:37:03 GMT

Ok, found in Machine Learning User Guide:

"Known issue: Decreased performance on MobilenetV1, MobileNetV2, VGG16, VGG19, and NasNet Mobile"

But why?

Re: imx8mp: NPU / tensorflow-lite in new BSP (6.1) slower 3.8 vs 2.5ms

Zhiming_Liu — Tue, 25 Apr 2023 06:14:14 GMT

Hi @arno_0

Maybe the VX delegate haven't been updated to compatible with the new features.

You can use the L5.15.X and we will fix this known issue in next release.

Re: imx8mp: NPU / tensorflow-lite in new BSP (6.1) slower 3.8 vs 2.5ms

arno_0 — Tue, 25 Apr 2023 15:02:47 GMT

Is a mixed operation possible? So use 6.1 kernel and just the older delegates and libs?
I need the 6.1 kernel for other tthings.

Re: imx8mp: NPU / tensorflow-lite in new BSP (6.1) slower 3.8 vs 2.5ms

lfant — Wed, 26 Apr 2023 08:04:19 GMT

Hello,

I will also have to use L6.1.x together with a performance-critical application using the NPU: can you please elaborate what "Maybe the VC delegate haven't been updated" exactly means?

When will the release containing the fix that you mentioned be available?

Re: imx8mp: NPU / tensorflow-lite in new BSP (6.1) slower 3.8 vs 2.5ms

lfant — Tue, 30 May 2023 09:47:26 GMT

@Zhiming_Liu : do you have any updates on:

1) the root cause of this performance regression

2) any potential workarounds, if kernel 6.1 needs to be used

3) timeline for a fix of the performance regression

Re: imx8mp: NPU / tensorflow-lite in new BSP (6.1) slower 3.8 vs 2.5ms

arno_0 — Wed, 31 May 2023 06:16:29 GMT

There is an update to L6.1.1_1.0.1 - maybe it contains a fix. If you test this, let us know.

Re: imx8mp: NPU / tensorflow-lite in new BSP (6.1) slower 3.8 vs 2.5ms

lfant — Tue, 04 Jul 2023 12:39:35 GMT

I'm answering the question myself: new release (LF6.1.22_2.0.0) is out which claims to include a fix for the decreased performance (not verified yet if actually true)

@Zhiming_Liu : thanks for the support, and quick response.