i.MX95 Verdin EVKにおけるNPUサポート こんにちは、 tfliteモデルをi.MX95 NPUで動かそうとしています。モデルは変換可能です。ベンチマークで加速は見えますが、出力は全く使えません(顔検出や顔のランドマークモデルで常に同じです)。 次に、このユーザーガイドに従って例を実行しようとしました:https://www.nxp.com/docs/en/user-guide/UG10166.pdf root@imx95-19x19-verdin-47:/usr/bin/tensorflow-lite-2.19.0/examples# ./label_image -m mobilenet_v1_1.0_224_quant.tflite -i grace_hopper.bmp -l labels.txt --external_delegate_path=/usr/lib/libneutron_delegate.so
INFO: Loaded model mobilenet_v1_1.0_224_quant.tflite
INFO: resolved reporter
INFO: EXTERNAL delegate created.
INFO: NeutronDelegate delegate: 1 nodes delegated out of 4 nodes with 1 partitions.
INFO: Neutron delegate version: v1.0.0-f24d08e5, zerocp enabled.
INFO: Applied EXTERNAL delegate.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: invoked
INFO: average time: 0.37 ms -> ご覧の通り推論は正常に進みましたが、ユーザーガイドに記載されているようなモデルの実際の動作を確認する分類はありません。 正しい変換版のSDK 2.2.2を使ってモデルを変換しました: NeutronSDK_2.2.2+LF_6.12.49_2.2.0/neutron-converter --input input/mobilenet_v1_1.0_224_quant.tflite --target imx95 --output output/mobilenet_v1_1.0_224_quant.tflite --dump-statistics
Performance estimates:
Clock Frequency: 0.000000 MHz
Clock cycles per inference: 0
Latency per inference: -nan ms
Inferences per second: -nan
Memory footprint:
Variables size: 0.000000 MB
Constants size: 0.000000 MB
Microcode size: 0.000000 MB
Statistics for NeutronGraph "subgraph_030":
Operators:
Number of Neutron operators = 29
Number of builtin operators = 44
Memory:
Inputs = 150,528 (bytes)
Microcode = 23,944 (bytes)
Weights = 4,329,648 (bytes)
Kernels = 11,088 (bytes)
Outputs = 381,913 (bytes)
Scratch = 380,912 (bytes) (Allocation efficiency: 1)
Total data = 913,353 (bytes) (Inputs + Outputs + Scratch)
Total weights = 4,364,680 (bytes) (Microcode + Weights + Kernels)
Total size = 5,278,033 (bytes) (All)
Latency:
Cycle estimation = 1,066,681 (cycles)
Latency estimation = 1.067 (ms) (@ 1000.000 MHz)
Overall statistics for graph "":
Operators:
Number of operators after import = 31
Number of operators after optimize = 47
Number of operators after extract = 4
Number of Neutron graphs = 1
Number of operators total = 47
Number of operators converted = 44
Number of operators NOT converted = 3
Operator conversion ratio = 44 / 47 = 0.93617
Operators converted = 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,
Memory:
Total data = 532,448 (bytes) (Inputs + Outputs + Intermediate Variable Tensors)
Total weights = 4,364,688 (bytes) (Weights)
Total size = 4,897,136 (bytes) (All)
Latency:
Cycle estimation = 1,066,681 (cycles) (NPU only)
Latency estimation = 1.067 (ms) (@ 1000.000 MHz) (NPU only)
Conversion time:
Optimization = 2.00387 (seconds)
Extraction = 0.0308523 (seconds)
Generation = 5.56016 (seconds)
Total = 7.59489 (seconds) 私のハードウェアとBSPの設定： SOC : iMX ヴェルディン EVK SoM V1.0C キャリア：iMX Verdin EVK v1.2A ( https://www.toradex.com/de/computer-on-modules/verdin-arm-family/nxp-imx95-evaluation-kit?srsltid=AfmBOookfBHOVzIUBJhrVmtYSV45ohtPJy_f2ymYGR956gzD0XzxS0jP ) 私が使用しているビルドについては、以下を参照してください。 MACHINE = "imx95-19x19-verdin" 私はこちらのBSPバージョンを使用しています： https://www.toradex.com/de/news/bsp-layers-reference-images-walnascar ?srsltid=AfmBOoqJF2YHULiIe7sA1L4gPBasmZrepuGIRBLrtcIuFPnz39jctBX6 そしてNXPのレイヤー： メタイムクス rel_imx_6.12.49_2.2.0 カーネルバージョン: 6.12.49-lts-next-g759f4038100f -> 私の主な質問は、私のシリコンリビジョン(A0)が本当にサポートされているのか、あるいはモデルが静かに失敗してひどい出力を出すときに何が原因なのかということです: 顔検出 (UltraFace-Ultraslim、 [1,128,128,3] int8/uint8 入力) は、アンカーごとに[bg_score、face_score、xmin、ymin、xmax、ymax]の単一の融合 (172,6) テンソルを出力します。これは既に NMS 化され、[0,1] に正規化されています。CPU では、これにより顔にぴったりと囲まれた、明確な高信頼度検出 (~0.996) が 1 つ得られますが、NPU では 172 個のアンカーすべてが同一の定数 (~0.50 スコア、~0.227,0.227,0.227 のほぼゼロサイズのボックス) に縮退します。顔ランドマーク (NXP facial_landmarks_35 、 [1,60,60,3] uint8 入力) は、顔の切り抜きに合わせて正規化された 35 個のインターリーブされた (x,y) ポイントの [1,70] テンソルを出力します。CPU では、これらを画像に重ねると認識可能な顔ポイントパターンが形成されますが、NPU では、70 個の値の出力全体がポイントごとに変化する代わりに、単一の繰り返し定数に縮退します。 Yocto Project Re: NPU support on the i.MX95 Verdin EVK 詳細出力： root@imx95-19x19-verdin-4798be6ce85542d2:/usr/bin/tensorflow-lite-2.19.0/examples# ./label_image -m demo_converted -i grace_hopper.bmp -l labels.txt --external_delegate_path=/usr/lib/libneutron_delegate.so -v 1 -r 5
INFO: Loaded model demo_converted
INFO: resolved reporter
INFO: tensors size: 11
INFO: nodes size: 4
INFO: inputs: 1
INFO: input(0) name: input
INFO: 0: MobilenetV1/Logits/SpatialSqueeze, 1001, 9, 0.166099, -62
INFO: 1: MobilenetV1/Predictions/Reshape_1, 1001, 3, 0.00390625, 0
INFO: 2: input, 150528, 3, 0.0078125, 128
INFO: 3: MobilenetV1/Predictions/Reshape_1/requantize, 1001, 9, 0.00390625, -128
INFO: 4: input, 150528, 9, 0.0078125, 0
INFO: 5: NeutronMicrocode, 23944, 3, 0, 0
INFO: 6: NeutronWeights, 4329648, 3, 0, 0
INFO: 7: NeutronKernels, 11088, 3, 0, 0
INFO: 8: NeutronScratch, 380912, 3, 0, 0
INFO: 9: NeutronProfile, 0, 3, 0, 0
INFO: 10: NeutronDebug, 0, 3, 0, 0
INFO: len: 940650
INFO: width, height, channels: 517, 606, 3
INFO: input: 2
INFO: number of inputs: 1
INFO: number of outputs: 1
INFO: EXTERNAL delegate created.
INFO: NeutronDelegate delegate: 1 nodes delegated out of 4 nodes with 1 partitions.
INFO: Neutron delegate version: v1.0.0-f24d08e5, zerocp enabled.
INFO: Applied EXTERNAL delegate.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
Interpreter has 1 subgraphs.
-----------Subgraph-0 has 11 tensors and 5 nodes------------
1 Inputs: [2] -> 150528B (0.14MB)
1 Outputs: [1] -> 1001B (0.00MB)
Tensor ID Name Type AllocType Size (Bytes/MB) Shape MemAddr-Offset
Tensor 0 MobilenetV1/Logits/Spa... kTfLiteInt8 kTfLiteCustom 1001 / 0.00 [1,1001] [-1, -1)
Tensor 1 MobilenetV1/Prediction... kTfLiteUInt8 kTfLiteArenaRw 1001 / 0.00 [1,1001] [151552, 152553)
Tensor 2 input kTfLiteUInt8 kTfLiteArenaRw 150528 / 0.14 [1,224,224,3] [0, 150528)
Tensor 3 MobilenetV1/Prediction... kTfLiteInt8 kTfLiteArenaRw 1001 / 0.00 [1,1001] [150528, 151529)
Tensor 4 input kTfLiteInt8 kTfLiteCustom 150528 / 0.14 [1,224,224,3] [-1, -1)
Tensor 5 NeutronMicrocode kTfLiteUInt8 kTfLiteMmapRo 23944 / 0.02 [23944] [4340768, 4364712)
Tensor 6 NeutronWeights kTfLiteUInt8 kTfLiteMmapRo 4329648 / 4.13 [4329648] [11104, 4340752)
Tensor 7 NeutronKernels kTfLiteUInt8 kTfLiteMmapRo 11088 / 0.01 [11088] [0, 11088)
Tensor 8 NeutronScratch kTfLiteUInt8 kTfLiteArenaRw 380912 / 0.36 [380912] [-1, -1)
Tensor 9 NeutronProfile kTfLiteUInt8 kTfLiteArenaRw 0 / 0.00 [0] [-1, -1)
Tensor 10 NeutronDebug kTfLiteUInt8 kTfLiteArenaRw 0 / 0.00 [0] [-1, -1)
kTfLiteArenaRw Info:
Tensor 2 has the max size 150528 bytes (0.144 MB).
This memory arena is estimated as[0xaaaafd73ffa9, 0xaaaafd71abc0), taking 152553 bytes (0.145 MB).
One possible set of tensors that have non-overlapping memory spaces with each other, and they take up the whole arena:
Tensor 2 -> 3 -> 1.
kTfLiteArenaRwPersistent Info: not holding any allocation.
kTfLiteMmapRo Info:
Tensor 6 has the max size 4329648 bytes (4.129 MB).
This memory arena is estimated as[0xffff7ec299f8, 0xffff7e800050), taking 4364712 bytes (4.163 MB).
One possible set of tensors that have non-overlapping memory spaces with each other, and they take up the whole arena:
Tensor 7 -> 6 -> 5.
kTfLiteDynamic Info: not holding any allocation.
=== Beginning of kTfLiteArenaRw Dump: ===
Total size is 152553 bytes (0.145 MB), holding 3 tensors.
tensor 2: life_span: node [0, 4], size: 150528 bytes (0.144 MB).
tensor 3: life_span: node [2, 3], size: 1001 bytes (0.001 MB).
tensor 1: life_span: node [3, 4], size: 1001 bytes (0.001 MB).
1 tensors are of same max size (150528 B (0.144 MB)): [2]
Per-layer-info in the order of op execution:
Node 0: 150528 bytes (0.144 MB), utilization rate: 98.673%, 1 live tensors: [2]
Node 4: 151529 bytes (0.145 MB), utilization rate: 99.329%, 2 live tensors: [1,2]
Node 2: 151529 bytes (0.145 MB), utilization rate: 99.329%, 2 live tensors: [2,3]
Node 3: 152530 bytes (0.145 MB), utilization rate: 99.985%, 3 live tensors: [1-3]
Top 4 memory-consuming layers:
Node 3: 152530 bytes (0.145 MB), utilization rate: 99.985%, 3 live tensors: [1-3]
Node 4: 151529 bytes (0.145 MB), utilization rate: 99.329%, 2 live tensors: [1,2]
Node 2: 151529 bytes (0.145 MB), utilization rate: 99.329%, 2 live tensors: [2,3]
Node 0: 150528 bytes (0.144 MB), utilization rate: 98.673%, 1 live tensors: [2]
===End of kTfLiteArenaRw Dump: ===
Node 0 Operator Builtin Code 114 QUANTIZE (not delegated)
1 Input Tensors:[2] -> 150528B (0.14MB)
1 Output Tensors:[4] -> 150528B (0.14MB)
Node 1 Operator Custom Name NeutronGraph (delegated by node 4)
4 Input Tensors:[4,5,6,7] -> 0B (0.00MB)
4 Output Tensors:[0,8-10] -> 0B (0.00MB)
Node 2 Operator Builtin Code 25 SOFTMAX (not delegated)
1 Input Tensors:[0] -> 1001B (0.00MB)
1 Output Tensors:[3] -> 1001B (0.00MB)
Node 3 Operator Builtin Code 114 QUANTIZE (not delegated)
1 Input Tensors:[3] -> 1001B (0.00MB)
1 Output Tensors:[1] -> 1001B (0.00MB)
Node 4 Operator Custom Name NeutronDelegate
4 Input Tensors:[4-7] -> 4515208B (4.31MB)
1 Output Tensors:[0] -> 1001B (0.00MB)
Execution plan as the list of 4 nodes invoked in-order: [0,4,2,3]
Among these nodes in the execution plan:
Node 4 is a NeutronDelegate node (0xaaaafd6f5c30), which has delegated 1 nodes: [1]
--------------Subgraph-0 dump has completed--------------
--------------Memory Arena Status Start--------------
Total memory usage: 152553 bytes (0.145 MB)
- Total arena memory usage: 152553 bytes (0.145 MB)
- Total dynamic memory usage: 0 bytes (0.000 MB)
Subgraph#0 Arena (Normal) 152553 (100.00%)
--------------Memory Arena Status End--------------
INFO: invoked
INFO: average time: 0.326 ms
記事全体を表示
