Which device is supported to run Quantized Gemma Model Inference

キャンセル
次の結果を表示 
表示  限定  | 次の代わりに検索 
もしかして: 

Which device is supported to run Quantized Gemma Model Inference

707件の閲覧回数
ramkumarkoppu_p
Contributor III

Hi, 

Out of i.MX RT700 and i.MX 95 devices, which device has full software support to run inference of Gen AI models like Quantized Google's Gemma Model — first in Python, then in C/C++ using the these devices NPU? Specifically:

  • Which device NPU support transformer-based architectures, or is it limited to CNNs?

  • Which inference frameworks are supported for GenAI on this e.IQ platform?

タグ(5)
0 件の賞賛
返信
1 返信

692件の閲覧回数
ramkumarkoppu_p
Contributor III

especially has any of these devices NPU ported by NXP for llama.cpp ?

0 件の賞賛
返信