Which device is supported to run Quantized Gemma Model Inference

取消
显示结果 
显示  仅  | 搜索替代 
您的意思是: 

Which device is supported to run Quantized Gemma Model Inference

480 次查看
ramkumarkoppu_p
Contributor III

Hi, 

Out of i.MX RT700 and i.MX 95 devices, which device has full software support to run inference of Gen AI models like Quantized Google's Gemma Model — first in Python, then in C/C++ using the these devices NPU? Specifically:

  • Which device NPU support transformer-based architectures, or is it limited to CNNs?

  • Which inference frameworks are supported for GenAI on this e.IQ platform?

标记 (5)
0 项奖励
回复
1 回复

465 次查看
ramkumarkoppu_p
Contributor III

especially has any of these devices NPU ported by NXP for llama.cpp ?

0 项奖励
回复