DNPU Training Hub

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

DNPU Training Hub

Discussions

Sort by:
Applies to: LLM Edge Studio Go Point Demo Workaround: Requires custom llm_params configuration The model Qwen2.5-Coder-1.5B is not supported out-of-the-box in LLM Edge Studio using the default configuration. The model loading will fail when attempting to run the model with the default  server_config.json fix: update the  llm_params  section in  server_config.json  with compatible sampling values: - Open config file: vi llm-edge-studio/usr/share/llm-edge-studio/server_config.json - Replace the parameters as follows: "llm_params": { - "temperature": 0.0, - "top_k": 0, - "top_p": 0.0, + "temperature": 1.0, + "top_k": 50, + "top_p": 0.95 } Qwen2.5-Coder-1.5B should run with proper output quality and sampling behavior after applying the updated configuration.  Remarks:  This configuration enables proper behavior for Qwen2.5-Coder-1.5B It may generate warning messages when running Qwen2.5-Instruct-7B, but the model will continue to function normally BR,
View full article
Applies to: LLM / VLM Edge Studio Go Point Demo Workaround: Update demos via deb package In the current BSP release, users may encounter a runtime failure when the requesting inference metrics after running an inference in the LLM / VLM Edge Studio demo, as a result, metrics are not displayed at all. This issue is caused by a change in the metrics response format from the AAF Connector, which is not compatible with the current demo version.  To fix, update the demo by installed the patched Debian package attached to this thread.  - Download and copy deb packages to target board:  scp vlm-edge-studio_1.0.1.deb [email protected]: scp llm-edge-studio_2.0.1.deb [email protected]: - Overwrite the demo installation: dpkg -i --force-overwrite vlm-edge-studio_1.0.1.deb dpkg -i --force-overwrite llm-edge-studio_2.0.1.deb BR,
View full article
  Applies to: VLM Edge Studio Demo using Qwen‑VL backend Workaround: Apply the attached  apply_qwenvl_fix.sh  script   In the current VLM Edge Studio Demo, users may encounter a runtime failure when starting the Qwen‑VL Vision Language Model. The issue is caused by incorrect processor initialization, where the code attempts to load tokenizer and image processor assets directly from a hard‑coded Hugging Face model reference instead of the locally deployed tokenizer path. This typically manifests as: Model startup failures Tokenizer / processor load errors Inconsistent behavior when running in offline environments To fix no configuration changes are required, copy and apply the provided patch on the board: root@imx95evk:~# chmod a+x apply_qwenvl_fix.sh root@imx95evk:~# sh apply_qwenvl_fix.sh remarks: please make sure the path in the script match your installation: PROD_FILE="/usr/share/eiq/aaf-connector/venv/lib/python3.13/site-packages/eiq_aaf_connector/llm_engines/QwenVL.py"
View full article