2385176_en-US

[#1]

LLM Edge Studio - Qwen2.5-Coder-1.5B not supported out-of-the-box

Applies to: LLM Edge Studio Go Point Demo
Workaround: Requires custom llm_params configuration

The model Qwen2.5-Coder-1.5B is not supported out-of-the-box in LLM Edge Studio using the default configuration. The model loading will fail when attempting to run the model with the default server_config.json

fix: update the llm_params section in server_config.json with compatible sampling values:

- Open config file:

vi llm-edge-studio/usr/share/llm-edge-studio/server_config.json

- Replace the parameters as follows:

"llm_params": {
-    "temperature": 0.0,
-    "top_k": 0,
-    "top_p": 0.0,
+    "temperature": 1.0,
+    "top_k": 50,
+    "top_p": 0.95
}

Qwen2.5-Coder-1.5B should run with proper output quality and sampling behavior after applying the updated configuration. 

Remarks: 

• This configuration enables proper behavior for Qwen2.5-Coder-1.5B
• It may generate warning messages when running Qwen2.5-Instruct-7B, but the model will continue to function normally

BR,

ARA240DNPU