Hostfix: remove not needed params from load_model by qnixsynapse · Pull Request #2209 · janhq/cortex.cpp

added 2 commits

June 12, 2025 10:11
The --pooling flag was removed as the mean pooling functionality not needed in chat models. This fixes the regression
Adds support for the ctx_len parameter by appending --ctx-size with its value. Removed outdated parameter mappings from the kParamsMap to reflect current implementation details and ensure consistency.
When the model path contains both "jan" and "nano" (case-insensitive), automatically add
speculative decoding parameters to adjust generation behavior. This improves
flexibility by enabling environment-specific configurations without manual
parameter tuning. Also includes necessary headers for string manipulation and
fixes whitespace in ctx_len handling.
The comment was redundant as the code's purpose is clear without it, improving readability.
This commit introduces new configuration parameters and their corresponding command-line flags for the local engine. The changes include:
- Adding "flash_attn" to ignored parameters
- Mapping UI parameters to CLI flags (e.g., cpu_threads → --threads)
- Expanding support for various model configuration options

These additions enhance the flexibility of the local engine by enabling fine-grained control over performance and behavior through both UI and CLI interfaces.
The condition was updated to include 'qwen' in the check for triggering specific parameters
('--temp', '--top-p', etc.), aligning it with the existing 'jan' and 'nano' validation logic. This allows
the same parameter configuration to apply to 'qwen' models as well as the original keywords.
Removed deprecated parameters such as "dynatemp_exponent" and "ctx_len" handling logic,
which were no longer needed. Added "flash_attn" back to the ignored parameters list.
Cleaned up the parameter conversion logic by removing conditional blocks for
specific model optimizations that are no longer required.

@louis-jan

@louis-jan

@louis-jan

louis-jan