Hostfix: remove not needed params from load_model by qnixsynapse · Pull Request #2209 · janhq/cortex.cpp
added 2 commits
June 12, 2025 10:11The --pooling flag was removed as the mean pooling functionality not needed in chat models. This fixes the regression
When the model path contains both "jan" and "nano" (case-insensitive), automatically add speculative decoding parameters to adjust generation behavior. This improves flexibility by enabling environment-specific configurations without manual parameter tuning. Also includes necessary headers for string manipulation and fixes whitespace in ctx_len handling.
This commit introduces new configuration parameters and their corresponding command-line flags for the local engine. The changes include: - Adding "flash_attn" to ignored parameters - Mapping UI parameters to CLI flags (e.g., cpu_threads → --threads) - Expanding support for various model configuration options These additions enhance the flexibility of the local engine by enabling fine-grained control over performance and behavior through both UI and CLI interfaces.
The condition was updated to include 'qwen' in the check for triggering specific parameters
('--temp', '--top-p', etc.), aligning it with the existing 'jan' and 'nano' validation logic. This allows
the same parameter configuration to apply to 'qwen' models as well as the original keywords.
Removed deprecated parameters such as "dynatemp_exponent" and "ctx_len" handling logic, which were no longer needed. Added "flash_attn" back to the ignored parameters list. Cleaned up the parameter conversion logic by removing conditional blocks for specific model optimizations that are no longer required.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters