Bug: Bug: `max_tokens` is not forwarded to Custom (OpenAI-compatible) Providers

Elvean does construct a full parameter payload — sampling parameters (`temperature`, top_p) and tool definitions are forwarded correctly. Only max_tokens is omitted, so the slider value never reaches the API. Row 2 reproduces the behaviour exactly. Row 1 shows the backend handles long generations without issue when the parameter is present. ## Controlled A/B through a logging proxy Elvean was pointed at a small proxy in front of mlx_lm.server. The proxy logs the incoming payload and can optionally inject max_tokens. Nothing in Elvean was changed between the two runs — same conversation settings, same Max Tokens slider value, same prompt. Run A — proxy passes through unchanged: ``` keys : ['messages', 'model', 'stream', 'stream_options', 'temperature', 'tool_choice', 'tools', 'top_p'] max_tokens : ABSENT ``` Result: output stops mid-sentence inside the reasoning block.

Please authenticate to join the conversation.

Upvoters
Status

In Review

Board
💡

Feature Request

Date

About 7 hours ago

Author

Nathan

Subscribe to post

Get notified by email when there are changes.