fix(gemini): map thinkingLevel to thinkingBudget for Gemini 2.5 models#3005
fix(gemini): map thinkingLevel to thinkingBudget for Gemini 2.5 models#3005maxronner wants to merge 3 commits intoolimorris:mainfrom
Conversation
Gemini 2.5 models expect `generationConfig.thinkingConfig` to be an
object with a `thinkingBudget` field, not a raw level string. Stage
`thinkingLevel` in `adapter.temp` and serialize it in `form_parameters`,
emitting `{ thinkingBudget = N }` for models tagged with a budget tier
via `opts.thinking_budget`. Other reasoning models keep the existing
`{ thinkingLevel = "<level>" }` shape.
On Gemini 2.5 Pro, `none` maps to the minimum valid budget (128) because
the API rejects `thinkingBudget = 0` for that model. Flash and Flash-Lite
accept 0 and map accordingly.
| flash_lite = { none = 0, low = 512, medium = 8192, high = 24576 }, | ||
| } | ||
|
|
||
| local function resolve_thinking_config(opts, level) |
There was a problem hiding this comment.
Can we add some type annotations here? And it's common practice in the plugin to have level, opts as the parameter order
| ---`opts.thinking_budget`, which selects a row from `THINKING_LEVEL_BUDGETS`. | ||
| ---Other reasoning models continue to use the `{ thinkingLevel = "<level>" }` shape. | ||
| --- | ||
| ---Gemini 2.5 Pro cannot fully disable thinking: the API rejects thinkingBudget = 0, |
There was a problem hiding this comment.
This is excessive commenting. A single line explaining what they do and a link to Google's own docs will suffice here
I've just seen that these models are being shutdown in 2-3 months time |
|
Happy to fix the remaining blockers if you still want to take this for the 2.5 models. I can see both sides, though. With Gemini 2.5 this close to EOL, adding model-specific handling here probably only makes sense if we’re okay with a bit of temporary special-casing. If that’s the preference, I think it’d be reasonable to land this with a follow-up note to remove the internal mapping once 2.5 support is gone. |
|
I'll give this a test later on tonight and merge. I think there's still value in having this in the adapter. |
2 participants
Description
This fixes Gemini reasoning config serialization for
gemini-2.5-*models.Previously, the adapter mapped
thinkingLeveldirectly intogenerationConfig.thinkingConfig, which caused Gemini 2.5 requests to fail because those models expectthinkingConfigto contain a numericthinkingBudget. This change movesthinkingLevelhandling intoform_parameters, where the adapter now builds the correctthinkingConfigshape for Gemini 2.5 models while preserving the existing{ thinkingLevel = ... }shape for other reasoning models, including Gemini 3.x.A model-specific budget mapping is added for Gemini 2.5 Pro, Flash, and Flash-Lite. Serialization is also guarded so
thinkingConfigis only added for reasoning models and only whenthinkingLevelis present.Test coverage was added for the Gemini 2.5 budget mappings, the existing Gemini 3.x
thinkingLevelshape, non-reasoning models, preservation of existinggenerationConfigkeys, and the absence ofthinkingConfigwhenthinkingLevelis unset.Note: for
gemini-2.5-pro,nonemaps to128, not0, because the API rejectsthinkingBudget = 0.Docs were regenerated via
make allper the project guidelines, otherwise untouched.References
https://ai.google.dev/gemini-api/docs/thinking
AI Usage
I used
claude-opus-4-6to cross-check this change against the request-shaping patterns used inlua/codecompanion/adapters/http/anthropic.luaand to draft the test cases. I reviewed and revised those tests before including them. I also usedgpt-5-4for wording and review feedback, but the implementation and final changes are mine.Related Issue(s)
#2994
Screenshots
N/A
Checklist
make allto ensure docs are generated, tests pass and StyLua has formatted the code