fix: update _evals_common to be compatible with litellm >=1.83.0#6599
fix: update _evals_common to be compatible with litellm >=1.83.0#6599quad2524 wants to merge 6 commits intogoogleapis:mainfrom
Conversation
9590ca7 to
24dff9c
Compare
|
Can you resolve the merge conflicts? Then I can run the workflow checks |
Done |
|
LGTM |
|
Hi, the lint has failed. Can you fix it? |
Fixed linting |
|
Appreciate the quick turnarounds on the PR feedback. We have received some urgent internal guidance that we want to maintain the same lower bound, but we need to have a stricter upper bound on a minor version given the impact that the litellm vulnerability had. I plan to proceed with #6617, and then we can merge in your PR with the updates to the evals files. Is that okay with you? |
|
That works for me. Thanks! |
|
Great, #6617 is now merged in. Can you revert your changes in setup.py? |
-- 24dff9c by Alan <argnarf@gmail.com>: fix: update litellm to >=1.83.0 to resolve security vulnerability -- 8cda891 by Alan <argnarf@gmail.com>: fix linting errors -- fb1efd0 by Alan <argnarf@gmail.com>: Removed version pinning from PR COPYBARA_INTEGRATE_REVIEW=#6599 from quad2524:issue-6598-litellm-version 18c9d68 PiperOrigin-RevId: 903440345
|
Thanks for the contribution! It's now merged as ac5a5e4 |
-- 68eaca8 by Casey West <caseywest@google.com>: fix(deps): bump litellm cap to >=1.83.7 for additional CVE remediation The current cap of <1.83.7 (set in #6617) clears CVE-2026-35030 in litellm 1.83.0 but excludes four additional CVEs patched in 1.83.7: GHSA-r75f-5x8p-qvmc, GHSA-jjhc-v7c2-5hh6, GHSA-xqmj-j6mv-4862, GHSA-69x8-hrgq-fjj8 (disclosed 2026-04-11/24). Required by google/adk-python#5489, which pins litellm>=1.83.7,<=1.83.14 in its own dependencies and currently fails to install alongside google-cloud-aiplatform[evaluation] because of this cap. Requested by @sasha-gitg in the ADK PR review. The code adaptation for litellm 1.83.x already shipped in #6599 (vertexai/_genai/_evals_common.py via get_llm_provider), so this is purely a version-pin change. Verified: nox -s lint and nox -s lint_setup_py pass; the litellm-touching tests in tests/unit/vertexai/genai/test_evals.py pass against installed litellm at both 1.83.7 (lower bound) and 1.83.14 (upper bound). COPYBARA_INTEGRATE_REVIEW=#6645 from cwest:topic/bump-litellm-cap 638e6fa PiperOrigin-RevId: 906452948
Summary
This PR updates the litellm dependency to version 1.83.0 or higher. This upgrade is necessary to bypass known security vulnerabilities present in version 1.82.7.
Because litellm introduced changes to how models and providers are validated, I have also updated the internal utility functions and associated tests to maintain compatibility.
Changes
Dependency Update: Bumped litellm version requirement in setup files.
Core Logic (_evals_common): Updated _is_litellm_model to utilize the newer get_llm_provider pattern. This ensures we accurately validate model strings against LiteLLM’s supported provider list.
Test Suite:
Refactored mocks to account for the new return signature of litellm.get_llm_provider, which now includes additional metadata (model, provider, etc.).
Updated get_valid_models mocks to ensure consistent behavior during unit testing.
Fixes #6598 🦕