(fix) auth checks - Add regex matching for models on virtual keys / teams

[ishaan-jaff]

Add regex matching for models on virtual keys / teams

E2E Scenario

The following flow was raising an error - Fixed with this PR

Key Generate

curl -X POST https://endpoint/key/generate \
  -H 'Authorization: Bearer sk-token' \
  -H 'Content-Type: application/json' \
  --data-raw '{"models": ["llmengine/*"]}'

Chat Completions Request with new key

curl -X POST https://endpoint/v1/chat/completions \
  -H "Authorization: Bearer generated-key" \
  -d '{
    "model": "llmengine/specific-model",
    "messages": [{"role": "user", "content": "Test prompt"}]
  }'

Error Raised

{
  "error": {
    "message": "Authentication Error, API Key not allowed to access model. This token can only access models=['llmengine/*']. Tried to access llmengine/llama-3-1-8b-instruct",
    "type": "auth_error",
    "param": "None",
    "code": "401"
  }
}

Key Changes

• Implemented regex pattern-based model access (e.g., "custom-llm-engine/*")
• Ensure keys/teams with regex patterns can access all models in that regex pattern

Relevant issues

Type

🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test

Changes

[REQUIRED] Testing - Attach a screenshot of any new tests passing locall

If UI changes, send a screenshot/GIF of working UI fixes

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
litellm	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Nov 28, 2024 2:39am

[codecov]

Codecov Report

Attention: Patch coverage is 89.28571% with 3 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/proxy/auth/auth_checks.py	89.28%	3 Missing ⚠️

📢 Thoughts on this report? Let us know!

[krrishdholakia]

Hey @ishaan-jaff i believe your missing testing for /model/info and /models endpoints -

litellm/tests/proxy_unit_tests/test_proxy_server.py

Line 1844 in dcea31e

async def test_proxy_model_group_alias_checks(prisma_client, hidden):

[krrishdholakia]

missing testing for /models and /model/info endpoints

[krrishdholakia]

why remove this?

[ishaan-jaff]

It's not removed, it's just moved to _model_is_within_list_of_allowed_models

if len(allowed_models) == 0:
        return True