OpenFox looks for max_model_len in /v1/models response to determine
context window size. This commit:
- Adds max_model_len field to Jean models
- Adds max_model_len field to Ollama models
- Removes dead code after return statement
- Add context_length and context_window fields at model root level
- Keep them in meta for backward compatibility
- This should allow OpenFox to properly detect max context size
The function was accidentally truncated when adding get_jean_context_lengths().
This caused TypeError: 'NoneType' object is not iterable in list_models().
- Add get_jean_context_lengths() to fetch n_ctx from Jean /v1/models
- Add preset_ctx_map with known CTX values for each preset
- Add context_length to Jean model metadata in /v1/models
- Add context_length to Ollama model metadata from details.context_length
- This allows OpenFox to know the max tokens each model can handle
- Add OLLAMA_KEEP_ALIVE_MINUTES config (default: 5)
- schedule_ollama_unload(): delays Ollama unload when switching to Jean
- cancel_ollama_unload(): cancels timer if returning to Ollama
- Ollama models stay in VRAM for X minutes after last use
- Timer resets on each Ollama request
- stop_jean(): stops Jean service to free VRAM before Ollama use
- start_jean(): starts Jean service when switching back from Ollama
- unload_ollama(): forces Ollama to unload models before Jean use
- ensure_vram_for_backend(): orchestrates VRAM switching
- Tracks active backend state to avoid unnecessary switches
- Configurable timeouts via environment variables
- Prevents VRAM contention between Jean and Ollama
- jean-router interroge /api/presets pour lister dynamiquement les presets Jean
- Support Ollama via /api/tags pour les modèles vision
- Switch automatique de preset quand nécessaire
- Expose /v1/models avec tous les modèles disponibles
- Route les requêtes vers Jean ou Ollama selon le modèle demandé
- Nouveaux presets Jean automatiquement visibles dans OpenWebUI
Architecture:
open-webui → LiteLLM:4000 → jean-router:3014
├→ jean-proxy:3012 → Jean (.11:8080)
└→ wol-proxy:3011 → Ollama (.11:11434)
Folzo