feat: add VRAM management for Jean/Ollama switching
- stop_jean(): stops Jean service to free VRAM before Ollama use - start_jean(): starts Jean service when switching back from Ollama - unload_ollama(): forces Ollama to unload models before Jean use - ensure_vram_for_backend(): orchestrates VRAM switching - Tracks active backend state to avoid unnecessary switches - Configurable timeouts via environment variables - Prevents VRAM contention between Jean and Ollama
This commit is contained in:
@@ -63,7 +63,12 @@ services:
|
||||
- JEAN_PROXY_PORT=3012
|
||||
- OLLAMA_PROXY_HOST=localhost
|
||||
- OLLAMA_PROXY_PORT=3011
|
||||
- OLLAMA_DIRECT_HOST=${OLLAMA_HOST}
|
||||
- OLLAMA_DIRECT_PORT=11434
|
||||
- ENABLE_OLLAMA=true
|
||||
- ENABLE_VRAM_MANAGEMENT=true
|
||||
- LISTEN_PORT=3014
|
||||
- SWITCH_TIMEOUT=90
|
||||
- CHECK_INTERVAL=2
|
||||
- JEAN_STOP_TIMEOUT=30
|
||||
- OLLAMA_UNLOAD_TIMEOUT=15
|
||||
|
||||
Reference in New Issue
Block a user