Folzo/Effi
Public Access
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
main
Effi/archive/Perplexity/Bonjour j'ai besoin d'aide pour continuer le proje.md
T
Folzo 97348e9a12 Add ComfyUI models documentation and download script 
- Complete MODELS.md with detailed model descriptions
- Interactive download script for 5+ specialized SDXL models
- Updated README with model selection guide
- Fix docker-compose volumes path (/root/ComfyUI)
- API workflows for OpenWebUI integration

Models documented:
- RealVisXL (portraits)
- Juggernaut XL (versatile)
- DreamShaper (art/fantasy)
- SDXL Lightning (fast)
- Proteus (realism)
2026-04-06 01:05:18 +02:00

3764 lines
192 KiB
Markdown
Raw Permalink Blame History

This file contains invisible Unicode characters
This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
<img src="https://r2cdn.perplexity.ai/pplx-full-logo-primary-dark%402x.png" style="height:64px;margin-right:32px"/>
# Bonjour j'ai besoin d'aide pour continuer le projet Effi, voici les infos :
Projet initial :
📋 CAHIER DES CHARGES - PROJET EFFI v2.0
Assistant IA Personnel Local avec Prise de Notes Intelligente
🎯 OBJECTIFS GÉNÉRAUX
Créer une assistante IA locale nommée Effi qui :
Gère automatiquement une base de connaissances personnelle (notes)
Connaît parfaitement Folzo et son contexte (vie pro + perso)
Prend des initiatives autonomes (noter, corriger, enrichir, résumer)
Recherche intelligemment dans la mémoire avec compréhension sémantique
Accessible partout (desktop + mobile) via reverse proxy sécurisé
💻 INFRASTRUCTURE EXISTANTE
Matériel disponible
PC Gaming : AMD Ryzen 5800X3D, 32GB RAM, AMD Radeon 7800XT 16GB VRAM
NAS Synology : hébergement applications, reverse proxy configuré, nom de domaine
Réseau : reverse proxy fonctionnel pour accès externe sécurisé
Stack actuelle à faire évoluer
Open WebUI (NAS:3000) → WOL Proxy (NAS:3011) → Ollama PC (11434)
└─ mcpo (NAS:8000) → siyuan-mcp → SiYuan (NAS:6806)
📝 BESOINS FONCTIONNELS - PRISE DE NOTES
Types de contenus à gérer
Vie Personnelle :
Organisation quotidienne (agenda, tâches)
To-do liste personnelle
Idées de voyages
Liste films/séries à regarder
Documentation tests IA et infrastructure perso
Journal personnel (optionnel)
Vie Professionnelle :
Suivi de projets (quelques projets actifs)
To-do liste professionnelle
Wiki personnel (aide-mémoires techniques, procédures)
Notes de réunions/échanges
Caractéristiques techniques requises
Format : Markdown pur (fichiers .md) → futur-proof et portable
Structure : flexible (pas encore d'habitudes établies)
Mobilité : accès desktop + mobile (iOS/Android) obligatoire​​
Synchronisation : temps réel ou quasi temps réel entre appareils
Sécurité : self-hosted, accessible via reverse proxy existant
projet "Effi - Assistante IA Personnelle Locale"
🤖 BESOINS FONCTIONNELS - INTERACTION IA
Niveau d'autonomie
Maximum souhaité avec confiance :
L'IA doit corriger automatiquement (orthographe, formulation, structure)
L'IA doit enrichir automatiquement (ajout contexte, liens entre notes)
L'IA doit créer des notes spontanément si échange pertinent détecté
L'IA doit proposer des améliorations visibles/validables
L'IA doit connaître profondément Folzo (préférences, projets, historique)
Workflows d'interaction
Mode 1 - Prise de note assistée (principal) :
Folzo écrit une note brute
Effi corrige/améliore en temps réel ou à la demande
Validation/ajustement par Folzo
Sauvegarde finale
Mode 2 - Capture automatique (autonome) :
Conversation Folzo ↔ Effi
Effi détecte information importante
Effi crée note automatiquement (avec notification)
Folzo peut consulter/modifier après
Mode 3 - Recherche sémantique :
Requête en langage naturel : "trouve mes problèmes Docker avec réseau"
Effi comprend contexte même sans mots-clés exacts
Résultats pertinents avec extraits contextuels
🔍 FONCTIONNALITÉS TECHNIQUES CRITIQUES
1. Recherche Avancée (RAG Sémantique)
Embeddings locaux pour recherche par sens, pas par mots-clés​
Modèle d'embedding : nomic-embed-text ou similaire (léger, performant)​
Vector store : Qdrant ou intégré à l'app de notes​
Compréhension contextuelle : "projets Docker réseau" = trouve notes avec "container networking issues", "bridge problems", etc.​
2. Mémoire Persistante Multi-Sessions
Historique complet conversations Folzo ↔ Effi accessible
Base de connaissances sur Folzo enrichie automatiquement
Contexte long-terme (projets en cours, préférences, historique décisions)
3. Synchronisation Multi-Devices
Desktop : Linux (Fedora/Bazzite) + Windows si nécessaire
Mobile : iOS/Android via app native
Sync method : self-hosted, temps réel, sans conflit​
4. Actions Autonomes
Auto-tagging intelligent des notes (par projet, type, urgence)
Détection automatique informations à noter
Résumés automatiques de conversations longues
Création daily notes automatique chaque matin
Proposition Claude sonnet 4.6 :
Voici une évaluation complète des solutions pour Effi v2.0, structurée par composant critique, puis une architecture cible recommandée.
1. Couche Notes : Quel backend choisir ?
C'est la décision la plus structurante car elle conditionne la portabilité, la sync mobile et l'intégration IA.
Critère
SiYuan (actuel)
Obsidian
Logseq
Format fichiers
❌ .sy (pas du vrai MD)
✅ .md pur
✅ .md
App mobile iOS/Android
✅ native
✅ native \& stable
⚠️ instable sur mobile
Sync self-hosted
✅ intégré
✅ LiveSync + CouchDB
⚠️ Syncthing (complexe)
API pour IA/MCP
✅ siyuan-mcp
✅ Local REST API plugin
❌ très limité
Futur-proof / portabilité
❌ lock-in format
✅ maximal
✅ bon
Écosystème plugins IA
Faible
✅ énorme (Adaptive Memory…)
Moyen
Verdict : Obsidian est le choix optimal pour respecter le cahier des charges. SiYuan ne stocke pas en markdown pur (format .sy propriétaire), ce qui contredit directement l'exigence de futur-proof. Obsidian LiveSync via CouchDB auto-hébergé sur le NAS offre une sync temps réel complète sur iOS, Android et desktop sans aucun service cloud.
2. Mémoire Persistante \& RAG Sémantique
C'est le cœur technique d'Effi. Trois approches à comparer :
Solution
Sémantique
Local 100%
Complexité
Verdict
Open WebUI RAG natif
⚠️ partiel (ChromaDB)
✅
Faible
Insuffisant pour les besoins
mem0 + Qdrant + Ollama
✅ complet
✅
Moyenne
✅ Recommandé
pgvector (PostgreSQL)
✅
✅
Moyenne
Alternative viable
mem0 + Qdrant est la combinaison idéale : mem0 gère la mémoire multi-sessions avec enrichissement automatique, Qdrant sert de vector store performant, et nomic-embed-text via Ollama génère les embeddings localement. Un serveur MCP pour mem0 existe également et peut s'intégrer dans le pipeline mcpo existant. Il y a quelques bugs connus avec Ollama (réponse mal parsée), mais ils sont documentés et corrigeables.
3. Orchestration Autonome (le cerveau d'Effi)
C'est le composant manquant dans le stack actuel pour les modes 2 et 3 (capture automatique, daily notes, auto-tagging).
n8n est le seul outil self-hosted qui combine : agent IA Ollama natif, triggers planifiés, lecture/écriture de fichiers Obsidian, webhooks Open WebUI, et connexion Qdrant — le tout sans code. Il permet de construire exactement les workflows décrits (détection info importante → création note → notification).​
4. Architecture Cible Recommandée
PC GAMING (Ollama)
└─ LLM : qwen2.5:14b (raison + génération)
└─ Embedding : nomic-embed-text (vecteurs RAG)
NAS SYNOLOGY (containers Docker)
├─ Open WebUI :3000 → Interface chat Effi
├─ WOL Proxy :3011 → (garder existant)
├─ mcpo :8000 → outils MCP (obsidian-local-rest-api)
├─ mem0 :8765 → mémoire persistante Folzo ↔ Effi
├─ Qdrant :6333 → vector store RAG sémantique
├─ n8n :5678 → automatisation autonome
├─ CouchDB :5984 → backend sync Obsidian LiveSync
└─ Obsidian Vault (volume partagé .md)
DEVICES
├─ Desktop : Obsidian app + Open WebUI (navigateur)
└─ Mobile : Obsidian iOS/Android ← LiveSync → CouchDB
5. Évaluation par Workflow Fonctionnel
Mode 1 – Prise de note assistée : Open WebUI + pipeline Adaptive Memory v2 + mcpo → obsidian-local-rest-api pour lire/écrire les fichiers .md directement depuis le chat.​
Mode 2 – Capture automatique : n8n détecte les conversations Open WebUI via webhook, envoie à un agent Ollama qui juge la pertinence, et écrit la note via l'API Obsidian avec notification.​
Mode 3 – Recherche sémantique : Requête naturelle → Open WebUI → mem0 recherche dans Qdrant par similarité sémantique (nomic-embed-text) → retourne les extraits contextuels.​
Daily notes + auto-tagging : n8n schedule trigger chaque matin → création note journalière dans Obsidian + re-indexation des nouvelles notes dans Qdrant via embedding Ollama.
Plan de Migration
Déployer CouchDB sur NAS → configurer Obsidian LiveSync (desktop + mobile)​
Migrer les données SiYuan vers Obsidian (export markdown existant dans SiYuan)
Déployer Qdrant sur NAS (container Docker, volume persistant)
Déployer mem0 configuré avec Ollama + Qdrant + nomic-embed-text
Déployer n8n et construire les 3 workflows (capture, daily notes, auto-tagging)
Brancher mcpo sur obsidian-local-rest-api plugin (remplace siyuan-mcp)
Configurer Open WebUI avec pipeline Adaptive Memory v2 + RAG Knowledge Collection pointant vers le vault Obsidian
Charge Système Estimée sur NAS
Le NAS reste léger (mem0, Qdrant, n8n, CouchDB sont peu gourmands). Tout le compute lourd (inférence LLM, génération embeddings) reste sur le PC via Ollama, ce qui est parfaitement cohérent avec ton infrastructure. Le 7800XT avec 16GB VRAM peut faire tourner qwen2.5:14b confortablement, laissant de la marge pour nomic-embed-text en parallèle.​
Avancement 1 :
# 📋 Projet Effi — Documentation complète
## 🎯 Objectif
Effi est un assistant IA personnel **100% self-hosted**, avec :
- Personnalité joyeuse, mémoire persistante entre sessions
- Discussion vocale (STT + TTS)
- Recherche internet (Brave Search)
- Prise de notes automatique dans AFFiNE
- Analyse d'images
- RAG sur fichiers personnels (PDF/DOC depuis Nextcloud)
---
## 🖥️ Infrastructure matérielle
| Machine | IP | Rôle | CPU/GPU |
| :-- | :-- | :-- | :-- |
| **effi-server** | `192.168.1.27` | Serveur principal Docker | Intel J5040 / ~7.6 GB RAM |
| **folzo-Corp** (PC gaming) | `192.168.1.11` | Ollama LLM + STT + TTS | AMD Ryzen 5800X3D / RX 7800 XT 16 GB VRAM |
---
## 🐳 Stack Docker — effi-server (`~/effi/docker-compose.yml`)
## Services actifs
| Service | Image | Port | Rôle |
| :-- | :-- | :-- | :-- |
| `open-webui` | `ghcr.io/open-webui/open-webui:main` | `3000` | Interface principale Effi |
| `qdrant` | `qdrant/qdrant` | `6333-6334` | Base vectorielle RAG |
| `couchdb` | `couchdb:3.3` | `5984` | Sync Obsidian LiveSync |
| `n8n` | `n8nio/n8n` | `5678` | Automatisation workflows |
| `mcpo` | `ghcr.io/open-webui/mcpo` | `8000` | Proxy MCP pour outils |
| `wol-proxy` | `effi-wol-proxy` (build local) | `3011` | Wake-on-LAN PC gaming |
## Fichier `.env` (`~/effi/.env`)
text
`OLLAMA_HOST=192.168.1.11 PC_GAMING_MAC=XX:XX:XX:XX:XX:XX WEBUI_SECRET_KEY=<clé_aléatoire> COUCHDB_USER=master COUCHDB_PASSWORD=<mot_de_passe> N8N_USER=master N8N_PASSWORD=<mot_de_passe>`
## wol-proxy
Image buildée localement depuis `~/effi/wol-proxy/` — pinge `192.168.1.11:11434`, envoie un magic packet WoL si Ollama ne répond pas.
---
## 🎮 Stack Docker — folzo-Corp (`~/speaches/docker-compose.yml`)
## speaches — STT (Speech To Text)
text
`services: # ───────────────────────────────────────────────────────── # 1. SPEACHES - Speech To Text (STT) OpenAI-compatible # → [http://192.168.1.11:8200](http://192.168.1.11:8200) # ───────────────────────────────────────────────────────── speaches-stt: image: speaches:rocm-ct2 # image buildée localement (voir ci-dessous) container_name: speaches-stt restart: unless-stopped ports: - "8200:8000" devices: - /dev/kfd - /dev/dri group_add: - "44" # GID video - "991" # GID render security_opt: - seccomp=unconfined environment: - HSA_OVERRIDE_GFX_VERSION=11.0.0 - WHISPER__MODEL=Systran/faster-whisper-large-v3 - WHISPER__LANGUAGE=fr - WHISPER__COMPUTE_TYPE=int8 # float16 non supporté en CPU - WHISPER__DEVICE=cpu # GPU ROCm en cours (rebuild) volumes: - speaches-models:/root/.cache/huggingface speaches-init: image: curlimages/curl:latest container_name: speaches-init restart: "no" depends_on: - speaches-stt entrypoint: > sh -c "sleep 5 && curl -sf -X POST http://speaches-stt:8000/v1/models/Systran%2Ffaster-whisper-large-v3 && echo 'Modèle enregistré'" volumes: speaches-models:`
## openedai-speech — TTS (Text To Speech) (`~/openedai-speech/docker-compose.yml`)
text
`services: openedai-speech: image: ghcr.io/matatonic/openedai-speech container_name: openedai-speech restart: unless-stopped ports: - "8100:8000" volumes: - ./tts-voices:/app/voices - ./tts-config:/app/config devices: - /dev/kfd - /dev/dri group_add: - "44" # GID video - "991" # GID render environment: - HSA_OVERRIDE_GFX_VERSION=11.0.0 - HIP_VISIBLE_DEVICES=0`
---
## ⚙️ Configuration Open WebUI
## Admin Panel → Settings → Connections
- **Ollama URL** : `http://192.168.1.11:3011`
## Admin Panel → Settings → Audio
- **STT Engine** : OpenAI
- **STT URL** : `http://192.168.1.11:8200/v1`
- **STT Model** : `Systran/faster-whisper-large-v3`
- **TTS Engine** : OpenAI
- **TTS URL** : `http://192.168.1.11:8100/v1`
- **TTS Model** : `tts-1-hd`
## Admin Panel → Settings → Documents
- **Embedding Engine** : Ollama
- **Embedding Model** : `bge-m3:latest`
- **Vector Database** : Qdrant
- **Qdrant URL** : `http://qdrant:6333`​
## Modèles Ollama disponibles sur folzo-Corp​
- `qwen2.5:14b` — LLM principal d'Effi
- `bge-m3` — Embeddings RAG
## Workspace Effi (Open WebUI)
- Compte admin : `master`
- Compte utilisateur : `folzo`
- System prompt global configuré avec personnalité + contexte infrastructure​
---
## 🔧 Build speaches ROCm — CTranslate2 gfx1100
## Problème rencontré
L'image speaches ne publie pas d'image ROCm. Il faut builder manuellement. La RX 7800 XT est `gfx1100` mais le Dockerfile par défaut cible `gfx1101` → `hipErrorInvalidDeviceFunction`.
## Solution
bash
`cd ~/speaches git clone [https://github.com/speaches-ai/speaches](https://github.com/speaches-ai/speaches) . cd src # Corriger l'architecture GPU : gfx1101 → gfx1100 sed -i 's/gfx1101/gfx1100/g' Dockerfile.rocm-ct2 grep gfx Dockerfile.rocm-ct2 # vérification # Build (~50 min la première fois) docker build -f Dockerfile.rocm-ct2 -t speaches:rocm-ct2 .`
## Ajout MIOpen (fix cuDNN)​
text
`# Dans Dockerfile.rocm-ct2, ajouter dans le cmake : -DWITH_CUDNN=ON -DCUDnn_LIBRARY=/opt/rocm-6.3.4/lib/libMIOpen.so`
> ⚠️ **Statut actuel** : Le rebuild avec `gfx1100` est en cours. speaches fonctionne en **CPU** (int8) en attendant — ~10s pour 5s d'audio sur le Ryzen 5800X3D.​
---
## 🐛 Problèmes rencontrés et solutions
## 1. wol-proxy — image inexistante
**Problème** : Image Docker inexistante sur le registry​
**Solution** : Build local depuis le code Python maison
bash
`cd ~/effi && docker compose up -d --build`
## 2. Interface réseau Ubuntu plantée​
**Problème** : La carte réseau d'effi-server tombe silencieusement
**Solution** : Off/On via NetworkManager
bash
`sudo systemctl restart NetworkManager`
**Fix permanent** (pour NIC Intel i225/i226) :
bash
`echo 'ACTION=="add", SUBSYSTEM=="net", NAME=="enp3s0", RUN+="/usr/sbin/ethtool -s enp3s0 wol g"' \\ | sudo tee /etc/udev/rules.d/99-disable-net-powersave.rules`
## 3. Whisper — mauvais repo HuggingFace
**Problème** : Open WebUI envoyait `Systeme/faster-whisper-tiny` (inexistant)
**Solution** : Corriger dans Settings → Audio → STT Model + changer `WHISPER__MODEL` dans docker-compose
## 4. speaches — `PRELOAD_MODELS` format invalide​
**Problème** : La variable `PRELOAD_MODELS` avec un objet JSON crash le conteneur
**Solution** : Supprimer `PRELOAD_MODELS` entièrement, utiliser un service `speaches-init` pour enregistrer le modèle via API au démarrage
## 5. openedai-speech — `group render` introuvable
**Problème** : `group_add: render` échoue car le groupe n'existe pas dans le conteneur
**Solution** : Utiliser les GID numériques `"44"` et `"991"` au lieu des noms
## 6. CTranslate2 float16 sur CPU​
**Problème** : `ValueError: float16 not supported on CPU backend`
**Solution** : `WHISPER__COMPUTE_TYPE=int8`
## 7. mcpo redémarre en boucle​
**Problème** : Config `mcpo-config/config.json` vide au premier démarrage
**Solution** :
bash
`cat > ~/effi/mcpo-config/config.json << 'EOF' {"mcpServers": {}} EOF docker compose restart mcpo`
## 8. Open WebUI lent après reboot​
**Problème** : Conteneurs sans `restart: unless-stopped`
**Solution** : Ajouter `restart: unless-stopped` à tous les services du docker-compose
---
## 📦 Initialisation CouchDB (une seule fois)
bash
`curl -X PUT [http://master:MOT_DE_PASSE@192.168.1.27:5984/_users](http://master:MOT_DE_PASSE@192.168.1.27:5984/_users) curl -X PUT [http://master:MOT_DE_PASSE@192.168.1.27:5984/_replicator](http://master:MOT_DE_PASSE@192.168.1.27:5984/_replicator) curl -X PUT http://master:MOT_DE_PASSE@192.168.1.27:5984/obsidian-livesync`
---
## 🔌 Ouverture de ports firewall
Sur **folzo-Corp** (Bazzite/Fedora) :
bash
`sudo firewall-cmd --add-port=8200/tcp --permanent # speaches STT sudo firewall-cmd --add-port=8100/tcp --permanent # openedai-speech TTS sudo firewall-cmd --reload`
---
## ✅ État actuel du pipeline vocal
| Étape | Status | Détail |
| :-- | :-- | :-- |
| STT (Whisper) | ✅ Fonctionnel | speaches CPU, large-v3, ~10s/phrase |
| LLM (Ollama) | ✅ Fonctionnel | qwen2.5:14b sur folzo-Corp |
| TTS (openedai-speech) | ✅ Fonctionnel | tts-1-hd, ROCm activé |
| WOL PC gaming | ✅ Fonctionnel | wol-proxy sur port 3011 |
| RAG Qdrant | 🔧 En cours | Nextcloud → indexation à configurer |
---
## 🚧 Prochaines étapes
- Finaliser le rebuild `speaches:rocm-ct2` avec `gfx1100` → activer GPU pour STT
- Configurer le RAG Qdrant avec les fichiers Nextcloud (~1 GB, ~1000 fichiers)
- Tester mem0 / MCP pour la mémoire persistante d'Effi​
- Configurer n8n pour les workflows automatiques
- Sync Obsidian LiveSync sur mobile​
Avancement 2 :
## 📋 Résumé Complet du Thread
---
## 🖥️ Infrastructure
| Machine | IP | Rôle | GPU/CPU |
| :-- | :-- | :-- | :-- |
| **folzo-Corp** | 192.168.1.11 | PC gaming / STT+TTS | AMD RX 7800 XT (gfx1100) |
| **effi-server** | 192.168.1.27 | Serveur IA principal | CPU Intel 4 cores / 7.6GB RAM |
| **PC Gaming Ollama** | variable | LLM Ollama | GPU dédié |
---
## 🎯 Objectif du Thread
Faire tourner Whisper (STT) sur la RX 7800 XT de folzo-Corp via ROCm, en construisant CTranslate2-ROCm depuis les sources (speaches), puis migrer le TTS (openedai-speech) d'effi-server vers folzo-Corp pour libérer le CPU.
---
## 🔨 Étapes Réalisées
## 1. Build CTranslate2-ROCm pour speaches
**Problème initial :** `RuntimeError: Conv1D on GPU currently requires the cuDNN library which is not integrated in this build`
**Cause :** CTranslate2-ROCm compilé sans support MIOpen (équivalent ROCm de cuDNN)
**Solution :** Dockerfile custom `~/speaches/src/Dockerfile.rocm-ct2` avec :
text
`-DWITH_CUDNN=ON -DCUDnn_INCLUDE_DIR=/opt/rocm-6.3.4/include -DCUDnn_LIBRARY=/opt/rocm-6.3.4/lib/libMIOpen.so`
Image base : `rocm/dev-ubuntu-24.04:6.3.4-complete`
Source CTranslate2 : `github.com/arlo-phoenix/CTranslate2-rocm`
---
## 2. Erreur Architecture GPU Incorrecte
**Problème :** `RuntimeError: parallel_for failed: hipErrorInvalidDeviceFunction: invalid device function`
**Cause :** Build compilé pour `gfx1101` mais la RX 7800 XT est `gfx1100`
**Diagnostic :**
bash
`docker run --device /dev/kfd --device /dev/dri \\ -e HSA_OVERRIDE_GFX_VERSION=11.0.0 speaches:rocm-ct2 \\ /opt/rocm-6.3.4/bin/rocminfo | grep -E "gfx|Name" # → gfx1100 (AMD Radeon RX 7800 XT)`
**Fix :**
bash
`sed -i 's/gfx1101/gfx1100/g' ~/speaches/src/Dockerfile.rocm-ct2 cd ~/speaches/src docker build -f Dockerfile.rocm-ct2 -t speaches:rocm-ct2 .`
---
## 3. Transcription fonctionnelle (CPU en attendant rebuild)
Après le premier build, speaches fonctionne en CPU (fallback) :
- `200 OK` sur `/v1/audio/transcriptions`
- Langue `fr` détectée à 99%
- RTF > 1 (13s pour 5.4s audio) → confirme CPU, pas GPU
- ONNX Runtime cherche `libcublasLt.so.12` (CUDA) → warning non bloquant, fallback CPU automatique
**Rebuild avec gfx1100 en cours** pour activer l'accélération GPU (~10x plus rapide attendu).
---
## 4. Problème réseau effi-server
**Symptômes :** Interface filaire effi-server plantée après reboot, OpenWebUI inaccessible
**Diagnostic :**
text
`conflict detected for IP address 192.168.1.11 with host 5C:87:9C:65:95:23`
**Cause :** Conflit DHCP — deux machines avec la même IP `192.168.1.11`
**Fix court terme :** `nmcli device disconnect enp6s0 && nmcli device connect enp6s0`
**Fix permanent recommandé :** Réservations DHCP par MAC sur le routeur + `ethtool wol g` pour la carte Realtek RTL8111
---
## 5. OpenWebUI lent / erreurs 500
**Symptômes :** `500: Open WebUI: Server Connection Error`, lenteur UI
**Diagnostic :**
text
`ERROR: open_webui.routers.ollama: Connection error`
**Cause :** Ollama inaccessible — le PC gaming était éteint. Le `wol-proxy` sur effi-server assure le Wake-on-LAN mais le PC n'avait pas répondu à temps.
**Architecture WOL :**
text
`OpenWebUI → port 3011 effi → wol-proxy → WOL magic packet → PC Gaming → proxy TCP vers 11434`
---
## 6. Deux processus zombies sur effi-server
Détectés via `ps aux` :
text
`[mcpo] <defunct> [python] <defunct>`
**Fix :**
bash
`docker restart mcpo`
---
## 7. Migration openedai-speech → folzo-Corp
**Raison :** `openedai-speech` consommait ~100% CPU sur effi-server (1.06GB RAM, CPU intensif)
**Étapes :**
bash
`# Arrêt sur effi docker compose stop openedai-speech && docker compose rm -f openedai-speech # Copie des données depuis folzo-Corp scp -r effi@192.168.1.27:~/effi/tts-voices ~/openedai-speech/ scp -r effi@192.168.1.27:~/effi/tts-config ~/openedai-speech/`
**Contenu transféré :** modèles XTTS (~1.8GB), voix WAV (alloy/echo/nova/shimmer/fable/onyx), modèles ONNX piper, configs YAML
**Problème group_add :** `Unable to find group render: no matching entries in group file`
**Fix :** Utiliser GID numériques au lieu des noms :
text
`group_add: - "44" # video - "991" # render`
---
## 📁 Fichiers Clés
text
`folzo-Corp: ├── ~/speaches/ │ ├── src/Dockerfile.rocm-ct2 # Build CTranslate2 gfx1100 + MIOpen │ └── docker-compose.yml # speaches-stt:8200 └── ~/openedai-speech/ ├── docker-compose.yml # TTS ROCm gfx1100 ├── tts-voices/ # Voix XTTS + ONNX └── tts-config/ # voice_to_speaker.yaml etc. effi-server: └── ~/effi/docker-compose.yml # OpenWebUI + WOL + n8n + qdrant...`
---
## ✅ État Final des Services
| Service | Hôte | Port | État | Notes |
| :-- | :-- | :-- | :-- | :-- |
| **speaches-stt** | folzo-Corp | 8200 | ✅ Fonctionne | CPU actuellement, rebuild gfx1100 pending |
| **openedai-speech** | folzo-Corp | 8100 | ✅ Démarré | ROCm gfx1100, group_add 44+991 |
| **OpenWebUI** | effi-server | 3000 | ✅ Fonctionne | Pointé STT+TTS vers 192.168.1.11 |
| **wol-proxy** | effi-server | 3011 | ✅ Fonctionne | Proxy Ollama + WOL PC gaming |
| **n8n** | effi-server | 5678 | ✅ Running | |
| **qdrant** | effi-server | 6333 | ✅ Running | Vector store RAG |
| **pipelines** | effi-server | 9099 | ✅ Running | mem0 memory |
| **couchdb** | effi-server | 5984 | ✅ Running | Obsidian LiveSync |
| **mcpo** | effi-server | 8000 | ⚠️ Zombie détecté | Redémarré |
---
## 🔜 Prochaines Étapes
1. **Finaliser rebuild speaches gfx1100** → activer GPU sur Whisper large-v3
2. **Tester TTS ROCm** → vérifier que openedai-speech utilise bien la RX 7800 XT
3. **Mettre à jour l'URL TTS** dans OpenWebUI Admin → Audio → `http://192.168.1.11:8100`
4. **Réservations DHCP** sur le routeur pour fixer les IPs de façon permanente
5. **Tester pipeline vocal complet** : micro → STT folzo → OpenWebUI effi → LLM Ollama → TTS folzo → speaker
Le probleme est que lorsque je fais un vocal avec Effi ma charge CPU explose entre le llm d'effi, le llm de la memoire, le whisper pour la comprehension et la generation vocal, mon pc et ma 7800xt ne suive pas. Alors j'ai besoin de toi pour faire le point et voire quel solution nous pouvons trouver pour ameliorer ca
Le goulot d'étranglement est clair : ton pipeline vocal empile **4 processus lourds séquentiels** sur la même RX 7800 XT, dont deux qui mobilisent le GPU simultanément. Voici l'analyse et le plan d'optimisation par ordre d'impact.
## Diagnostic : qui consomme quoi
| Composant | Modèle actuel | CPU/GPU | VRAM estimée |
| :-- | :-- | :-- | :-- |
| STT (speaches) | `large-v3` sur **CPU** | CPU 100% | — |
| LLM principal | `qwen2.5:14b` | GPU | ~8.5 GB |
| TTS (openedai-speech) | `tts-1-hd` = XTTS v2 | GPU | ~4 GB [^1_1] |
| mem0 mémoire | `qwen2.5:14b` (même modèle) | GPU | (séquentiel, mais occupe le slot) |
| **Total** | | | **~12.5 GB** |
Le 7800 XT avec 16 GB est donc à ~78% de VRAM occupée **au repos** entre le LLM et le TTS, sans même compter Whisper sur GPU. Le CPU explose à cause de `large-v3` en mode software. Il y a **3 fixes immédiats** et **1 optimisation structurelle**.
***
## Fix \#1 — STT : `large-v3-turbo` (gain x5 immédiat)
`large-v3-turbo` a **exactement la même précision** que `large-v3` mais est 5x plus rapide. C'est le fix le plus impactant à faire maintenant, sans changer d'architecture.[^1_2]
Dans `~/speaches/docker-compose.yml` sur folzo-Corp :
```yaml
environment:
- WHISPER__MODEL=Systran/faster-whisper-large-v3-turbo # ← était large-v3
- WHISPER__COMPUTE_TYPE=int8
- WHISPER__DEVICE=cpu
```
```bash
docker compose pull speaches-stt && docker compose up -d speaches-stt
```
Résultat attendu : ~2-3s pour 5s d'audio au lieu de ~10-13s, sur le même CPU.[^1_2]
***
## Fix \#2 — TTS : basculer de XTTS vers Piper (libère 4 GB VRAM)
`openedai-speech` supporte deux backends : `tts-1-hd` = XTTS v2 (GPU, ~4 GB VRAM), et `tts-1` = Piper (CPU uniquement, très rapide, négligeable en ressources). Piper génère la parole quasi en temps réel sur CPU, la qualité est légèrement inférieure à XTTS mais amplement suffisante pour un usage conversationnel.[^1_1][^1_3]
**Dans Open WebUI Admin → Settings → Audio :**
→ Changer **TTS Model** de `tts-1-hd` vers `tts-1`
Résultat : **4 GB VRAM libérés**, le CPU de folzo-Corp reste très peu chargé (Piper est ultra-léger ). Après ce fix, le budget VRAM tombe à ~8.5 GB utilisés.[^1_4]
***
## Fix \#3 — mem0 : modèle dédié léger pour la mémoire
mem0 appelle le LLM pour extraire/stocker les souvenirs — tâche simple qui n'a pas besoin de 14B paramètres. La config mem0 permet de spécifier un modèle séparé :[^1_5]
```python
# Dans la config mem0 (pipeline Open WebUI ou service dédié)
config = {
"llm": {
"provider": "ollama",
"config": {
"model": "qwen2.5:3b", # ← au lieu de qwen2.5:14b
"ollama_base_url": "http://192.168.1.11:11434"
}
},
"embedder": {
"provider": "ollama",
"config": {
"model": "bge-m3",
"ollama_base_url": "http://192.168.1.11:11434"
}
}
}
```
Télécharge d'abord le modèle sur folzo-Corp :
```bash
ollama pull qwen2.5:3b
```
`qwen2.5:3b` occupe ~2 GB VRAM. Avec les fixes \#2 et \#3, le budget VRAM total devient : **8.5 GB (14b) + 2 GB (3b) = ~10.5 GB**, ce qui laisse **5.5 GB libres** — suffisamment pour enfin activer Whisper sur GPU (voir étape suivante).
***
## Fix \#4 — STT sur GPU (après les fixes \#2 et \#3)
Une fois 5.5 GB libres, Whisper `large-v3-turbo` sur GPU ne prend que ~1.5 GB VRAM. Le rebuild speaches ROCm `gfx1100` devient utile ici. Avec Piper sur CPU et mem0 sur un 3b :[^1_6]
```
VRAM après tous les fixes :
qwen2.5:14b → 8.5 GB
qwen2.5:3b (mem0) → 2.0 GB
Whisper large-v3-t → 1.5 GB
─────────────────────────────
Total → 12.0 GB / 16 GB ✅ confortable
```
***
## Plan d'action prioritaire
1. **Maintenant** : changer `WHISPER__MODEL` → `large-v3-turbo` (restart speaches, 2 min)
2. **Maintenant** : changer TTS model → `tts-1` dans Open WebUI (instantané)
3. **Après** : `ollama pull qwen2.5:3b` + reconfigurer mem0 pour pointer sur ce modèle
4. **Ensuite** : finaliser le rebuild `speaches:rocm-ct2 gfx1100` pour passer Whisper sur GPU
Les fixes \#1 et \#2 seuls devraient **diviser par 3-4 la charge ressentie** lors d'une conversation vocale, sans aucun impact notable sur la qualité perçue.
<span style="display:none">[^1_10][^1_11][^1_12][^1_13][^1_14][^1_15][^1_7][^1_8][^1_9]</span>
<div align="center">⁂</div>
[^1_1]: https://git.robby.blue/ml/openedai-speech
[^1_2]: https://whispernotes.app/blog/introducing-whisper-large-v3-turbo
[^1_3]: https://github.com/matatonic/openedai-speech
[^1_4]: https://github.com/mdmonsurali/Offline-Fast-CPU-PIPER-TTS
[^1_5]: https://docs.mem0.ai/components/llms/config
[^1_6]: https://whishper.net/reference/models/
[^1_7]: https://blog.stephane-robert.info/docs/developper/programmation/python/ollama/
[^1_8]: https://dev.to/ajitkumar/the-complete-guide-to-ollama-run-large-language-models-locally-2mge
[^1_9]: https://docs.ollama.com/faq
[^1_10]: https://docs.mem0.ai/components/embedders/models/ollama
[^1_11]: https://github.com/mem0ai/mem0/issues/2758
[^1_12]: https://github.com/ggml-org/whisper.cpp/discussions/3074
[^1_13]: https://komelin.com/blog/using-small-llms-locally
[^1_14]: https://www.youtube.com/watch?v=Q-4pkwHiAkA
[^1_15]: https://www.whishper.net/reference/models/
---
# go pour ce plan d'action, je te laisse m'accompagner etape par etape
Parfait, on y va étape par étape. On commence par le **Fix \#1** — c'est le plus rapide et le gain est immédiat.
***
## Étape 1 — Whisper `large-v3-turbo`
Sur **folzo-Corp**, édite le docker-compose de speaches :
```bash
nano ~/speaches/docker-compose.yml
```
Repère la ligne `WHISPER__MODEL` et remplace :
```yaml
# Avant
- WHISPER__MODEL=Systran/faster-whisper-large-v3
# Après
- WHISPER__MODEL=Systran/faster-whisper-large-v3-turbo
```
Puis redémarre le service pour qu'il télécharge le nouveau modèle :
```bash
cd ~/speaches && docker compose up -d speaches-stt
```
Surveille le téléchargement du modèle (environ 1.5 GB) :
```bash
docker logs -f speaches-stt
```
Le conteneur est prêt quand tu vois une ligne indiquant que le modèle est chargé et qu'il écoute.
***
**Pour tester rapidement** une fois chargé :
```bash
curl -s http://192.168.1.11:8200/v1/audio/transcriptions \
-F "file=@/un/fichier.wav" \
-F "model=Systran/faster-whisper-large-v3-turbo" | jq .
```
Dis-moi ce que tu observes dans les logs, et si la transcription de test fonctionne — on passera ensuite au Fix \#2 (basculer TTS vers Piper).
---
# voici le docker compose : "cd ~/speaches \&\& docker compose up -d speaches-stt
" est-ce bon ?
---
# je me suis louper voici le docker compose : "\# ═══════════════════════════════════════════════════════════
# PC GAMING - Stack STT ROCm
# Machine : folzo-Corp (192.168.1.11)
# GPU : AMD RX 7800 XT (16 GB VRAM) — ROCm gfx1101
# ═══════════════════════════════════════════════════════════
services:
# ─────────────────────────────────────────────────────────
# 1. SPEACHES - Speech To Text (STT) OpenAI-compatible
# → [http://192.168.1.11:8200](http://192.168.1.11:8200)
# → Utilisé par Open WebUI sur effi-server
# ─────────────────────────────────────────────────────────
speaches:
image: speaches:rocm-ct2
container_name: speaches-stt
restart: unless-stopped
ports:
- "8200:8000"
devices:
- /dev/kfd
- /dev/dri
group_add:
- video
security_opt:
- seccomp=unconfined
environment:
- HSA_OVERRIDE_GFX_VERSION=11.0.0
- WHISPER__MODEL=Systran/faster-whisper-large-v3-turbo
- WHISPER__LANGUAGE=fr
- WHISPER__COMPUTE_TYPE=int8
- WHISPER__DEVICE=cpu
volumes:
- speaches-models:/root/.cache/huggingface
# ─────────────────────────────────────────────────────────
# 2. SPEACHES-INIT - Enregistre le modèle au démarrage
# → S'exécute une fois puis s'arrête
# ─────────────────────────────────────────────────────────
speaches-init:
image: curlimages/curl:latest
container_name: speaches-init
restart: "no"
depends_on:
- speaches
entrypoint: >
sh -c "sleep 5 \&\&
curl -sf -X POST http://speaches:8000/v1/models/Systran%2Ffaster-whisper-large-v3 \&\&
echo 'Modèle enregistré'"
volumes:
speaches-models:
"
est-ce bon ?
Presque bon ! Il y a **2 corrections à faire** avant de lancer.
***
## Correction 1 — `speaches-init` pointe encore l'ancien modèle
La ligne du `curl` dans `speaches-init` n'a pas été mise à jour. Elle va tenter d'enregistrer `large-v3` au lieu de `large-v3-turbo` :
```yaml
# ❌ Avant
curl -sf -X POST http://speaches:8000/v1/models/Systran%2Ffaster-whisper-large-v3
# ✅ Après
curl -sf -X POST http://speaches:8000/v1/models/Systran%2Ffaster-whisper-large-v3-turbo
```
***
## Correction 2 — `group_add` doit utiliser les GID numériques
Comme on l'a déjà résolu pour `openedai-speech`, les noms de groupes ne sont pas résolus dans le conteneur ROCm :
```yaml
# ❌ Avant
group_add:
- video
# ✅ Après
group_add:
- "44" # video
- "991" # render
```
***
## Fichier final corrigé
```yaml
services:
speaches:
image: speaches:rocm-ct2
container_name: speaches-stt
restart: unless-stopped
ports:
- "8200:8000"
devices:
- /dev/kfd
- /dev/dri
group_add:
- "44"
- "991"
security_opt:
- seccomp=unconfined
environment:
- HSA_OVERRIDE_GFX_VERSION=11.0.0
- WHISPER__MODEL=Systran/faster-whisper-large-v3-turbo
- WHISPER__LANGUAGE=fr
- WHISPER__COMPUTE_TYPE=int8
- WHISPER__DEVICE=cpu
volumes:
- speaches-models:/root/.cache/huggingface
speaches-init:
image: curlimages/curl:latest
container_name: speaches-init
restart: "no"
depends_on:
- speaches
entrypoint: >
sh -c "sleep 5 &&
curl -sf -X POST http://speaches:8000/v1/models/Systran%2Ffaster-whisper-large-v3-turbo &&
echo 'Modèle enregistré'"
volumes:
speaches-models:
```
***
Une fois le fichier sauvegardé, lance :
```bash
cd ~/speaches && docker compose up -d
```
Puis surveille les logs :
```bash
docker logs -f speaches-stt
```
Tu devrais voir le téléchargement du modèle `large-v3-turbo` (~1.5 GB). Dis-moi quand c'est prêt !
---
# Invalid model-index. Not loading eval results into CardData.
2026-02-22 11:50:36,197:WARNING:huggingface_hub.repocard_data:__init__:368:Invalid model-index. Not loading eval results into CardData.
2026-02-22 11:50:36,303:DEBUG:urllib3.connectionpool:_make_request:546:https://huggingface.co:443 "GET /api/models?filter=automatic-speech-recognition\&search=istupakov%2Fparakeet-tdt\&cardData=True HTTP/11" 200 1633
2026-02-22 11:50:36,417:DEBUG:urllib3.connectionpool:_make_request:546:https://huggingface.co:443 "GET /api/models?filter=onnx\&filter=text-to-speech\&filter=speaches\&filter=piper\&cardData=True HTTP/11" 200 63637
2026-02-22 11:50:36,546:DEBUG:urllib3.connectionpool:_make_request:546:https://huggingface.co:443 "GET /api/models?filter=onnx\&filter=text-to-speech\&filter=speaches\&filter=kokoro\&cardData=True HTTP/11" 200 1911
2026-02-22 11:50:36,546:ERROR:speaches.main:_custom_http_exception_handler:154:HTTP error: 404: Model 'Systran/faster-whisper-large-v3-turbo' not found
INFO: 172.18.0.3:36816 - "POST /v1/models/Systran/faster-whisper-large-v3-turbo HTTP/1.1" 404 Not Found
2026-02-22 11:55:36,172:DEBUG:speaches.dependencies:audio_file_dependency:89:Decoding audio file: 1020a792-24f0-4380-a1c6-a8961d5fc7a9.mp3, content_type: None, header: Headers({'content-disposition': 'form-data; name="file"; filename="1020a792-24f0-4380-a1c6-a8961d5fc7a9.mp3"'}), size: 27885
2026-02-22 11:55:36,254:DEBUG:speaches.dependencies:audio_file_dependency:103:Decoded 3.42s of audio in 0.08189s (RTF: 0.023945732459518514)
2026-02-22 11:55:36,254:DEBUG:asyncio:__init__:64:Using selector: EpollSelector
2026-02-22 11:55:36,255:ERROR:speaches.main:_custom_http_exception_handler:154:HTTP error: 404: Model 'Systran/faster-whisper-large-v3-turbo' is not installed locally. You can download the model using `POST /v1/models`
INFO: 192.168.1.27:47886 - "POST /v1/audio/transcriptions HTTP/1.1" 404 Not Found
2026-02-22 11:55:36,259:DEBUG:speaches.dependencies:audio_file_dependency:89:Decoding audio file: 1020a792-24f0-4380-a1c6-a8961d5fc7a9.mp3, content_type: None, header: Headers({'content-disposition': 'form-data; name="file"; filename="1020a792-24f0-4380-a1c6-a8961d5fc7a9.mp3"'}), size: 27885
2026-02-22 11:55:36,317:DEBUG:speaches.dependencies:audio_file_dependency:103:Decoded 3.42s of audio in 0.05753s (RTF: 0.01682135614254022)
2026-02-22 11:55:36,317:DEBUG:asyncio:__init__:64:Using selector: EpollSelector
2026-02-22 11:55:36,318:ERROR:speaches.main:_custom_http_exception_handler:154:HTTP error: 404: Model 'Systran/faster-whisper-large-v3-turbo' is not installed locally. You can download the model using `POST /v1/models`
INFO: 192.168.1.27:47892 - "POST /v1/audio/transcriptions HTTP/1.1" 404 Not Found
2026-02-22 11:55:55,631:DEBUG:speaches.dependencies:audio_file_dependency:89:Decoding audio file: 597236d0-11de-45db-8bfc-038ebb55e7c5.mp3, content_type: None, header: Headers({'content-disposition': 'form-data; name="file"; filename="597236d0-11de-45db-8bfc-038ebb55e7c5.mp3"'}), size: 27885
2026-02-22 11:55:55,702:DEBUG:speaches.dependencies:audio_file_dependency:103:Decoded 3.42s of audio in 0.07090s (RTF: 0.020731435089619908)
2026-02-22 11:55:55,702:DEBUG:asyncio:__init__:64:Using selector: EpollSelector
2026-02-22 11:55:55,703:ERROR:speaches.main:_custom_http_exception_handler:154:HTTP error: 404: Model 'Systran/faster-whisper-large-v3-turbo' is not installed locally. You can download the model using `POST /v1/models`
INFO: 192.168.1.27:50708 - "POST /v1/audio/transcriptions HTTP/1.1" 404 Not Found
2026-02-22 11:55:55,707:DEBUG:speaches.dependencies:audio_file_dependency:89:Decoding audio file: 597236d0-11de-45db-8bfc-038ebb55e7c5.mp3, content_type: None, header: Headers({'content-disposition': 'form-data; name="file"; filename="597236d0-11de-45db-8bfc-038ebb55e7c5.mp3"'}), size: 27885
2026-02-22 11:55:55,762:DEBUG:speaches.dependencies:audio_file_dependency:103:Decoded 3.42s of audio in 0.05481s (RTF: 0.016025735087636914)
2026-02-22 11:55:55,762:DEBUG:asyncio:__init__:64:Using selector: EpollSelector
2026-02-22 11:55:55,763:ERROR:speaches.main:_custom_http_exception_handler:154:HTTP error: 404: Model 'Systran/faster-whisper-large-v3-turbo' is not installed locally. You can download the model using `POST /v1/models`
INFO: 192.168.1.27:50712 - "POST /v1/audio/transcriptions HTTP/1.1" 404 Not Found
2026-02-22 11:56:03,959:DEBUG:speaches.dependencies:audio_file_dependency:89:Decoding audio file: 3a76309e-53f9-409d-a80c-b13a605ae34d.mp3, content_type: None, header: Headers({'content-disposition': 'form-data; name="file"; filename="3a76309e-53f9-409d-a80c-b13a605ae34d.mp3"'}), size: 6189
2026-02-22 11:56:04,026:DEBUG:speaches.dependencies:audio_file_dependency:103:Decoded 0.72s of audio in 0.06635s (RTF: 0.09215139444778389)
2026-02-22 11:56:04,026:DEBUG:asyncio:__init__:64:Using selector: EpollSelector
2026-02-22 11:56:04,026:ERROR:speaches.main:_custom_http_exception_handler:154:HTTP error: 404: Model 'Systran/faster-whisper-large-v3-turbo' is not installed locally. You can download the model using `POST /v1/models`
INFO: 192.168.1.27:47664 - "POST /v1/audio/transcriptions HTTP/1.1" 404 Not Found
2026-02-22 11:56:04,031:DEBUG:speaches.dependencies:audio_file_dependency:89:Decoding audio file: 3a76309e-53f9-409d-a80c-b13a605ae34d.mp3, content_type: None, header: Headers({'content-disposition': 'form-data; name="file"; filename="3a76309e-53f9-409d-a80c-b13a605ae34d.mp3"'}), size: 6189
2026-02-22 11:56:04,080:DEBUG:speaches.dependencies:audio_file_dependency:103:Decoded 0.72s of audio in 0.04893s (RTF: 0.06796120000621562)
2026-02-22 11:56:04,080:DEBUG:asyncio:__init__:64:Using selector: EpollSelector
2026-02-22 11:56:04,080:ERROR:speaches.main:_custom_http_exception_handler:154:HTTP error: 404: Model 'Systran/faster-whisper-large-v3-turbo' is not installed locally. You can download the model using `POST /v1/models`
INFO: 192.168.1.27:47674 - "POST /v1/audio/transcriptions HTTP/1.1" 404 Not Found
Le problème est clair : **Systran n'a pas publié de version officielle `faster-whisper-large-v3-turbo`**. Le modèle n'existe pas sous ce nom sur HuggingFace. Il faut utiliser une conversion communautaire en CTranslate2 — `deepdml/faster-whisper-large-v3-turbo-ct2`.[^5_1][^5_2]
## Correction à appliquer
Deux lignes à changer dans `~/speaches/docker-compose.yml` :
```yaml
services:
speaches:
# ...
environment:
- WHISPER__MODEL=deepdml/faster-whisper-large-v3-turbo-ct2 # ← corrigé
speaches-init:
entrypoint: >
sh -c "sleep 5 &&
curl -sf -X POST http://speaches:8000/v1/models/deepdml%2Ffaster-whisper-large-v3-turbo-ct2 &&
echo 'Modèle enregistré'"
```
Puis redémarre tout proprement (le modèle en cache `large-v3` reste intact, on ajoute juste le nouveau) :
```bash
cd ~/speaches
docker compose down
docker compose up -d
```
Surveille le téléchargement (~800 MB, plus léger que large-v3) :
```bash
docker logs -f speaches-stt
```
Tu devrais voir `speaches-init` confirmer `Modèle enregistré` au bout de ~30s, puis speaches-stt être prêt. Dis-moi ce que donnent les logs !
<span style="display:none">[^5_10][^5_11][^5_12][^5_13][^5_14][^5_15][^5_3][^5_4][^5_5][^5_6][^5_7][^5_8][^5_9]</span>
<div align="center">⁂</div>
[^5_1]: https://huggingface.co/deepdml/faster-whisper-large-v3-turbo-ct2
[^5_2]: https://github.com/SYSTRAN/faster-whisper/issues/1030
[^5_3]: https://huggingface.co/Systran/faster-whisper-large-v3
[^5_4]: https://www.toolify.ai/ai-model/systran-faster-whisper-large-v3
[^5_5]: https://github.com/SYSTRAN/faster-whisper
[^5_6]: https://console.groq.com/docs/model/whisper-large-v3-turbo
[^5_7]: https://github.com/SYSTRAN/faster-whisper/issues/1025
[^5_8]: https://catalog.ngc.nvidia.com/orgs/nvidia/teams/riva/models/whisper_large_turbo
[^5_9]: https://github.com/SYSTRAN/faster-whisper/issues/567
[^5_10]: https://www.oreateai.com/blog/downloading-whisperlargev3turbo-a-guide-to-enhanced-speech-recognition/5f278388c08fe192dc30d0c0844ee0a4
[^5_11]: https://www.kaggle.com/datasets/rikstillman/faster-whisper-large-v3-model
[^5_12]: https://github.com/openai/whisper
[^5_13]: https://www.reddit.com/r/LocalLLaMA/comments/1ftjqg9/whisper_turbo_now_supported_in_transformers/
[^5_14]: https://huggingface.co/openai/whisper-large-v3-turbo
[^5_15]: https://github.com/systran/faster-whisper
---
# GNU nano 8.4 /home/folzo/speaches/docker-compose.yml
services:
speaches:
image: speaches:rocm-ct2
container_name: speaches-stt
restart: unless-stopped
ports:
- "8200:8000"
devices:
- /dev/kfd
- /dev/dri
group_add:
- "44"
- "991"
security_opt:
- seccomp=unconfined
environment:
- HSA_OVERRIDE_GFX_VERSION=11.0.0
- WHISPER__MODEL=deepdml/faster-whisper-large-v3-turbo-ct2
- WHISPER__LANGUAGE=fr
- WHISPER__COMPUTE_TYPE=int8
- WHISPER__DEVICE=cpu
volumes:
- speaches-models:/root/.cache/huggingface
speaches-init:
entrypoint: >
sh -c "sleep 5 \&\&
curl -sf -X POST http://speaches:8000/v1/models/deepdml%2Ffaster-whisper-large-v3-turbo-ct2 \&\&
echo 'Modèle enregistré'"
volumes:
speaches-models:
^Cfolzo@folzo-Corp:~/speachesnano ~/speaches/docker-compose.yml
folzo@folzo-Corp:~/speaches\$ cd ~/speaches
docker compose down
docker compose up -d
service "speaches-init" has neither an image nor a build context specified: invalid compose project
service "speaches-init" has neither an image nor a build context specified: invalid compose project
Le service `speaches-init` a perdu son champ `image` pendant l'édition. Voici le fichier complet corrigé :
```bash
nano ~/speaches/docker-compose.yml
```
Remplace tout le contenu par :
```yaml
services:
speaches:
image: speaches:rocm-ct2
container_name: speaches-stt
restart: unless-stopped
ports:
- "8200:8000"
devices:
- /dev/kfd
- /dev/dri
group_add:
- "44"
- "991"
security_opt:
- seccomp=unconfined
environment:
- HSA_OVERRIDE_GFX_VERSION=11.0.0
- WHISPER__MODEL=deepdml/faster-whisper-large-v3-turbo-ct2
- WHISPER__LANGUAGE=fr
- WHISPER__COMPUTE_TYPE=int8
- WHISPER__DEVICE=cpu
volumes:
- speaches-models:/root/.cache/huggingface
speaches-init:
image: curlimages/curl:latest
container_name: speaches-init
restart: "no"
depends_on:
- speaches
entrypoint: >
sh -c "sleep 5 &&
curl -sf -X POST http://speaches:8000/v1/models/deepdml%2Ffaster-whisper-large-v3-turbo-ct2 &&
echo 'Modèle enregistré'"
volumes:
speaches-models:
```
Sauvegarde avec `Ctrl+O` → `Entrée` → `Ctrl+X`, puis :
```bash
cd ~/speaches && docker compose up -d
```
---
# encore une erreur :
olzo@folzo-Corp:~/speaches\$ docker logs -f speaches-stt
2026-02-22 11:59:59,545:DEBUG:speaches.main:create_app:102:Config: stt_model_ttl=300 tts_model_ttl=300 vad_model_ttl=-1 api_key=None log_level='debug' host='0.0.0.0' port=8000 allow_origins=None enable_ui=True whisper=WhisperConfig(inference_device='auto', device_index=0, compute_type='int8', cpu_threads=0, num_workers=1) loopback_host_url=None chat_completion_base_url='http://localhost:11434/v1' chat_completion_api_key=SecretStr('**********') unstable_ort_opts=OrtOptions(exclude_providers=['TensorrtExecutionProvider'], provider_priority={'CUDAExecutionProvider': 100}, provider_opts={}) otel_exporter_otlp_endpoint=None otel_service_name='speaches' preload_models=[]
2026-02-22 12:00:00,459:DEBUG:urllib3.connectionpool:_new_conn:1051:Starting new HTTPS connection (1): huggingface.co:443
2026-02-22 12:00:00,513:DEBUG:urllib3.connectionpool:_new_conn:1051:Starting new HTTPS connection (1): huggingface.co:443
2026-02-22 12:00:00,631:DEBUG:urllib3.connectionpool:_make_request:546:https://huggingface.co:443 "HEAD /api/telemetry/https%3A/api.gradio.app/gradio-initiated-analytics HTTP/11" 200 0
2026-02-22 12:00:00,760:DEBUG:urllib3.connectionpool:_make_request:546:https://huggingface.co:443 "HEAD /api/telemetry/https%3A/api.gradio.app/gradio-initiated-analytics HTTP/11" 200 0
new
2026-02-22 12:00:00,771:INFO:speaches.main:create_app:202:
To view the gradio web ui of speaches open your browser and visit:
http://localhost:8000/
INFO: Started server process [1]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on [http://0.0.0.0:8000](http://0.0.0.0:8000) (Press CTRL+C to quit)
2026-02-22 12:00:01,198:INFO:httpx:_send_single_request:1025:HTTP Request: GET [https://api.gradio.app/pkg-version](https://api.gradio.app/pkg-version) "HTTP/1.1 200 OK"
2026-02-22 12:00:01,198:INFO:httpx:_send_single_request:1025:HTTP Request: GET [https://api.gradio.app/pkg-version](https://api.gradio.app/pkg-version) "HTTP/1.1 200 OK"
2026-02-22 12:00:01,320:INFO:httpx:_send_single_request:1025:HTTP Request: GET [https://api.gradio.app/pkg-version](https://api.gradio.app/pkg-version) "HTTP/1.1 200 OK"
2026-02-22 12:00:01,320:INFO:httpx:_send_single_request:1025:HTTP Request: GET [https://api.gradio.app/pkg-version](https://api.gradio.app/pkg-version) "HTTP/1.1 200 OK"
2026-02-22 12:00:02,520:DEBUG:urllib3.connectionpool:_new_conn:1051:Starting new HTTPS connection (1): huggingface.co:443
2026-02-22 12:00:02,714:DEBUG:urllib3.connectionpool:_make_request:546:https://huggingface.co:443 "GET /api/models?filter=ctranslate2\&filter=automatic-speech-recognition\&cardData=True HTTP/11" 200 244215
Invalid model-index. Not loading eval results into CardData.
2026-02-22 12:00:02,846:WARNING:huggingface_hub.repocard_data:__init__:368:Invalid model-index. Not loading eval results into CardData.
2026-02-22 12:00:02,972:DEBUG:urllib3.connectionpool:_make_request:546:https://huggingface.co:443 "GET /api/models/deepdml/faster-whisper-large-v3-turbo-ct2/revision/main HTTP/11" 200 2873
2026-02-22 12:00:02,974:DEBUG:urllib3.connectionpool:_new_conn:1051:Starting new HTTPS connection (1): huggingface.co:443
2026-02-22 12:00:02,976:DEBUG:urllib3.connectionpool:_new_conn:1051:Starting new HTTPS connection (1): huggingface.co:443
2026-02-22 12:00:02,978:DEBUG:urllib3.connectionpool:_new_conn:1051:Starting new HTTPS connection (1): huggingface.co:443
2026-02-22 12:00:02,978:DEBUG:urllib3.connectionpool:_new_conn:1051:Starting new HTTPS connection (1): huggingface.co:443
2026-02-22 12:00:03,079:DEBUG:urllib3.connectionpool:_make_request:546:https://huggingface.co:443 "HEAD /deepdml/faster-whisper-large-v3-turbo-ct2/resolve/4df90f75321148c3a29a9e2351b7ddf8f5b115a8/model.bin HTTP/11" 302 0
2026-02-22 12:00:03,079:DEBUG:urllib3.connectionpool:_make_request:546:https://huggingface.co:443 "HEAD /deepdml/faster-whisper-large-v3-turbo-ct2/resolve/4df90f75321148c3a29a9e2351b7ddf8f5b115a8/config.json HTTP/11" 307 0
2026-02-22 12:00:03,085:DEBUG:filelock:acquire:331:Attempting to acquire lock 131631311507184 on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock
2026-02-22 12:00:03,085:DEBUG:filelock:acquire:334:Lock 131631311507184 acquired on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock
2026-02-22 12:00:03,094:DEBUG:urllib3.connectionpool:_make_request:546:https://huggingface.co:443 "HEAD /api/resolve-cache/models/deepdml/faster-whisper-large-v3-turbo-ct2/4df90f75321148c3a29a9e2351b7ddf8f5b115a8/config.json HTTP/11" 200 0
2026-02-22 12:00:03,095:DEBUG:filelock:acquire:331:Attempting to acquire lock 131631311513904 on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/0351d1d6870005e865747b781b5d7c23ea0459cd.lock
2026-02-22 12:00:03,095:DEBUG:filelock:acquire:334:Lock 131631311513904 acquired on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/0351d1d6870005e865747b781b5d7c23ea0459cd.lock
2026-02-22 12:00:03,128:DEBUG:urllib3.connectionpool:_make_request:546:https://huggingface.co:443 "HEAD /deepdml/faster-whisper-large-v3-turbo-ct2/resolve/4df90f75321148c3a29a9e2351b7ddf8f5b115a8/tokenizer.json HTTP/11" 307 0
2026-02-22 12:00:03,131:DEBUG:urllib3.connectionpool:_make_request:546:https://huggingface.co:443 "HEAD /deepdml/faster-whisper-large-v3-turbo-ct2/resolve/4df90f75321148c3a29a9e2351b7ddf8f5b115a8/vocabulary.json HTTP/11" 307 0
2026-02-22 12:00:03,143:DEBUG:urllib3.connectionpool:_make_request:546:https://huggingface.co:443 "HEAD /api/resolve-cache/models/deepdml/faster-whisper-large-v3-turbo-ct2/4df90f75321148c3a29a9e2351b7ddf8f5b115a8/tokenizer.json HTTP/11" 200 0
2026-02-22 12:00:03,143:DEBUG:filelock:acquire:331:Attempting to acquire lock 131631311512464 on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/17456db595adc78a973f97d69d8cb50bc87c0b1c.lock
2026-02-22 12:00:03,143:DEBUG:filelock:acquire:334:Lock 131631311512464 acquired on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/17456db595adc78a973f97d69d8cb50bc87c0b1c.lock
2026-02-22 12:00:03,146:DEBUG:urllib3.connectionpool:_make_request:546:https://huggingface.co:443 "HEAD /api/resolve-cache/models/deepdml/faster-whisper-large-v3-turbo-ct2/4df90f75321148c3a29a9e2351b7ddf8f5b115a8/vocabulary.json HTTP/11" 200 0
2026-02-22 12:00:03,146:DEBUG:filelock:acquire:331:Attempting to acquire lock 131631311512944 on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/0adcd01e7c237205d593b707e66dd5d7bc785d2d.lock
2026-02-22 12:00:03,146:DEBUG:filelock:acquire:334:Lock 131631311512944 acquired on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/0adcd01e7c237205d593b707e66dd5d7bc785d2d.lock
2026-02-22 12:00:03,155:DEBUG:urllib3.connectionpool:_make_request:546:https://huggingface.co:443 "HEAD /deepdml/faster-whisper-large-v3-turbo-ct2/resolve/4df90f75321148c3a29a9e2351b7ddf8f5b115a8/README.md HTTP/11" 307 0
2026-02-22 12:00:03,183:DEBUG:urllib3.connectionpool:_make_request:546:https://huggingface.co:443 "HEAD /deepdml/faster-whisper-large-v3-turbo-ct2/resolve/4df90f75321148c3a29a9e2351b7ddf8f5b115a8/preprocessor_config.json HTTP/11" 307 0
2026-02-22 12:00:03,194:DEBUG:urllib3.connectionpool:_make_request:546:https://huggingface.co:443 "GET /api/resolve-cache/models/deepdml/faster-whisper-large-v3-turbo-ct2/4df90f75321148c3a29a9e2351b7ddf8f5b115a8/config.json HTTP/11" 200 None
2026-02-22 12:00:03,195:DEBUG:filelock:release:364:Attempting to release lock 131631311513904 on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/0351d1d6870005e865747b781b5d7c23ea0459cd.lock
2026-02-22 12:00:03,195:DEBUG:filelock:release:367:Lock 131631311513904 released on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/0351d1d6870005e865747b781b5d7c23ea0459cd.lock
2026-02-22 12:00:03,196:DEBUG:urllib3.connectionpool:_make_request:546:https://huggingface.co:443 "GET /api/models/deepdml/faster-whisper-large-v3-turbo-ct2/xet-read-token/4df90f75321148c3a29a9e2351b7ddf8f5b115a8 HTTP/11" 200 365
2026-02-22 12:00:03,198:DEBUG:urllib3.connectionpool:_make_request:546:https://huggingface.co:443 "HEAD /api/resolve-cache/models/deepdml/faster-whisper-large-v3-turbo-ct2/4df90f75321148c3a29a9e2351b7ddf8f5b115a8/preprocessor_config.json HTTP/11" 200 0
2026-02-22 12:00:03,198:DEBUG:filelock:acquire:331:Attempting to acquire lock 131631311506848 on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/931c77a740890c46365c7ae0c9d350ba3cca908f.lock
2026-02-22 12:00:03,198:DEBUG:filelock:acquire:334:Lock 131631311506848 acquired on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/931c77a740890c46365c7ae0c9d350ba3cca908f.lock
2026-02-22 12:00:03,243:DEBUG:urllib3.connectionpool:_make_request:546:https://huggingface.co:443 "GET /api/resolve-cache/models/deepdml/faster-whisper-large-v3-turbo-ct2/4df90f75321148c3a29a9e2351b7ddf8f5b115a8/tokenizer.json HTTP/11" 200 None
2026-02-22 12:00:03,248:DEBUG:urllib3.connectionpool:_make_request:546:https://huggingface.co:443 "GET /api/resolve-cache/models/deepdml/faster-whisper-large-v3-turbo-ct2/4df90f75321148c3a29a9e2351b7ddf8f5b115a8/vocabulary.json HTTP/11" 200 None
2026-02-22 12:00:03,254:DEBUG:urllib3.connectionpool:_make_request:546:https://huggingface.co:443 "HEAD /api/resolve-cache/models/deepdml/faster-whisper-large-v3-turbo-ct2/4df90f75321148c3a29a9e2351b7ddf8f5b115a8/README.md HTTP/11" 200 0
2026-02-22 12:00:03,255:DEBUG:filelock:acquire:331:Attempting to acquire lock 131631311404352 on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/fb53844bcca20e84e58c86859351b0af33771b87.lock
2026-02-22 12:00:03,255:DEBUG:filelock:acquire:334:Lock 131631311404352 acquired on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/fb53844bcca20e84e58c86859351b0af33771b87.lock
2026-02-22 12:00:03,286:DEBUG:filelock:release:364:Attempting to release lock 131631311512944 on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/0adcd01e7c237205d593b707e66dd5d7bc785d2d.lock
2026-02-22 12:00:03,286:DEBUG:filelock:release:367:Lock 131631311512944 released on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/0adcd01e7c237205d593b707e66dd5d7bc785d2d.lock
2026-02-22 12:00:03,299:DEBUG:urllib3.connectionpool:_make_request:546:https://huggingface.co:443 "GET /api/resolve-cache/models/deepdml/faster-whisper-large-v3-turbo-ct2/4df90f75321148c3a29a9e2351b7ddf8f5b115a8/preprocessor_config.json HTTP/11" 200 340
2026-02-22 12:00:03,300:DEBUG:filelock:release:364:Attempting to release lock 131631311506848 on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/931c77a740890c46365c7ae0c9d350ba3cca908f.lock
2026-02-22 12:00:03,300:DEBUG:filelock:release:367:Lock 131631311506848 released on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/931c77a740890c46365c7ae0c9d350ba3cca908f.lock
2026-02-22 12:00:03,300:DEBUG:filelock:release:364:Attempting to release lock 131631311512464 on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/17456db595adc78a973f97d69d8cb50bc87c0b1c.lock
2026-02-22 12:00:03,300:DEBUG:filelock:release:367:Lock 131631311512464 released on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/17456db595adc78a973f97d69d8cb50bc87c0b1c.lock
2026-02-22 12:00:03,357:DEBUG:urllib3.connectionpool:_make_request:546:https://huggingface.co:443 "GET /api/resolve-cache/models/deepdml/faster-whisper-large-v3-turbo-ct2/4df90f75321148c3a29a9e2351b7ddf8f5b115a8/README.md HTTP/11" 200 None
2026-02-22 12:00:03,358:DEBUG:filelock:release:364:Attempting to release lock 131631311404352 on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/fb53844bcca20e84e58c86859351b0af33771b87.lock
2026-02-22 12:00:03,358:DEBUG:filelock:release:367:Lock 131631311404352 released on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/fb53844bcca20e84e58c86859351b0af33771b87.lock
2026-02-22 12:00:17,998:DEBUG:speaches.dependencies:audio_file_dependency:89:Decoding audio file: c557b30f-5a36-4f95-8a19-2180897c60ea.mp3, content_type: None, header: Headers({'content-disposition': 'form-data; name="file"; filename="c557b30f-5a36-4f95-8a19-2180897c60ea.mp3"'}), size: 45165
2026-02-22 12:00:18,106:DEBUG:speaches.dependencies:audio_file_dependency:103:Decoded 5.58s of audio in 0.10827s (RTF: 0.01940251379999863)
2026-02-22 12:00:18,107:DEBUG:asyncio:__init__:64:Using selector: EpollSelector
2026-02-22 12:00:18,107:ERROR:speaches.main:_custom_http_exception_handler:154:HTTP error: 404: Model 'Systran/faster-whisper-large-v3-turbo' is not installed locally. You can download the model using `POST /v1/models`
INFO: 192.168.1.27:53486 - "POST /v1/audio/transcriptions HTTP/1.1" 404 Not Found
2026-02-22 12:00:18,112:DEBUG:speaches.dependencies:audio_file_dependency:89:Decoding audio file: c557b30f-5a36-4f95-8a19-2180897c60ea.mp3, content_type: None, header: Headers({'content-disposition': 'form-data; name="file"; filename="c557b30f-5a36-4f95-8a19-2180897c60ea.mp3"'}), size: 45165
2026-02-22 12:00:18,181:DEBUG:speaches.dependencies:audio_file_dependency:103:Decoded 5.58s of audio in 0.06847s (RTF: 0.01227030716826021)
2026-02-22 12:00:18,181:DEBUG:asyncio:__init__:64:Using selector: EpollSelector
2026-02-22 12:00:18,181:ERROR:speaches.main:_custom_http_exception_handler:154:HTTP error: 404: Model 'Systran/faster-whisper-large-v3-turbo' is not installed locally. You can download the model using `POST /v1/models`
INFO: 192.168.1.27:53496 - "POST /v1/audio/transcriptions HTTP/1.1" 404 Not Found
2026-02-22 12:00:32,620:DEBUG:filelock:release:364:Attempting to release lock 131631311507184 on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock
2026-02-22 12:00:32,620:DEBUG:filelock:release:367:Lock 131631311507184 released on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock
Fetching 6 files: 100%|██████████| 6/6 [00:29<00:00, 4.94s/it]
INFO: 172.18.0.3:55182 - "POST /v1/models/deepdml/faster-whisper-large-v3-turbo-ct2 HTTP/1.1" 200 OK
Tout se passe bien ! Voici ce que les logs indiquent :
- ✅ **Modèle téléchargé** : `Fetching 6 files: 100%` en 29 secondes
- ✅ **speaches-init a enregistré le modèle** : `POST /v1/models/deepdml/faster-whisper-large-v3-turbo-ct2` → **200 OK**
- ⚠️ **Les erreurs 404** en cours de route viennent d'Open WebUI qui cherchait encore l'ancien nom `Systran/faster-whisper-large-v3-turbo` — c'est normal, il faut mettre à jour la config
## Action : mettre à jour Open WebUI
Dans l'interface Open WebUI sur **effi-server** :
**Admin Panel → Settings → Audio → STT Model**
```
# Avant
Systran/faster-whisper-large-v3
# Après
deepdml/faster-whisper-large-v3-turbo-ct2
```
Sauvegarde, puis **teste une transcription vocale** dans Effi.
Dans les logs speaches, tu devrais maintenant voir apparaître un bloc du type :
```
Transcribing audio...
RTF: 0.0X ← doit être < 0.5 (beaucoup plus rapide qu'avant)
```
Dis-moi le RTF que tu observes — ça confirmera le gain de vitesse — et on passera au **Fix \#2** (basculer TTS sur Piper pour libérer 4 GB VRAM).
---
## Fix \#2 — Basculer TTS vers Piper (libère 4 GB VRAM)
C'est le fix le plus simple, tout se passe dans Open WebUI.
**Admin Panel → Settings → Audio → TTS Model**
```
# Avant
tts-1-hd ← XTTS v2, ~4 GB VRAM sur la 7800 XT
# Après
tts-1 ← Piper, tourne sur CPU, quasi aucune ressource
```
Sauvegarde. C'est tout — pas de redémarrage de conteneur nécessaire.
***
## Ce que ça change concrètement
| | Avant | Après |
| :-- | :-- | :-- |
| TTS backend | XTTS v2 (GPU) | Piper (CPU) |
| VRAM consommée | ~4 GB | ~0 GB |
| Qualité voix | Très naturelle | Correcte, légèrement synthétique |
| Latence génération | ~2-3s | ~0.3s (quasi instantané) |
La qualité Piper est un cran en dessous d'XTTS, mais la latence est **bien meilleure** pour un usage conversationnel. Si la qualité vocale te manque vraiment, on pourra revenir à `tts-1-hd` une fois le budget VRAM optimisé avec le Fix \#3.
***
Dis-moi quand c'est fait, on attaquera le **Fix \#3** — configurer mem0 pour utiliser `qwen2.5:3b` au lieu du 14b pour les opérations mémoire.
---
# la configuration est bonne ? :
L'image est très petite et difficile à lire en détail. Les deux points clés à vérifier :
**✅ Ce qui doit être configuré :**
| Champ | Valeur attendue |
| :-- | :-- |
| **STT Engine** | OpenAI |
| **STT URL** | `http://192.168.1.11:8200/v1` |
| **STT Model** | `deepdml/faster-whisper-large-v3-turbo-ct2` |
| **TTS Engine** | OpenAI |
| **TTS URL** | `http://192.168.1.11:8100/v1` |
| **TTS Model** | `tts-1` |
Peux-tu confirmer que ces valeurs correspondent à ce que tu vois à l'écran ? En particulier :
- Le **STT Model** est bien `deepdml/faster-whisper-large-v3-turbo-ct2` (et non l'ancien `Systran/...`) ?
- Le **TTS Model** est bien `tts-1` (et non `tts-1-hd`) ?
Si tout est bon, on passe au **Fix \#3** — mem0 avec `qwen2.5:3b`.
<span style="display:none">[^9_1]</span>
<div align="center">⁂</div>
[^9_1]: image.jpg
---
# et la voix est reglé sur alloy c'est la bonne ?
**Oui, `alloy` fonctionne** avec `tts-1` (Piper). C'est un bon choix pour commencer.
Une petite nuance à avoir en tête : avec `tts-1-hd` (XTTS), les voix utilisaient les fichiers WAV que tu avais dans `tts-voices/` comme référence de clonage vocal. Avec `tts-1` (Piper), `alloy` pointe vers un modèle ONNX anglais — ça fonctionnera en français mais avec une légère "sonorité anglophone" possible sur certains mots.
Si la voix te semble trop robotique ou mal accentuée en français, on pourra plus tard configurer une voix Piper française native (ex : `fr_FR-upmc-medium`). Mais pour l'instant c'est largement suffisant pour valider le pipeline.
***
**Fix \#2 est terminé ✅**
Récap de ce qu'on a fait :
- ✅ Fix \#1 — STT : `large-v3-turbo-ct2` (CPU beaucoup plus rapide)
- ✅ Fix \#2 — TTS : `tts-1` Piper (libère ~4 GB VRAM)
***
## Fix \#3 — mem0 avec `qwen2.5:3b`
On attaque la partie la plus impactante sur le GPU. D'abord, télécharge le petit modèle sur **folzo-Corp** :
```bash
ollama pull qwen2.5:3b
```
Pendant ce temps, montre-moi comment mem0 est configuré actuellement — soit le fichier de config, soit le pipeline Open WebUI qui l'utilise. C'est sur **effi-server**, probablement dans `~/effi/` ou dans les pipelines Open WebUI.
---
# ok parfait, mais j'ai toujours un probleme mon gpu touurne a fon longtemps apres avoir demander quelque chose a effi, est-ce normal comment voir ce qui demande du gpu ?
**GPU à 100% longtemps après une requête, c'est anormal.** La VRAM à 8.93 GB correspond au modèle `qwen2.5:14b` chargé — c'est normal. Mais le 100% d'utilisation persistant pointe vers un process qui tourne encore. Il faut identifier lequel.
## Diagnostic en temps réel
Sur **folzo-Corp**, ouvre deux terminaux :
**Terminal 1 — identifier le process GPU :**
```bash
watch -n 1 rocm-smi --showpidgpus
```
**Terminal 2 — voir ce que font les conteneurs Docker :**
```bash
docker stats
```
**Terminal 3 — logs Ollama (tourne en natif sur folzo-Corp) :**
```bash
journalctl -u ollama -f --no-pager
# ou si lancé manuellement :
ollama ps
```
***
## Les coupables les plus probables
| Process | Signe dans les logs | Pourquoi ça dure |
| :-- | :-- | :-- |
| **Ollama (qwen2.5:14b)** | `ollama ps` montre le modèle actif | mem0 déclenche un 2ème appel LLM après la réponse |
| **openedai-speech** | Conteneur à >50% CPU/GPU dans `docker stats` | Génère l'audio XTTS en post-traitement |
| **bge-m3 embeddings** | Pic GPU après la réponse | Open WebUI indexe la conversation dans Qdrant |
Lance ces commandes pendant qu'Effi "refroidit" et dis-moi ce que `rocm-smi --showpidgpus` et `ollama ps` affichent — on identifiera le coupable précis.
<span style="display:none">[^11_1]</span>
<div align="center">⁂</div>
[^11_1]: image.jpg
---
# eral.languages arr[str,1] = ["en"]
févr. 22 13:11:11 folzo-Corp ollama[4303]: llama_model_loader: - kv 15: qwen2.block_count u32 = 36
févr. 22 13:11:11 folzo-Corp ollama[4303]: llama_model_loader: - kv 16: qwen2.context_length u32 = 32768
févr. 22 13:11:11 folzo-Corp ollama[4303]: llama_model_loader: - kv 17: qwen2.embedding_length u32 = 2048
févr. 22 13:11:11 folzo-Corp ollama[4303]: llama_model_loader: - kv 18: qwen2.feed_forward_length u32 = 11008
févr. 22 13:11:11 folzo-Corp ollama[4303]: llama_model_loader: - kv 19: qwen2.attention.head_count u32 = 16
févr. 22 13:11:11 folzo-Corp ollama[4303]: llama_model_loader: - kv 20: qwen2.attention.head_count_kv u32 = 2
févr. 22 13:11:11 folzo-Corp ollama[4303]: llama_model_loader: - kv 21: qwen2.rope.freq_base f32 = 1000000.000000
févr. 22 13:11:11 folzo-Corp ollama[4303]: llama_model_loader: - kv 22: qwen2.attention.layer_norm_rms_epsilon f32 = 0.000001
févr. 22 13:11:11 folzo-Corp ollama[4303]: llama_model_loader: - kv 23: general.file_type u32 = 15
févr. 22 13:11:11 folzo-Corp ollama[4303]: llama_model_loader: - kv 24: tokenizer.ggml.model str = gpt2
févr. 22 13:11:11 folzo-Corp ollama[4303]: llama_model_loader: - kv 25: tokenizer.ggml.pre str = qwen2
févr. 22 13:11:11 folzo-Corp ollama[4303]: llama_model_loader: - kv 26: tokenizer.ggml.tokens arr[str,151936] = ["!", "\\"", "\#", "\$", "%", "\&", "'", ...
févr. 22 13:11:11 folzo-Corp ollama[4303]: llama_model_loader: - kv 27: tokenizer.ggml.token_type arr[i32,151936] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
févr. 22 13:11:11 folzo-Corp ollama[4303]: llama_model_loader: - kv 28: tokenizer.ggml.merges arr[str,151387] = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",...
févr. 22 13:11:11 folzo-Corp ollama[4303]: llama_model_loader: - kv 29: tokenizer.ggml.eos_token_id u32 = 151645
févr. 22 13:11:11 folzo-Corp ollama[4303]: llama_model_loader: - kv 30: tokenizer.ggml.padding_token_id u32 = 151643
févr. 22 13:11:11 folzo-Corp ollama[4303]: llama_model_loader: - kv 31: tokenizer.ggml.bos_token_id u32 = 151643
févr. 22 13:11:11 folzo-Corp ollama[4303]: llama_model_loader: - kv 32: tokenizer.ggml.add_bos_token bool = false
févr. 22 13:11:11 folzo-Corp ollama[4303]: llama_model_loader: - kv 33: tokenizer.chat_template str = {%- if tools %}\\n {{- '<|im_start|>...
févr. 22 13:11:11 folzo-Corp ollama[4303]: llama_model_loader: - kv 34: general.quantization_version u32 = 2
févr. 22 13:11:11 folzo-Corp ollama[4303]: llama_model_loader: - type f32: 181 tensors
févr. 22 13:11:11 folzo-Corp ollama[4303]: llama_model_loader: - type q4_K: 216 tensors
févr. 22 13:11:11 folzo-Corp ollama[4303]: llama_model_loader: - type q6_K: 37 tensors
févr. 22 13:11:11 folzo-Corp ollama[4303]: print_info: file format = GGUF V3 (latest)
févr. 22 13:11:11 folzo-Corp ollama[4303]: print_info: file type = Q4_K - Medium
févr. 22 13:11:11 folzo-Corp ollama[4303]: print_info: file size = 1.79 GiB (4.99 BPW)
févr. 22 13:11:12 folzo-Corp ollama[4303]: load: printing all EOG tokens:
févr. 22 13:11:12 folzo-Corp ollama[4303]: load: - 151643 ('<|endoftext|>')
févr. 22 13:11:12 folzo-Corp ollama[4303]: load: - 151645 ('<|im_end|>')
févr. 22 13:11:12 folzo-Corp ollama[4303]: load: - 151662 ('<|fim_pad|>')
févr. 22 13:11:12 folzo-Corp ollama[4303]: load: - 151663 ('<|repo_name|>')
févr. 22 13:11:12 folzo-Corp ollama[4303]: load: - 151664 ('<|file_sep|>')
févr. 22 13:11:12 folzo-Corp ollama[4303]: load: special tokens cache size = 22
févr. 22 13:11:12 folzo-Corp ollama[4303]: load: token to piece cache size = 0.9310 MB
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: arch = qwen2
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: vocab_only = 0
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: no_alloc = 0
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: n_ctx_train = 32768
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: n_embd = 2048
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: n_embd_inp = 2048
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: n_layer = 36
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: n_head = 16
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: n_head_kv = 2
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: n_rot = 128
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: n_swa = 0
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: is_swa_any = 0
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: n_embd_head_k = 128
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: n_embd_head_v = 128
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: n_gqa = 8
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: n_embd_k_gqa = 256
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: n_embd_v_gqa = 256
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: f_norm_eps = 0.0e+00
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: f_norm_rms_eps = 1.0e-06
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: f_clamp_kqv = 0.0e+00
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: f_max_alibi_bias = 0.0e+00
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: f_logit_scale = 0.0e+00
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: f_attn_scale = 0.0e+00
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: n_ff = 11008
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: n_expert = 0
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: n_expert_used = 0
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: n_expert_groups = 0
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: n_group_used = 0
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: causal attn = 1
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: pooling type = -1
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: rope type = 2
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: rope scaling = linear
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: freq_base_train = 1000000.0
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: freq_scale_train = 1
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: n_ctx_orig_yarn = 32768
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: rope_yarn_log_mul= 0.0000
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: rope_finetuned = unknown
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: model type = 3B
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: model params = 3.09 B
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: general.name = Qwen2.5 3B Instruct
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: vocab type = BPE
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: n_vocab = 151936
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: n_merges = 151387
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: BOS token = 151643 '<|endoftext|>'
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: EOS token = 151645 '<|im_end|>'
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: EOT token = 151645 '<|im_end|>'
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: PAD token = 151643 '<|endoftext|>'
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: LF token = 198 'Ċ'
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: FIM PRE token = 151659 '<|fim_prefix|>'
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: FIM SUF token = 151661 '<|fim_suffix|>'
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: FIM MID token = 151660 '<|fim_middle|>'
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: FIM PAD token = 151662 '<|fim_pad|>'
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: FIM REP token = 151663 '<|repo_name|>'
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: FIM SEP token = 151664 '<|file_sep|>'
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: EOG token = 151643 '<|endoftext|>'
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: EOG token = 151645 '<|im_end|>'
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: EOG token = 151662 '<|fim_pad|>'
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: EOG token = 151663 '<|repo_name|>'
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: EOG token = 151664 '<|file_sep|>'
févr. 22 13:11:12 folzo-Corp ollama[4303]: print_info: max token length = 256
févr. 22 13:11:12 folzo-Corp ollama[4303]: load_tensors: loading model tensors, this can take a while... (mmap = true)
févr. 22 13:11:12 folzo-Corp ollama[4303]: load_tensors: offloading 36 repeating layers to GPU
févr. 22 13:11:12 folzo-Corp ollama[4303]: load_tensors: offloading output layer to GPU
févr. 22 13:11:12 folzo-Corp ollama[4303]: load_tensors: offloaded 37/37 layers to GPU
févr. 22 13:11:12 folzo-Corp ollama[4303]: load_tensors: CPU_Mapped model buffer size = 243.43 MiB
févr. 22 13:11:12 folzo-Corp ollama[4303]: load_tensors: ROCm0 model buffer size = 1834.83 MiB
févr. 22 13:11:12 folzo-Corp ollama[4303]: llama_context: constructing llama_context
févr. 22 13:11:12 folzo-Corp ollama[4303]: llama_context: n_seq_max = 1
févr. 22 13:11:12 folzo-Corp ollama[4303]: llama_context: n_ctx = 4096
févr. 22 13:11:12 folzo-Corp ollama[4303]: llama_context: n_ctx_seq = 4096
févr. 22 13:11:12 folzo-Corp ollama[4303]: llama_context: n_batch = 512
févr. 22 13:11:12 folzo-Corp ollama[4303]: llama_context: n_ubatch = 512
févr. 22 13:11:12 folzo-Corp ollama[4303]: llama_context: causal_attn = 1
févr. 22 13:11:12 folzo-Corp ollama[4303]: llama_context: flash_attn = auto
févr. 22 13:11:12 folzo-Corp ollama[4303]: llama_context: kv_unified = false
févr. 22 13:11:12 folzo-Corp ollama[4303]: llama_context: freq_base = 1000000.0
févr. 22 13:11:12 folzo-Corp ollama[4303]: llama_context: freq_scale = 1
févr. 22 13:11:12 folzo-Corp ollama[4303]: llama_context: n_ctx_seq (4096) < n_ctx_train (32768) -- the full capacity of the model will not be utilized
févr. 22 13:11:12 folzo-Corp ollama[4303]: llama_context: ROCm_Host output buffer size = 0.59 MiB
févr. 22 13:11:12 folzo-Corp ollama[4303]: llama_kv_cache: ROCm0 KV buffer size = 144.00 MiB
févr. 22 13:11:12 folzo-Corp ollama[4303]: llama_kv_cache: size = 144.00 MiB ( 4096 cells, 36 layers, 1/1 seqs), K (f16): 72.00 MiB, V (f16): 72.00 MiB
févr. 22 13:11:12 folzo-Corp ollama[4303]: llama_context: Flash Attention was auto, set to enabled
févr. 22 13:11:12 folzo-Corp ollama[4303]: llama_context: ROCm0 compute buffer size = 300.75 MiB
févr. 22 13:11:12 folzo-Corp ollama[4303]: llama_context: ROCm_Host compute buffer size = 12.01 MiB
févr. 22 13:11:12 folzo-Corp ollama[4303]: llama_context: graph nodes = 1231
févr. 22 13:11:12 folzo-Corp ollama[4303]: llama_context: graph splits = 2
févr. 22 13:11:12 folzo-Corp ollama[4303]: time=2026-02-22T13:11:12.929+01:00 level=INFO source=server.go:1388 msg="llama runner started in 2.46 seconds"
févr. 22 13:11:12 folzo-Corp ollama[4303]: time=2026-02-22T13:11:12.929+01:00 level=INFO source=sched.go:537 msg="loaded runners" count=2
févr. 22 13:11:12 folzo-Corp ollama[4303]: time=2026-02-22T13:11:12.929+01:00 level=INFO source=server.go:1350 msg="waiting for llama runner to start responding"
févr. 22 13:11:12 folzo-Corp ollama[4303]: time=2026-02-22T13:11:12.929+01:00 level=INFO source=server.go:1388 msg="llama runner started in 2.46 seconds"
févr. 22 13:11:12 folzo-Corp ollama[4303]: time=2026-02-22T13:11:12.960+01:00 level=WARN source=runner.go:153 msg="truncating input prompt" limit=4096 prompt=5856 keep=4 new=4096
févr. 22 13:11:23 folzo-Corp ollama[4303]: [GIN] 2026/02/22 - 13:11:23 | 200 | 15.107434932s | 192.168.1.27 | POST "/v1/chat/completions"
Toutes les 1,0s: rocm-smi --showpidgpus folzo-Corp: Sun Feb 22 13:11:39 2026
============================ ROCm System Management Interface ============================
================================== GPUs Indexed by PID ===================================
PID 2330840 is using 1 DRM device(s):
0
PID 2331861 is using 1 DRM device(s):
0
PID 2278476 is using 1 DRM device(s):
0
==========================================================================================
================================== End of ROCm SMI Log ===================================
folzo@folzo-Corp:~\$ docker stats
permission denied while trying to connect to the docker API at unix:///var/run/docker.sock
folzo@folzo-Corp:~\$
Excellente capture ! Plusieurs infos importantes à analyser.
## Ce que révèlent les logs
**Bonne nouvelle — `qwen2.5:3b` est bien chargé :**
```
loaded runners count=2 ← 14b ET 3b en VRAM simultanément
ROCm0 model buffer size = 1834.83 MiB ← c'est le 3b
```
**Problème détecté — contexte tronqué :**
```
truncating input prompt limit=4096 prompt=5856
```
mem0 envoie **5856 tokens** au 3b qui n'a que 4096 de contexte — il tronque silencieusement. On corrigera ça.
**3 PIDs GPU — il faut identifier lesquels :**
```bash
for pid in 2330840 2331861 2278476; do
echo "PID $pid: $(cat /proc/$pid/cmdline | tr '\0' ' ')"
done
```
**Fix docker stats (manque sudo) :**
```bash
sudo docker stats --no-stream
```
***
## Action immédiate — augmenter le contexte du 3b
Le 3b a un contexte limité par défaut à 4096. Il faut créer un Modelfile custom :
```bash
ollama show qwen2.5:3b --modelfile > /tmp/qwen3b.modelfile
echo "PARAMETER num_ctx 8192" >> /tmp/qwen3b.modelfile
ollama create qwen2.5:3b-mem0 -f /tmp/qwen3b.modelfile
```
Puis dans la config mem0, pointer vers `qwen2.5:3b-mem0` au lieu de `qwen2.5:3b`.
Lance d'abord les deux commandes de diagnostic et montre-moi les résultats — surtout les noms des PIDs GPU.
---
# folzo@folzo-Corp:~\$ for pid in 2330840 2331861 2278476; do
echo "PID \$pid: $(cat /proc/$pid/cmdline | tr '\\0' ' ')"
done
PID 2330840: /usr/local/bin/ollama runner --model /usr/share/ollama/.ollama/models/blobs/sha256-daec91ffb5dd0c27411bd71f29932917c49cf529a641d0168496c3a501e3062c --port 38881
PID 2331861: /usr/local/bin/ollama runner --model /usr/share/ollama/.ollama/models/blobs/sha256-5ee4f07cdb9beadbbb293e85803c569b01bd37ed059d2715faa7bb405f31caa6 --port 33289
PID 2278476: /home/ubuntu/speaches/.venv/bin/python .venv/bin/uvicorn speaches.main:create_app --host 0.0.0.0 --port 8000 --factory
folzo@folzo-Corp:~\$ sudo docker stats --no-stream
[sudo: authenticate] Password:
CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
3a5af44bc8bb speaches-stt 0.14% 2.701GiB / 30.25GiB 8.93% 1.62GB / 14.8MB 1.28GB / 3.3GB 73
36c0ce71710d openedai-speech 0.05% 3.124GiB / 30.25GiB 10.33% 439kB / 394kB 1.2GB / 1.2GB 48
folzo@folzo-Corp:~\$
Très bonne capture ! Je vois **3 problèmes clés**.
## Analyse des PIDs GPU
```
PID 2330840 → Ollama runner (modèle sha256-daec91...) ← 14b ou 3b ?
PID 2331861 → Ollama runner (modèle sha256-5ee4f0...) ← 14b ou 3b ?
PID 2278476 → speaches STT ← utilise le GPU malgré WHISPER__DEVICE=cpu !
```
**Problème \#1 — speaches utilise quand même le GPU.** Malgré `WHISPER__DEVICE=cpu`, les devices `/dev/kfd` et `/dev/dri` sont passés au conteneur, ROCm s'initialise quand même et consomme de la VRAM. C'est lui qui maintient le GPU actif.
**Problème \#2 — openedai-speech consomme 3.1 GB RAM** malgré le passage à `tts-1` → les modèles XTTS sont probablement encore chargés en mémoire.
**Problème \#3 — deux runners Ollama actifs simultanément** → 14b + 3b en VRAM en même temps.
***
## Diagnostics immédiats
**Identifier les deux modèles Ollama chargés :**
```bash
ollama ps
```
**Fix speaches — retirer les devices GPU si on reste en CPU :**
```bash
nano ~/speaches/docker-compose.yml
```
Commenter les devices et group_add :
```yaml
#devices:
# - /dev/kfd
# - /dev/dri
#group_add:
# - "44"
# - "991"
#security_opt:
# - seccomp=unconfined
```
```bash
cd ~/speaches && docker compose up -d
```
Ça libérera immédiatement la VRAM utilisée par speaches. Montre-moi d'abord `ollama ps` pour qu'on voit quels modèles sont actifs.
---
# j'ai changé qwen 2.5:3b par ministral pour la limitation de token et pour la rapidité, voici les resultats :
folzo@folzo-Corp:~\$ ollama ps
NAME ID SIZE PROCESSOR CONTEXT UNTIL
qwen2.5:14b 7cdf5a0187d5 13 GB 20%/80% CPU/GPU 16690 4 minutes from now
folzo@folzo-Corp:~\$ \#devices:
# - /dev/kfd
# - /dev/dri
\#group_add:
# - "44"
# - "991"
\#security_opt:
# - seccomp=unconfined
folzo@folzo-Corp:~\$ ollama ps
NAME ID SIZE PROCESSOR CONTEXT UNTIL
ministral-3:3b f04aa1c738f6 4.6 GB 100% GPU 4096 4 minutes from now
bge-m3:latest 790764642607 1.2 GB 100% GPU 4096 4 minutes from now
folzo@folzo-Corp:~\$ ollama ps
Par contre ca tourne toujours longtemps apres les interaction avec effi. Mais concernant la voit c'est plus rapide par contre effi a une voix anglaise...
Tu as déjà fait deux excellents moves : **ministral-3:3b** pour la mémoire et **Piper (`tts-1`)** pour le TTS. On voit maintenant clairement ce qui consomme le GPU :
```text
ministral-3:3b → 4.6 GB VRAM (mémoire / outils)
bge-m3:latest → 1.2 GB VRAM (embeddings RAG)
```
Ces deux modèles restent chargés ~5 minutes après chaque utilisation (`UNTIL 4 minutes from now`). C’est le comportement normal d’Ollama : il garde les modèles « chauds » pour accélérer les requêtes suivantes. Mais si tu enchaînes les interactions avec Effi, ils restent quasiment tout le temps en VRAM.
### Ce que tu peux faire tout de suite
1. **Vérifier qui appelle quoi :**
- `ministral-3:3b` peut être :
- le modèle de mem0 (mémoire),
- un modèle d’outils dans Open WebUI (pipelines MCP),
- ou un modèle de fallback pour le chat.
- `bge-m3:latest` est appelé par :
- Open WebUI pour le RAG,
- et/ou mem0 pour les embeddings.
2. **Voir la charge réelle GPU vs « juste chargé »**
Quand tu ne parles plus à Effi, regarde :
```bash
rocm-smi --showuse
```
Si l’occupation GPU est proche de 0% mais la VRAM reste haute, ce n’est pas un problème de charge, juste de modèles en cache. Le 100% que tu vois dans le moniteur GNOME est souvent la **file d’attente GPU saturée** au moment des requêtes, pas en permanence.
***
### Rendre Effi moins « anglophone »
Piper (`tts-1`) utilise par défaut un modèle anglais, d’où l’accent. On peut lui forcer une voix française.
1. Ouvre `~/openedai-speech/tts-config/config.yaml` (ou fichier similaire) sur folzo-Corp.
2. Cherche la section de la voix `alloy` (ou la voix par défaut) et bascule-la sur un modèle Piper français, par exemple un de ceux listés ici.[^14_1]
Exemple de mapping (structure typique)  :[^14_2][^14_3]
```yaml
voices:
alloy:
engine: piper
model: fr_FR-upmc-medium # ou autre modèle fr_* installé
```
3. Redémarre le conteneur :
```bash
cd ~/openedai-speech && docker compose restart openedai-speech
```
Si tu veux, on peut faire étape par étape sur le fichier de config Piper : tu peux me coller le contenu de `tts-config` (ou un extrait anonymisé), et je te dirai exactement quoi modifier pour une voix française propre.
<div align="center">⁂</div>
[^14_1]: https://github.com/mdmonsurali/Offline-Fast-CPU-PIPER-TTS
[^14_2]: https://github.com/matatonic/openedai-speech
[^14_3]: https://git.robby.blue/ml/openedai-speech
---
# quand je parle plus a effi :
folzo@folzo-Corp:~\$ rocm-smi --showuse
============================ ROCm System Management Interface ============================
=================================== % time GPU is busy ===================================
GPU[0] : GPU use (%): 100
==========================================================================================
================================== End of ROCm SMI Log ===================================
folzo@folzo-Corp:~\$
Là on a deux choses différentes :
1. **VRAM occupée** (modèles chargés) → normal si `ollama ps` montre des modèles.
2. **`GPU use 100%` dans `rocm-smi` même au repos** → ça ressemble à un bug de monitoring / DPM, pas forcément à un calcul réel, surtout si la conso reste basse.
Avant de tout chambouler, vérifions si le GPU travaille vraiment.
***
## 1. Vérifier la conso réelle du GPU
Lance :
```bash
rocm-smi --showpower --showclk
```
Si tu vois un truc du genre :
- **Power** autour de 20–40 W
- Fréquence mémoire / core basse
… alors le « 100% » est juste un compteur mal interprété, le GPU est en fait idle.[^15_1][^15_2]
***
## 2. Limiter le temps de cache des modèles Ollama
Par défaut, Ollama garde les modèles jusqu’à **5 minutes** après la dernière requête. Tu peux réduire ce délai globalement.[^15_3][^15_4]
Édite le service systemd d’Ollama :
```bash
sudo systemctl edit ollama
```
Ajoute :
```ini
[Service]
Environment="OLLAMA_KEEP_ALIVE=60"
```
Puis :
```bash
sudo systemctl daemon-reload
sudo systemctl restart ollama
```
Effet : tout modèle est déchargé **60s après la dernière requête**. Tu peux même mettre `OLLAMA_KEEP_ALIVE=0` si tu veux qu’il soit déchargé tout de suite après chaque réponse (au prix d’un temps de première token plus long).[^15_4][^15_3]
***
## 3. Pour forcer un déchargement immédiat
À la main, tu peux aussi éjecter les modèles dès que tu as fini ta session Effi :
```bash
ollama stop ministral-3:3b
ollama stop bge-m3:latest
```
***
Si tu m’indiques les valeurs de `rocm-smi --showpower --showclk`, je te dirai si ton GPU est réellement en charge ou juste « marqué à 100% » alors qu’il roupille.
<span style="display:none">[^15_10][^15_5][^15_6][^15_7][^15_8][^15_9]</span>
<div align="center">⁂</div>
[^15_1]: https://community.amd.com/t5/gpu-developer-tools/rocm-smi-reports-low-power-usage/m-p/213365
[^15_2]: https://github.com/ROCm/ROCm/issues/504
[^15_3]: https://docs.ollama.com/faq
[^15_4]: https://www.reddit.com/r/ollama/comments/1cnxnrv/how_to_set_keepalive_1_on_ollama_linux/
[^15_5]: https://www.reddit.com/r/ollama/comments/1fh040f/question_how_to_keep_ollama_from_unloading_model/
[^15_6]: https://github.com/ollama/ollama/issues/11085
[^15_7]: https://github.com/ollama/ollama/issues/5272
[^15_8]: https://github.com/ollama/ollama/issues/10597
[^15_9]: https://www.reddit.com/r/ollama/comments/1bt1bir/unload_model_from_memory/
[^15_10]: https://blog.csdn.net/gitblog_00488/article/details/151533510
---
# pour le probleme de voi c'est bien ce fichier que je dois modifier ?
tts-1:
some_other_voice_name_you_want:
model: voices/choose your own model.onnx
speaker: set your own speaker
alloy:
model: voices/en_US-libritts_r-medium.onnx
speaker: 79 \# 64, 79, 80, 101, 130
echo:
model: voices/en_US-libritts_r-medium.onnx
speaker: 134 \# 52, 102, 134
echo-alt:
model: voices/en_US-ryan-high.onnx
speaker: \# default speaker
fable:
model: voices/en_GB-northern_english_male-medium.onnx
speaker: \# default speaker
onyx:
model: voices/en_US-libritts_r-medium.onnx
speaker: 159 \# 55, 90, 132, 136, 137, 159
nova:
model: voices/en_US-libritts_r-medium.onnx
speaker: 107 \# 57, 61, 107, 150, 162
shimmer:
model: voices/en_US-libritts_r-medium.onnx
speaker: 163
tts-1-hd:
alloy-alt:
model: xtts
speaker: voices/alloy-alt.wav
alloy:
model: xtts
speaker: voices/alloy.wav
echo:
model: xtts
speaker: voices/echo.wav
fable:
model: xtts
speaker: voices/fable.wav
onyx:
model: xtts
speaker: voices/onyx.wav
nova:
model: xtts
speaker: voices/nova.wav
shimmer:
model: xtts
speaker: voices/shimmer.wav
me:
model: xtts_v2.0.2 \# you can specify an older xtts version
speaker: voices/me.wav \# this could be you
language: auto
enable_text_splitting: True
length_penalty: 1.0
repetition_penalty: 10
speed: 1.0
temperature: 0.75
top_k: 50
top_p: 0.85
comment: You can add a comment here also, which will be persistent and otherwise ignored.?
Oui, c’est exactement **ce fichier** qu’il faut modifier.
Actuellement, pour `tts-1` tu as uniquement des modèles anglais :
```yaml
tts-1:
alloy:
model: voices/en_US-libritts_r-medium.onnx
speaker: 79
# ...
```
Pour avoir une voix française, tu dois :
1. Mettre un modèle Piper français dans `voices/` (par ex. `fr_FR-upmc-medium.onnx` ou autre modèle `fr_*` que tu as téléchargé ).[^16_1]
2. Pointer `alloy` dessus :
```yaml
tts-1:
alloy:
model: voices/fr_FR-upmc-medium.onnx # ← modèle FR
speaker: 0 # ou l’index de speaker adapté
```
Tu peux laisser le bloc `tts-1-hd` tel quel, il ne sera pas utilisé tant que tu restes sur `tts-1` dans Open WebUI.
Après modification :
```bash
cd ~/openedai-speech
docker compose restart openedai-speech
```
Ensuite, teste Effi : si la voix reste anglaise, envoie-moi la liste des fichiers dans `~/openedai-speech/tts-voices` pour qu’on choisisse un modèle FR précis.
<div align="center">⁂</div>
[^16_1]: https://github.com/mdmonsurali/Offline-Fast-CPU-PIPER-TTS
---
# si je veux cette voix comment je peux faire ?
La voix que tu montres est le modèle Piper **`fr_FR-upmc-medium`**, locutrice “jessica”. Pour l’utiliser avec `tts-1` :
### 1. Télécharger le modèle ONNX
Sur folzo-Corp, dans le dossier `openedai-speech` :
```bash
cd ~/openedai-speech/tts-voices
# adapte l’URL si besoin, mais le fichier doit s’appeler ainsi :
wget https://github.com/rhasspy/piper/releases/download/v1.2.0/fr_FR-upmc-medium.onnx
```
(Le nom exact du fichier doit correspondre à ce que tu mettras dans la config.)
### 2. Modifier la config `tts-1`
Édite ton fichier de config (celui que tu as collé) :
```bash
nano ~/openedai-speech/tts-config/config.yaml # ou le nom équivalent
```
Remplace le bloc `tts-1` par :
```yaml
tts-1:
alloy:
model: voices/fr_FR-upmc-medium.onnx
speaker: 0 # Jessica (0), Pierre (1) selon le modèle
```
Tu peux laisser les autres voix telles quelles ou les supprimer, ce n’est pas obligatoire.
### 3. Redémarrer openedai-speech
```bash
cd ~/openedai-speech
docker compose restart openedai-speech
```
Dans Open WebUI, laisse **TTS Model = `tts-1`** et voix **alloy**. Effi devrait maintenant parler avec la voix française `fr_FR-upmc-medium`. Si tu entends encore de l’anglais ou un mauvais speaker, on ajustera juste l’index `speaker` (0 ou 1).
<span style="display:none">[^17_1]</span>
<div align="center">⁂</div>
[^17_1]: image.jpg
---
# le lien que tu m'a donné nme marche plus. voici les derniers telechargement sur piper qui dattent de 2024 : [https://github.com/rhasspy/piper/releases/](https://github.com/rhasspy/piper/releases/)
Parfait, on va le prendre directement à la source Hugging Face.
### 1. Télécharger les fichiers de la voix `fr_FR-upmc-medium`
Sur **folzo-Corp** :
```bash
cd ~/openedai-speech/tts-voices
# Modèle
wget https://huggingface.co/rhasspy/piper-voices/resolve/main/fr/fr_FR/upmc/medium/fr_FR-upmc-medium.onnx
# Config JSON associée (souvent utilisée par Piper)
wget https://huggingface.co/rhasspy/piper-voices/resolve/main/fr/fr_FR/upmc/medium/fr_FR-upmc-medium.onnx.json
```
Ces fichiers sont listés ici.[^18_1]
### 2. Configurer `tts-1` pour utiliser cette voix
Édite ta config :
```bash
nano ~/openedai-speech/tts-config/config.yaml
```
Et adapte la section `tts-1` comme ceci :
```yaml
tts-1:
alloy:
model: voices/fr_FR-upmc-medium.onnx
speaker: 0 # 0 = Jessica, 1 = Pierre (tu pourras changer si besoin)
```
Tu peux laisser les autres entrées, elles ne seront pas utilisées tant que tu choisis la voix **alloy** côté Open WebUI.
### 3. Redémarrer le TTS
```bash
cd ~/openedai-speech
docker compose restart openedai-speech
```
Ensuite, dans Open WebUI :
- TTS Engine : OpenAI
- TTS Model : `tts-1`
- Voice : `alloy`
Teste une phrase française avec Effi. Si la voix ne te plaît pas ou que tu tombes sur Pierre au lieu de Jessica, on ajustera juste `speaker: 1` au lieu de `0`.
<span style="display:none">[^18_10][^18_2][^18_3][^18_4][^18_5][^18_6][^18_7][^18_8][^18_9]</span>
<div align="center">⁂</div>
[^18_1]: https://huggingface.co/rhasspy/piper-voices/tree/main/fr/fr_FR/upmc/medium
[^18_2]: https://ftp.us2.freshrpms.net/linux/RPM/openmandriva/cooker/aarch64/main/release/piper-voice-fr_FR-upmc-2023.09.23-1.noarch.html
[^18_3]: https://huggingface.co/rhasspy/piper-voices/blame/0c9c5d340496a47205a799eb046aab69334a88e9/fr/fr_FR/upmc/medium/MODEL_CARD
[^18_4]: https://github.com/JohnnySn0w/piper-rocm
[^18_5]: https://github.com/rhasspy/piper/releases
[^18_6]: https://github.com/home-assistant/core/issues/105819
[^18_7]: https://github.com/rhasspy/piper/blob/master/README.md
[^18_8]: https://github.com/rhasspy/piper
[^18_9]: https://huggingface.co/rhasspy/piper-voices/blob/49a0d23f04e055f1766e5ad15ea2d121a90fa17a/fr/fr_FR/upmc/medium/fr_FR-upmc-medium.onnx.json
[^18_10]: https://github.com/home-assistant/addons/issues/3411
---
# folzo@folzo-Corp:~/openedai-speech/tts-voices\$ cd ~/openedai-speech
docker compose restart openedai-speech
permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get "http://%2Fvar%2Frun%2Fdocker.sock/v1.51/containers/json?all=1\&filters=%7B%22label%22%3A%7B%22com.docker.compose.config-hash%22%3Atrue%2C%22com.docker.compose.oneoff%3DFalse%22%3Atrue%2C%22com.docker.compose.project%3Dopenedai-speech%22%3Atrue%7D%7D": dial unix /var/run/docker.sock: connect: permission denied
folzo@folzo-Corp:~/openedai-speech\$
Il te manque juste les droits Docker, il faut ajouter `sudo`.
Dans `~/openedai-speech` :
```bash
cd ~/openedai-speech
sudo docker compose restart openedai-speech
```
Entre ton mot de passe, attends que le conteneur redémarre, puis teste Effi avec :
- TTS Model = `tts-1`
- Voice = `alloy`
Si la voix n’est pas celle que tu veux (Jessica vs Pierre), dis-le et on ajustera juste la valeur `speaker` (0 ou 1) dans la config.
---
# top pour la voix merci c'est vriament pas mal pour le moment. Maintenant concentron nous sur l'utilisation de la carte graphique :
============================ ROCm System Management Interface ============================
================================== GPUs Indexed by PID ===================================
PID 2278476 is using 1 DRM device(s):
0
PID 2526746 is using 1 DRM device(s):
0
==========================================================================================
================================== End of ROCm SMI Log ===================================
Parfait pour la voix, on revient sur le GPU.
On voit encore **2 PIDs qui utilisent le GPU** :
```text
PID 2278476
PID 2526746
```
On veut savoir qui c’est exactement.
## 1. Identifier les processus GPU
Sur **folzo-Corp** :
```bash
for pid in 2278476 2526746; do
echo "PID $pid: $(cat /proc/$pid/cmdline | tr '\0' ' ')"
done
```
Colle-moi la sortie. On saura si c’est :
- un **runner Ollama** (qwen/ministral/bge),
- ou encore **speaches** qui tient le GPU.
## 2. Vérifier la conso réelle
En parallèle :
```bash
rocm-smi --showuse --showpower --showclk
```
Ça nous dira si :
- le GPU est vraiment à fond (power élevé, fréquences hautes),
- ou juste marqué à 100% avec une faible conso (bug de reporting).
<span style="display:none">[^20_1]</span>
<div align="center">⁂</div>
[^20_1]: paste.txt
---
# ca c'est juste apres la conversation
folzo@folzo-Corp:~\$ rocm-smi --showpower --showclkfrq
============================ ROCm System Management Interface ============================
=================================== Power Consumption ====================================
GPU[0] : Average Graphics Package Power (W): 53.0
==========================================================================================
============================== Supported clock frequencies ===============================
GPU[0] : Supported dcefclk frequencies on GPU0
GPU[0] : 0: 145Mhz
GPU[0] : 1: 1563Mhz *
GPU[0] :
GPU[0] : Supported fclk frequencies on GPU0
GPU[0] : 0: 601Mhz
GPU[0] : 1: 1000Mhz
GPU[0] : 2: 1200Mhz
GPU[0] : 3: 1600Mhz
GPU[0] : 4: 1801Mhz
GPU[0] : 5: 1950Mhz
GPU[0] : 6: 2201Mhz
GPU[0] : 7: 2301Mhz *
GPU[0] :
GPU[0] : Supported mclk frequencies on GPU0
GPU[0] : 0: 96Mhz
GPU[0] : 1: 456Mhz
GPU[0] : 2: 772Mhz
GPU[0] : 3: 1218Mhz *
GPU[0] :
GPU[0] : Supported sclk frequencies on GPU0
GPU[0] : 0: 500Mhz
GPU[0] : 1: 43Mhz *
GPU[0] : 2: 2124Mhz
GPU[0] :
GPU[0] : Supported socclk frequencies on GPU0
GPU[0] : 0: 300Mhz
GPU[0] : 1: 1500Mhz *
GPU[0] :
GPU[0] : Supported PCIe frequencies on GPU0
GPU[0] : 0: 2.5GT/s x1
GPU[0] : 1: 5.0GT/s x4
GPU[0] : 2: 16.0GT/s x16 *
GPU[0] :
------------------------------------------------------------------------------------------
==========================================================================================
================================== End of ROCm SMI Log ===================================
folzo@folzo-Corp:~\$
et ca c'est 1min apres la conversation :
folzo@folzo-Corp:~\$ rocm-smi --showpower --showclkfrq
============================ ROCm System Management Interface ============================
=================================== Power Consumption ====================================
GPU[0] : Average Graphics Package Power (W): 93.0
==========================================================================================
============================== Supported clock frequencies ===============================
GPU[0] : Supported dcefclk frequencies on GPU0
GPU[0] : 0: 145Mhz
GPU[0] : 1: 1563Mhz *
GPU[0] :
GPU[0] : Supported fclk frequencies on GPU0
GPU[0] : 0: 601Mhz
GPU[0] : 1: 1000Mhz
GPU[0] : 2: 1200Mhz
GPU[0] : 3: 1600Mhz
GPU[0] : 4: 1801Mhz
GPU[0] : 5: 1950Mhz
GPU[0] : 6: 2201Mhz
GPU[0] : 7: 2301Mhz *
GPU[0] :
GPU[0] : Supported mclk frequencies on GPU0
GPU[0] : 0: 96Mhz
GPU[0] : 1: 456Mhz
GPU[0] : 2: 772Mhz
GPU[0] : 3: 1218Mhz *
GPU[0] :
GPU[0] : Supported sclk frequencies on GPU0
GPU[0] : 0: 500Mhz
GPU[0] : 1: 2734Mhz *
GPU[0] : 2: 2124Mhz
GPU[0] :
GPU[0] : Supported socclk frequencies on GPU0
GPU[0] : 0: 300Mhz
GPU[0] : 1: 1500Mhz *
GPU[0] :
GPU[0] : Supported PCIe frequencies on GPU0
GPU[0] : 0: 2.5GT/s x1
GPU[0] : 1: 5.0GT/s x4
GPU[0] : 2: 16.0GT/s x16 *
GPU[0] :
------------------------------------------------------------------------------------------
==========================================================================================
================================== End of ROCm SMI Log ===================================
folzo@folzo-Corp:~\$
La conso à **93 W** une minute après la fin de la conversation montre que le GPU calcule encore sérieusement, pas juste un bug d’affichage. Il faut maintenant identifier **qui** le fait.
## 1. Identifier les process GPU actuels
Tu avais ça :
```text
PID 2278476
PID 2526746
```
Lance à nouveau :
```bash
for pid in 2278476 2526746; do
echo "PID $pid: $(cat /proc/$pid/cmdline | tr '\0' ' ')"
done
```
Ça va nous dire si :
- c’est un **runner Ollama** (ministral / bge / qwen),
- ou encore **speaches** (STT),
- ou autre chose.
Colle-moi la sortie.
***
## 2. Vérifier quels modèles Ollama sont actifs
Juste après une conversation avec Effi, lance :
```bash
ollama ps
```
On doit voir si un modèle (par ex. `ministral-3:3b` ou `qwen2.5:14b`) reste en génération ou boucle.
Colle aussi cette sortie, et on décidera si on :
- coupe un process fautif,
- réduit `keep_alive`,
- ou corrige un pipeline (mem0 / RAG) qui spamme le LLM.
---
# folzo@folzo-Corp:~\$ for pid in 2278476 2526746; do
echo "PID \$pid: $(cat /proc/$pid/cmdline | tr '\\0' ' ')"
done
PID 2278476: /home/ubuntu/speaches/.venv/bin/python .venv/bin/uvicorn speaches.main:create_app --host 0.0.0.0 --port 8000 --factory
cat: /proc/2526746/cmdline: No such file or directory (os error 2)
PID 2526746:
folzo@folzo-Corp:~\$ ollama ps
NAME ID SIZE PROCESSOR CONTEXT UNTIL
bge-m3:latest 790764642607 1.2 GB 100% GPU 4096 2 minutes from now
ministral-3:3b f04aa1c738f6 4.6 GB 100% GPU 4096 2 minutes from now
folzo@folzo-Corp:~\$
On a enfin les coupables clairement identifiés :
- **PID 2278476 = speaches (STT)** → tient le GPU inutilement.
- **`ministral-3:3b` + `bge-m3:latest`** → modèles Ollama encore chauds (normal), mais ils ne devraient pas continuer à consommer 90 W longtemps après.
On va réduire la consommation en 3 petits réglages.
***
## 1. Forcer speaches à ne plus toucher le GPU
Tu as déjà commenté les `devices` dans `~/speaches/docker-compose.yml`. Vérifions qu’il n’est plus attaché au GPU.
1. Vérifier le docker-compose :
```bash
cat ~/speaches/docker-compose.yml
```
Assure-toi que tu n’as bien **aucune** section `devices:` active dans `speaches:` (tout commenté ou supprimé).
2. Recrée le conteneur proprement :
```bash
cd ~/speaches
sudo docker compose down
sudo docker compose up -d
```
3. Vérifie côté ROCm :
```bash
rocm-smi --showpidgpus
```
PID `speaches` (python/uvicorn) ne doit plus apparaître. Si c’est bon, tu as supprimé une source de charge GPU permanente.
***
## 2. Réduire le temps de cache des modèles Ollama
On veut que `bge-m3` (embeddings) et `ministral-3:3b` (mémoire/outils) soient **déchargés** plus vite quand Effi est idle.
1. Crée un override systemd pour Ollama :
```bash
sudo systemctl edit ollama
```
Ajoute :
```ini
[Service]
Environment="OLLAMA_KEEP_ALIVE=60"
```
2. Recharge et redémarre Ollama :
```bash
sudo systemctl daemon-reload
sudo systemctl restart ollama
```
Résultat : chaque modèle est déchargé **60 secondes** après la dernière requête au lieu de rester plusieurs minutes. Si tu veux plus agressif, tu pourras descendre à `OLLAMA_KEEP_ALIVE=10`.[^22_1][^22_2]
***
## 3. Forcer le calme après une session Effi
En attendant de voir l’effet de `KEEP_ALIVE`, tu peux explicitement vider la VRAM juste après un long échange vocal :
```bash
ollama stop ministral-3:3b
ollama stop bge-m3:latest
```
Puis vérifier :
```bash
ollama ps # doit être vide
rocm-smi --showpower --showuse
```
Tu devrais voir la puissance retomber autour de 20–30 W et `GPU use` proche de 0% après quelques secondes.
***
Quand tu auras :
1) vérifié que `speaches` n’apparaît plus dans `rocm-smi --showpidgpus`,
2) mis `OLLAMA_KEEP_ALIVE=60` et redémarré Ollama,
on pourra refaire un test complet (conversation Effi → attendre 1–2 minutes → `rocm-smi --showpower --showpidgpus`) pour confirmer que le GPU se calme enfin comme prévu.
<div align="center">⁂</div>
[^22_1]: https://docs.ollama.com/faq
[^22_2]: https://www.reddit.com/r/ollama/comments/1cnxnrv/how_to_set_keepalive_1_on_ollama_linux/
---
# folzo@folzo-Corp:~\$ cat ~/speaches/docker-compose.yml
services:
speaches:
image: speaches:rocm-ct2
container_name: speaches-stt
restart: unless-stopped
ports:
- "8200:8000"
devices:
- /dev/kfd
- /dev/dri
group_add:
- "44"
- "991"
security_opt:
- seccomp=unconfined
environment:
- HSA_OVERRIDE_GFX_VERSION=11.0.0
- WHISPER__MODEL=deepdml/faster-whisper-large-v3-turbo-ct2
- WHISPER__LANGUAGE=fr
- WHISPER__COMPUTE_TYPE=int8
- WHISPER__DEVICE=cpu
volumes:
- speaches-models:/root/.cache/huggingface
speaches-init:
image: curlimages/curl:latest
container_name: speaches-init
restart: "no"
depends_on:
- speaches
entrypoint: >
sh -c "sleep 5 \&\&
curl -sf -X POST http://speaches:8000/v1/models/deepdml%2Ffaster-whisper-large-v3-turbo-ct2 \&\&
echo 'Modèle enregistré'"
volumes:
speaches-models:
folzo@folzo-Corp:~\$
Je n'ai pas envi de reduire le temps des models ollama, je veux garder en fluidité lors des echanges ecrit avec effi, ca ne me derange que les modeles restent chargé en memoire, quand ils sont pas utilisé ils utilise pas de gpu. par contre la le gpu reste utilisé a 100% longtemps apres les echange ce que je ne comprenais pas
Parfait, on va garder la fluidité côté Ollama et juste **éteindre la source de charge permanente : speaches sur GPU**.
Actuellement ton docker-compose attache encore le GPU :
```yaml
devices:
- /dev/kfd
- /dev/dri
group_add:
- "44"
- "991"
security_opt:
- seccomp=unconfined
```
Même avec `WHISPER__DEVICE=cpu`, la présence de `/dev/kfd` fait que ROCm initialise la carte et la garde active, d’où la conso élevée.
## 1. Désactiver complètement le GPU pour speaches
Édite le fichier :
```bash
nano ~/speaches/docker-compose.yml
```
Remplace la partie `speaches` par :
```yaml
speaches:
image: speaches:rocm-ct2
container_name: speaches-stt
restart: unless-stopped
ports:
- "8200:8000"
# AUCUN accès GPU ici
environment:
- HSA_OVERRIDE_GFX_VERSION=11.0.0
- WHISPER__MODEL=deepdml/faster-whisper-large-v3-turbo-ct2
- WHISPER__LANGUAGE=fr
- WHISPER__COMPUTE_TYPE=int8
- WHISPER__DEVICE=cpu
volumes:
- speaches-models:/root/.cache/huggingface
```
Donc tu supprimes complètement :
- `devices:`
- `group_add:`
- `security_opt:`
Puis :
```bash
cd ~/speaches
sudo docker compose down
sudo docker compose up -d
```
## 2. Vérifier que le GPU est enfin au repos après STT
1–2 minutes après une conversation Effi :
```bash
rocm-smi --showpidgpus
rocm-smi --showpower --showuse
```
Tu devrais voir :
- plus aucun PID `python ... speaches` dans `--showpidgpus`,
- **power** qui redescend, et `GPU use` proche de 0% lorsque tu ne parles pas.
Les modèles Ollama (`ministral-3:3b`, `bge-m3`) resteront chargés en VRAM pour la fluidité, mais ne consommeront presque rien tant qu’il n’y a pas de requêtes, ce qui correspond exactement à ce que tu veux.[^23_1]
<div align="center">⁂</div>
[^23_1]: https://docs.ollama.com/faq
---
# folzo@folzo-Corp:~\$ nano ~/speaches/docker-compose.yml
folzo@folzo-Corp:~\$ cd ~/speaches
sudo docker compose down
sudo docker compose up -d
[sudo: authenticate] Password:
[+] down 3/3
✔ Container speaches-init Removed 0.1s
✔ Container speaches-stt Removed 1.4s
✔ Network speaches_default Removed 0.1s
[+] up 3/3
✔ Network speaches_default Created 0.0s
✔ Container speaches-stt Created 0.1s
✔ Container speaches-init Created 0.1s
folzo@folzo-Corp:~/speaches\$
normalement c'est bon ?
Oui, le redéploiement est OK. Vérifions que le GPU est enfin tranquille quand tu ne parles pas.
Sur **folzo-Corp**, sans interaction avec Effi pendant 30–60s :
```bash
rocm-smi --showpidgpus
rocm-smi --showpower --showuse
```
Tu devrais voir :
- plus **aucun PID python/uvicorn speaches** dans `--showpidgpus`,
- une **puissance bien plus basse** (idéalement <40–50 W) et un `GPU use` qui retombe vers 0% quand tu es idle.
Si c’est le cas, on a réglé la cause principale : speaches ne garde plus la 7800 XT en charge permanente, les modèles Ollama restent en VRAM pour la fluidité mais ne consomment presque rien en dehors des requêtes.
---
# j'a iune erreur je dois configurer quelque chose dans openwebui ?
L’erreur vient de **speaches** (STT) qui renvoie un 500 sur `/v1/audio/transcriptions`, pas d’Open WebUI.
On vient de modifier le conteneur (plus de GPU), donc il y a probablement une stacktrace côté logs.
Sur **folzo-Corp** :
```bash
docker logs -f speaches-stt
```
Refais un test vocal dans Effi et regarde ce qui s’affiche au moment de l’erreur (les 10–20 dernières lignes).
Colle-moi ce bloc de logs, on va voir si :
- le modèle `deepdml/faster-whisper-large-v3-turbo-ct2` se charge bien en CPU,
- ou si speaches s’attend à un provider GPU et plante sans `/dev/kfd`.
<span style="display:none">[^25_1]</span>
<div align="center">⁂</div>
[^25_1]: image.jpg
---
# folzo@folzo-Corp:~/speaches\$ sudo docker logs -f speaches-stt
2026-02-22 13:05:13,331:DEBUG:speaches.main:create_app:102:Config: stt_model_ttl=300 tts_model_ttl=300 vad_model_ttl=-1 api_key=None log_level='debug' host='0.0.0.0' port=8000 allow_origins=None enable_ui=True whisper=WhisperConfig(inference_device='auto', device_index=0, compute_type='int8', cpu_threads=0, num_workers=1) loopback_host_url=None chat_completion_base_url='http://localhost:11434/v1' chat_completion_api_key=SecretStr('**********') unstable_ort_opts=OrtOptions(exclude_providers=['TensorrtExecutionProvider'], provider_priority={'CUDAExecutionProvider': 100}, provider_opts={}) otel_exporter_otlp_endpoint=None otel_service_name='speaches' preload_models=[]
2026-02-22 13:05:14,290:DEBUG:urllib3.connectionpool:_new_conn:1051:Starting new HTTPS connection (1): huggingface.co:443
2026-02-22 13:05:14,321:DEBUG:urllib3.connectionpool:_new_conn:1051:Starting new HTTPS connection (1): huggingface.co:443
2026-02-22 13:05:14,537:DEBUG:urllib3.connectionpool:_make_request:546:https://huggingface.co:443 "HEAD /api/telemetry/https%3A/api.gradio.app/gradio-initiated-analytics HTTP/11" 200 0
2026-02-22 13:05:14,547:DEBUG:urllib3.connectionpool:_make_request:546:https://huggingface.co:443 "HEAD /api/telemetry/https%3A/api.gradio.app/gradio-initiated-analytics HTTP/11" 200 0
new
2026-02-22 13:05:14,571:INFO:speaches.main:create_app:202:
To view the gradio web ui of speaches open your browser and visit:
http://localhost:8000/
INFO: Started server process [1]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on [http://0.0.0.0:8000](http://0.0.0.0:8000) (Press CTRL+C to quit)
2026-02-22 13:05:15,016:INFO:httpx:_send_single_request:1025:HTTP Request: GET [https://api.gradio.app/pkg-version](https://api.gradio.app/pkg-version) "HTTP/1.1 200 OK"
2026-02-22 13:05:15,016:INFO:httpx:_send_single_request:1025:HTTP Request: GET [https://api.gradio.app/pkg-version](https://api.gradio.app/pkg-version) "HTTP/1.1 200 OK"
2026-02-22 13:05:15,063:INFO:httpx:_send_single_request:1025:HTTP Request: GET [https://api.gradio.app/pkg-version](https://api.gradio.app/pkg-version) "HTTP/1.1 200 OK"
2026-02-22 13:05:15,063:INFO:httpx:_send_single_request:1025:HTTP Request: GET [https://api.gradio.app/pkg-version](https://api.gradio.app/pkg-version) "HTTP/1.1 200 OK"
2026-02-22 13:05:15,854:DEBUG:urllib3.connectionpool:_new_conn:1051:Starting new HTTPS connection (1): huggingface.co:443
2026-02-22 13:05:16,031:DEBUG:urllib3.connectionpool:_make_request:546:https://huggingface.co:443 "GET /api/models?filter=ctranslate2\&filter=automatic-speech-recognition\&cardData=True HTTP/11" 200 244215
2026-02-22 13:05:16,083:WARNING:huggingface_hub.repocard_data:__init__:368:Invalid model-index. Not loading eval results into CardData.
Invalid model-index. Not loading eval results into CardData.
2026-02-22 13:05:16,228:DEBUG:urllib3.connectionpool:_make_request:546:https://huggingface.co:443 "GET /api/models/deepdml/faster-whisper-large-v3-turbo-ct2/revision/main HTTP/11" 200 2873
Fetching 6 files: 100%|██████████| 6/6 [00:00<00:00, 67108.86it/s]
INFO: 172.18.0.3:42604 - "POST /v1/models/deepdml/faster-whisper-large-v3-turbo-ct2 HTTP/1.1" 200 OK
2026-02-22 13:06:38,231:DEBUG:speaches.dependencies:audio_file_dependency:89:Decoding audio file: f876c35d-0a33-43d9-a9f8-d5c35ba83fa4.mp3, content_type: None, header: Headers({'content-disposition': 'form-data; name="file"; filename="f876c35d-0a33-43d9-a9f8-d5c35ba83fa4.mp3"'}), size: 39405
2026-02-22 13:06:38,316:DEBUG:speaches.dependencies:audio_file_dependency:103:Decoded 4.86s of audio in 0.08465s (RTF: 0.01741720123301306)
2026-02-22 13:06:38,316:DEBUG:asyncio:__init__:64:Using selector: EpollSelector
2026-02-22 13:06:38,320:DEBUG:speaches.executors.shared.base_model_manager:_load:72:Loading model silero_vad_v5
2026-02-22 13:06:38,321:DEBUG:speaches.executors.shared.base_model_manager:get_ort_providers_with_options:23:Available ONNX Runtime providers: ['TensorrtExecutionProvider', 'CUDAExecutionProvider', 'CPUExecutionProvider']
2026-02-22 13:06:38,321:DEBUG:speaches.executors.shared.base_model_manager:get_ort_providers_with_options:33:Using ONNX Runtime providers: [('CUDAExecutionProvider', {}), ('CPUExecutionProvider', {})]
2026-02-22 13:06:38.323054163 [E:onnxruntime:Default, provider_bridge_ort.cc:2195 TryGetProviderInfo_CUDA] /onnxruntime_src/onnxruntime/core/session/provider_bridge_ort.cc:1778 onnxruntime::Provider\& onnxruntime::ProviderLibrary::Get() [ONNXRuntimeError] : 1 : FAIL : Failed to load library libonnxruntime_providers_cuda.so with error: libcublasLt.so.12: cannot open shared object file: No such file or directory
2026-02-22 13:06:38.323066733 [W:onnxruntime:Default, onnxruntime_pybind_state.cc:1055 CreateExecutionProviderInstance] Failed to create CUDAExecutionProvider. Require cuDNN 9.* and CUDA 12.*. Please install all dependencies as mentioned in the GPU requirements page (https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html\#requirements), make sure they're in the PATH, and that your GPU is supported.
2026-02-22 13:06:38.328087859 [E:onnxruntime:Default, provider_bridge_ort.cc:2195 TryGetProviderInfo_CUDA] /onnxruntime_src/onnxruntime/core/session/provider_bridge_ort.cc:1778 onnxruntime::Provider\& onnxruntime::ProviderLibrary::Get() [ONNXRuntimeError] : 1 : FAIL : Failed to load library libonnxruntime_providers_cuda.so with error: libcublasLt.so.12: cannot open shared object file: No such file or directory
2026-02-22 13:06:38.328097539 [W:onnxruntime:Default, onnxruntime_pybind_state.cc:1055 CreateExecutionProviderInstance] Failed to create CUDAExecutionProvider. Require cuDNN 9.* and CUDA 12.*. Please install all dependencies as mentioned in the GPU requirements page (https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html\#requirements), make sure they're in the PATH, and that your GPU is supported.
2026-02-22 13:06:38,329:INFO:speaches.executors.shared.base_model_manager:_load:75:Model silero_vad_v5 loaded in 0.01s
2026-02-22 13:06:38,329:DEBUG:speaches.executors.shared.base_model_manager:_increment_ref:83:Incremented ref count for silero_vad_v5, self.ref_count=1
2026-02-22 13:06:38,346:DEBUG:speaches.executors.silero_vad_v5:__call__:138:VAD model inference took 0.0167s
2026-02-22 13:06:38,346:DEBUG:speaches.executors.shared.base_model_manager:_decrement_ref:88:Decremented ref count for silero_vad_v5, self.ref_count=0
2026-02-22 13:06:38,346:INFO:speaches.executors.shared.base_model_manager:_decrement_ref:98:Model silero_vad_v5 is idle, not unloading
2026-02-22 13:06:38,346:DEBUG:speaches.executors.silero_vad_v5:get_speech_timestamps:307:VAD processing took 0.0260s for 4.86s audio
2026-02-22 13:06:38,347:DEBUG:speaches.executors.shared.base_model_manager:_load:72:Loading model deepdml/faster-whisper-large-v3-turbo-ct2
2026-02-22 13:06:38,461:DEBUG:urllib3.connectionpool:_make_request:546:https://huggingface.co:443 "GET /api/models/deepdml/faster-whisper-large-v3-turbo-ct2/revision/main HTTP/11" 200 2873
INFO: 192.168.1.27:51524 - "POST /v1/audio/transcriptions HTTP/1.1" 500 Internal Server Error
ERROR: Exception in ASGI application
Traceback (most recent call last):
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/uvicorn/protocols/http/h11_impl.py", line 403, in run_asgi
result = await app( \# type: ignore[func-returns-value]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__
return await self.app(scope, receive, send)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/fastapi/applications.py", line 1134, in __call__
await super().__call__(scope, receive, send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/applications.py", line 113, in __call__
await self.middleware_stack(scope, receive, send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 187, in __call__
raise exc
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 165, in __call__
await self.app(scope, receive, _send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
raise exc
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
await app(scope, receive, sender)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
await self.app(scope, receive, send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/routing.py", line 715, in __call__
await self.middleware_stack(scope, receive, send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/routing.py", line 735, in app
await route.handle(scope, receive, send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/routing.py", line 288, in handle
await self.app(scope, receive, send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 125, in app
await wrap_app_handling_exceptions(app, request)(scope, receive, send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
raise exc
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
await app(scope, receive, sender)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 111, in app
response = await f(request)
^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 391, in app
raw_response = await run_endpoint_function(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 292, in run_endpoint_function
return await run_in_threadpool(dependant.call, **values)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/concurrency.py", line 39, in run_in_threadpool
return await anyio.to_thread.run_sync(func, *args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/anyio/to_thread.py", line 56, in run_sync
return await get_async_backend().run_sync_in_worker_thread(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 2470, in run_sync_in_worker_thread
return await future
^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 967, in run
result = context.run(func, *args)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/speaches/routers/stt.py", line 171, in transcribe_file
res = transcription_executor.model_manager.handle_transcription_request(transcription_request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/speaches/executors/whisper.py", line 231, in handle_transcription_request
return self.handle_non_streaming_transcription_request(request, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/speaches/tracing.py", line 31, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/speaches/executors/whisper.py", line 154, in handle_non_streaming_transcription_request
with self.load_model(request.model) as whisper:
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/speaches/executors/shared/base_model_manager.py", line 103, in __enter__
self._load()
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/speaches/executors/shared/base_model_manager.py", line 74, in _load
self.model = self.load_fn()
^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/speaches/executors/shared/base_model_manager.py", line 142, in <lambda>
load_fn=lambda: self._load_fn(model_id),
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/speaches/executors/whisper.py", line 134, in _load_fn
return WhisperModel(
^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/faster_whisper/transcribe.py", line 647, in __init__
self.model = ctranslate2.models.Whisper(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: Requested int8 compute type, but the target device or backend do not support efficient int8 computation.
2026-02-22 13:06:39,503:DEBUG:speaches.dependencies:audio_file_dependency:89:Decoding audio file: f876c35d-0a33-43d9-a9f8-d5c35ba83fa4.mp3, content_type: None, header: Headers({'content-disposition': 'form-data; name="file"; filename="f876c35d-0a33-43d9-a9f8-d5c35ba83fa4.mp3"'}), size: 39405
2026-02-22 13:06:39,579:DEBUG:speaches.dependencies:audio_file_dependency:103:Decoded 4.86s of audio in 0.07575s (RTF: 0.01558649218104657)
2026-02-22 13:06:39,579:DEBUG:asyncio:__init__:64:Using selector: EpollSelector
2026-02-22 13:06:39,583:DEBUG:speaches.executors.shared.base_model_manager:load_model:138:silero_vad_v5 model already loaded
2026-02-22 13:06:39,583:DEBUG:speaches.executors.shared.base_model_manager:_increment_ref:83:Incremented ref count for silero_vad_v5, self.ref_count=1
2026-02-22 13:06:39,592:DEBUG:speaches.executors.silero_vad_v5:__call__:138:VAD model inference took 0.0097s
2026-02-22 13:06:39,593:DEBUG:speaches.executors.shared.base_model_manager:_decrement_ref:88:Decremented ref count for silero_vad_v5, self.ref_count=0
2026-02-22 13:06:39,593:INFO:speaches.executors.shared.base_model_manager:_decrement_ref:98:Model silero_vad_v5 is idle, not unloading
2026-02-22 13:06:39,593:DEBUG:speaches.executors.silero_vad_v5:get_speech_timestamps:307:VAD processing took 0.0104s for 4.86s audio
2026-02-22 13:06:39,593:DEBUG:speaches.executors.shared.base_model_manager:load_model:138:deepdml/faster-whisper-large-v3-turbo-ct2 model already loaded
2026-02-22 13:06:39,593:DEBUG:speaches.executors.shared.base_model_manager:_load:72:Loading model deepdml/faster-whisper-large-v3-turbo-ct2
2026-02-22 13:06:39,700:DEBUG:urllib3.connectionpool:_make_request:546:https://huggingface.co:443 "GET /api/models/deepdml/faster-whisper-large-v3-turbo-ct2/revision/main HTTP/11" 200 2873
INFO: 192.168.1.27:51540 - "POST /v1/audio/transcriptions HTTP/1.1" 500 Internal Server Error
ERROR: Exception in ASGI application
Traceback (most recent call last):
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/uvicorn/protocols/http/h11_impl.py", line 403, in run_asgi
result = await app( \# type: ignore[func-returns-value]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__
return await self.app(scope, receive, send)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/fastapi/applications.py", line 1134, in __call__
await super().__call__(scope, receive, send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/applications.py", line 113, in __call__
await self.middleware_stack(scope, receive, send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 187, in __call__
raise exc
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 165, in __call__
await self.app(scope, receive, _send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
raise exc
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
await app(scope, receive, sender)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
await self.app(scope, receive, send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/routing.py", line 715, in __call__
await self.middleware_stack(scope, receive, send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/routing.py", line 735, in app
await route.handle(scope, receive, send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/routing.py", line 288, in handle
await self.app(scope, receive, send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 125, in app
await wrap_app_handling_exceptions(app, request)(scope, receive, send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
raise exc
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
await app(scope, receive, sender)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 111, in app
response = await f(request)
^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 391, in app
raw_response = await run_endpoint_function(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 292, in run_endpoint_function
return await run_in_threadpool(dependant.call, **values)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/concurrency.py", line 39, in run_in_threadpool
return await anyio.to_thread.run_sync(func, *args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/anyio/to_thread.py", line 56, in run_sync
return await get_async_backend().run_sync_in_worker_thread(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 2470, in run_sync_in_worker_thread
return await future
^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 967, in run
result = context.run(func, *args)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/speaches/routers/stt.py", line 171, in transcribe_file
res = transcription_executor.model_manager.handle_transcription_request(transcription_request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/speaches/executors/whisper.py", line 231, in handle_transcription_request
return self.handle_non_streaming_transcription_request(request, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/speaches/tracing.py", line 31, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/speaches/executors/whisper.py", line 154, in handle_non_streaming_transcription_request
with self.load_model(request.model) as whisper:
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/speaches/executors/shared/base_model_manager.py", line 103, in __enter__
self._load()
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/speaches/executors/shared/base_model_manager.py", line 74, in _load
self.model = self.load_fn()
^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/speaches/executors/shared/base_model_manager.py", line 142, in <lambda>
load_fn=lambda: self._load_fn(model_id),
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/speaches/executors/whisper.py", line 134, in _load_fn
return WhisperModel(
^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/faster_whisper/transcribe.py", line 647, in __init__
self.model = ctranslate2.models.Whisper(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: Requested int8 compute type, but the target device or backend do not support efficient int8 computation.
2026-02-22 13:08:09,485:DEBUG:speaches.dependencies:audio_file_dependency:89:Decoding audio file: 4e0c529c-2fa4-4183-b569-b3379565bfe4.mp3, content_type: None, header: Headers({'content-disposition': 'form-data; name="file"; filename="4e0c529c-2fa4-4183-b569-b3379565bfe4.mp3"'}), size: 34029
2026-02-22 13:08:09,585:DEBUG:speaches.dependencies:audio_file_dependency:103:Decoded 4.2s of audio in 0.09908s (RTF: 0.023590347381763247)
2026-02-22 13:08:09,585:DEBUG:asyncio:__init__:64:Using selector: EpollSelector
2026-02-22 13:08:09,588:DEBUG:speaches.executors.shared.base_model_manager:load_model:138:silero_vad_v5 model already loaded
2026-02-22 13:08:09,588:DEBUG:speaches.executors.shared.base_model_manager:_increment_ref:83:Incremented ref count for silero_vad_v5, self.ref_count=1
2026-02-22 13:08:09,595:DEBUG:speaches.executors.silero_vad_v5:__call__:138:VAD model inference took 0.0066s
2026-02-22 13:08:09,660:DEBUG:speaches.executors.shared.base_model_manager:_decrement_ref:88:Decremented ref count for silero_vad_v5, self.ref_count=0
2026-02-22 13:08:09,660:INFO:speaches.executors.shared.base_model_manager:_decrement_ref:98:Model silero_vad_v5 is idle, not unloading
2026-02-22 13:08:09,660:DEBUG:speaches.executors.silero_vad_v5:get_speech_timestamps:307:VAD processing took 0.0713s for 4.20s audio
2026-02-22 13:08:09,660:DEBUG:speaches.executors.shared.base_model_manager:load_model:138:deepdml/faster-whisper-large-v3-turbo-ct2 model already loaded
2026-02-22 13:08:09,660:DEBUG:speaches.executors.shared.base_model_manager:_load:72:Loading model deepdml/faster-whisper-large-v3-turbo-ct2
2026-02-22 13:08:09,767:DEBUG:urllib3.connectionpool:_make_request:546:https://huggingface.co:443 "GET /api/models/deepdml/faster-whisper-large-v3-turbo-ct2/revision/main HTTP/11" 200 2873
INFO: 192.168.1.27:55642 - "POST /v1/audio/transcriptions HTTP/1.1" 500 Internal Server Error
ERROR: Exception in ASGI application
Traceback (most recent call last):
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/uvicorn/protocols/http/h11_impl.py", line 403, in run_asgi
result = await app( \# type: ignore[func-returns-value]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__
return await self.app(scope, receive, send)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/fastapi/applications.py", line 1134, in __call__
await super().__call__(scope, receive, send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/applications.py", line 113, in __call__
await self.middleware_stack(scope, receive, send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 187, in __call__
raise exc
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 165, in __call__
await self.app(scope, receive, _send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
raise exc
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
await app(scope, receive, sender)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
await self.app(scope, receive, send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/routing.py", line 715, in __call__
await self.middleware_stack(scope, receive, send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/routing.py", line 735, in app
await route.handle(scope, receive, send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/routing.py", line 288, in handle
await self.app(scope, receive, send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 125, in app
await wrap_app_handling_exceptions(app, request)(scope, receive, send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
raise exc
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
await app(scope, receive, sender)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 111, in app
response = await f(request)
^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 391, in app
raw_response = await run_endpoint_function(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 292, in run_endpoint_function
return await run_in_threadpool(dependant.call, **values)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/concurrency.py", line 39, in run_in_threadpool
return await anyio.to_thread.run_sync(func, *args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/anyio/to_thread.py", line 56, in run_sync
return await get_async_backend().run_sync_in_worker_thread(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 2470, in run_sync_in_worker_thread
return await future
^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 967, in run
result = context.run(func, *args)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/speaches/routers/stt.py", line 171, in transcribe_file
res = transcription_executor.model_manager.handle_transcription_request(transcription_request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/speaches/executors/whisper.py", line 231, in handle_transcription_request
return self.handle_non_streaming_transcription_request(request, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/speaches/tracing.py", line 31, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/speaches/executors/whisper.py", line 154, in handle_non_streaming_transcription_request
with self.load_model(request.model) as whisper:
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/speaches/executors/shared/base_model_manager.py", line 103, in __enter__
self._load()
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/speaches/executors/shared/base_model_manager.py", line 74, in _load
self.model = self.load_fn()
^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/speaches/executors/shared/base_model_manager.py", line 142, in <lambda>
load_fn=lambda: self._load_fn(model_id),
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/speaches/executors/whisper.py", line 134, in _load_fn
return WhisperModel(
^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/faster_whisper/transcribe.py", line 647, in __init__
self.model = ctranslate2.models.Whisper(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: Requested int8 compute type, but the target device or backend do not support efficient int8 computation.
2026-02-22 13:08:10,409:DEBUG:speaches.dependencies:audio_file_dependency:89:Decoding audio file: 4e0c529c-2fa4-4183-b569-b3379565bfe4.mp3, content_type: None, header: Headers({'content-disposition': 'form-data; name="file"; filename="4e0c529c-2fa4-4183-b569-b3379565bfe4.mp3"'}), size: 34029
2026-02-22 13:08:10,491:DEBUG:speaches.dependencies:audio_file_dependency:103:Decoded 4.2s of audio in 0.08204s (RTF: 0.01953266142837016)
2026-02-22 13:08:10,491:DEBUG:asyncio:__init__:64:Using selector: EpollSelector
2026-02-22 13:08:10,495:DEBUG:speaches.executors.shared.base_model_manager:load_model:138:silero_vad_v5 model already loaded
2026-02-22 13:08:10,495:DEBUG:speaches.executors.shared.base_model_manager:_increment_ref:83:Incremented ref count for silero_vad_v5, self.ref_count=1
2026-02-22 13:08:10,501:DEBUG:speaches.executors.silero_vad_v5:__call__:138:VAD model inference took 0.0066s
2026-02-22 13:08:10,527:DEBUG:speaches.executors.shared.base_model_manager:_decrement_ref:88:Decremented ref count for silero_vad_v5, self.ref_count=0
2026-02-22 13:08:10,527:INFO:speaches.executors.shared.base_model_manager:_decrement_ref:98:Model silero_vad_v5 is idle, not unloading
2026-02-22 13:08:10,527:DEBUG:speaches.executors.silero_vad_v5:get_speech_timestamps:307:VAD processing took 0.0324s for 4.20s audio
2026-02-22 13:08:10,527:DEBUG:speaches.executors.shared.base_model_manager:load_model:138:deepdml/faster-whisper-large-v3-turbo-ct2 model already loaded
2026-02-22 13:08:10,527:DEBUG:speaches.executors.shared.base_model_manager:_load:72:Loading model deepdml/faster-whisper-large-v3-turbo-ct2
2026-02-22 13:08:10,637:DEBUG:urllib3.connectionpool:_make_request:546:https://huggingface.co:443 "GET /api/models/deepdml/faster-whisper-large-v3-turbo-ct2/revision/main HTTP/11" 200 2873
INFO: 192.168.1.27:55650 - "POST /v1/audio/transcriptions HTTP/1.1" 500 Internal Server Error
ERROR: Exception in ASGI application
Traceback (most recent call last):
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/uvicorn/protocols/http/h11_impl.py", line 403, in run_asgi
result = await app( \# type: ignore[func-returns-value]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__
return await self.app(scope, receive, send)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/fastapi/applications.py", line 1134, in __call__
await super().__call__(scope, receive, send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/applications.py", line 113, in __call__
await self.middleware_stack(scope, receive, send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 187, in __call__
raise exc
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 165, in __call__
await self.app(scope, receive, _send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
raise exc
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
await app(scope, receive, sender)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
await self.app(scope, receive, send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/routing.py", line 715, in __call__
await self.middleware_stack(scope, receive, send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/routing.py", line 735, in app
await route.handle(scope, receive, send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/routing.py", line 288, in handle
await self.app(scope, receive, send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 125, in app
await wrap_app_handling_exceptions(app, request)(scope, receive, send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
raise exc
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
await app(scope, receive, sender)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 111, in app
response = await f(request)
^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 391, in app
raw_response = await run_endpoint_function(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 292, in run_endpoint_function
return await run_in_threadpool(dependant.call, **values)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/concurrency.py", line 39, in run_in_threadpool
return await anyio.to_thread.run_sync(func, *args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/anyio/to_thread.py", line 56, in run_sync
return await get_async_backend().run_sync_in_worker_thread(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 2470, in run_sync_in_worker_thread
return await future
^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 967, in run
result = context.run(func, *args)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/speaches/routers/stt.py", line 171, in transcribe_file
res = transcription_executor.model_manager.handle_transcription_request(transcription_request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/speaches/executors/whisper.py", line 231, in handle_transcription_request
return self.handle_non_streaming_transcription_request(request, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/speaches/tracing.py", line 31, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/speaches/executors/whisper.py", line 154, in handle_non_streaming_transcription_request
with self.load_model(request.model) as whisper:
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/speaches/executors/shared/base_model_manager.py", line 103, in __enter__
self._load()
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/speaches/executors/shared/base_model_manager.py", line 74, in _load
self.model = self.load_fn()
^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/speaches/executors/shared/base_model_manager.py", line 142, in <lambda>
load_fn=lambda: self._load_fn(model_id),
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/speaches/executors/whisper.py", line 134, in _load_fn
return WhisperModel(
^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/faster_whisper/transcribe.py", line 647, in __init__
self.model = ctranslate2.models.Whisper(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: Requested int8 compute type, but the target device or backend do not support efficient int8 computation.
Parfait, on a l’erreur clé ici :
> `ValueError: Requested int8 compute type, but the target device or backend do not support efficient int8 computation.`
En gros : **le modèle CTranslate2 que tu utilises (`deepdml/faster-whisper-large-v3-turbo-ct2`) ne supporte pas vraiment `int8` sur CPU dans ce conteneur**. On vient de retirer le GPU, donc il n’a plus de backend adapté pour l’int8.
Il suffit de passer le compute type en `float32` (ou `int16`) côté speaches.
***
## 1. Modifier le compute type dans speaches
Sur folzo-Corp :
```bash
nano ~/speaches/docker-compose.yml
```
Dans `environment:` de `speaches`, remplace :
```yaml
- WHISPER__COMPUTE_TYPE=int8
```
par :
```yaml
- WHISPER__COMPUTE_TYPE=float32
```
(Éventuellement tu peux tester `int16` plus tard pour un compromis perf/précision.)
Sauvegarde, puis redémarre :
```bash
cd ~/speaches
sudo docker compose down
sudo docker compose up -d
```
***
## 2. Tester à nouveau le vocal
Ensuite :
1. Vérifie vite fait les logs au démarrage :
```bash
sudo docker logs -f speaches-stt
```
Tu ne dois plus voir l’exception `Requested int8 compute type`.
2. Re-teste un vocal avec Effi.
Si ça fonctionne, on a :
- STT en **CPU** (plus de charge GPU résiduelle),
- modèle **large-v3-turbo** (donc rapide),
- TTS **Piper FR**,
- et la 7800XT qui ne bosse que pour les LLM/embeddings.
Dis-moi juste si le vocal repasse bien après ce changement.
<span style="display:none">[^26_1][^26_10][^26_2][^26_3][^26_4][^26_5][^26_6][^26_7][^26_8][^26_9]</span>
<div align="center">⁂</div>
[^26_1]: https://docs.speechmatics.com/deployments/container/gpu-speech-to-text
[^26_2]: https://forums.docker.com/t/enabling-gpu-support-in-docker-containers-for-accelerated-computations/138686
[^26_3]: https://docs.docker.com/desktop/features/gpu/
[^26_4]: https://www.reddit.com/r/docker/comments/htmrj0/how_to_allow_docker_to_access_gpu/
[^26_5]: https://speech.fish.audio/install/
[^26_6]: https://github.com/manzolo/openai-whisper-docker
[^26_7]: https://speaches.ai/installation/
[^26_8]: https://github.com/speaches-ai/speaches/issues/561
[^26_9]: https://docs-beta.internal.speechmatics.com/on-prem/containers/gpu-container/gpu-inference-container
[^26_10]: https://learn.microsoft.com/de-de/virtualization/windowscontainers/deploy-containers/gpu-acceleration
---
# toujours une erreur ; folzo@folzo-Corp:~/speaches\$ sudo docker logs -f speaches-stt
2026-02-22 13:10:20,319:DEBUG:speaches.main:create_app:102:Config: stt_model_ttl=300 tts_model_ttl=300 vad_model_ttl=-1 api_key=None log_level='debug' host='0.0.0.0' port=8000 allow_origins=None enable_ui=True whisper=WhisperConfig(inference_device='auto', device_index=0, compute_type='float32', cpu_threads=0, num_workers=1) loopback_host_url=None chat_completion_base_url='http://localhost:11434/v1' chat_completion_api_key=SecretStr('**********') unstable_ort_opts=OrtOptions(exclude_providers=['TensorrtExecutionProvider'], provider_priority={'CUDAExecutionProvider': 100}, provider_opts={}) otel_exporter_otlp_endpoint=None otel_service_name='speaches' preload_models=[]
2026-02-22 13:10:21,202:DEBUG:urllib3.connectionpool:_new_conn:1051:Starting new HTTPS connection (1): huggingface.co:443
2026-02-22 13:10:21,253:DEBUG:urllib3.connectionpool:_new_conn:1051:Starting new HTTPS connection (1): huggingface.co:443
2026-02-22 13:10:21,380:DEBUG:urllib3.connectionpool:_make_request:546:https://huggingface.co:443 "HEAD /api/telemetry/https%3A/api.gradio.app/gradio-initiated-analytics HTTP/11" 200 0
2026-02-22 13:10:21,496:DEBUG:urllib3.connectionpool:_make_request:546:https://huggingface.co:443 "HEAD /api/telemetry/https%3A/api.gradio.app/gradio-initiated-analytics HTTP/11" 200 0
new
2026-02-22 13:10:21,522:INFO:speaches.main:create_app:202:
To view the gradio web ui of speaches open your browser and visit:
http://localhost:8000/
INFO: Started server process [1]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on [http://0.0.0.0:8000](http://0.0.0.0:8000) (Press CTRL+C to quit)
2026-02-22 13:10:21,936:INFO:httpx:_send_single_request:1025:HTTP Request: GET [https://api.gradio.app/pkg-version](https://api.gradio.app/pkg-version) "HTTP/1.1 200 OK"
2026-02-22 13:10:21,936:INFO:httpx:_send_single_request:1025:HTTP Request: GET [https://api.gradio.app/pkg-version](https://api.gradio.app/pkg-version) "HTTP/1.1 200 OK"
2026-02-22 13:10:21,994:INFO:httpx:_send_single_request:1025:HTTP Request: GET [https://api.gradio.app/pkg-version](https://api.gradio.app/pkg-version) "HTTP/1.1 200 OK"
2026-02-22 13:10:21,994:INFO:httpx:_send_single_request:1025:HTTP Request: GET [https://api.gradio.app/pkg-version](https://api.gradio.app/pkg-version) "HTTP/1.1 200 OK"
2026-02-22 13:10:23,321:DEBUG:urllib3.connectionpool:_new_conn:1051:Starting new HTTPS connection (1): huggingface.co:443
2026-02-22 13:10:23,499:DEBUG:urllib3.connectionpool:_make_request:546:https://huggingface.co:443 "GET /api/models?filter=ctranslate2\&filter=automatic-speech-recognition\&cardData=True HTTP/11" 200 244215
Invalid model-index. Not loading eval results into CardData.
2026-02-22 13:10:23,559:WARNING:huggingface_hub.repocard_data:__init__:368:Invalid model-index. Not loading eval results into CardData.
2026-02-22 13:10:23,674:DEBUG:urllib3.connectionpool:_make_request:546:https://huggingface.co:443 "GET /api/models/deepdml/faster-whisper-large-v3-turbo-ct2/revision/main HTTP/11" 200 2873
Fetching 6 files: 100%|██████████| 6/6 [00:00<00:00, 92521.41it/s]
INFO: 172.18.0.3:32990 - "POST /v1/models/deepdml/faster-whisper-large-v3-turbo-ct2 HTTP/1.1" 200 OK
2026-02-22 13:10:38,591:DEBUG:speaches.dependencies:audio_file_dependency:89:Decoding audio file: a8eff92c-6bac-4320-b80d-e66e9b9c1c55.mp3, content_type: None, header: Headers({'content-disposition': 'form-data; name="file"; filename="a8eff92c-6bac-4320-b80d-e66e9b9c1c55.mp3"'}), size: 25389
2026-02-22 13:10:38,669:DEBUG:speaches.dependencies:audio_file_dependency:103:Decoded 3.12s of audio in 0.07752s (RTF: 0.024845482053373974)
2026-02-22 13:10:38,669:DEBUG:asyncio:__init__:64:Using selector: EpollSelector
2026-02-22 13:10:38,674:DEBUG:speaches.executors.shared.base_model_manager:_load:72:Loading model silero_vad_v5
2026-02-22 13:10:38,674:DEBUG:speaches.executors.shared.base_model_manager:get_ort_providers_with_options:23:Available ONNX Runtime providers: ['TensorrtExecutionProvider', 'CUDAExecutionProvider', 'CPUExecutionProvider']
2026-02-22 13:10:38,674:DEBUG:speaches.executors.shared.base_model_manager:get_ort_providers_with_options:33:Using ONNX Runtime providers: [('CUDAExecutionProvider', {}), ('CPUExecutionProvider', {})]
2026-02-22 13:10:38.675920860 [E:onnxruntime:Default, provider_bridge_ort.cc:2195 TryGetProviderInfo_CUDA] /onnxruntime_src/onnxruntime/core/session/provider_bridge_ort.cc:1778 onnxruntime::Provider\& onnxruntime::ProviderLibrary::Get() [ONNXRuntimeError] : 1 : FAIL : Failed to load library libonnxruntime_providers_cuda.so with error: libcublasLt.so.12: cannot open shared object file: No such file or directory
2026-02-22 13:10:38.675933100 [W:onnxruntime:Default, onnxruntime_pybind_state.cc:1055 CreateExecutionProviderInstance] Failed to create CUDAExecutionProvider. Require cuDNN 9.* and CUDA 12.*. Please install all dependencies as mentioned in the GPU requirements page (https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html\#requirements), make sure they're in the PATH, and that your GPU is supported.
2026-02-22 13:10:38.679662590 [E:onnxruntime:Default, provider_bridge_ort.cc:2195 TryGetProviderInfo_CUDA] /onnxruntime_src/onnxruntime/core/session/provider_bridge_ort.cc:1778 onnxruntime::Provider\& onnxruntime::ProviderLibrary::Get() [ONNXRuntimeError] : 1 : FAIL : Failed to load library libonnxruntime_providers_cuda.so with error: libcublasLt.so.12: cannot open shared object file: No such file or directory
2026-02-22 13:10:38.679701570 [W:onnxruntime:Default, onnxruntime_pybind_state.cc:1055 CreateExecutionProviderInstance] Failed to create CUDAExecutionProvider. Require cuDNN 9.* and CUDA 12.*. Please install all dependencies as mentioned in the GPU requirements page (https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html\#requirements), make sure they're in the PATH, and that your GPU is supported.
2026-02-22 13:10:38,681:INFO:speaches.executors.shared.base_model_manager:_load:75:Model silero_vad_v5 loaded in 0.01s
2026-02-22 13:10:38,681:DEBUG:speaches.executors.shared.base_model_manager:_increment_ref:83:Incremented ref count for silero_vad_v5, self.ref_count=1
2026-02-22 13:10:38,691:DEBUG:speaches.executors.silero_vad_v5:__call__:138:VAD model inference took 0.0102s
2026-02-22 13:10:38,691:DEBUG:speaches.executors.shared.base_model_manager:_decrement_ref:88:Decremented ref count for silero_vad_v5, self.ref_count=0
2026-02-22 13:10:38,692:INFO:speaches.executors.shared.base_model_manager:_decrement_ref:98:Model silero_vad_v5 is idle, not unloading
2026-02-22 13:10:38,692:DEBUG:speaches.executors.silero_vad_v5:get_speech_timestamps:307:VAD processing took 0.0181s for 3.12s audio
2026-02-22 13:10:38,692:DEBUG:speaches.executors.shared.base_model_manager:_load:72:Loading model deepdml/faster-whisper-large-v3-turbo-ct2
2026-02-22 13:10:38,800:DEBUG:urllib3.connectionpool:_make_request:546:https://huggingface.co:443 "GET /api/models/deepdml/faster-whisper-large-v3-turbo-ct2/revision/main HTTP/11" 200 2873
2026-02-22 13:10:41,242:INFO:speaches.executors.shared.base_model_manager:_load:75:Model deepdml/faster-whisper-large-v3-turbo-ct2 loaded in 2.55s
2026-02-22 13:10:41,242:DEBUG:speaches.executors.shared.base_model_manager:_increment_ref:83:Incremented ref count for deepdml/faster-whisper-large-v3-turbo-ct2, self.ref_count=1
2026-02-22 13:10:41,242:INFO:faster_whisper:transcribe:391:Processing audio with duration 00:03.120
2026-02-22 13:10:41,242:INFO:faster_whisper:transcribe:428:VAD filter removed 00:00.880 of audio
2026-02-22 13:10:41,262:DEBUG:speaches.executors.shared.base_model_manager:_decrement_ref:88:Decremented ref count for deepdml/faster-whisper-large-v3-turbo-ct2, self.ref_count=0
2026-02-22 13:10:41,262:DEBUG:speaches.executors.shared.base_model_manager:_decrement_ref:91:Model deepdml/faster-whisper-large-v3-turbo-ct2 is idle, scheduling offload in 300s
INFO: 192.168.1.27:45124 - "POST /v1/audio/transcriptions HTTP/1.1" 500 Internal Server Error
ERROR: Exception in ASGI application
Traceback (most recent call last):
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/uvicorn/protocols/http/h11_impl.py", line 403, in run_asgi
result = await app( \# type: ignore[func-returns-value]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__
return await self.app(scope, receive, send)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/fastapi/applications.py", line 1134, in __call__
await super().__call__(scope, receive, send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/applications.py", line 113, in __call__
await self.middleware_stack(scope, receive, send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 187, in __call__
raise exc
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 165, in __call__
await self.app(scope, receive, _send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
raise exc
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
await app(scope, receive, sender)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
await self.app(scope, receive, send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/routing.py", line 715, in __call__
await self.middleware_stack(scope, receive, send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/routing.py", line 735, in app
await route.handle(scope, receive, send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/routing.py", line 288, in handle
await self.app(scope, receive, send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 125, in app
await wrap_app_handling_exceptions(app, request)(scope, receive, send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
raise exc
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
await app(scope, receive, sender)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 111, in app
response = await f(request)
^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 391, in app
raw_response = await run_endpoint_function(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 292, in run_endpoint_function
return await run_in_threadpool(dependant.call, **values)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/concurrency.py", line 39, in run_in_threadpool
return await anyio.to_thread.run_sync(func, *args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/anyio/to_thread.py", line 56, in run_sync
return await get_async_backend().run_sync_in_worker_thread(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 2470, in run_sync_in_worker_thread
return await future
^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 967, in run
result = context.run(func, *args)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/speaches/routers/stt.py", line 171, in transcribe_file
res = transcription_executor.model_manager.handle_transcription_request(transcription_request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/speaches/executors/whisper.py", line 231, in handle_transcription_request
return self.handle_non_streaming_transcription_request(request, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/speaches/tracing.py", line 31, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/speaches/executors/whisper.py", line 161, in handle_non_streaming_transcription_request
segments, transcription_info = whisper_model.transcribe(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/faster_whisper/transcribe.py", line 451, in transcribe
) = self.model.detect_language(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/faster_whisper/transcribe.py", line 1777, in detect_language
encoder_output = self.encode(
^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/faster_whisper/transcribe.py", line 1358, in encode
return self.model.encode(features, to_cpu=to_cpu)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: No SGEMM backend on CPU
2026-02-22 13:10:41,277:DEBUG:speaches.dependencies:audio_file_dependency:89:Decoding audio file: a8eff92c-6bac-4320-b80d-e66e9b9c1c55.mp3, content_type: None, header: Headers({'content-disposition': 'form-data; name="file"; filename="a8eff92c-6bac-4320-b80d-e66e9b9c1c55.mp3"'}), size: 25389
2026-02-22 13:10:41,367:DEBUG:speaches.dependencies:audio_file_dependency:103:Decoded 3.12s of audio in 0.08957s (RTF: 0.028709025000586197)
2026-02-22 13:10:41,368:DEBUG:asyncio:__init__:64:Using selector: EpollSelector
2026-02-22 13:10:41,371:DEBUG:speaches.executors.shared.base_model_manager:load_model:138:silero_vad_v5 model already loaded
2026-02-22 13:10:41,371:DEBUG:speaches.executors.shared.base_model_manager:_increment_ref:83:Incremented ref count for silero_vad_v5, self.ref_count=1
2026-02-22 13:10:41,378:DEBUG:speaches.executors.silero_vad_v5:__call__:138:VAD model inference took 0.0068s
2026-02-22 13:10:41,378:DEBUG:speaches.executors.shared.base_model_manager:_decrement_ref:88:Decremented ref count for silero_vad_v5, self.ref_count=0
2026-02-22 13:10:41,378:INFO:speaches.executors.shared.base_model_manager:_decrement_ref:98:Model silero_vad_v5 is idle, not unloading
2026-02-22 13:10:41,379:DEBUG:speaches.executors.silero_vad_v5:get_speech_timestamps:307:VAD processing took 0.0074s for 3.12s audio
2026-02-22 13:10:41,379:DEBUG:speaches.executors.shared.base_model_manager:load_model:138:deepdml/faster-whisper-large-v3-turbo-ct2 model already loaded
2026-02-22 13:10:41,379:DEBUG:speaches.executors.shared.base_model_manager:_increment_ref:81:Model was set to expire in 300s, cancelling
2026-02-22 13:10:41,379:DEBUG:speaches.executors.shared.base_model_manager:_increment_ref:83:Incremented ref count for deepdml/faster-whisper-large-v3-turbo-ct2, self.ref_count=1
2026-02-22 13:10:41,379:INFO:faster_whisper:transcribe:391:Processing audio with duration 00:03.120
2026-02-22 13:10:41,379:INFO:faster_whisper:transcribe:428:VAD filter removed 00:00.880 of audio
2026-02-22 13:10:41,401:DEBUG:speaches.executors.shared.base_model_manager:_decrement_ref:88:Decremented ref count for deepdml/faster-whisper-large-v3-turbo-ct2, self.ref_count=0
2026-02-22 13:10:41,401:DEBUG:speaches.executors.shared.base_model_manager:_decrement_ref:91:Model deepdml/faster-whisper-large-v3-turbo-ct2 is idle, scheduling offload in 300s
INFO: 192.168.1.27:45128 - "POST /v1/audio/transcriptions HTTP/1.1" 500 Internal Server Error
ERROR: Exception in ASGI application
Traceback (most recent call last):
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/uvicorn/protocols/http/h11_impl.py", line 403, in run_asgi
result = await app( \# type: ignore[func-returns-value]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__
return await self.app(scope, receive, send)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/fastapi/applications.py", line 1134, in __call__
await super().__call__(scope, receive, send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/applications.py", line 113, in __call__
await self.middleware_stack(scope, receive, send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 187, in __call__
raise exc
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 165, in __call__
await self.app(scope, receive, _send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
raise exc
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
await app(scope, receive, sender)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
await self.app(scope, receive, send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/routing.py", line 715, in __call__
await self.middleware_stack(scope, receive, send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/routing.py", line 735, in app
await route.handle(scope, receive, send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/routing.py", line 288, in handle
await self.app(scope, receive, send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 125, in app
await wrap_app_handling_exceptions(app, request)(scope, receive, send)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
raise exc
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
await app(scope, receive, sender)
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 111, in app
response = await f(request)
^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 391, in app
raw_response = await run_endpoint_function(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 292, in run_endpoint_function
return await run_in_threadpool(dependant.call, **values)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/starlette/concurrency.py", line 39, in run_in_threadpool
return await anyio.to_thread.run_sync(func, *args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/anyio/to_thread.py", line 56, in run_sync
return await get_async_backend().run_sync_in_worker_thread(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 2470, in run_sync_in_worker_thread
return await future
^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 967, in run
result = context.run(func, *args)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/speaches/routers/stt.py", line 171, in transcribe_file
res = transcription_executor.model_manager.handle_transcription_request(transcription_request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/speaches/executors/whisper.py", line 231, in handle_transcription_request
return self.handle_non_streaming_transcription_request(request, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/speaches/tracing.py", line 31, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/speaches/executors/whisper.py", line 161, in handle_non_streaming_transcription_request
segments, transcription_info = whisper_model.transcribe(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/faster_whisper/transcribe.py", line 451, in transcribe
) = self.model.detect_language(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/faster_whisper/transcribe.py", line 1777, in detect_language
encoder_output = self.encode(
^^^^^^^^^^^^
File "/home/ubuntu/speaches/.venv/lib/python3.12/site-packages/faster_whisper/transcribe.py", line 1358, in encode
return self.model.encode(features, to_cpu=to_cpu)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: No SGEMM backend on CPU
Là on a enfin quelque chose de propre : le modèle tourne bien, mais il manque une brique BLAS CPU dans l’image speaches.
L’erreur clé est :
- `RuntimeError: No SGEMM backend on CPU`
Ça vient de CTranslate2/faster-whisper qui a besoin d’un backend SGEMM (OpenBLAS / MKL / etc.) pour faire les multiplications de matrices en float32 sur CPU. Dans l’image `speaches:rocm-ct2` que tu utilises, tout est câblé pour le GPU (ROCm), et le chemin CPU « pur » n’a pas ce backend, donc ça plante dès qu’on enlève le GPU.
Tu as donc deux options réalistes :
***
## Option A — Revenir à l’usage GPU pour STT (simple, efficace)
Comme tu as déjà optimisé la charge globale (LLM + TTS + mem), remettre Whisper sur GPU n’est plus un drame, surtout avec `large-v3-turbo` qui est plus léger.
1. **Remettre l’accès GPU dans le compose :**
```yaml
speaches:
image: speaches:rocm-ct2
container_name: speaches-stt
restart: unless-stopped
ports:
- "8200:8000"
devices:
- /dev/kfd
- /dev/dri
group_add:
- "44"
- "991"
security_opt:
- seccomp=unconfined
environment:
- HSA_OVERRIDE_GFX_VERSION=11.0.0
- WHISPER__MODEL=deepdml/faster-whisper-large-v3-turbo-ct2
- WHISPER__LANGUAGE=fr
- WHISPER__COMPUTE_TYPE=int8
- WHISPER__DEVICE=auto
```
2. Redémarrer :
```bash
cd ~/speaches
sudo docker compose down
sudo docker compose up -d
```
3. Surveiller la charge GPU : elle montera pendant la transcription, puis retomber à un niveau bas après quelques secondes, les autres optimisations (Piper CPU, ministral 3B pour mem, etc.) faisant leur effet.
C’est la voie la plus pragmatique : tu profites à fond de la 7800XT pour STT + LLM, tout en gardant le CPU tranquille.
***
## Option B — Rebuilder une image speaches CPU-only (plus long)
Si tu tiens absolument à garder STT 100% CPU dans un conteneur séparé, il faudrait :
- repartir d’une image de base CPU (pas la ROCm),
- installer CTranslate2 compilé avec support BLAS CPU,
- et reconstruire speaches avec cette stack.
C’est faisable mais assez long (rebuild complet, ajustements de Dockerfile, tests). On peut le faire ensemble, mais ce sera plusieurs étapes de compilation.
***
À mon avis, au vu de l’état actuel de ta stack (LLM optimisé, TTS sur CPU, mémoire sur petit modèle), **Option A** est un bon compromis : tu retrouves un vocal fonctionnel, très rapide, avec une charge GPU cohérente et le CPU globalement cool. Si tu veux qu’on parte sur l’option B (image CPU-only), dis-le et on montera un plan de build pas à pas.
<span style="display:none">[^27_1][^27_10][^27_2][^27_3][^27_4][^27_5][^27_6][^27_7][^27_8][^27_9]</span>
<div align="center">⁂</div>
[^27_1]: https://github.com/docker/for-linux/issues/841
[^27_2]: https://stackoverflow.com/questions/20096632/limit-memory-on-a-docker-container-doesnt-work
[^27_3]: https://www.devopsroles.com/fix-docker-cannot-allocate-memory-error/
[^27_4]: https://community.atlassian.com/forums/Bitbucket-questions/Container-docker-exceeded-memory-limit-Confusing-cause-remedy/qaq-p/2273380
[^27_5]: https://docs.docker.com/engine/containers/resource_constraints/
[^27_6]: https://forums.docker.com/t/cannot-allocate-memory/1163
[^27_7]: https://stackoverflow.com/questions/77893652/how-to-solve-memory-error-in-docker-container-in-linux-for-error-was-sent-sigk
[^27_8]: https://www.reddit.com/r/docker/comments/1fidnei/cant_set_a_memory_limit_in_dockercomposeyaml_am_i/
[^27_9]: https://notes.kodekloud.com/docs/Docker-Certified-Associate-Exam-Course/Docker-Engine-Security/Resource-Limits-Memory
[^27_10]: https://www.reddit.com/r/docker/comments/13ud72v/creating_a_docker_container_with_memory_limits/
---
# 026-02-22 13:12:57,132:DEBUG:filelock:acquire:331:Attempting to acquire lock 140358771067648 on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock
2026-02-22 13:12:57,133:DEBUG:filelock:acquire:343:Lock 140358771067648 not acquired on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock, waiting 0.05 seconds ...
2026-02-22 13:12:57,183:DEBUG:filelock:acquire:331:Attempting to acquire lock 140358771067648 on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock
2026-02-22 13:12:57,183:DEBUG:filelock:acquire:343:Lock 140358771067648 not acquired on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock, waiting 0.05 seconds ...
2026-02-22 13:12:57,233:DEBUG:filelock:acquire:331:Attempting to acquire lock 140358771067648 on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock
2026-02-22 13:12:57,233:DEBUG:filelock:acquire:343:Lock 140358771067648 not acquired on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock, waiting 0.05 seconds ...
2026-02-22 13:12:57,283:DEBUG:filelock:acquire:331:Attempting to acquire lock 140358771067648 on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock
2026-02-22 13:12:57,283:DEBUG:filelock:acquire:343:Lock 140358771067648 not acquired on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock, waiting 0.05 seconds ...
2026-02-22 13:12:57,333:DEBUG:filelock:acquire:331:Attempting to acquire lock 140358771067648 on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock
2026-02-22 13:12:57,334:DEBUG:filelock:acquire:343:Lock 140358771067648 not acquired on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock, waiting 0.05 seconds ...
2026-02-22 13:12:57,384:DEBUG:filelock:acquire:331:Attempting to acquire lock 140358771067648 on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock
2026-02-22 13:12:57,384:DEBUG:filelock:acquire:343:Lock 140358771067648 not acquired on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock, waiting 0.05 seconds ...
2026-02-22 13:12:57,434:DEBUG:filelock:acquire:331:Attempting to acquire lock 140358771067648 on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock
2026-02-22 13:12:57,434:DEBUG:filelock:acquire:343:Lock 140358771067648 not acquired on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock, waiting 0.05 seconds ...
2026-02-22 13:12:57,484:DEBUG:filelock:acquire:331:Attempting to acquire lock 140358771067648 on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock
2026-02-22 13:12:57,484:DEBUG:filelock:acquire:343:Lock 140358771067648 not acquired on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock, waiting 0.05 seconds ...
2026-02-22 13:12:57,534:DEBUG:filelock:acquire:331:Attempting to acquire lock 140358771067648 on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock
2026-02-22 13:12:57,534:DEBUG:filelock:acquire:343:Lock 140358771067648 not acquired on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock, waiting 0.05 seconds ...
2026-02-22 13:12:57,585:DEBUG:filelock:acquire:331:Attempting to acquire lock 140358771067648 on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock
2026-02-22 13:12:57,585:DEBUG:filelock:acquire:343:Lock 140358771067648 not acquired on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock, waiting 0.05 seconds ...
2026-02-22 13:12:57,635:DEBUG:filelock:acquire:331:Attempting to acquire lock 140358771067648 on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock
2026-02-22 13:12:57,635:DEBUG:filelock:acquire:343:Lock 140358771067648 not acquired on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock, waiting 0.05 seconds ...
2026-02-22 13:12:57,685:DEBUG:filelock:acquire:331:Attempting to acquire lock 140358771067648 on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock
2026-02-22 13:12:57,685:DEBUG:filelock:acquire:343:Lock 140358771067648 not acquired on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock, waiting 0.05 seconds ...
2026-02-22 13:12:57,735:DEBUG:filelock:acquire:331:Attempting to acquire lock 140358771067648 on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock
2026-02-22 13:12:57,736:DEBUG:filelock:acquire:343:Lock 140358771067648 not acquired on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock, waiting 0.05 seconds ...
2026-02-22 13:12:57,786:DEBUG:filelock:acquire:331:Attempting to acquire lock 140358771067648 on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock
2026-02-22 13:12:57,786:DEBUG:filelock:acquire:343:Lock 140358771067648 not acquired on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock, waiting 0.05 seconds ...
2026-02-22 13:12:57,836:DEBUG:filelock:acquire:331:Attempting to acquire lock 140358771067648 on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock
2026-02-22 13:12:57,836:DEBUG:filelock:acquire:343:Lock 140358771067648 not acquired on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock, waiting 0.05 seconds ...
2026-02-22 13:12:57,886:DEBUG:filelock:acquire:331:Attempting to acquire lock 140358771067648 on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock
2026-02-22 13:12:57,887:DEBUG:filelock:acquire:343:Lock 140358771067648 not acquired on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock, waiting 0.05 seconds ...
2026-02-22 13:12:57,937:DEBUG:filelock:acquire:331:Attempting to acquire lock 140358771067648 on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock
2026-02-22 13:12:57,937:DEBUG:filelock:acquire:343:Lock 140358771067648 not acquired on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock, waiting 0.05 seconds ...
2026-02-22 13:12:57,987:DEBUG:filelock:acquire:331:Attempting to acquire lock 140358771067648 on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock
2026-02-22 13:12:57,987:DEBUG:filelock:acquire:343:Lock 140358771067648 not acquired on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock, waiting 0.05 seconds ...
2026-02-22 13:12:58,037:DEBUG:filelock:acquire:331:Attempting to acquire lock 140358771067648 on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock
2026-02-22 13:12:58,037:DEBUG:filelock:acquire:343:Lock 140358771067648 not acquired on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock, waiting 0.05 seconds ...
2026-02-22 13:12:58,088:DEBUG:filelock:acquire:331:Attempting to acquire lock 140358771067648 on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock
2026-02-22 13:12:58,088:DEBUG:filelock:acquire:343:Lock 140358771067648 not acquired on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock, waiting 0.05 seconds ...
2026-02-22 13:12:58,093:DEBUG:filelock:release:364:Attempting to release lock 140358748802176 on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock
2026-02-22 13:12:58,093:DEBUG:filelock:release:367:Lock 140358748802176 released on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock
Fetching 6 files: 100%|██████████| 6/6 [00:28<00:00, 4.79s/it]
INFO: 172.18.0.3:49268 - "POST /v1/models/deepdml/faster-whisper-large-v3-turbo-ct2 HTTP/1.1" 200 OK
2026-02-22 13:12:58,138:DEBUG:filelock:acquire:331:Attempting to acquire lock 140358771067648 on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock
2026-02-22 13:12:58,138:DEBUG:filelock:acquire:334:Lock 140358771067648 acquired on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock
2026-02-22 13:12:58,138:DEBUG:filelock:release:364:Attempting to release lock 140358771067648 on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock
2026-02-22 13:12:58,138:DEBUG:filelock:release:367:Lock 140358771067648 released on /root/.cache/huggingface/hub/.locks/models--deepdml--faster-whisper-large-v3-turbo-ct2/e76620f83d5f5b69efd3d87e3dc180c1bd21df9fbebacfd4335e5e1efcc018da.lock
2026-02-22 13:13:06,496:INFO:speaches.executors.shared.base_model_manager:_load:75:Model deepdml/faster-whisper-large-v3-turbo-ct2 loaded in 14.80s
2026-02-22 13:13:06,496:DEBUG:speaches.executors.shared.base_model_manager:_increment_ref:83:Incremented ref count for deepdml/faster-whisper-large-v3-turbo-ct2, self.ref_count=1
2026-02-22 13:13:06,496:INFO:faster_whisper:transcribe:391:Processing audio with duration 00:05.280
2026-02-22 13:13:06,496:INFO:faster_whisper:transcribe:428:VAD filter removed 00:01.184 of audio
2026-02-22 13:13:11,105:INFO:faster_whisper:transcribe:463:Detected language 'fr' with probability 0.99
2026-02-22 13:13:11,328:INFO:speaches.executors.whisper:handle_non_streaming_transcription_request:181:Transcribed 5.28 seconds of audio in 19.632597505988088 seconds
2026-02-22 13:13:11,328:DEBUG:speaches.executors.shared.base_model_manager:_decrement_ref:88:Decremented ref count for deepdml/faster-whisper-large-v3-turbo-ct2, self.ref_count=0
2026-02-22 13:13:11,328:DEBUG:speaches.executors.shared.base_model_manager:_decrement_ref:91:Model deepdml/faster-whisper-large-v3-turbo-ct2 is idle, scheduling offload in 300s
2026-02-22 13:13:11,328:ERROR:speaches.routers.stt:transcription_response_to_http_response:105:Unexpected streaming transcription response type: <class 'openai.types.audio.transcription.Transcription'>
INFO: 192.168.1.27:37238 - "POST /v1/audio/transcriptions HTTP/1.1" 200 OK
Là on est bon côté backend : Whisper s’exécute correctement, en français, sur `deepdml/faster-whisper-large-v3-turbo-ct2` (5.28 s audio, 19.6 s de traitement CPU). L’erreur que tu vois encore côté Open WebUI vient juste d’un **mauvais type de réponse retourné par speaches**.
Ligne clé :
> `Unexpected streaming transcription response type: <class 'openai.types.audio.transcription.Transcription'>`
Speaches renvoie un objet `Transcription` (réponse non-streaming), alors qu’il a cru être en mode « streaming chunk ». Open WebUI, lui, considère la requête comme streaming (il envoie des chunks) et s’attend à un flux d’events.
### Fix simple côté Open WebUI
Dans **Admin Panel → Settings → Audio** (STT) :
- Assure-toi que :
- **STT Engine** : OpenAI
- **STT URL** : `http://192.168.1.11:8200/v1`
- **STT Model** : `deepdml/faster-whisper-large-v3-turbo-ct2`
- Désactive (si présent) tout mode « streaming » pour STT (certains builds d’Open WebUI ont un toggle « Streaming STT » ou similaire). Il faut que la requête soit de type **/v1/audio/transcriptions** classique, pas streaming.
Ensuite, dans Open WebUI, teste le micro en **mode message unique** (un clic, tu parles, tu relâches, ça envoie), et non en dictée continue.
Le log speaches montre déjà :
```text
"POST /v1/audio/transcriptions HTTP/1.1" 200 OK
```
Donc côté serveur tout est bon, la transcription est bien produite. Il reste juste à ce qu’Open WebUI consomme la réponse comme une transcription simple, pas comme un flux.
Si après avoir vérifié ça tu as encore une erreur, envoie-moi la notification exacte affichée dans l’UI (copier/coller du message rouge complet), et on ajustera le paramétrage Open WebUI (certains thèmes ajoutent un flag `chunked` dans la requête STT qu’on peut désactiver).
---
# ok c'est bon ca marche, on est bon niveau reconnaissance vocale et voix. FAit moi un descriptif complet de cette configuration audio pour que je puisse facilement la reproduire stp
Voici le descriptif complet de ta **config audio Effi** (STT + TTS), telle qu’on l’a stabilisée.
***
## 1. STT – Speaches (Whisper) sur folzo-Corp
### Conteneur Docker `speaches-stt` (folzo-Corp)
Fichier `~/speaches/docker-compose.yml` :
```yaml
services:
speaches:
image: speaches:rocm-ct2
container_name: speaches-stt
restart: unless-stopped
ports:
- "8200:8000"
devices:
- /dev/kfd
- /dev/dri
group_add:
- "44"
- "991"
security_opt:
- seccomp=unconfined
environment:
- HSA_OVERRIDE_GFX_VERSION=11.0.0
- WHISPER__MODEL=deepdml/faster-whisper-large-v3-turbo-ct2
- WHISPER__LANGUAGE=fr
- WHISPER__COMPUTE_TYPE=float32
- WHISPER__DEVICE=auto
volumes:
- speaches-models:/root/.cache/huggingface
speaches-init:
image: curlimages/curl:latest
container_name: speaches-init
restart: "no"
depends_on:
- speaches
entrypoint: >
sh -c "sleep 5 &&
curl -sf -X POST http://speaches:8000/v1/models/deepdml%2Ffaster-whisper-large-v3-turbo-ct2 &&
echo 'Modèle enregistré'"
volumes:
speaches-models:
```
Points importants :
- Modèle STT : `deepdml/faster-whisper-large-v3-turbo-ct2` (CT2, turbo).[^29_1]
- Langue forcée : `WHISPER__LANGUAGE=fr`.
- Compute type CPU : `float32` (évite les erreurs int8).
- `speaches-init` enregistre le modèle au démarrage via `POST /v1/models/...`.
Commandes de déploiement :
```bash
cd ~/speaches
sudo docker compose up -d
```
***
## 2. TTS – openedai-speech + Piper FR sur folzo-Corp
### Fichiers de voix Piper
Dans `~/openedai-speech/tts-voices` :
```bash
cd ~/openedai-speech/tts-voices
wget https://huggingface.co/rhasspy/piper-voices/resolve/main/fr/fr_FR/upmc/medium/fr_FR-upmc-medium.onnx
wget https://huggingface.co/rhasspy/piper-voices/resolve/main/fr/fr_FR/upmc/medium/fr_FR-upmc-medium.onnx.json
```
### Config openedai-speech
Fichier `~/openedai-speech/tts-config/config.yaml` (partie utile) :
```yaml
tts-1:
alloy:
model: voices/fr_FR-upmc-medium.onnx
speaker: 0 # 0 = Jessica, 1 = Pierre (tu peux changer si besoin)
tts-1-hd:
alloy:
model: xtts
speaker: voices/alloy.wav
# … autres voix XTTS inchangées
```
- `tts-1` = backend Piper (CPU), voix par défaut **alloy** ↦ modèle FR `fr_FR-upmc-medium`.
- L’ancienne config `tts-1-hd` (XTTS GPU) reste, mais n’est plus utilisée.
Docker compose (rappel) :
```yaml
services:
openedai-speech:
image: ghcr.io/matatonic/openedai-speech
container_name: openedai-speech
restart: unless-stopped
ports:
- "8100:8000"
volumes:
- ./tts-voices:/app/voices
- ./tts-config:/app/config
devices:
- /dev/kfd
- /dev/dri
group_add:
- "44"
- "991"
environment:
- HSA_OVERRIDE_GFX_VERSION=11.0.0
- HIP_VISIBLE_DEVICES=0
```
Redémarrage :
```bash
cd ~/openedai-speech
sudo docker compose restart openedai-speech
```
***
## 3. Configuration Open WebUI (effi-server)
Dans **Admin Panel → Settings → Audio** :
- **STT Engine** : `OpenAI`
- **STT URL** : `http://192.168.1.11:8200/v1`
- **STT Model** : `deepdml/faster-whisper-large-v3-turbo-ct2`
- **TTS Engine** : `OpenAI`
- **TTS URL** : `http://192.168.1.11:8100/v1`
- **TTS Model** : `tts-1`
- **Voice / Speaker** : `alloy` (qui correspond à `fr_FR-upmc-medium` côté Piper)
C’est cette combinaison qui donne :
- STT rapide, précis en français (large-v3-turbo CT2 sur GPU).[^29_2][^29_1]
- TTS français fluide, léger côté CPU (Piper `fr_FR-upmc-medium`).[^29_3]
- GPU principalement utilisé par les LLM (qwen/ministral, bge-m3), plus de charge « fantôme » après les échanges.
Avec ce descriptif, tu peux reconstruire l’audio d’Effi sur une autre machine en reprenant exactement ces trois blocs : compose speaches, compose openedai-speech + config, et réglages Open WebUI.
<span style="display:none">[^29_4]</span>
<div align="center">⁂</div>
[^29_1]: https://huggingface.co/deepdml/faster-whisper-large-v3-turbo-ct2
[^29_2]: https://github.com/SYSTRAN/faster-whisper/issues/1030
[^29_3]: https://huggingface.co/rhasspy/piper-voices/tree/main/fr/fr_FR/upmc/medium
[^29_4]: https://huggingface.co/rhasspy/piper-voices/blob/49a0d23f04e055f1766e5ad15ea2d121a90fa17a/fr/fr_FR/upmc/medium/fr_FR-upmc-medium.onnx.json
Reference in New Issue View Git Blame Copy Permalink