🧠 My LocalAI Setup
#This LocalAI setup runs in a Docker container and provides a self-hosted AI model server that can be utilized for running machine learning models locally. The setup includes features such as health checks, volume mounts, and easy-to-configure environment variables for your project.
⚙️ My Configuration Details
#🌍 Environment Variables
#| Variable | Description |
|---|
DEBUG=true | Enables debug mode for detailed logs |
MODEL_PATH=./models | Path where your machine learning models are stored |
📂 Volume Mounts
#| Volume | Description |
|---|
~/local-ai/config/models:/build/models:cached | Mounts the models directory to ensure persistent storage and caching |
🔗 Networking
#| Port Mapping | Function |
|---|
8300:8080 | API web interface for interacting with the models |
📜 My Docker Compose Configuration
#services:
api:
image: localai/localai:latest-aio-cpu # CPU version
# For a specific version:
# image: localai/localai:v2.16.0-aio-cpu
# For Nvidia GPUs, uncomment one of the following lines (cuda11 or cuda12):
# image: localai/localai:v2.16.0-aio-gpu-nvidia-cuda-11
# image: localai/localai:v2.16.0-aio-gpu-nvidia-cuda-12
# image: localai/localai:latest-aio-gpu-nvidia-cuda-11
# image: localai/localai:latest-aio-gpu-nvidia-cuda-12
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8080/readyz"] # Health check for API readiness
interval: 1m # Interval between health checks
timeout: 20m # Timeout for the health check
retries: 5 # Number of retries before failing
ports:
- 8300:8080 # Expose port 8080 to 8300 for local access
environment:
- DEBUG=true # Enables debug mode
# Add more environment variables as needed
volumes:
- ~/local-ai/config/models:/build/models:cached # Mount models directory for persistent storage
# Uncomment below if running with Nvidia GPUs
# deploy:
# resources:
# reservations:
# devices:
# - driver: nvidia
# count: 1
# capabilities: [gpu] # GPU configuration
