61 lines
3.9 KiB
Markdown
61 lines
3.9 KiB
Markdown
# AI Frontend Nomad Job
|
|
|
|
This Nomad job defines the deployment for two AI frontend applications: Open WebUI and LobeChat. Both frontends are designed to interact with an Ollama backend (like the one defined in `ai-backend.nomad`).
|
|
|
|
## What is this file?
|
|
|
|
The [`ai-frontend.nomad`](stacks/ai/ai-frontend.nomad) file is a HashiCorp Nomad job specification written in HCL. It describes how to deploy and manage the Open WebUI and LobeChat services.
|
|
|
|
Key configurations:
|
|
|
|
### Open WebUI Group
|
|
- **`group "openwebui"`**: Defines the task group for Open WebUI.
|
|
- **`constraint { attribute = "${attr.unique.hostname}"; value = "hp1-home" }`**: Ensures Open WebUI runs on the `hp1-home` node.
|
|
- **`network { port "http" { static = 8080; to = 8080 } }`**: Exposes port 8080 for Open WebUI.
|
|
- **`service "openwebui"`**: Registers the service with Consul and Traefik.
|
|
- `tags = ["traefik.enable=true"]`: Enables Traefik ingress.
|
|
- **`task "server"`**: The Open WebUI container.
|
|
- **`driver = "podman"`**: Uses Podman.
|
|
- **`env { OLLAMA_BASE_URL = "http://ollama:11434" }`**: Configures Open WebUI to connect to the Ollama service.
|
|
- **`config { image = "ghcr.io/open-webui/open-webui:main" }`**: Uses the official Open WebUI image.
|
|
- **`volumes = ["/mnt/local-ssd/nomad/stacks/ai/ai-frontend/openwebui:/app/backend/data"]`**: Persistent storage for Open WebUI data.
|
|
|
|
### LobeChat Group
|
|
- **`group "lobechat"`**: Defines the task group for LobeChat.
|
|
- **`constraint { attribute = "${attr.unique.hostname}"; value = "hp1-home" }`**: Ensures LobeChat runs on the `hp1-home` node.
|
|
- **`network { port "http" { static = 3210; to = 3210 } }`**: Exposes port 3210 for LobeChat.
|
|
- **`service "lobechat"`**: Registers the service with Consul.
|
|
- *No Traefik tags*: This service is not exposed via Traefik by default.
|
|
- **`task "server"`**: The LobeChat container.
|
|
- **`driver = "podman"`**: Uses Podman.
|
|
- **`env { OLLAMA_PROXY_URL = "http://ollama.service.consul:11434" }`**: Configures LobeChat to connect to the Ollama service via Consul DNS.
|
|
- **`config { image = "lobehub/lobe-chat:latest" }`**: Uses the official LobeChat image.
|
|
- **`volumes = ["/mnt/local-ssd/nomad/stacks/ai/ai-frontend/lobechat/data:/data"]`**: Persistent storage for LobeChat data.
|
|
|
|
## How to use it
|
|
|
|
To deploy these AI frontend applications:
|
|
|
|
1. Ensure you have a Nomad cluster running with a client node tagged `hp1-home` that has Podman installed.
|
|
2. Make sure the following directories exist on the host for persistent data:
|
|
- `/mnt/local-ssd/nomad/stacks/ai/ai-frontend/openwebui`
|
|
- `/mnt/local-ssd/nomad/stacks/ai/ai-frontend/lobechat/data`
|
|
3. Ensure your Ollama backend is deployed and accessible (e.g., via the `ai-backend.nomad` job).
|
|
4. Execute the following command on your Nomad server (or a machine with Nomad CLI access configured to connect to your server):
|
|
|
|
```bash
|
|
nomad job run stacks/ai/ai-frontend.nomad
|
|
```
|
|
|
|
After deployment:
|
|
- Open WebUI will be accessible on port 8080 on the host machine, and via Traefik if properly configured.
|
|
- LobeChat will be accessible on port 3210 on the host machine. If you wish to expose LobeChat externally, you will need to add appropriate Traefik tags to its `service` block.
|
|
|
|
## Projects Involved
|
|
|
|
- **[HashiCorp Nomad](https://www.nomadproject.io/)**: A workload orchestrator.
|
|
- **[Open WebUI](https://docs.openwebui.com/)**: A user-friendly, open-source web interface for LLMs.
|
|
- **[LobeChat](https://github.com/lobehub/lobe-chat)**: An open-source, high-performance, extensible LLM chatbot framework.
|
|
- **[Podman](https://podman.io/)**: A daemonless container engine.
|
|
- **[Traefik](https://traefik.io/traefik/)**: An open-source Edge Router (used by Open WebUI).
|
|
- **[HashiCorp Consul](https://www.consul.io/)**: A service mesh solution providing service discovery, configuration, and segmentation (used for internal service discovery for Ollama by LobeChat). |