nomad/stacks/ai/ai-frontend

AI Frontend Nomad Job

This Nomad job defines the deployment for two AI frontend applications: Open WebUI and LobeChat. Both frontends are designed to interact with an Ollama backend (like the one defined in ai-backend.nomad).

What is this file?

The ai-frontend.nomad file is a HashiCorp Nomad job specification written in HCL. It describes how to deploy and manage the Open WebUI and LobeChat services.

Key configurations:

Open WebUI Group

• group "openwebui": Defines the task group for Open WebUI.
• constraint { attribute = "${attr.unique.hostname}"; value = "hp1-home" }: Ensures Open WebUI runs on the hp1-home node.
• network { port "http" { static = 8080; to = 8080 } }: Exposes port 8080 for Open WebUI.
• service "openwebui": Registers the service with Consul and Traefik.
  • tags = ["traefik.enable=true"]: Enables Traefik ingress.
• task "server": The Open WebUI container.
  • driver = "podman": Uses Podman.
  • env { OLLAMA_BASE_URL = "http://ollama:11434" }: Configures Open WebUI to connect to the Ollama service.
  • config { image = "ghcr.io/open-webui/open-webui:main" }: Uses the official Open WebUI image.
  • volumes = ["/mnt/local-ssd/nomad/stacks/ai/ai-frontend/openwebui:/app/backend/data"]: Persistent storage for Open WebUI data.

LobeChat Group

• group "lobechat": Defines the task group for LobeChat.
• constraint { attribute = "${attr.unique.hostname}"; value = "hp1-home" }: Ensures LobeChat runs on the hp1-home node.
• network { port "http" { static = 3210; to = 3210 } }: Exposes port 3210 for LobeChat.
• service "lobechat": Registers the service with Consul.
  • No Traefik tags: This service is not exposed via Traefik by default.
• task "server": The LobeChat container.
  • driver = "podman": Uses Podman.
  • env { OLLAMA_PROXY_URL = "http://ollama.service.consul:11434" }: Configures LobeChat to connect to the Ollama service via Consul DNS.
  • config { image = "lobehub/lobe-chat:latest" }: Uses the official LobeChat image.
  • volumes = ["/mnt/local-ssd/nomad/stacks/ai/ai-frontend/lobechat/data:/data"]: Persistent storage for LobeChat data.

How to use it

To deploy these AI frontend applications:

1. Ensure you have a Nomad cluster running with a client node tagged hp1-home that has Podman installed.

2. Make sure the following directories exist on the host for persistent data:

   • /mnt/local-ssd/nomad/stacks/ai/ai-frontend/openwebui
   • /mnt/local-ssd/nomad/stacks/ai/ai-frontend/lobechat/data

3. Ensure your Ollama backend is deployed and accessible (e.g., via the ai-backend.nomad job).

4. Execute the following command on your Nomad server (or a machine with Nomad CLI access configured to connect to your server):

   nomad job run stacks/ai/ai-frontend.nomad
   

After deployment:

• Open WebUI will be accessible on port 8080 on the host machine, and via Traefik if properly configured.
• LobeChat will be accessible on port 3210 on the host machine. If you wish to expose LobeChat externally, you will need to add appropriate Traefik tags to its service block.

Projects Involved

• HashiCorp Nomad: A workload orchestrator.
• Open WebUI: A user-friendly, open-source web interface for LLMs.
• LobeChat: An open-source, high-performance, extensible LLM chatbot framework.
• Podman: A daemonless container engine.
• Traefik: An open-source Edge Router (used by Open WebUI).
• HashiCorp Consul: A service mesh solution providing service discovery, configuration, and segmentation (used for internal service discovery for Ollama by LobeChat).