Tuesday 27 October 2026 José Carrasquel Vera 720 views
AI Containers Kubernetes Ansible
https://cylab.be/blog/533/deploy-an-ai-chatbot-for-zulip-in-kubernetes
Cylab has currently deployed a local LLM which can be accessed through Open WebUI. The next obvious step is to use it as the brain of a chatbot.
Zulip offers Generic bots whose logic can be coded with python using the zulip library.
For info on how to deploy Zulip in Kubernetes read this. To populate the ollama_api_key, follow this and save it in an ansible-vault file.
The following python code is a more than reasonable start for a chat bot.
# ollama_bot_with_context.py
import os
from collections import defaultdict
import requests
import zulip
# .zuliprc defines the connection settings for the client.
client = zulip.Client(config_file="/root/.zuliprc")
# Load env variables
API_KEY = os.environ.get("OLLAMA_API_KEY")
OLLAMA_URL = os.environ.get("OLLAMA_URL")
MODEL = os.environ.get("OLLAMA_MODEL")
DEBUG = os.environ.get("BOT_DEBUG", False)
system_prompt=os.environ.get("OLLAMA_SYSTEM_PROMPT", "You are a chat bot on a research institute chatting system. As a chat bot, your answers should be short, unless instructed otherwise.")
HEADERS = {
"Authorization": f"Bearer {API_KEY}",
"Content-Type": "application/json",
}
# Maximum number of past turns (user + assistant) to retain per thread
MAX_HISTORY_MESSAGES = int(os.environ.get("MAX_HISTORY_MESSAGES", 15))
# Stores conversation history: {conversation_key: [{"role": "...", "content": "..."}, ...]}
conversations = defaultdict(list)
def get_conversation_key(msg):
"""Generates a unique identifier for a channel topic or a DM thread."""
if msg["type"] == "stream":
stream_id = msg.get("stream_id", msg.get("display_recipient"))
topic = msg.get("subject", "").lower()
return f"stream:{stream_id}:{topic}"
# Direct messages: display_recipient is a list of participant dicts [{'id': ...}, ...]
if isinstance(msg.get("display_recipient"), list):
user_ids = sorted(user["id"] for user in msg["display_recipient"])
return f"dm:{'-'.join(map(str, user_ids))}"
# Fallback to sender_id if display_recipient isn't a list
return f"dm:{msg['sender_id']}"
def handle_message(msg):
if DEBUG:
print(msg)
# Ignore messages sent by the bot itself
if msg["sender_email"] == client.email:
return
user_prompt = f"{msg["sender_full_name"]}: {msg["content"].strip()}"
thread_key = get_conversation_key(msg)
# Optional: reset context on demand
if user_prompt.lower() in ["/reset", "/clear"]:
conversations[thread_key] = []
reply_payload = {"type": msg["type"], "content": "Conversation history cleared."}
if msg["type"] == "stream":
reply_payload["to"] = msg["display_recipient"]
reply_payload["topic"] = msg.get("subject", "")
else:
reply_payload["to"] = [msg["sender_id"]]
client.send_message(reply_payload)
return
# Append user prompt to thread history. Even if the bot is not mentioned, it will still take the message into accout if mentioned in the future.
conversations[thread_key].append({"role": "user", "content": user_prompt})
# Keep only the last N messages to respect the model's context window
if len(conversations[thread_key]) > MAX_HISTORY_MESSAGES:
conversations[thread_key] = conversations[thread_key][-MAX_HISTORY_MESSAGES:]
# reply to all private messages or to non-private messages where the bot is mentioned.
if msg['type']=='private' or '@**ai-bot**' in user_prompt.lower():
try:
response = requests.post(
OLLAMA_URL,
headers=HEADERS,
json={
"model": MODEL,
"messages": conversations[thread_key] + [{"role": "system", "content": system_prompt}],
"stream": False,
},
timeout=60,
)
response.raise_for_status()
reply = response.json().get("message", {}).get("content", "No response received.")
# Save model reply to context
conversations[thread_key].append({"role": "assistant", "content": reply})
except Exception as e:
reply = f"Error reaching Ollama: {e}"
# Pop user message if the request failed so it doesn't corrupt history
if conversations[thread_key]:
conversations[thread_key].pop()
reply_payload = {
"type": msg["type"],
"content": reply,
}
if msg["type"] == "stream":
reply_payload["to"] = msg["display_recipient"]
reply_payload["topic"] = msg.get("subject", "")
else:
reply_payload["to"] = [msg["sender_id"]]
client.send_message(reply_payload)
client.call_on_each_message(handle_message, all_public_streams=True)
The following Ansible tasks create the bot if it doesn’t exsits and register its API key. We will use it to later on to populate the .zuliprc template.
- name: Create AI bot
ansible.builtin.uri:
url: "https://chat.{{ k8s_top_level_domain }}/api/v1/bots"
method: POST
user: "{{ cluster_administrator_email }}"
password: "{{ api_key.json.api_key }}"
ca_path: "/etc/ssl/certs/cluster_CA.crt"
body_format: form-urlencoded
body:
full_name: "AI-Bot"
short_name: "ai"
bot_type: "1"
ignore_errors: true
- name: Get AI bot
ansible.builtin.uri:
url: "https://chat.{{ k8s_top_level_domain }}/api/v1/bots"
method: GET
user: "{{ cluster_administrator_email }}"
password: "{{ api_key.json.api_key }}"
ca_path: "/etc/ssl/certs/cluster_CA.crt"
register: ai_bot_response
ignore_errors: true
- name: Extract API key for specific bot
ansible.builtin.set_fact:
ai_bot_key: "{{ ai_bot_response.json.bots | selectattr('username', 'equalto', 'ai-bot@chat.' + k8s_top_level_domain ) | map(attribute='api_key') | first }}"
The app is stateless so it can be deployed as a Kubernetes Deployment:
# bot-deployment.yaml (Part 1)
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: "{{ bot_name }}-bot"
namespace: zulip
spec:
replicas: 1
selector:
matchLabels:
app: "{{ bot_name }}-bot"
template:
metadata:
labels:
app: "{{ bot_name }}-bot"
spec:
restartPolicy: Always
containers:
- name: bot
image: python:3.12-slim
command:
- /bin/sh
- -c
- pip install --no-cache-dir zulip && python3 -u /opt/bot/bot.py
workingDir: /opt/bot
volumeMounts:
- name: zuliprc
mountPath: /root/.zuliprc
subPath: zuliprc
readOnly: true
- name: bot-code
mountPath: /opt/bot
readOnly: true
env:
- name: PYTHONUNBUFFERED
value: "1" # Ensure logs go to stdout immediately
- name: OLLAMA_API_KEY
valueFrom:
secretKeyRef:
name: "{{ bot_name }}-bot-credentials"
key: ollama_api_key
- name: OLLAMA_URL
value: "{{ ai_bot_ollama_url }}"
- name: OLLAMA_MODEL
value: "{{ ai_bot_ollama_model }}"
- name: MAX_HISTORY_MESSAGES
value: "{{ ai_bot_max_history_messages | default('50')}}"
- name: OLLAMA_SYSTEM_PROMPT
value: "{{ ai_bot_system_prompt }}"
resources:
limits:
memory: "256Mi"
cpu: "200m"
tolerations:
- key: "node.kubernetes.io/unreachable"
operator: "Exists"
effect: "NoExecute"
tolerationSeconds: 5
- key: "node.kubernetes.io/not-ready"
operator: "Exists"
effect: "NoExecute"
tolerationSeconds: 5
volumes:
- name: zuliprc
secret:
secretName: "{{ bot_name }}-bot-credentials"
- name: bot-code
configMap:
name: "{{ bot_name }}-bot-code"
Where the secret and configMap are as follows:
# bot-deployment.yaml (Part 2)
---
apiVersion: v1
kind: Secret
metadata:
name: "{{ bot_name }}-bot-credentials"
namespace: "zulip"
type: Opaque
stringData:
ollama_api_key: "{{ ai_bot_ollama_api_key }}"
data:
zuliprc: "{{ lookup('template', 'zuliprc') | b64encode }}"
---
apiVersion: v1
kind: ConfigMap
metadata:
name: "{{ bot_name }}-bot-code"
namespace: zulip
binaryData:
bot.py: "{{ lookup('file', bot_code) | b64encode }}"
and the zuliprc template is
[api]
email={{ bot_name }}-bot@chat.{{ k8s_top_level_domain }}
key={{ bot_key }}
site=https://chat.{{ k8s_top_level_domain }}
insecure=true
The following Ansible task will deploy the above
- name: Install ai-bot
kubernetes.core.k8s:
template: "bot-deployment.yaml"
state: present
vars:
bot_name: ai
bot_code: ollama_bot_with_context.py
bot_key: "{{ ai_bot_key }}"
This blog post is licensed under
CC BY-SA 4.0