Part 5: AI

21. Run the model as a service#

What you'll do: make the model start by itself at boot, run as the locked ai account, listen only on 127.0.0.1:8095, and stay within memory and CPU limits. Time: 15 minutes.

Step 1: Choose which model to load#

Type on: the server

sudo nano /etc/ai-runtime/model.env

File: /etc/ai-runtime/model.env

MODEL_PATH=/srv/ai/models/Ministral-3-3B-Instruct-2512-Q4_K_M.gguf

Use your exact file name. To switch models later, change only this line and restart the service (step 4).

Step 2: Create the service#

Type on: the server

sudo nano /etc/systemd/system/llama-model.service

File: /etc/systemd/system/llama-model.service

[Unit]
Description=Local AI model (llama.cpp)
After=network.target

[Service]
User=ai
Group=ai
EnvironmentFile=/etc/ai-runtime/model.env
WorkingDirectory=/var/lib/ai-workspace
ExecStart=/srv/ai/runtime/llama-server -m ${MODEL_PATH} --host 127.0.0.1 --port 8095 --ctx-size 2048 --threads 2 --parallel 1 --no-webui
Restart=on-failure
RestartSec=10

# Limits
MemoryMax=4G
CPUQuota=150%
TasksMax=64
LimitNOFILE=4096

# Lock-down
NoNewPrivileges=yes
CapabilityBoundingSet=
PrivateTmp=yes
PrivateDevices=yes
ProtectSystem=strict
ProtectHome=yes
ReadWritePaths=/var/lib/ai-workspace
ProtectKernelTunables=yes
ProtectKernelModules=yes
ProtectControlGroups=yes
RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX
IPAddressDeny=any
IPAddressAllow=localhost

[Install]
WantedBy=multi-user.target

What the important lines do:

Line	Meaning
`User=ai`	Runs as the locked account, not as you or root
`--host 127.0.0.1 --port 8095`	Only reachable from the server itself
`--ctx-size 2048`	How much text it can read at once (about 1,500 words). `4096` doubles that and uses more RAM
`--threads 2`	CPU threads to use. Set it to your physical core count
`--no-webui`	No built-in chat page. Chapter 23 shows how to turn it on
`MemoryMax=4G`, `CPUQuota=150%`	Caps on memory and CPU (150% = one and a half cores), so the rest of the server stays responsive
`ProtectHome=yes`	It can't see anyone's home folder
`IPAddressAllow=localhost`	Network traffic only to and from this computer

Step 3: Start it#

Type on: the server

sudo systemctl daemon-reload
sudo systemctl enable --now llama-model
systemctl is-active llama-model

It should say active. Loading the model takes 10–60 seconds.

Step 4: Check it answers#

Type on: the server

curl -s http://127.0.0.1:8095/health

Expected: {"status":"ok"}. If it says the model is still loading, wait 30 seconds and try again.

Now ask a real question:

Type on: the server

curl -s http://127.0.0.1:8095/v1/chat/completions -H 'Content-Type: application/json' -d '{"messages":[{"role":"user","content":"Reply with the single word READY."}],"max_tokens":10,"temperature":0}'

Somewhere in the reply you should see "content":"READY". The first question after a start can take about a minute.

Use /v1/chat/completions, not /completion

Chat ("Instruct") models expect the chat format. The older /completion address can return an empty answer.

Step 5: Prove it's not on the network#

Type on: the server

sudo ss -ltnp | grep 8095

It must show 127.0.0.1:8095. If it shows 0.0.0.0:8095 or *:8095, fix --host in the service file.

Everyday commands#

Do this	Command
Is it running?	`systemctl is-active llama-model`
Restart (e.g. after changing model)	`sudo systemctl restart llama-model`
Stop	`sudo systemctl stop llama-model`
Read its log	`sudo journalctl -u llama-model -n 50`
Switch model	edit `/etc/ai-runtime/model.env`, then restart

Done when:

/health says ok
The READY test answers
ss shows 127.0.0.1:8095 only

Next: 22. Block the AI from the internet