Part 5: AI

24. Search your documents and ask about them#

What you'll do: make your PDFs searchable with Recoll, then ask the AI questions about the exact pages it found. Time: 30 minutes.

The idea#

A small model can't read a whole library at once (it reads about 1,500 words at a time). So you split the job in two:

flowchart LR
  Q["Your question"] --> R["1. Recoll search<br/>finds file + page"]
  R --> P["2. ask-pages<br/>sends only those pages"]
  P --> M["AI model"]
  M --> A["Answer +<br/>which file and page"]

Recoll does the finding (fast and exact). The AI only does the reading of a few pages.

Step 1: Install the tools#

sudo apt install recollcmd poppler-utils jq

Step 2: Tell Recoll which folder to search#

Put your documents in one folder, e.g. /srv/files/papers. Then create a settings folder for Recoll:

mkdir -p ~/.recoll-docs
nano ~/.recoll-docs/recoll.conf

topdirs = /srv/files/papers
indexstemminglanguages = english french

topdirs is the only folder Recoll will look in. Keep it narrow: only what you want searchable. The second line lets a search for impact also find impacts. Change the languages to match your documents.

Step 3: Build the index#

recollindex -c ~/.recoll-docs

It reads every document once. That can take a while for a big folder. Run the same command again whenever you add files; it only processes what's new.

Step 4: Search#

recollq -c ~/.recoll-docs -A -p 2 -n 5 cobalt sulfate

You get the file path of each match and snippets like 3 : ...cobalt sulfate emissions in.... The number in front is the PDF page.

Step 5: Create the ask-pages script#

mkdir -p ~/bin
nano ~/bin/ask-pages

#!/bin/bash
# ask-pages: ask the local AI a question about some pages of a PDF.
# Usage: ask-pages FILE.pdf FIRST-PAGE LAST-PAGE "your question"
set -euo pipefail

if [ $# -ne 4 ]; then
  echo 'Usage: ask-pages FILE.pdf FIRST-PAGE LAST-PAGE "your question"'
  exit 1
fi

pdf=$1; first=$2; last=$3; question=$4
source="$(basename "$pdf"), PDF pages $first-$last"

# Take the text of those pages, squeeze spaces, keep it short enough for the model.
text=$(pdftotext -f "$first" -l "$last" "$pdf" - | tr -s ' \n' | head -c 5000)

jq -n --arg doc "$text" --arg q "$question" --arg src "$source" '{
  temperature: 0,
  max_tokens: 400,
  messages: [
    {role: "system", content: "Answer only from the document text you are given. If the answer is not in it, reply: Not found in these pages. Keep the answer short."},
    {role: "user", content: ("Document (" + $src + "):\n" + $doc + "\n\nQuestion: " + $q)}
  ]
}' | curl -s --max-time 300 http://127.0.0.1:8095/v1/chat/completions \
       -H 'Content-Type: application/json' -d @- \
   | jq -r '.choices[0].message.content'

echo
echo "Source: $source"

Save, exit, and make it runnable:

chmod +x ~/bin/ask-pages

Step 6: Ask#

Take a file and page from the Step 4 search, and ask about two or three pages around it:

~/bin/ask-pages /srv/files/papers/report.pdf 3 5 "What are the main impacts of cobalt sulfate?"

Wait (up to a few minutes). You get a short answer, then the line Source: report.pdf, PDF pages 3-5.

Next: 25. The agent system (optional)