**Building a Streaming Local AI Agent: Real-Time Wikipedia Vandal Detection**
In this article, I walk through the design and implementation of a **streaming local AI agent** that watches Wikipedia’s live edit stream, detects likely vandalism, and explains its reasoning in real time—all running on your machine using Ollama and no API keys.
—
## Key Design Principles
### Streaming Means Two Things
When people say “streaming agent,” they often refer to only one aspect:
– Streaming *input* (listening to events over time).
– Streaming *output* (token-by-token reasoning).
This build does **both**, because they solve different problems:
– **Input streaming** keeps the agent responsive to live events.
– **Output streaming** gives immediate visibility into the model’s reasoning.
### Ambient Agent Pattern
Inspired by LangChain’s “ambient agent” concept and Google’s Agent Development Kit, the agent is **triggered by events**, not by human requests. It stays awake, watches a stream, and acts only when needed.
### Two-Stage Funnel (The Core Insight)
To avoid wasting compute and falling behind the live stream:
1. **Stage 1 – Cheap filtering (no LLM):**
– Runs on every event using plain Python.
– Detects boring edits (e.g., small changes or bot edits).
– Only events that cross simple thresholds move to the next stage.
2. **Stage 2 – Local LLM reasoning:**
– Fires only on a tiny fraction of events.
– Uses a local model (e.g., Llama 3.1 via Ollama).
– Returns structured, validated judgments.
This pattern mirrors real monitoring systems: cheap filters first, expensive reasoning last.
—
## Architecture Overview
“`
Live Wikipedia Event Stream
↓
Stage 1 Filter (cheap math)
↓
(Only suspicious edits proceed)
↓
Stage 2 Local LLM (Ollama)
↓
Structured Verdict + Reasoning
↓
Broadcast to connected clients (SSE)
“`
Each component maps cleanly to a single file in the pipeline, making the system easy to understand, test, and extend.
—
## Build Section 1: Event Stream Consumer (`stream_source.py`)
Wikipedia publishes live edits via **Server-Sent Events (SSE)**:
– Plain HTTP GET with no API key.
– One long-lived connection pushing edits as they happen.
Key implementation details:
– `parse_sse_line`: Handles comment lines and keep-alive messages.
– `to_event`: Normalizes Wikimedia payloads and skips irrelevant event types.
– `wikipedia_event_stream`: Auto-reconnects on network failures.
– **Anonymous edit detection:** Since Wikimedia doesn’t provide a dedicated flag, we infer anonymity by checking whether the username looks like an IPv4/IPv6 address.
—
## Build Section 2: Stage 1 Filter (`filters.py`)
### `Stage1Filter` + `EditVelocityTracker`
– Maintains a **per-user sliding window** of edit timestamps.
– Evicts old users to bound memory usage.
– Flags events that meet either condition:
– Large byte removal (`BYTES_REMOVED_THRESHOLD`).
– High edit velocity (`EDIT_VELOCITY_THRESHOLD` within a window).
Most events return `None` immediately, avoiding any LLM involvement.
—
## Build Section 3: Local Reasoner (`agent.py` + `schemas.py`)
### Structured Output via JSON Schema
– `AgentVerdict` is a Pydantic model with strict fields.
– Ollama’s `format=AgentVerdict.model_json_schema()` forces valid JSON output.
### Streaming Tokens
– `evaluate_signal` yields raw model tokens for live UI display.
– After the stream ends, it yields a single validated `AgentVerdict`.
This gives users real-time insight while keeping downstream code type-safe.
—
## Build Section 4: Broadcasting (`broadcaster.py`)
– Each connected client gets its own `asyncio.Queue`.
– `publish` uses `put_nowait` so slow clients never block the pipeline.
– `Broadcaster.stream()` is an async generator consumed by the SSE endpoint.
**Why this matters:** A single stuck subscriber used to stall entire systems; now backpressure is isolated.
—
## Wiring It Together (`main.py`)
– `run_pipeline` consumes Wikipedia events forever.
– Stage 1 runs on every event.
– Stage 2 runs only for flagged events.
– All outputs are published through the broadcaster.
FastAPI endpoints:
– `GET /events` → SSE stream of flags, tokens, and verdicts.
– `GET /health` → simple liveness check.
Lifecycle management starts and stops the pipeline cleanly with the app.
—
## How to Run
“`bash
# 1. Pull the model
ollama pull llama3.1:8b
# 2. Install dependencies
python -m venv venv
source venv/bin/activate
pip install -r requirements.txt
# 3. Start the server
uvicorn src.main:app –reload
“`
Then open:
– `http://localhost:8000/events` to watch the live stream.
– Or point a browser at the same URL to see SSE text flow.
You should see:
– `flagged` messages for suspicious edits.
– `token` messages showing the model reasoning.
– `verdict` messages with structured severity scores.
Boring edits never appear.
—
## FAQ
### Why not use an API key?
The goal is a zero-cost, fully local system. Wikipedia’s EventStreams is public and requires no authentication.
### How is “anonymous” detected without a dedicated field?
Wikipedia uses the username field to store IP addresses for anonymous editors. We detect them with regex patterns matching IPv4 and IPv6 formats.
### What if the model crashes or produces bad output?
Stage 2 is wrapped in a try/except. Errors are logged, and the pipeline continues to the next event.
### Can I watch multiple wikis or add new detection rules?
Yes. Both are configurable via `config.py` and filter thresholds are adjustable without touching core logic.
### Why not use a queue like Kafka now?
For a single-machine demo, an in-process queue is simplest. At scale, Kafka or Pulsa would replace it without changing the pipeline design.
—
## Conclusion
This project shows that truly “always-on” agents require **efficiency by design**, not as an afterthought. Every line of code serves either:
– Low-cost event filtering,
– Reliable streaming,
– Bounded resource usage, or
– Graceful degradation.
The result is a local, real-time Wikipedia vandal detector that is fast, transparent, and sustainable to run indefinitely.
If you’re building streaming agents, remember:
> **An always-on system must earn its uptime every second—not just in the first minutes of testing.**
By prioritizing simple, cheap filters and structured reasoning, you get an agent that scales in practice, not just in theory.
—
*Shittu Olumide is a software engineer and technical writer passionate about leveraging cutting-edge technologies to craft compelling narratives.*
*Follow him on Twitter [@Shittu_Olumide](https://twitter.com/Shittu_Olumide).*
