> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tracectrl.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Intelligence Engine

> FastAPI backend that processes spans into agent intelligence

## Overview

The TraceCtrl Engine is a FastAPI application that:

1. Runs a **background pipeline** every `PIPELINE_INTERVAL_SECONDS` to process spans.
2. Serves a **REST API** under `/api/v1` for the dashboard and external integrations.
3. Maintains **cumulative state** in ClickHouse using `ReplacingMergeTree` tables.

## Pipeline

Scheduled via APScheduler at the interval set by `PIPELINE_INTERVAL_SECONDS` (default: `60`). Each tick re-reads all spans — there is no watermark on the main path; `ReplacingMergeTree` handles deduplication via background merges. {/* Sources: engine/scheduler.py:23, engine/pipeline/runner.py:14-37 */}

**1. Fetch spans**

Read every span from `otel_traces` (via `fetch_all_spans()`).

**2. Update agent inventory**

Group spans by `tracectrl.agent.id` and upsert agent records. Tracks cumulative `observation_count`, `run_count`, `tools_observed`, and computes maturity (`LEARNING` → `MATURE` after 10 observations).

**3. Build topology**

Create agent-to-agent edges (via `tracectrl.caller.agent_id` or parent span resolution) and agent-to-tool edges. Edges carry cumulative counts and confidence levels (`LOW` → `MEDIUM` → `HIGH`).

**4. Ingest guardrail violations**

Scan `otel_traces` for `tracectrl.guardrail.evaluation` spans with `decision in ('fail', 'error')` and insert into `guardrail_violations`. A 5-minute lookback off the latest `observed_at` keeps it from rescanning the whole table every tick. {/* Sources: engine/db/violations.py:21-155 */}

**5. Ingest guardrail registry**

Scan `otel_traces` for `tracectrl.guardrail.registered` spans and upsert into `guardrail_registry`. A health-override pass flips `health='error'` for any guardrail with a decision=error violation in the last hour. {/* Sources: engine/db/guardrail_registry.py:39-189 */}

**6. Attack graph + risk**

Evaluate TAGAAI rules, build multi-hop attack paths, score risk, and write `attack_paths`, `agent_risk_scores`, `system_risk`. {/* Sources: engine/pipeline/attack_graph_runner.py, engine/rules/ */}

## TAGAAI Rules

The engine ships four detection rules. Rule names and base CVSS come from `engine/rules/`.

| Rule (`rule_name`) | OWASP | Base CVSS | Trigger |
| - | - | - | - |
| `vulnerableToPromptInjection` | ASI01 | 7.2 | Agent has a tool edge with `external` call-context > 0 AND no `human_interaction` tool. {/* Sources: engine/rules/prompt_injection.py:1-20 */} |
| `vulnerableToExcessiveAgency` | ASI02 | 8.1 | Agent has access to `code_execution` or `email` tools without a `human_interaction` guardrail. {/* Sources: engine/rules/excessive_agency.py:1, 41-53 */} |
| `vulnerableToDataLeakage` | ASI01 + ASI02 | 6.8 | Agent chain where one reads from `memory_read` and another writes to `external_api` or `email`. {/* Sources: engine/rules/data_leakage.py:1 */} |
| `ingressToEndpointAttackPath` | ASI01/ASI02 (impact-dependent) | varies (5.0–9.0) | Multi-hop ingress → high-impact endpoint chain. Base CVSS varies by impact category. {/* Sources: engine/rules/ingress_to_endpoint.py:572-604 */} |

## Guardrails Pipeline

The engine has two ingestion paths feeding the Alerts and Guardrails UI pages, both driven from spans the SDK already emits.

### Registration ingestion

`update_guardrail_registry()` reads `tracectrl.guardrail.registered` spans and writes one row per `(agent_id, guardrail_name)` to `guardrail_registry`. The ReplacingMergeTree version column is `last_seen_at`, so re-emitting the same registration just refreshes the row. Spans carry: `severity`, `mode` (`monitoring`/`blocking`), `timing` (`pre_input`/`post_output`), `judge_model`, `description`, `judge_prompt`, `health` (`active`/`error`/`disabled`), and `provider` (`judge_llm` or `protector_plus`). {/* Sources: engine/db/guardrail_registry.py:39-183, engine/db/client.py:110-125 */}

### Violation ingestion

`update_violations()` reads `tracectrl.guardrail.evaluation` spans with `decision in ('fail', 'error')` and writes them to `guardrail_violations`. The `violation_id` is the eval span id, so the ReplacingMergeTree dedupes re-inserts. Each row carries `reason`, `evidence`, `severity`, `judge_model`, and `provider`. {/* Sources: engine/db/violations.py:21-155, engine/db/client.py:94-109 */}

### ClickHouse tables (guardrails)

| Table | Engine | Purpose |
| - | - | - |
| `guardrail_registry` | `ReplacingMergeTree(last_seen_at)` ORDER BY `(agent_id, guardrail_name)` | One row per registered guardrail |
| `guardrail_violations` | `ReplacingMergeTree()` ORDER BY `(observed_at, violation_id)` | One row per failing/erroring evaluation |
| `protector_plus_config` | `ReplacingMergeTree(updated_at)` ORDER BY `(id)` | Single global row: endpoint, API key, enabled guardrails |

## Configuration

| Variable | Default | Description |
| - | - | - |
| `CLICKHOUSE_HOST` | `localhost` (`clickhouse` in compose) | ClickHouse host |
| `CLICKHOUSE_PORT` | `9000` | ClickHouse native port |
| `CLICKHOUSE_DB` | `tracectrl` | Database name |
| `PIPELINE_INTERVAL_SECONDS` | `60` | Pipeline cycle interval {/* Sources: engine/scheduler.py:23, docker-compose.yml:39 */} |

## Running Locally

```bash theme={"dark"}
cd tracectrl
source .venv/bin/activate
pip install -r engine/requirements.txt
uvicorn engine.main:app --reload --port 8000
```

> [!NOTE]
>
> When running locally, ensure ClickHouse is accessible at the configured host/port. The engine's `ensure_schema()` will create the `tracectrl` database and all tables on startup.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.