Observability

Brim Internals

Deep visibility into the semantic search pipeline execution

Pipeline Architecture

BRIM SEMANTIC SEARCH PIPELINE

EMBEDDING GENERATION

1. Text Extraction Model::toEmbeddableText()
2. Text Chunking ~0 chunks
3. Embedding (Ollama) 0ms avg
4. Storage (pgvector) 0ms avg
Total 0ms avg

SEMANTIC SEARCH

1. Query Embedding 537.7ms avg
2. Vector Search (cosine) 5.29ms avg
3. Model Hydration 2.77ms avg
4. Results avg 10 @ 0.636
Total 545.76ms avg

Embedding Generation

0
Total Generated
0ms
Avg Duration
Embedding Time (Ollama) 0ms
Storage Time (pgvector) 0ms
Total Chunks Created 0
Min / Max Duration 0ms / 0ms

Semantic Search

16
Total Searches
545.76ms
Avg Duration
Query Embedding Time 537.7ms
Vector Search Time 5.29ms
Model Hydration Time 2.77ms
Avg Results / Top Score 10 / 0.636
Min / Max Duration 91.99ms / 840ms

Pipeline Timing Breakdown

Embedding Generation

Text Chunking ~5ms
chunk
Ollama Embedding 0ms
embed
pgvector Insert 0ms
store

Semantic Search

Query Embedding 537.7ms
embed
Vector Search (cosine) 5.29ms
search
Model Hydration 2.77ms
hydrate

Event Stream

Time Event Duration Details
09:34:53 search.completed 114.7ms Concept | 10 results | top: 0.558 | embed: 107.4ms | search: 4.8ms | hydrate: 2.5ms
09:34:53 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 27 chars
09:30:42 search.completed 92.0ms Concept | 10 results | top: 0.691 | embed: 85.8ms | search: 4.1ms | hydrate: 2.0ms
09:30:42 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 32 chars
09:30:42 search.completed 104.6ms Concept | 10 results | top: 0.558 | embed: 97.4ms | search: 4.7ms | hydrate: 2.5ms
09:30:42 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 27 chars
09:30:24 search.completed 104.7ms Concept | 10 results | top: 0.685 | embed: 96.5ms | search: 5.1ms | hydrate: 3.2ms
09:30:24 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 14 chars
09:28:38 search.completed 122.1ms Concept | 10 results | top: 0.617 | embed: 114.0ms | search: 5.5ms | hydrate: 2.6ms
09:28:38 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 28 chars
09:27:07 search.completed 125.4ms Concept | 10 results | top: 0.628 | embed: 117.3ms | search: 5.3ms | hydrate: 2.8ms
09:27:07 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 23 chars
09:26:40 search.completed 823.8ms Concept | 10 results | top: 0.628 | embed: 815.3ms | search: 5.6ms | hydrate: 2.9ms
09:26:39 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 27 chars
08:37:39 search.completed 821.6ms Concept | 10 results | top: 0.617 | embed: 813.5ms | search: 5.3ms | hydrate: 2.8ms
08:37:39 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 28 chars
02:24:31 search.completed 793.1ms Concept | 10 results | top: 0.691 | embed: 785.1ms | search: 5.4ms | hydrate: 2.6ms
02:24:30 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 32 chars
22:24:31 search.completed 812.8ms Concept | 10 results | top: 0.685 | embed: 803.6ms | search: 6.1ms | hydrate: 3.1ms
22:24:30 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 14 chars
20:00:32 search.completed 800.7ms Concept | 10 results | top: 0.628 | embed: 794.0ms | search: 4.3ms | hydrate: 2.4ms
20:00:31 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 27 chars
16:11:02 search.completed 840.0ms Concept | 10 results | top: 0.617 | embed: 829.9ms | search: 6.8ms | hydrate: 3.3ms
16:11:01 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 28 chars
15:29:12 search.completed 795.5ms Concept | 10 results | top: 0.685 | embed: 788.1ms | search: 5.0ms | hydrate: 2.4ms
15:29:11 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 14 chars
14:33:41 search.completed 766.1ms Concept | 10 results | top: 0.628 | embed: 756.7ms | search: 5.9ms | hydrate: 3.5ms
14:33:40 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 23 chars
12:52:51 search.completed 805.4ms Concept | 10 results | top: 0.558 | embed: 797.3ms | search: 5.2ms | hydrate: 2.8ms
12:52:50 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 27 chars
11:21:02 search.completed 809.7ms Concept | 10 results | top: 0.691 | embed: 801.3ms | search: 5.4ms | hydrate: 3.0ms
11:21:01 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 32 chars
07:13:15 search.completed 742.0ms Concept | 10 results | top: 0.628 | embed: 734.2ms | search: 5.3ms | hydrate: 2.5ms
07:13:14 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 27 chars
05:37:11 search.completed 799.7ms Concept | 10 results | top: 0.685 | embed: 789.9ms | search: 6.6ms | hydrate: 3.1ms
05:37:11 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 14 chars
00:07:51 search.completed 770.1ms Concept | 10 results | top: 0.558 | embed: 763.0ms | search: 4.8ms | hydrate: 2.3ms
00:07:50 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 27 chars
23:15:22 search.completed 785.9ms Concept | 10 results | top: 0.628 | embed: 778.3ms | search: 4.8ms | hydrate: 2.8ms
23:15:21 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 27 chars
21:46:11 search.completed 778.3ms Concept | 10 results | top: 0.628 | embed: 770.5ms | search: 5.2ms | hydrate: 2.6ms
21:46:10 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 23 chars
21:13:34 search.completed 878.9ms Concept | 10 results | top: 0.617 | embed: 871.5ms | search: 5.2ms | hydrate: 2.3ms
21:13:34 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 28 chars
13:30:53 search.completed 100.2ms Concept | 10 results | top: 0.685 | embed: 93.2ms | search: 4.7ms | hydrate: 2.3ms
13:30:53 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 14 chars
13:30:34 search.completed 103.4ms Concept | 10 results | top: 0.628 | embed: 95.7ms | search: 5.2ms | hydrate: 2.6ms
13:30:34 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 27 chars
13:30:32 search.completed 815.3ms Concept | 10 results | top: 0.617 | embed: 808.2ms | search: 4.8ms | hydrate: 2.3ms
13:30:31 search.started - Concept | limit: 10 | min_sim: 0.00 | query: 28 chars

Telemetry API

GET /api/telemetry

Aggregated statistics for the specified period

GET /api/telemetry/recent

Recent event stream with optional type filter

Artisan Command
php artisan brim:telemetry stats --period=24h

Telemetry Configuration

telemetry.enabled true
telemetry.store.enabled true
telemetry.logging.enabled false
telemetry.debug false
telemetry.sample_rate 1
telemetry.store.retention_days 30