Data Visualizationbeta

Streaming Analytics Dashboard

A real-time analytics dashboard pulling events through Kafka, aggregating in ClickHouse, and pushing updates over WebSockets at sub-second refresh rates. I built it to learn how the columnar-store world has changed: ClickHouse is dramatically faster than the OLAP setups I was running five years ago, and it makes a single-node setup feel like a small data warehouse. The dashboard itself is intentionally simple, the interesting part is the pipeline behind it.

Apache KafkaClickHouseReactD3.jsWebSocket

What this is

A lab, not a product.

Features

Learnings

Technologies

Capabilities

What it does

The features that actually got built and run in this prototype.

feature_01.ts

Sub-second data refresh over WebSockets with backpressure handling

feature_02.ts

Streaming aggregations in ClickHouse materialised views, see the data pipeline blueprint

feature_03.ts

Interactive drill-downs that rewrite the query on click instead of preloading everything

feature_04.ts

Custom metric definitions stored in Postgres, applied in ClickHouse at query time

feature_05.ts

Alerting and anomaly detection using simple rolling z-scores, not a separate ML service

The stack

What it is built with

The libraries and runtimes I picked for this lab and why they earned their place.

Apache Kafka

ClickHouse

React

D3.js

WebSocket

What I learned

Learnings, in order of how much they surprised me

The things I would tell another engineer before they tried the same experiment.

ClickHouse handles real-time aggregations exceptionally well, especially with proper partitioning

WebSocket connection pooling is essential at scale. One socket per chart will melt your server

Client-side data windowing keeps the browser sane. Never trust the user not to leave a dashboard open for a week

The expensive part of a dashboard is not the queries, it is the people asking for new ones

Note: This is an experimental project in the beta stage. It is a learning exercise and technical exploration rather than a production-ready solution. Patterns and code may change.

Data Visualization

Related labs

Other explorations in this area.

alpha

Embedding Explorer

An interactive 3D visualiser for vector embeddings. Paste a corpus, pick an embedding model, watch UMAP project it into 3D, and click clusters to inspect what is inside. I built it as a debugging tool for RAG systems, because the fastest way to understand why a retrieval is bad is to look at the embeddings themselves. It turns into a teaching tool too: showing someone the cluster of 'shipping' tickets sitting next to 'returns' explains a vector database faster than a slide deck.

Want me to build something like this for you?

If this kind of work fits your roadmap, I take on a small number of paid projects each quarter.

Start a project Just say hello

AI Form Generator

Voice-First Interface