clusterpath metric clusters

MetricClusterer provides a tenant-scoped URL dimension with a hard upper bound, suitable for aggregating hit counters. It is separate from Clusterer: URL normalization can learn precise templates while metrics emit a fixed, dashboard-safe number of cluster IDs.

See clusterpath for the underlying normalizer.

Default budget (96 IDs per tenant/site)

Route and family selection weights hit share. Active IDs receive a promotion advantage so a new route must be meaningfully more important before it displaces an existing cluster. Query strings are never included in metric labels.

Lifecycle

1. Train the normalizer from representative history.
2. FreezeNormalizer so the template model is immutable.
3. Observe URLs to accumulate selection counters.
4. Rebalance at a reporting-window boundary to select the dictionary.
5. Persist the dictionary with its effective time.
6. ResetWindow before the next window.

With an immutable template model, Rebalance is the only operation that changes assignments. IDs can be reused after a future rebalance, so historical dashboard queries must resolve an ID with the dictionary that was active at the data timestamp.

Library usage

Train from history before serving live events so the initial dictionary is useful rather than fallback-heavy:

m := clusterpath.NewMetricClusterer(clusterpath.MetricConfig{MaxClusters: 96})

for _, event := range historicalEvents {
    m.Train([]byte(event.URL))
}
m.FreezeNormalizer()
for _, event := range historicalEvents {
    m.Observe([]byte(event.URL))
}
m.Rebalance()
dictionaryStore.Save(tenantID, time.Now(), m.Clusters())
m.ResetWindow()

For every live event, write additive counters keyed by tenant, time bucket, and the returned ID:

cluster := m.Observe([]byte(event.URL))
metricStore.Add(event.Timestamp, tenantID, cluster.ID, 1)

Flush the current window before changing its dictionary:

metricStore.Flush()
m.Rebalance()
dictionaryStore.Save(tenantID, time.Now(), m.Clusters())
m.ResetWindow()

• Use Assign to replay URLs against the current dictionary without changing selection counters.
• A MetricClusterer belongs to one goroutine. Run one per tenant or serialize access to a shared tenant dictionary.

CLI: offline report

go run ./cmd/metriccluster \
  -in testdata/url-clusters.txt \
  -out metric_clusters.tsv

The command reads its input three times: train normalization, select a metric dictionary with the frozen model, then assign every URL and write aggregate counts. Input must be a seekable regular file.

CLI: online windowed report

For a single-pass stream, pass -window-size. The command learns templates online and rebalances the 96-ID dictionary after each window. The report keeps each dictionary version separate — the same cluster ID may represent a different route after a later rebalance:

cat requests.tsv | go run ./cmd/metriccluster \
  -in - \
  -window-size 100000 \
  -out windowed_metric_clusters.tsv

Output has one row for every active ID in every dictionary version, with hit totals. The initial window uses only fallback IDs while candidates are gathered; subsequent windows use the dictionary selected from the preceding window.

This mode is appropriate when each version is stored with its reporting window. Use a persisted frozen dictionary for stable IDs across all windows.

Operational checklist

• Train on representative traffic before the first live rebalance.
• Always pair metric IDs with the dictionary version (and time range) that produced them.
• Flush stores before Rebalance + ResetWindow.
• Do not share one MetricClusterer across goroutines without external locking.
• Keep query strings out of labels — that is intentional for cardinality safety.

Next

• Back to clusterpath for normalization, CLI, and tuning.
• Open source overview for other projects.