Monitoring with Prometheus

Scrape your whole fleet from one endpoint — every caching server, labelled and ready for Grafana.

Overview

Content Caching Monitor exposes a /metrics endpoint in the Prometheus text format. One target covers the entire fleet: every enrolled server appears as a labelled time series, so a single scrape config keeps working as servers are added and removed.

The endpoint is served by the collector — the background service that receives your servers' reports — on the same port it uses for reporting. There is nothing extra to install on the caching servers themselves.

Scrape one host, not many. The caching servers report to your Mac; Prometheus reads from your Mac. You do not scrape the caching servers, and they need no inbound access from your monitoring host.

Enable the Endpoint

In Settings > Monitoring, tick Prometheus and compatible systems. The pane then shows the endpoint URL, and a username and password under Access, shared with the Nagios endpoint.

Configure Prometheus

scrape_configs:
  - job_name: content_caching
    metrics_path: /metrics
    basic_auth:
      username: monitor
      password: your-password
    static_configs:
      - targets: ['your-mac.local:17440']

Verify

curl -u monitor:your-password http://your-mac.local:17440/metrics

A healthy response looks like this — one line per server, labelled by GUID and display name:

# HELP content_caching_server_up 1 when the server is currently reporting, 0 when it has never reported or has gone quiet
# TYPE content_caching_server_up gauge
content_caching_server_up{server="4C2A…",name="library-cache"} 1
content_caching_server_up{server="9F81…",name="lab-cache"} 0

Metrics Reference

Every metric below carries server (the stable GUID) and name (the display name) labels, except content_caching_servers, content_caching_gaps and content_caching_gaps_recent, which describe the fleet as a whole. content_caching_servers carries a state label instead: active, idle, stale, offline, retired, awaiting_first_report, never_reported, not_registered.

Fleet Health

MetricTypeMeaning
content_caching_serversgaugeServers enrolled in the fleet
content_caching_server_upgauge1 reporting, 0 gone quiet or never reported
content_caching_last_report_secondsgaugeSeconds since this server's most recent report; -1 if it has never reported

Traffic by Source

Where the bytes came from. These four sum to content_caching_bytes_served_1h, and together they are what separates a cache that is working from one that is merely busy.

MetricTypeMeaning
content_caching_bytes_served_1hgaugeBytes served in the last hour, all sources and destinations
content_caching_bytes_from_cache_1hgaugeServed from local cache
content_caching_bytes_from_origin_1hgaugeFetched from Apple — the bandwidth actually spent
content_caching_bytes_from_parents_1hgaugeServed from a parent cache
content_caching_bytes_from_peers_1hgaugeServed from a peer cache

Traffic by Destination

Who they went to. These three also sum to the total, along the other axis — a server whose traffic is mostly to children is a parent doing its job, not a cache with idle clients.

MetricTypeMeaning
content_caching_bytes_to_clients_1hgaugeServed to end-user devices
content_caching_bytes_to_children_1hgaugeServed to child caches
content_caching_bytes_to_peers_1hgaugeServed to peer caches
Why _1h gauges and not _total counters? These are sums over a rolling one-hour window, not monotonic totals. rate() over them is meaningless, and the suffix says so at the point of use — a name like bytes_served_total would invite exactly that mistake.

Cache Health

MetricTypeMeaning
content_caching_hit_ratiogaugeShare of bytes served from local cache, 0 to 1. Reports 1 when nothing was served — an idle cache has missed nothing
content_caching_pressuregaugeCache pressure on Apple's tiered 0–100 scale over the last day
content_caching_bytes_purged_1hgaugeBytes evicted in the last hour — the input to pressure
content_caching_peak_throughput_bpsgaugePeak 15-minute throughput in bits per second over the last hour
content_caching_agent_spool_reportsgaugeReports the server's agent holds and has not delivered yet, as of its last heartbeat
content_caching_agent_spool_oldest_secondsgaugeAge of the oldest report the agent holds; 0 when it holds none
content_caching_link_speed_bpsgaugeThe server's network link in bits per second: set on it in the app, else measured by its agent, else its site's default, else 1 Gbps

Delivery Integrity

Whether the reports themselves arrived. These have no equivalent in a per-server exporter, and they are the ones worth alerting on.

MetricTypeMeaning
content_caching_reports_accepted_totalcounterReports the agent has taken since it started
content_caching_reports_lost_totalcounterReports the caching service abandoned before the agent could answer — data that no longer exists
content_caching_gapsgaugeReporting windows destroyed, in the retained history
content_caching_gaps_recentgaugeThe same, within the last 24 hours — this one can return to zero
The two counters, and the two spool gauges, are emitted only for agent-assisted reporting servers. They are written by the agent's authenticated heartbeat, which a direct reporting server has nothing to send. Publishing 0 for those servers would be a confident lie about exactly the machines that lose reports while your Mac is unavailable, so they are absent instead — use absent() or content_caching_server_up to cover them.

PromQL Examples

# Bandwidth saved across the fleet, last hour — the app's figure: served from cache
sum(content_caching_bytes_from_cache_1h)

# Internet traffic avoided, parents and peers included
sum(content_caching_bytes_served_1h) - sum(content_caching_bytes_from_origin_1h)

# Fleet-wide hit ratio, weighted by traffic rather than averaged per server
sum(content_caching_bytes_from_cache_1h) / sum(content_caching_bytes_served_1h)

# Servers whose traffic is mostly upstream — a cache that is not caching
content_caching_hit_ratio < 0.5 and content_caching_bytes_served_1h > 0

# Which servers act as parents
content_caching_bytes_to_children_1h > 0

# Peak throughput as a share of each server's own link
content_caching_peak_throughput_bps / content_caching_link_speed_bps

Alerting Rules

groups:
  - name: content_caching
    rules:
      - alert: CachingServerDown
        expr: content_caching_server_up == 0
        for: 15m
        annotations:
          summary: '{{ $labels.name }} has stopped reporting'

      - alert: CacheAtCapacity
        expr: content_caching_pressure >= 100
        for: 1h
        annotations:
          summary: '{{ $labels.name }} is rejecting content for want of space'

      - alert: ReportsBeingLost
        expr: increase(content_caching_reports_lost_total[1h]) > 0
        annotations:
          summary: '{{ $labels.name }} is discarding reports before the agent can take them'

      - alert: CollectorUnreachable
        expr: up{job="content_caching"} == 0
        for: 10m
        annotations:
          summary: 'The collector is not answering — the whole fleet is dark'
Alert on up as well as on server_up. The collector runs while you are signed in. If your Mac is asleep or logged out, the scrape fails and every per-server metric goes stale — which is a different problem from a caching server going quiet, and only up distinguishes them.

Grafana

Useful starting panels: