rickub
nandi/atop-analyzepublic⑂ Fork 0
⑂ 098793c
Branches
main View all branches → View all tags →
Commits
⬇ Clone ▾
git clone https://git.rickub.com/nandi/atop-analyze.git
git clone ssh://git@rickub.com/nandi/atop-analyze.git

Host key fingerprint (ed25519): SHA256:iycHnxEyq0Q7uyVpB7JlznP0G7JrTPXLYRcAU5CSLhc — verify it before your first connect.

Add atop CPU-history analyzer

Linux stores no CPU history by default, so this reads atop's logs and
asks Claude what happened.

Parses `atop -P` output into per-interval samples, builds a bounded
digest (timeline, top processes by CPU-seconds, peak-sample breakdown),
and pipes it through `claude -p`. Running the model through the Claude
Code CLI means it uses that login's credential, so no API key or
`anthropic` package is needed; `--digest-only` skips the model entirely.

Field offsets come from atop(1) PARSABLE OUTPUT. Two non-obvious ones:
PRC emits a line per thread as well as per process (so filtering on
is_process is required to avoid double-counting), and -b/-e makes atop
interleave its interactive report into stdout on lines that can start
with a valid label.

Verified against a synthetic log with four known busy-loop processes:
each reads as one saturated core of eight, and the totals match.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
nandithebull committed 2026-09-21T18:49:17-07:00 Browse files
098793c
added .gitignore +3 -0
new file mode 100644
@@ -0,0 +1,3 @@
1+__pycache__/
2+*.pyc
3+*.atop
new file mode 100644
@@ -0,0 +1,3 @@
1+__pycache__/
2+*.pyc
3+*.atop
added README.md +107 -0
new file mode 100644
@@ -0,0 +1,107 @@
1+# atop-analyze
2+
3+Ask Claude what used your CPU last night.
4+
5+Linux keeps no CPU history of its own — `/proc/stat` holds only cumulative
6+counters since boot, so `top` and `vmstat` derive their percentages by sampling
7+those twice. History exists only if something writes it down. [atop][atop] does,
8+including per-process CPU, which means you can go back and ask *what* was eating
9+the CPU at 03:00 rather than only *that* something was.
10+
11+This script reads those logs, builds a compact digest, and hands it to Claude
12+for interpretation.
13+
14+## Requirements
15+
16+- `atop`, with its logging service running (see [Setup](#setup))
17+- Python 3.10+ (standard library only)
18+- [Claude Code][cc] on `PATH`, logged in
19+
20+The model runs through `claude -p`, so this uses whatever credential Claude Code
21+is logged in with — a Pro/Max subscription included. No `ANTHROPIC_API_KEY` and
22+no `anthropic` package. `--digest-only` needs neither Claude Code nor a network.
23+
24+## Usage
25+
26+```sh
27+./atop_analyze.py # today
28+./atop_analyze.py --date 20260920 # a past day
29+./atop_analyze.py -b 03:00 -e 05:00 # a window
30+./atop_analyze.py -q "why the iowait spike at 04:10?"
31+./atop_analyze.py --digest-only # just the numbers, no model
32+```
33+
34+Useful flags: `--top N` (processes to rank), `--max-rows N` (timeline rows),
35+`--effort low|medium|high|xhigh|max`, `--model`, `--claude-bin`.
36+
37+## What it sends
38+
39+Not the raw log — a day of samples would be millions of tokens. It sends a
40+bounded digest with three parts:
41+
42+- **Timeline** — per-interval sys/user/wait/irq/steal, load average, top 3
43+ consumers. Samples beyond `--max-rows` (default 60) are averaged into buckets,
44+ so a full day stays compact.
45+- **Top processes** — ranked by CPU-seconds over the window, with average cores
46+ and distinct pid count.
47+- **Peak sample** — the busiest interval broken down by process.
48+
49+Percentages are of the **whole machine**: one saturated core on an 8-core box is
50+~12.5%, not 100%. The system prompt tells the model as much, along with the other
51+traps that make this analysis go wrong — iowait is I/O blocking rather than CPU
52+pressure, load average counts uninterruptible sleep too, and coarse intervals
53+average short spikes away.
54+
55+## Setup
56+
57+```sh
58+# Arch
59+sudo pacman -S atop
60+sudo systemctl enable --now atop.service atopacct.service atop-rotate.timer
61+```
62+
63+`atopacct.service` adds process accounting, so processes that exited during an
64+interval are still attributed. `atop-rotate.timer` rolls the log at midnight and
65+prunes old ones — without it atop appends to one file forever.
66+
67+The default sample interval is 600s, which averages a two-minute spike into
68+near-invisibility. For chasing spikes, drop it to 60s:
69+
70+```sh
71+sudo sed -i 's/^LOGINTERVAL=.*/LOGINTERVAL=60/' /etc/default/atop
72+sudo systemctl restart atop.service
73+```
74+
75+That is ~10x the samples; `LOGGENERATIONS` in the same file controls retention
76+(28 days by default).
77+
78+## Reading atop directly
79+
80+The digest is a summary. For the full picture:
81+
82+```sh
83+atop -r /var/log/atop/atop_20260921 # t / T to step samples, c for command lines
84+atop -r /var/log/atop/atop_20260921 -b 03:00 # jump to a time
85+```
86+
87+## Notes
88+
89+Field offsets come from `atop(1)`'s PARSABLE OUTPUT section. Two things that bite
90+when parsing it yourself:
91+
92+- `PRC` emits a line per **thread** as well as per process, and a
93+ single-threaded process's thread line duplicates it exactly. Filter on the
94+ `is_process` field or you will double-count.
95+- With `-b`/`-e`, atop interleaves its *interactive* report into stdout, and
96+ those lines can begin with a valid label (lowercase `cpu`). A label whitelist
97+ alone is not enough.
98+
99+`-Z` keeps the field count constant at 27; without it, spaces in process names
100+make it vary.
101+
102+atop's first sample after a `RESET` marker holds totals since boot rather than
103+interval deltas, so it is skipped. A freshly started atop therefore has nothing
104+to report until one more interval elapses.
105+
106+[atop]: https://www.atoptool.nl/
107+[cc]: https://claude.com/claude-code
new file mode 100644
@@ -0,0 +1,107 @@
1+# atop-analyze
2+
3+Ask Claude what used your CPU last night.
4+
5+Linux keeps no CPU history of its own — `/proc/stat` holds only cumulative
6+counters since boot, so `top` and `vmstat` derive their percentages by sampling
7+those twice. History exists only if something writes it down. [atop][atop] does,
8+including per-process CPU, which means you can go back and ask *what* was eating
9+the CPU at 03:00 rather than only *that* something was.
10+
11+This script reads those logs, builds a compact digest, and hands it to Claude
12+for interpretation.
13+
14+## Requirements
15+
16+- `atop`, with its logging service running (see [Setup](#setup))
17+- Python 3.10+ (standard library only)
18+- [Claude Code][cc] on `PATH`, logged in
19+
20+The model runs through `claude -p`, so this uses whatever credential Claude Code
21+is logged in with — a Pro/Max subscription included. No `ANTHROPIC_API_KEY` and
22+no `anthropic` package. `--digest-only` needs neither Claude Code nor a network.
23+
24+## Usage
25+
26+```sh
27+./atop_analyze.py # today
28+./atop_analyze.py --date 20260920 # a past day
29+./atop_analyze.py -b 03:00 -e 05:00 # a window
30+./atop_analyze.py -q "why the iowait spike at 04:10?"
31+./atop_analyze.py --digest-only # just the numbers, no model
32+```
33+
34+Useful flags: `--top N` (processes to rank), `--max-rows N` (timeline rows),
35+`--effort low|medium|high|xhigh|max`, `--model`, `--claude-bin`.
36+
37+## What it sends
38+
39+Not the raw log — a day of samples would be millions of tokens. It sends a
40+bounded digest with three parts:
41+
42+- **Timeline** — per-interval sys/user/wait/irq/steal, load average, top 3
43+ consumers. Samples beyond `--max-rows` (default 60) are averaged into buckets,
44+ so a full day stays compact.
45+- **Top processes** — ranked by CPU-seconds over the window, with average cores
46+ and distinct pid count.
47+- **Peak sample** — the busiest interval broken down by process.
48+
49+Percentages are of the **whole machine**: one saturated core on an 8-core box is
50+~12.5%, not 100%. The system prompt tells the model as much, along with the other
51+traps that make this analysis go wrong — iowait is I/O blocking rather than CPU
52+pressure, load average counts uninterruptible sleep too, and coarse intervals
53+average short spikes away.
54+
55+## Setup
56+
57+```sh
58+# Arch
59+sudo pacman -S atop
60+sudo systemctl enable --now atop.service atopacct.service atop-rotate.timer
61+```
62+
63+`atopacct.service` adds process accounting, so processes that exited during an
64+interval are still attributed. `atop-rotate.timer` rolls the log at midnight and
65+prunes old ones — without it atop appends to one file forever.
66+
67+The default sample interval is 600s, which averages a two-minute spike into
68+near-invisibility. For chasing spikes, drop it to 60s:
69+
70+```sh
71+sudo sed -i 's/^LOGINTERVAL=.*/LOGINTERVAL=60/' /etc/default/atop
72+sudo systemctl restart atop.service
73+```
74+
75+That is ~10x the samples; `LOGGENERATIONS` in the same file controls retention
76+(28 days by default).
77+
78+## Reading atop directly
79+
80+The digest is a summary. For the full picture:
81+
82+```sh
83+atop -r /var/log/atop/atop_20260921 # t / T to step samples, c for command lines
84+atop -r /var/log/atop/atop_20260921 -b 03:00 # jump to a time
85+```
86+
87+## Notes
88+
89+Field offsets come from `atop(1)`'s PARSABLE OUTPUT section. Two things that bite
90+when parsing it yourself:
91+
92+- `PRC` emits a line per **thread** as well as per process, and a
93+ single-threaded process's thread line duplicates it exactly. Filter on the
94+ `is_process` field or you will double-count.
95+- With `-b`/`-e`, atop interleaves its *interactive* report into stdout, and
96+ those lines can begin with a valid label (lowercase `cpu`). A label whitelist
97+ alone is not enough.
98+
99+`-Z` keeps the field count constant at 27; without it, spaces in process names
100+make it vary.
101+
102+atop's first sample after a `RESET` marker holds totals since boot rather than
103+interval deltas, so it is skipped. A freshly started atop therefore has nothing
104+to report until one more interval elapses.
105+
106+[atop]: https://www.atoptool.nl/
107+[cc]: https://claude.com/claude-code
added atop_analyze.py +368 -0
new file mode 100755
@@ -0,0 +1,368 @@
1+#!/usr/bin/env python3
2+"""Analyze atop history with Claude.
3+
4+Reads atop's binary logs via `atop -P` (parsable output), builds a compact
5+digest of CPU behaviour over time, and asks Claude to interpret it.
6+
7+ ./atop_analyze.py # today's log
8+ ./atop_analyze.py --date 20260920 # a specific day
9+ ./atop_analyze.py -b 03:00 -e 05:00 # a window
10+ ./atop_analyze.py --digest-only # just the digest, no model call
11+ ./atop_analyze.py -q "why the iowait spike at 04:10?"
12+
13+The model runs through the Claude Code CLI (`claude -p`), so this uses whatever
14+credential Claude Code is logged in with -- including a Pro/Max subscription --
15+and needs no ANTHROPIC_API_KEY and no `anthropic` package.
16+
17+Field layouts come from atop(1) PARSABLE OUTPUT. Every line starts with six
18+fields -- label host epoch date time interval -- and the rest is label-specific.
19+"""
20+
21+from __future__ import annotations
22+
23+import argparse
24+import os
25+import re
26+import shutil
27+import subprocess
28+import sys
29+from collections import defaultdict
30+from dataclasses import dataclass, field
31+from datetime import datetime
32+
33+DEFAULT_LOGDIR = "/var/log/atop"
34+CLAUDE_BIN = "claude"
35+CLAUDE_MODEL = "opus"
36+
37+WANTED_LABELS = {"CPU", "CPL", "PRC"}
38+DATE_RE = re.compile(r"^\d{4}/\d{2}/\d{2}$")
39+
40+# Offsets into the label-specific fields (i.e. after the 6 common header fields).
41+# CPU: hertz ncpu sys user nice idle wait irq softirq steal guest freq freqperc ...
42+CPU_HERTZ, CPU_NCPU = 0, 1
43+CPU_MODES = { # name -> offset
44+ "sys": 2, "user": 3, "nice": 4, "idle": 5,
45+ "wait": 6, "irq": 7, "softirq": 8, "steal": 9,
46+}
47+CPU_GUEST = 10 # overlaps user mode -- excluded from the total on purpose
48+
49+# CPL: ncpu load1 load5 load15 ctxsw intr
50+CPL_LOAD1, CPL_LOAD5, CPL_LOAD15, CPL_CTXSW, CPL_INTR = 1, 2, 3, 4, 5
51+
52+# PRC: pid name state hertz utime stime nice prio rtprio policy curcpu
53+# sleepavg tgid is_process ...
54+# atop emits a line per thread as well as per process; only is_process == 'y'
55+# lines are real processes, and a single-threaded process's thread line
56+# duplicates it exactly. Counting both double-counts CPU time.
57+PRC_PID, PRC_NAME, PRC_STATE, PRC_HERTZ, PRC_UTIME, PRC_STIME = 0, 1, 2, 3, 4, 5
58+PRC_ISPROC = 13
59+
60+
61+@dataclass
62+class Sample:
63+ """One atop interval."""
64+
65+ epoch: int
66+ when: str
67+ interval: int
68+ ncpu: int = 0
69+ hertz: int = 100
70+ modes: dict[str, int] = field(default_factory=dict)
71+ load1: float = 0.0
72+ load5: float = 0.0
73+ load15: float = 0.0
74+ ctxsw: int = 0
75+ intr: int = 0
76+ procs: list[tuple[str, int, float]] = field(default_factory=list)
77+
78+ @property
79+ def total_ticks(self) -> int:
80+ return sum(self.modes.values())
81+
82+ def pct(self, mode: str) -> float:
83+ total = self.total_ticks
84+ return 100.0 * self.modes.get(mode, 0) / total if total else 0.0
85+
86+ @property
87+ def busy_pct(self) -> float:
88+ return 100.0 - self.pct("idle") - self.pct("wait")
89+
90+
91+class AtopError(RuntimeError):
92+ pass
93+
94+
95+def log_path(date: str | None, logdir: str) -> str:
96+ stamp = date or datetime.now().strftime("%Y%m%d")
97+ path = os.path.join(logdir, f"atop_{stamp}")
98+ if not os.path.exists(path):
99+ have = sorted(f for f in os.listdir(logdir)) if os.path.isdir(logdir) else []
100+ raise AtopError(
101+ f"no atop log at {path}."
102+ + (f" Available: {', '.join(have)}" if have else " Is atop.service running?")
103+ )
104+ return path
105+
106+
107+def run_atop(path: str, begin: str | None, end: str | None) -> str:
108+ if not shutil.which("atop"):
109+ raise AtopError("atop is not installed (pacman -S atop)")
110+ cmd = ["atop", "-r", path, "-Z", "-P", "CPU,CPL,PRC"]
111+ if begin:
112+ cmd += ["-b", begin]
113+ if end:
114+ cmd += ["-e", end]
115+ try:
116+ proc = subprocess.run(cmd, capture_output=True, text=True, timeout=180)
117+ except subprocess.TimeoutExpired:
118+ raise AtopError("atop took over 180s to read the log; narrow the range with -b/-e")
119+ if proc.returncode != 0:
120+ raise AtopError(f"atop failed ({proc.returncode}): {proc.stderr.strip()}")
121+ return proc.stdout
122+
123+
124+def parse(raw: str) -> list[Sample]:
125+ """Turn `atop -P` output into samples, dropping the since-boot RESET sample.
126+
127+ atop reports per-interval deltas for every sample except the first one after
128+ a RESET marker, which holds totals since boot. Mixing the two would make the
129+ first interval dominate everything, so the RESET sample is skipped.
130+ """
131+ samples: list[Sample] = []
132+ current: Sample | None = None
133+ skip_this = False
134+
135+ for line in raw.splitlines():
136+ if not line:
137+ continue
138+ if line == "RESET":
139+ skip_this = True
140+ continue
141+ if line == "SEP":
142+ if current and not skip_this:
143+ samples.append(current)
144+ current, skip_this = None, False
145+ continue
146+
147+ parts = line.split()
148+ if len(parts) < 7:
149+ continue
150+ label, _host, epoch, date, tm = parts[:5]
151+ # With -b/-e, atop interleaves its interactive-format report into
152+ # stdout. Those lines can start with a real label (lowercase "cpu"),
153+ # so also require the date field to actually be a date.
154+ if label not in WANTED_LABELS or not DATE_RE.match(date):
155+ continue
156+ try:
157+ interval = int(parts[5])
158+ rest = parts[6:]
159+ except ValueError:
160+ continue
161+
162+ if current is None:
163+ current = Sample(epoch=int(epoch), when=tm, interval=interval)
164+
165+ try:
166+ if label == "CPU":
167+ current.hertz = int(rest[CPU_HERTZ])
168+ current.ncpu = int(rest[CPU_NCPU])
169+ current.modes = {n: int(rest[i]) for n, i in CPU_MODES.items()}
170+ elif label == "CPL":
171+ current.load1 = float(rest[CPL_LOAD1])
172+ current.load5 = float(rest[CPL_LOAD5])
173+ current.load15 = float(rest[CPL_LOAD15])
174+ current.ctxsw = int(rest[CPL_CTXSW])
175+ current.intr = int(rest[CPL_INTR])
176+ elif label == "PRC":
177+ if rest[PRC_ISPROC] != "y":
178+ continue # thread line, not a process
179+ hertz = int(rest[PRC_HERTZ])
180+ ticks = int(rest[PRC_UTIME]) + int(rest[PRC_STIME])
181+ if ticks:
182+ capacity = hertz * interval * max(current.ncpu or 1, 1)
183+ pct = 100.0 * ticks / capacity if capacity else 0.0
184+ current.procs.append((rest[PRC_NAME], int(rest[PRC_PID]), pct))
185+ except (IndexError, ValueError):
186+ continue # a label whose layout this atop version changed
187+
188+ if current and not skip_this:
189+ samples.append(current)
190+ for s in samples:
191+ s.procs.sort(key=lambda p: -p[2])
192+ return samples
193+
194+
195+def bucket(samples: list[Sample], max_rows: int) -> list[list[Sample]]:
196+ """Group samples into at most max_rows contiguous buckets."""
197+ if len(samples) <= max_rows:
198+ return [[s] for s in samples]
199+ size = -(-len(samples) // max_rows) # ceil
200+ return [samples[i : i + size] for i in range(0, len(samples), size)]
201+
202+
203+def digest(samples: list[Sample], top: int, max_rows: int) -> str:
204+ """Build a compact, token-bounded text summary of the run."""
205+ ncpu = next((s.ncpu for s in samples if s.ncpu), 0)
206+ span = f"{samples[0].when}-{samples[-1].when}"
207+ out: list[str] = [
208+ f"host cores: {ncpu}",
209+ f"window: {span} samples: {len(samples)} interval: {samples[0].interval}s",
210+ "",
211+ "TIMELINE (CPU% of whole machine; load1 = 1-min load average)",
212+ f"{'time':>9} {'busy':>6} {'user':>6} {'sys':>6} {'wait':>6} {'irq':>5} "
213+ f"{'stl':>5} {'load1':>6} top consumers",
214+ ]
215+
216+ for group in bucket(samples, max_rows):
217+ n = len(group)
218+ avg = lambda fn: sum(fn(s) for s in group) / n # noqa: E731
219+ label = group[0].when if n == 1 else f"{group[0].when[:5]}+{n}"
220+ merged: dict[str, float] = defaultdict(float)
221+ for s in group:
222+ for name, _pid, pct in s.procs:
223+ merged[name] += pct / n
224+ leaders = sorted(merged.items(), key=lambda kv: -kv[1])[:3]
225+ tops = " ".join(f"{name}:{pct:.0f}%" for name, pct in leaders if pct >= 1.0)
226+ out.append(
227+ f"{label:>9} {avg(lambda s: s.busy_pct):6.1f} "
228+ f"{avg(lambda s: s.pct('user') + s.pct('nice')):6.1f} "
229+ f"{avg(lambda s: s.pct('sys')):6.1f} {avg(lambda s: s.pct('wait')):6.1f} "
230+ f"{avg(lambda s: s.pct('irq') + s.pct('softirq')):5.1f} "
231+ f"{avg(lambda s: s.pct('steal')):5.1f} "
232+ f"{avg(lambda s: s.load1):6.2f} {tops}"
233+ )
234+
235+ # Overall leaders, weighted by interval so uneven samples don't skew it.
236+ total_secs: dict[str, float] = defaultdict(float)
237+ pids: dict[str, set[int]] = defaultdict(set)
238+ for s in samples:
239+ cores = max(s.ncpu or 1, 1)
240+ for name, pid, pct in s.procs:
241+ total_secs[name] += pct / 100.0 * cores * s.interval
242+ pids[name].add(pid)
243+ wall = sum(s.interval for s in samples) or 1
244+
245+ out += ["", f"TOP PROCESSES over the whole window ({wall}s wall clock)",
246+ f"{'process':<24} {'cpu-sec':>9} {'avg-cores':>10} {'pids':>5}"]
247+ for name, secs in sorted(total_secs.items(), key=lambda kv: -kv[1])[:top]:
248+ out.append(f"{name:<24} {secs:9.1f} {secs / wall:10.2f} {len(pids[name]):5d}")
249+
250+ peak = max(samples, key=lambda s: s.busy_pct)
251+ out += ["", f"PEAK SAMPLE {peak.when} busy {peak.busy_pct:.1f}% "
252+ f"wait {peak.pct('wait'):.1f}% load1 {peak.load1:.2f} "
253+ f"ctxsw {peak.ctxsw} intr {peak.intr}"]
254+ for name, pid, pct in peak.procs[:10]:
255+ out.append(f" {name:<24} pid {pid:<8} {pct:5.1f}% of machine")
256+ return "\n".join(out)
257+
258+
259+SYSTEM = """You are a Linux performance engineer reading atop history.
260+
261+The digest gives per-interval CPU state plus per-process CPU attribution.
262+Ground every claim in the numbers shown and cite the timestamps you used.
263+
264+Interpret carefully:
265+- Percentages are of the WHOLE machine, so one saturated core on an 8-core box
266+ is ~12.5%, not 100%.
267+- High 'wait' (iowait) means blocked on I/O, not CPU pressure -- do not call it
268+ a CPU bottleneck.
269+- Load average counts runnable AND uninterruptible-sleep tasks, so load can be
270+ high while CPU sits idle. Say which one the data supports.
271+- Sampled intervals average away short spikes. If the interval is coarse
272+ (>= 300s) say so rather than reading precise events into it.
273+- Per-process rows only cover processes alive at sample time; short-lived
274+ processes may be missing from attribution while still showing in system time.
275+
276+Structure the answer as: verdict (one or two sentences), then the evidence,
277+then what to check next as concrete commands. Say plainly when the data is
278+insufficient instead of speculating."""
279+
280+
281+def analyze(
282+ text: str, question: str | None, effort: str, model: str, claude_bin: str
283+) -> int:
284+ """Pipe the digest through `claude -p`, inheriting Claude Code's own login."""
285+ if not shutil.which(claude_bin):
286+ print(f"error: {claude_bin} not found on PATH. Install Claude Code, or "
287+ f"use --digest-only for the numbers without analysis.",
288+ file=sys.stderr)
289+ return 1
290+
291+ ask = question or (
292+ "Analyze this CPU history. What used the CPU, was the machine actually "
293+ "CPU-bound, and is anything here worth investigating?"
294+ )
295+ # The digest goes in on stdin rather than argv -- a long window can run to
296+ # tens of kilobytes, past the point where argv limits get interesting.
297+ prompt = f"{ask}\n\n<atop_digest>\n{text}\n</atop_digest>"
298+ cmd = [
299+ claude_bin, "-p",
300+ "--model", model,
301+ "--effort", effort,
302+ "--system-prompt", SYSTEM, # replaces the coding-agent prompt wholesale
303+ "--restricted", # this is pure text analysis; no tools needed
304+ ]
305+ try:
306+ # stdout/stderr are inherited, so the response streams as it arrives.
307+ proc = subprocess.run(cmd, input=prompt, text=True)
308+ except OSError as e:
309+ print(f"error: could not run {claude_bin}: {e}", file=sys.stderr)
310+ return 1
311+ except KeyboardInterrupt:
312+ return 130
313+ if proc.returncode != 0:
314+ print(f"\nerror: {claude_bin} exited {proc.returncode}. If this is an "
315+ f"auth problem, run `claude` once interactively to log in.",
316+ file=sys.stderr)
317+ return proc.returncode
318+
319+
320+def main() -> int:
321+ p = argparse.ArgumentParser(description="Analyze atop CPU history with Claude.")
322+ src = p.add_mutually_exclusive_group()
323+ src.add_argument("--date", help="log day as YYYYMMDD (default: today)")
324+ src.add_argument("--file", help="path to a raw atop log")
325+ p.add_argument("--logdir", default=DEFAULT_LOGDIR)
326+ p.add_argument("-b", "--begin", help="start time HH:MM")
327+ p.add_argument("-e", "--end", help="end time HH:MM")
328+ p.add_argument("-q", "--question", help="ask something specific")
329+ p.add_argument("--top", type=int, default=15, help="processes to rank (default 15)")
330+ p.add_argument("--max-rows", type=int, default=60,
331+ help="timeline rows; extra samples are averaged into buckets")
332+ p.add_argument("--effort", default="high",
333+ choices=["low", "medium", "high", "xhigh", "max"])
334+ p.add_argument("--model", default=CLAUDE_MODEL,
335+ help=f"model alias or full name (default: {CLAUDE_MODEL})")
336+ p.add_argument("--claude-bin", default=CLAUDE_BIN,
337+ help="path to the claude CLI")
338+ p.add_argument("--digest-only", action="store_true",
339+ help="print the digest and exit; runs no model")
340+ args = p.parse_args()
341+
342+ try:
343+ path = args.file or log_path(args.date, args.logdir)
344+ samples = parse(run_atop(path, args.begin, args.end))
345+ except AtopError as e:
346+ print(f"error: {e}", file=sys.stderr)
347+ return 1
348+ except PermissionError:
349+ print(f"error: cannot read {args.logdir} -- try run0", file=sys.stderr)
350+ return 1
351+
352+ if not samples:
353+ print("error: no usable samples in range. atop's first sample holds "
354+ "since-boot totals and is skipped, so a freshly started atop has "
355+ "nothing to compare yet -- wait for one more interval.",
356+ file=sys.stderr)
357+ return 1
358+
359+ text = digest(samples, args.top, args.max_rows)
360+ if args.digest_only:
361+ print(text)
362+ return 0
363+ print(f"# {len(samples)} samples from {path}\n", file=sys.stderr)
364+ return analyze(text, args.question, args.effort, args.model, args.claude_bin)
365+
366+
367+if __name__ == "__main__":
368+ sys.exit(main())
new file mode 100755
@@ -0,0 +1,368 @@
1+#!/usr/bin/env python3
2+"""Analyze atop history with Claude.
3+
4+Reads atop's binary logs via `atop -P` (parsable output), builds a compact
5+digest of CPU behaviour over time, and asks Claude to interpret it.
6+
7+ ./atop_analyze.py # today's log
8+ ./atop_analyze.py --date 20260920 # a specific day
9+ ./atop_analyze.py -b 03:00 -e 05:00 # a window
10+ ./atop_analyze.py --digest-only # just the digest, no model call
11+ ./atop_analyze.py -q "why the iowait spike at 04:10?"
12+
13+The model runs through the Claude Code CLI (`claude -p`), so this uses whatever
14+credential Claude Code is logged in with -- including a Pro/Max subscription --
15+and needs no ANTHROPIC_API_KEY and no `anthropic` package.
16+
17+Field layouts come from atop(1) PARSABLE OUTPUT. Every line starts with six
18+fields -- label host epoch date time interval -- and the rest is label-specific.
19+"""
20+
21+from __future__ import annotations
22+
23+import argparse
24+import os
25+import re
26+import shutil
27+import subprocess
28+import sys
29+from collections import defaultdict
30+from dataclasses import dataclass, field
31+from datetime import datetime
32+
33+DEFAULT_LOGDIR = "/var/log/atop"
34+CLAUDE_BIN = "claude"
35+CLAUDE_MODEL = "opus"
36+
37+WANTED_LABELS = {"CPU", "CPL", "PRC"}
38+DATE_RE = re.compile(r"^\d{4}/\d{2}/\d{2}$")
39+
40+# Offsets into the label-specific fields (i.e. after the 6 common header fields).
41+# CPU: hertz ncpu sys user nice idle wait irq softirq steal guest freq freqperc ...
42+CPU_HERTZ, CPU_NCPU = 0, 1
43+CPU_MODES = { # name -> offset
44+ "sys": 2, "user": 3, "nice": 4, "idle": 5,
45+ "wait": 6, "irq": 7, "softirq": 8, "steal": 9,
46+}
47+CPU_GUEST = 10 # overlaps user mode -- excluded from the total on purpose
48+
49+# CPL: ncpu load1 load5 load15 ctxsw intr
50+CPL_LOAD1, CPL_LOAD5, CPL_LOAD15, CPL_CTXSW, CPL_INTR = 1, 2, 3, 4, 5
51+
52+# PRC: pid name state hertz utime stime nice prio rtprio policy curcpu
53+# sleepavg tgid is_process ...
54+# atop emits a line per thread as well as per process; only is_process == 'y'
55+# lines are real processes, and a single-threaded process's thread line
56+# duplicates it exactly. Counting both double-counts CPU time.
57+PRC_PID, PRC_NAME, PRC_STATE, PRC_HERTZ, PRC_UTIME, PRC_STIME = 0, 1, 2, 3, 4, 5
58+PRC_ISPROC = 13
59+
60+
61+@dataclass
62+class Sample:
63+ """One atop interval."""
64+
65+ epoch: int
66+ when: str
67+ interval: int
68+ ncpu: int = 0
69+ hertz: int = 100
70+ modes: dict[str, int] = field(default_factory=dict)
71+ load1: float = 0.0
72+ load5: float = 0.0
73+ load15: float = 0.0
74+ ctxsw: int = 0
75+ intr: int = 0
76+ procs: list[tuple[str, int, float]] = field(default_factory=list)
77+
78+ @property
79+ def total_ticks(self) -> int:
80+ return sum(self.modes.values())
81+
82+ def pct(self, mode: str) -> float:
83+ total = self.total_ticks
84+ return 100.0 * self.modes.get(mode, 0) / total if total else 0.0
85+
86+ @property
87+ def busy_pct(self) -> float:
88+ return 100.0 - self.pct("idle") - self.pct("wait")
89+
90+
91+class AtopError(RuntimeError):
92+ pass
93+
94+
95+def log_path(date: str | None, logdir: str) -> str:
96+ stamp = date or datetime.now().strftime("%Y%m%d")
97+ path = os.path.join(logdir, f"atop_{stamp}")
98+ if not os.path.exists(path):
99+ have = sorted(f for f in os.listdir(logdir)) if os.path.isdir(logdir) else []
100+ raise AtopError(
101+ f"no atop log at {path}."
102+ + (f" Available: {', '.join(have)}" if have else " Is atop.service running?")
103+ )
104+ return path
105+
106+
107+def run_atop(path: str, begin: str | None, end: str | None) -> str:
108+ if not shutil.which("atop"):
109+ raise AtopError("atop is not installed (pacman -S atop)")
110+ cmd = ["atop", "-r", path, "-Z", "-P", "CPU,CPL,PRC"]
111+ if begin:
112+ cmd += ["-b", begin]
113+ if end:
114+ cmd += ["-e", end]
115+ try:
116+ proc = subprocess.run(cmd, capture_output=True, text=True, timeout=180)
117+ except subprocess.TimeoutExpired:
118+ raise AtopError("atop took over 180s to read the log; narrow the range with -b/-e")
119+ if proc.returncode != 0:
120+ raise AtopError(f"atop failed ({proc.returncode}): {proc.stderr.strip()}")
121+ return proc.stdout
122+
123+
124+def parse(raw: str) -> list[Sample]:
125+ """Turn `atop -P` output into samples, dropping the since-boot RESET sample.
126+
127+ atop reports per-interval deltas for every sample except the first one after
128+ a RESET marker, which holds totals since boot. Mixing the two would make the
129+ first interval dominate everything, so the RESET sample is skipped.
130+ """
131+ samples: list[Sample] = []
132+ current: Sample | None = None
133+ skip_this = False
134+
135+ for line in raw.splitlines():
136+ if not line:
137+ continue
138+ if line == "RESET":
139+ skip_this = True
140+ continue
141+ if line == "SEP":
142+ if current and not skip_this:
143+ samples.append(current)
144+ current, skip_this = None, False
145+ continue
146+
147+ parts = line.split()
148+ if len(parts) < 7:
149+ continue
150+ label, _host, epoch, date, tm = parts[:5]
151+ # With -b/-e, atop interleaves its interactive-format report into
152+ # stdout. Those lines can start with a real label (lowercase "cpu"),
153+ # so also require the date field to actually be a date.
154+ if label not in WANTED_LABELS or not DATE_RE.match(date):
155+ continue
156+ try:
157+ interval = int(parts[5])
158+ rest = parts[6:]
159+ except ValueError:
160+ continue
161+
162+ if current is None:
163+ current = Sample(epoch=int(epoch), when=tm, interval=interval)
164+
165+ try:
166+ if label == "CPU":
167+ current.hertz = int(rest[CPU_HERTZ])
168+ current.ncpu = int(rest[CPU_NCPU])
169+ current.modes = {n: int(rest[i]) for n, i in CPU_MODES.items()}
170+ elif label == "CPL":
171+ current.load1 = float(rest[CPL_LOAD1])
172+ current.load5 = float(rest[CPL_LOAD5])
173+ current.load15 = float(rest[CPL_LOAD15])
174+ current.ctxsw = int(rest[CPL_CTXSW])
175+ current.intr = int(rest[CPL_INTR])
176+ elif label == "PRC":
177+ if rest[PRC_ISPROC] != "y":
178+ continue # thread line, not a process
179+ hertz = int(rest[PRC_HERTZ])
180+ ticks = int(rest[PRC_UTIME]) + int(rest[PRC_STIME])
181+ if ticks:
182+ capacity = hertz * interval * max(current.ncpu or 1, 1)
183+ pct = 100.0 * ticks / capacity if capacity else 0.0
184+ current.procs.append((rest[PRC_NAME], int(rest[PRC_PID]), pct))
185+ except (IndexError, ValueError):
186+ continue # a label whose layout this atop version changed
187+
188+ if current and not skip_this:
189+ samples.append(current)
190+ for s in samples:
191+ s.procs.sort(key=lambda p: -p[2])
192+ return samples
193+
194+
195+def bucket(samples: list[Sample], max_rows: int) -> list[list[Sample]]:
196+ """Group samples into at most max_rows contiguous buckets."""
197+ if len(samples) <= max_rows:
198+ return [[s] for s in samples]
199+ size = -(-len(samples) // max_rows) # ceil
200+ return [samples[i : i + size] for i in range(0, len(samples), size)]
201+
202+
203+def digest(samples: list[Sample], top: int, max_rows: int) -> str:
204+ """Build a compact, token-bounded text summary of the run."""
205+ ncpu = next((s.ncpu for s in samples if s.ncpu), 0)
206+ span = f"{samples[0].when}-{samples[-1].when}"
207+ out: list[str] = [
208+ f"host cores: {ncpu}",
209+ f"window: {span} samples: {len(samples)} interval: {samples[0].interval}s",
210+ "",
211+ "TIMELINE (CPU% of whole machine; load1 = 1-min load average)",
212+ f"{'time':>9} {'busy':>6} {'user':>6} {'sys':>6} {'wait':>6} {'irq':>5} "
213+ f"{'stl':>5} {'load1':>6} top consumers",
214+ ]
215+
216+ for group in bucket(samples, max_rows):
217+ n = len(group)
218+ avg = lambda fn: sum(fn(s) for s in group) / n # noqa: E731
219+ label = group[0].when if n == 1 else f"{group[0].when[:5]}+{n}"
220+ merged: dict[str, float] = defaultdict(float)
221+ for s in group:
222+ for name, _pid, pct in s.procs:
223+ merged[name] += pct / n
224+ leaders = sorted(merged.items(), key=lambda kv: -kv[1])[:3]
225+ tops = " ".join(f"{name}:{pct:.0f}%" for name, pct in leaders if pct >= 1.0)
226+ out.append(
227+ f"{label:>9} {avg(lambda s: s.busy_pct):6.1f} "
228+ f"{avg(lambda s: s.pct('user') + s.pct('nice')):6.1f} "
229+ f"{avg(lambda s: s.pct('sys')):6.1f} {avg(lambda s: s.pct('wait')):6.1f} "
230+ f"{avg(lambda s: s.pct('irq') + s.pct('softirq')):5.1f} "
231+ f"{avg(lambda s: s.pct('steal')):5.1f} "
232+ f"{avg(lambda s: s.load1):6.2f} {tops}"
233+ )
234+
235+ # Overall leaders, weighted by interval so uneven samples don't skew it.
236+ total_secs: dict[str, float] = defaultdict(float)
237+ pids: dict[str, set[int]] = defaultdict(set)
238+ for s in samples:
239+ cores = max(s.ncpu or 1, 1)
240+ for name, pid, pct in s.procs:
241+ total_secs[name] += pct / 100.0 * cores * s.interval
242+ pids[name].add(pid)
243+ wall = sum(s.interval for s in samples) or 1
244+
245+ out += ["", f"TOP PROCESSES over the whole window ({wall}s wall clock)",
246+ f"{'process':<24} {'cpu-sec':>9} {'avg-cores':>10} {'pids':>5}"]
247+ for name, secs in sorted(total_secs.items(), key=lambda kv: -kv[1])[:top]:
248+ out.append(f"{name:<24} {secs:9.1f} {secs / wall:10.2f} {len(pids[name]):5d}")
249+
250+ peak = max(samples, key=lambda s: s.busy_pct)
251+ out += ["", f"PEAK SAMPLE {peak.when} busy {peak.busy_pct:.1f}% "
252+ f"wait {peak.pct('wait'):.1f}% load1 {peak.load1:.2f} "
253+ f"ctxsw {peak.ctxsw} intr {peak.intr}"]
254+ for name, pid, pct in peak.procs[:10]:
255+ out.append(f" {name:<24} pid {pid:<8} {pct:5.1f}% of machine")
256+ return "\n".join(out)
257+
258+
259+SYSTEM = """You are a Linux performance engineer reading atop history.
260+
261+The digest gives per-interval CPU state plus per-process CPU attribution.
262+Ground every claim in the numbers shown and cite the timestamps you used.
263+
264+Interpret carefully:
265+- Percentages are of the WHOLE machine, so one saturated core on an 8-core box
266+ is ~12.5%, not 100%.
267+- High 'wait' (iowait) means blocked on I/O, not CPU pressure -- do not call it
268+ a CPU bottleneck.
269+- Load average counts runnable AND uninterruptible-sleep tasks, so load can be
270+ high while CPU sits idle. Say which one the data supports.
271+- Sampled intervals average away short spikes. If the interval is coarse
272+ (>= 300s) say so rather than reading precise events into it.
273+- Per-process rows only cover processes alive at sample time; short-lived
274+ processes may be missing from attribution while still showing in system time.
275+
276+Structure the answer as: verdict (one or two sentences), then the evidence,
277+then what to check next as concrete commands. Say plainly when the data is
278+insufficient instead of speculating."""
279+
280+
281+def analyze(
282+ text: str, question: str | None, effort: str, model: str, claude_bin: str
283+) -> int:
284+ """Pipe the digest through `claude -p`, inheriting Claude Code's own login."""
285+ if not shutil.which(claude_bin):
286+ print(f"error: {claude_bin} not found on PATH. Install Claude Code, or "
287+ f"use --digest-only for the numbers without analysis.",
288+ file=sys.stderr)
289+ return 1
290+
291+ ask = question or (
292+ "Analyze this CPU history. What used the CPU, was the machine actually "
293+ "CPU-bound, and is anything here worth investigating?"
294+ )
295+ # The digest goes in on stdin rather than argv -- a long window can run to
296+ # tens of kilobytes, past the point where argv limits get interesting.
297+ prompt = f"{ask}\n\n<atop_digest>\n{text}\n</atop_digest>"
298+ cmd = [
299+ claude_bin, "-p",
300+ "--model", model,
301+ "--effort", effort,
302+ "--system-prompt", SYSTEM, # replaces the coding-agent prompt wholesale
303+ "--restricted", # this is pure text analysis; no tools needed
304+ ]
305+ try:
306+ # stdout/stderr are inherited, so the response streams as it arrives.
307+ proc = subprocess.run(cmd, input=prompt, text=True)
308+ except OSError as e:
309+ print(f"error: could not run {claude_bin}: {e}", file=sys.stderr)
310+ return 1
311+ except KeyboardInterrupt:
312+ return 130
313+ if proc.returncode != 0:
314+ print(f"\nerror: {claude_bin} exited {proc.returncode}. If this is an "
315+ f"auth problem, run `claude` once interactively to log in.",
316+ file=sys.stderr)
317+ return proc.returncode
318+
319+
320+def main() -> int:
321+ p = argparse.ArgumentParser(description="Analyze atop CPU history with Claude.")
322+ src = p.add_mutually_exclusive_group()
323+ src.add_argument("--date", help="log day as YYYYMMDD (default: today)")
324+ src.add_argument("--file", help="path to a raw atop log")
325+ p.add_argument("--logdir", default=DEFAULT_LOGDIR)
326+ p.add_argument("-b", "--begin", help="start time HH:MM")
327+ p.add_argument("-e", "--end", help="end time HH:MM")
328+ p.add_argument("-q", "--question", help="ask something specific")
329+ p.add_argument("--top", type=int, default=15, help="processes to rank (default 15)")
330+ p.add_argument("--max-rows", type=int, default=60,
331+ help="timeline rows; extra samples are averaged into buckets")
332+ p.add_argument("--effort", default="high",
333+ choices=["low", "medium", "high", "xhigh", "max"])
334+ p.add_argument("--model", default=CLAUDE_MODEL,
335+ help=f"model alias or full name (default: {CLAUDE_MODEL})")
336+ p.add_argument("--claude-bin", default=CLAUDE_BIN,
337+ help="path to the claude CLI")
338+ p.add_argument("--digest-only", action="store_true",
339+ help="print the digest and exit; runs no model")
340+ args = p.parse_args()
341+
342+ try:
343+ path = args.file or log_path(args.date, args.logdir)
344+ samples = parse(run_atop(path, args.begin, args.end))
345+ except AtopError as e:
346+ print(f"error: {e}", file=sys.stderr)
347+ return 1
348+ except PermissionError:
349+ print(f"error: cannot read {args.logdir} -- try run0", file=sys.stderr)
350+ return 1
351+
352+ if not samples:
353+ print("error: no usable samples in range. atop's first sample holds "
354+ "since-boot totals and is skipped, so a freshly started atop has "
355+ "nothing to compare yet -- wait for one more interval.",
356+ file=sys.stderr)
357+ return 1
358+
359+ text = digest(samples, args.top, args.max_rows)
360+ if args.digest_only:
361+ print(text)
362+ return 0
363+ print(f"# {len(samples)} samples from {path}\n", file=sys.stderr)
364+ return analyze(text, args.question, args.effort, args.model, args.claude_bin)
365+
366+
367+if __name__ == "__main__":
368+ sys.exit(main())