glucose-dashboard/docs/design-reporting.md

# Design: Clinical reporting page

Status: **implementation in progress**
Implements: [requirements-reporting.md](requirements-reporting.md) (see that doc for the final, user-edited FR list)

## 1. Architecture overview

Streamlit's classic `pages/` directory convention (numbered filenames, auto-populated
sidebar nav) rather than the newer `st.navigation`/`st.Page` object API — same
end result (multipage nav), simpler to reason about since each page is an
independent script sharing only imported modules, and it's what the project's
own `homelab-cicd-werkinstructie.md` anticipated.

```
app.py                          # home/landing page — auth gate + short intro
pages/
  1_Live.py                     # existing rolling 6-48h view, moved as-is from current app.py
  2_Samenvatting.py             # FR2 (summary stats) + FR9 (data completeness)
  3_AGP_Profiel.py              # FR3 (percentile chart)
  4_Dagoverzicht.py             # FR4 (day-by-day graphs)
  5_Hypo_Events.py              # FR5
  6_Hyper_Events.py             # FR6
  7_Trends.py                   # FR7 (GMI trend) + FR8 (other stats)
  8_Export.py                   # FR10 (PDF export)
auth.py                         # password gate, imported by app.py and every page
reporting.py                    # new pure-logic module (mirrors data_processing.py's role)
nightscout_client.py            # + fetch_entries_range / fetch_treatments_range
```

Each report page has its **own** period control per the user's edit to FR1
(90-day default for statistics/TIR pages, 14-day default for the daily-log page),
rather than one global selector — consistent with "each page separately."

## 2. Auth (NFR6 — new requirement from the user's edit)

Streamlit has no built-in auth. Given this is a single-user-plus-a-few-clinicians
homelab app (not a multi-tenant product), a lightweight session-based password
gate is proportionate — consistent with how `nightscout_client._auth()` already
hashes the API secret rather than storing/sending it raw:

```python
# auth.py
import hashlib, os
import streamlit as st

def require_auth() -> None:
    if st.session_state.get("authenticated"):
        return
    st.title("Glucose dashboard")
    password = st.text_input("Wachtwoord", type="password")
    if not password:
        st.stop()
    expected = os.environ.get("DASHBOARD_PASSWORD_HASH", "")
    if hashlib.sha256(password.encode()).hexdigest() == expected:
        st.session_state["authenticated"] = True
        st.rerun()
    st.error("Onjuist wachtwoord")
    st.stop()
```

Called as the first line of `app.py` and every `pages/*.py` file (Streamlit runs
each page as an independent script, so the guard must repeat per-file — no
shared middleware). New env var `DASHBOARD_PASSWORD_HASH` (sha256 hex digest,
not the plaintext password) added to `.env.example`/README with instructions to
generate it (`python -c "import hashlib; print(hashlib.sha256(b'yourpassword').hexdigest())"`).
Session-based means each browser session re-enters the password once; this is a
basic gate against casual URL sharing, not a hardened auth system — acceptable
for this context but worth knowing if the diabetes team needs individual
accounts later.

## 3. `nightscout_client.py` additions

`fetch_entries`/`fetch_treatments` scale `count` linearly with `hours` — fine for
the existing rolling 6-48h view, not for a 90-day report (~26,000 5-min
readings). New functions follow the exact same "overfetch by count, then filter
locally by timestamp" pattern already used (rather than introducing Mongo
`find[]` date-query params, which the existing code's own comments flag as
inconsistent across Nightscout versions):

```python
def fetch_entries_range(start_date: date, end_date: date) -> pd.DataFrame:
    """Haal glucosemetingen op tussen start_date en end_date (inclusief)."""
    days = (end_date - start_date).days + 1
    count = min(days * 300, 100_000)  # ruime marge boven 5-min-interval
    ...  # zelfde _get/parse-logica als fetch_entries, lokaal gefilterd op [start_date, end_date]

def fetch_treatments_range(start_date: date, end_date: date) -> pd.DataFrame:
    ...  # analoog, count = min(days * 50, 50_000)
```

## 4. `reporting.py` (new module)

Pure functions only (no Streamlit imports), mirroring `data_processing.py`'s
existing style — testable with the same fixture-based pytest pattern.

- `gmi_percent(mean_mmol: float) -> float` — `3.31 + 0.4311 * mean_mmol`
- `percent_cv(glucose: pd.DataFrame) -> float` — `stdev/mean * 100`
- `tir_breakdown_5band(glucose) -> dict` — extends `data_processing.summarize_time_in_range`
  (which only has 3 bands) to the 5 clinical bands (<3.0, 3.0–3.9, 3.9–10.0,
  10.0–13.9, >13.9), each as % and as minutes/day
- `detect_events(glucose, threshold, direction, min_duration_minutes=15) -> pd.DataFrame`
  — generic engine for FR5/FR6 (direction="below" for hypo, "above" for hyper);
  finds runs of consecutive out-of-threshold readings ≥15 min, merging across
  brief in-range blips per the persistence rule (a return above/below threshold
  only "counts" once it has itself persisted ≥15 min)
- `summarize_events(events: pd.DataFrame, period_days: int) -> dict` — count, avg/longest
  duration, total time, events/week
- `overnight_daytime_split(glucose, overnight_start=0, overnight_end=6) -> dict`
  — TIR breakdown for each segment
- `day_of_week_pattern(glucose) -> pd.DataFrame` — per-weekday summary stats
- `bolus_stats(treatments) -> dict` — meal vs. correction bolus split, using the
  existing `INSULIN_EVENT_TYPES` set from `nightscout_client.py` (currently
  defined but unused — this is its first real consumer)
- `site_change_glucose_impact(glucose, site_changes, window_hours=6) -> pd.DataFrame`
  — TIR in the N hours after each site change vs. the period baseline

**Scoping caveat surfaced during design** (flagging rather than silently
under-delivering): a true basal:bolus insulin ratio and total-daily-dose trend
need the *scheduled* basal rate (from the Nightscout profile) reconciled against
any temp-basal overrides — neither the profile nor "Temp Basal" treatments are
fetched anywhere in this codebase today, and profile access is explicitly the
other feature's territory ([requirements-profile-tuning.md](requirements-profile-tuning.md)).
Implementing this now would mean quietly duplicating profile-fetch logic between
two features that are supposed to stay separate. First pass ships **bolus-only**
stats (which are fully derivable from `treatments` alone) with an explicit label
that basal totals aren't included; true basal:bolus/TDD becomes a fast-follow
once the profile-tuning feature's Nightscout profile client exists and can be
reused read-only from here.

## 5. PDF export (FR10)

`kaleido` (Plotly's static-image renderer) bundles its own Chromium build —
meaningfully bigger image and more Trivy surface for not much benefit here.
Instead: `matplotlib` (already a transitive pandas dependency's peer, tiny
incremental footprint) renders simplified static charts specifically for the
PDF, and `fpdf2` (pure-Python, no system deps) assembles the PDF — same
approach, smaller/simpler Docker image, consistent with how carefully the
Trivy/registry-hostname issues were chased down in this project's CI history.
`pages/8_Export.py` lets the user check which sections to include, renders the
selected stats/charts via a new `pdf_export.py` (`build_report_pdf(sections: list[str], ...) -> bytes`),
and offers it via `st.download_button`.

## 6. Testing strategy

- `reporting.py` functions: unit tests in `tests/test_reporting.py`, same
  fixture style as `tests/test_data_processing.py`, explicitly covering the
  edge cases named in the requirements doc (14:59 vs 15:00 min events, two dips
  separated by a brief recovery, empty input, 100%-in-range).
- `nightscout_client.py` additions: tests in the existing
  `tests/test_nightscout_client.py`, same `monkeypatch`-on-`requests` pattern.
- `auth.py`: the hashing/compare logic is trivial enough to test directly
  (`hashlib.sha256(...).hexdigest() == expected`) without needing to drive
  Streamlit's UI.
- Page files (`pages/*.py`) are UI wiring only (like the existing `app.py`) —
  consistent with this project's existing convention of not unit-testing the
  Streamlit UI layer itself, only the logic it calls into.
- `pdf_export.py`: unit test that a given set of sections produces a non-empty,
  valid PDF (`fpdf2`'s output starts with `%PDF`), without asserting on exact
  rendered pixels.

## 7. Implementation order

1. Foundation: `auth.py`, multipage restructure (move current `app.py` →
   `pages/1_Live.py`, new minimal `app.py`), `nightscout_client.py` range-fetch
   additions — all independently testable, no visible feature yet.
2. `reporting.py` core: GMI, %CV, 5-band TIR, event detection + summaries —
   the clinically load-bearing pieces, fully unit tested.
3. Wire pages 2–3–4–5–6 (summary, AGP, daily, hypo/hyper events) to `reporting.py`.
4. `reporting.py` extras: overnight/daytime split, day-of-week, bolus stats,
   site-change impact; wire page 7 (trends).
5. PDF export (page 8) — last, since it's the most novel dependency and lowest
   clinical risk if delayed a step.

Each step lands as its own commit(s) behind `ruff check .` + `pytest tests/ -v`,
pushed through the existing `dev` → PR → `main` flow so the CI/CD pipeline
(test → SonarQube → build+Trivy+push → deploy) validates every increment, per
the user's original request to implement requirement-by-requirement with as
much automated testing as possible.