add reporting foundation: multipage nav, auth gate, date-range fetch
First increment of the clinical reporting feature (see docs/requirements-reporting.md and docs/design-reporting.md): moves the existing live view to pages/1_Live.py, adds a password gate since the dashboard URL will be shared with the diabetes team, adds fetch_entries_range/fetch_treatments_range for multi-day report queries, and adds reporting.py with the core clinical calculations (GMI, %CV, 5-band time-in-range, hypo/hyper event detection per the Battelino et al. 2019 persistence rule). No report pages wired up yet — that's next. Co-Authored-By: Claude Sonnet 5 <[email protected]>
This commit is contained in:
Claude Sonnet 5
@@ -0,0 +1,178 @@
|
||||
# Design: Clinical reporting page
|
||||
|
||||
Status: **implementation in progress**
|
||||
Implements: [requirements-reporting.md](requirements-reporting.md) (see that doc for the final, user-edited FR list)
|
||||
|
||||
## 1. Architecture overview
|
||||
|
||||
Streamlit's classic `pages/` directory convention (numbered filenames, auto-populated
|
||||
sidebar nav) rather than the newer `st.navigation`/`st.Page` object API — same
|
||||
end result (multipage nav), simpler to reason about since each page is an
|
||||
independent script sharing only imported modules, and it's what the project's
|
||||
own `homelab-cicd-werkinstructie.md` anticipated.
|
||||
|
||||
```
|
||||
app.py # home/landing page — auth gate + short intro
|
||||
pages/
|
||||
1_Live.py # existing rolling 6-48h view, moved as-is from current app.py
|
||||
2_Samenvatting.py # FR2 (summary stats) + FR9 (data completeness)
|
||||
3_AGP_Profiel.py # FR3 (percentile chart)
|
||||
4_Dagoverzicht.py # FR4 (day-by-day graphs)
|
||||
5_Hypo_Events.py # FR5
|
||||
6_Hyper_Events.py # FR6
|
||||
7_Trends.py # FR7 (GMI trend) + FR8 (other stats)
|
||||
8_Export.py # FR10 (PDF export)
|
||||
auth.py # password gate, imported by app.py and every page
|
||||
reporting.py # new pure-logic module (mirrors data_processing.py's role)
|
||||
nightscout_client.py # + fetch_entries_range / fetch_treatments_range
|
||||
```
|
||||
|
||||
Each report page has its **own** period control per the user's edit to FR1
|
||||
(90-day default for statistics/TIR pages, 14-day default for the daily-log page),
|
||||
rather than one global selector — consistent with "each page separately."
|
||||
|
||||
## 2. Auth (NFR6 — new requirement from the user's edit)
|
||||
|
||||
Streamlit has no built-in auth. Given this is a single-user-plus-a-few-clinicians
|
||||
homelab app (not a multi-tenant product), a lightweight session-based password
|
||||
gate is proportionate — consistent with how `nightscout_client._auth()` already
|
||||
hashes the API secret rather than storing/sending it raw:
|
||||
|
||||
```python
|
||||
# auth.py
|
||||
import hashlib, os
|
||||
import streamlit as st
|
||||
|
||||
def require_auth() -> None:
|
||||
if st.session_state.get("authenticated"):
|
||||
return
|
||||
st.title("Glucose dashboard")
|
||||
password = st.text_input("Wachtwoord", type="password")
|
||||
if not password:
|
||||
st.stop()
|
||||
expected = os.environ.get("DASHBOARD_PASSWORD_HASH", "")
|
||||
if hashlib.sha256(password.encode()).hexdigest() == expected:
|
||||
st.session_state["authenticated"] = True
|
||||
st.rerun()
|
||||
st.error("Onjuist wachtwoord")
|
||||
st.stop()
|
||||
```
|
||||
|
||||
Called as the first line of `app.py` and every `pages/*.py` file (Streamlit runs
|
||||
each page as an independent script, so the guard must repeat per-file — no
|
||||
shared middleware). New env var `DASHBOARD_PASSWORD_HASH` (sha256 hex digest,
|
||||
not the plaintext password) added to `.env.example`/README with instructions to
|
||||
generate it (`python -c "import hashlib; print(hashlib.sha256(b'yourpassword').hexdigest())"`).
|
||||
Session-based means each browser session re-enters the password once; this is a
|
||||
basic gate against casual URL sharing, not a hardened auth system — acceptable
|
||||
for this context but worth knowing if the diabetes team needs individual
|
||||
accounts later.
|
||||
|
||||
## 3. `nightscout_client.py` additions
|
||||
|
||||
`fetch_entries`/`fetch_treatments` scale `count` linearly with `hours` — fine for
|
||||
the existing rolling 6-48h view, not for a 90-day report (~26,000 5-min
|
||||
readings). New functions follow the exact same "overfetch by count, then filter
|
||||
locally by timestamp" pattern already used (rather than introducing Mongo
|
||||
`find[]` date-query params, which the existing code's own comments flag as
|
||||
inconsistent across Nightscout versions):
|
||||
|
||||
```python
|
||||
def fetch_entries_range(start_date: date, end_date: date) -> pd.DataFrame:
|
||||
"""Haal glucosemetingen op tussen start_date en end_date (inclusief)."""
|
||||
days = (end_date - start_date).days + 1
|
||||
count = min(days * 300, 100_000) # ruime marge boven 5-min-interval
|
||||
... # zelfde _get/parse-logica als fetch_entries, lokaal gefilterd op [start_date, end_date]
|
||||
|
||||
def fetch_treatments_range(start_date: date, end_date: date) -> pd.DataFrame:
|
||||
... # analoog, count = min(days * 50, 50_000)
|
||||
```
|
||||
|
||||
## 4. `reporting.py` (new module)
|
||||
|
||||
Pure functions only (no Streamlit imports), mirroring `data_processing.py`'s
|
||||
existing style — testable with the same fixture-based pytest pattern.
|
||||
|
||||
- `gmi_percent(mean_mmol: float) -> float` — `3.31 + 0.4311 * mean_mmol`
|
||||
- `percent_cv(glucose: pd.DataFrame) -> float` — `stdev/mean * 100`
|
||||
- `tir_breakdown_5band(glucose) -> dict` — extends `data_processing.summarize_time_in_range`
|
||||
(which only has 3 bands) to the 5 clinical bands (<3.0, 3.0–3.9, 3.9–10.0,
|
||||
10.0–13.9, >13.9), each as % and as minutes/day
|
||||
- `detect_events(glucose, threshold, direction, min_duration_minutes=15) -> pd.DataFrame`
|
||||
— generic engine for FR5/FR6 (direction="below" for hypo, "above" for hyper);
|
||||
finds runs of consecutive out-of-threshold readings ≥15 min, merging across
|
||||
brief in-range blips per the persistence rule (a return above/below threshold
|
||||
only "counts" once it has itself persisted ≥15 min)
|
||||
- `summarize_events(events: pd.DataFrame, period_days: int) -> dict` — count, avg/longest
|
||||
duration, total time, events/week
|
||||
- `overnight_daytime_split(glucose, overnight_start=0, overnight_end=6) -> dict`
|
||||
— TIR breakdown for each segment
|
||||
- `day_of_week_pattern(glucose) -> pd.DataFrame` — per-weekday summary stats
|
||||
- `bolus_stats(treatments) -> dict` — meal vs. correction bolus split, using the
|
||||
existing `INSULIN_EVENT_TYPES` set from `nightscout_client.py` (currently
|
||||
defined but unused — this is its first real consumer)
|
||||
- `site_change_glucose_impact(glucose, site_changes, window_hours=6) -> pd.DataFrame`
|
||||
— TIR in the N hours after each site change vs. the period baseline
|
||||
|
||||
**Scoping caveat surfaced during design** (flagging rather than silently
|
||||
under-delivering): a true basal:bolus insulin ratio and total-daily-dose trend
|
||||
need the *scheduled* basal rate (from the Nightscout profile) reconciled against
|
||||
any temp-basal overrides — neither the profile nor "Temp Basal" treatments are
|
||||
fetched anywhere in this codebase today, and profile access is explicitly the
|
||||
other feature's territory ([requirements-profile-tuning.md](requirements-profile-tuning.md)).
|
||||
Implementing this now would mean quietly duplicating profile-fetch logic between
|
||||
two features that are supposed to stay separate. First pass ships **bolus-only**
|
||||
stats (which are fully derivable from `treatments` alone) with an explicit label
|
||||
that basal totals aren't included; true basal:bolus/TDD becomes a fast-follow
|
||||
once the profile-tuning feature's Nightscout profile client exists and can be
|
||||
reused read-only from here.
|
||||
|
||||
## 5. PDF export (FR10)
|
||||
|
||||
`kaleido` (Plotly's static-image renderer) bundles its own Chromium build —
|
||||
meaningfully bigger image and more Trivy surface for not much benefit here.
|
||||
Instead: `matplotlib` (already a transitive pandas dependency's peer, tiny
|
||||
incremental footprint) renders simplified static charts specifically for the
|
||||
PDF, and `fpdf2` (pure-Python, no system deps) assembles the PDF — same
|
||||
approach, smaller/simpler Docker image, consistent with how carefully the
|
||||
Trivy/registry-hostname issues were chased down in this project's CI history.
|
||||
`pages/8_Export.py` lets the user check which sections to include, renders the
|
||||
selected stats/charts via a new `pdf_export.py` (`build_report_pdf(sections: list[str], ...) -> bytes`),
|
||||
and offers it via `st.download_button`.
|
||||
|
||||
## 6. Testing strategy
|
||||
|
||||
- `reporting.py` functions: unit tests in `tests/test_reporting.py`, same
|
||||
fixture style as `tests/test_data_processing.py`, explicitly covering the
|
||||
edge cases named in the requirements doc (14:59 vs 15:00 min events, two dips
|
||||
separated by a brief recovery, empty input, 100%-in-range).
|
||||
- `nightscout_client.py` additions: tests in the existing
|
||||
`tests/test_nightscout_client.py`, same `monkeypatch`-on-`requests` pattern.
|
||||
- `auth.py`: the hashing/compare logic is trivial enough to test directly
|
||||
(`hashlib.sha256(...).hexdigest() == expected`) without needing to drive
|
||||
Streamlit's UI.
|
||||
- Page files (`pages/*.py`) are UI wiring only (like the existing `app.py`) —
|
||||
consistent with this project's existing convention of not unit-testing the
|
||||
Streamlit UI layer itself, only the logic it calls into.
|
||||
- `pdf_export.py`: unit test that a given set of sections produces a non-empty,
|
||||
valid PDF (`fpdf2`'s output starts with `%PDF`), without asserting on exact
|
||||
rendered pixels.
|
||||
|
||||
## 7. Implementation order
|
||||
|
||||
1. Foundation: `auth.py`, multipage restructure (move current `app.py` →
|
||||
`pages/1_Live.py`, new minimal `app.py`), `nightscout_client.py` range-fetch
|
||||
additions — all independently testable, no visible feature yet.
|
||||
2. `reporting.py` core: GMI, %CV, 5-band TIR, event detection + summaries —
|
||||
the clinically load-bearing pieces, fully unit tested.
|
||||
3. Wire pages 2–3–4–5–6 (summary, AGP, daily, hypo/hyper events) to `reporting.py`.
|
||||
4. `reporting.py` extras: overnight/daytime split, day-of-week, bolus stats,
|
||||
site-change impact; wire page 7 (trends).
|
||||
5. PDF export (page 8) — last, since it's the most novel dependency and lowest
|
||||
clinical risk if delayed a step.
|
||||
|
||||
Each step lands as its own commit(s) behind `ruff check .` + `pytest tests/ -v`,
|
||||
pushed through the existing `dev` → PR → `main` flow so the CI/CD pipeline
|
||||
(test → SonarQube → build+Trivy+push → deploy) validates every increment, per
|
||||
the user's original request to implement requirement-by-requirement with as
|
||||
much automated testing as possible.
|
||||
Reference in New Issue
Block a user