add reporting foundation: multipage nav, auth gate, date-range fetch
First increment of the clinical reporting feature (see docs/requirements-reporting.md and docs/design-reporting.md): moves the existing live view to pages/1_Live.py, adds a password gate since the dashboard URL will be shared with the diabetes team, adds fetch_entries_range/fetch_treatments_range for multi-day report queries, and adds reporting.py with the core clinical calculations (GMI, %CV, 5-band time-in-range, hypo/hyper event detection per the Battelino et al. 2019 persistence rule). No report pages wired up yet — that's next. Co-Authored-By: Claude Sonnet 5 <[email protected]>
This commit is contained in:
Claude Sonnet 5
@@ -0,0 +1,178 @@
|
||||
# Design: Clinical reporting page
|
||||
|
||||
Status: **implementation in progress**
|
||||
Implements: [requirements-reporting.md](requirements-reporting.md) (see that doc for the final, user-edited FR list)
|
||||
|
||||
## 1. Architecture overview
|
||||
|
||||
Streamlit's classic `pages/` directory convention (numbered filenames, auto-populated
|
||||
sidebar nav) rather than the newer `st.navigation`/`st.Page` object API — same
|
||||
end result (multipage nav), simpler to reason about since each page is an
|
||||
independent script sharing only imported modules, and it's what the project's
|
||||
own `homelab-cicd-werkinstructie.md` anticipated.
|
||||
|
||||
```
|
||||
app.py # home/landing page — auth gate + short intro
|
||||
pages/
|
||||
1_Live.py # existing rolling 6-48h view, moved as-is from current app.py
|
||||
2_Samenvatting.py # FR2 (summary stats) + FR9 (data completeness)
|
||||
3_AGP_Profiel.py # FR3 (percentile chart)
|
||||
4_Dagoverzicht.py # FR4 (day-by-day graphs)
|
||||
5_Hypo_Events.py # FR5
|
||||
6_Hyper_Events.py # FR6
|
||||
7_Trends.py # FR7 (GMI trend) + FR8 (other stats)
|
||||
8_Export.py # FR10 (PDF export)
|
||||
auth.py # password gate, imported by app.py and every page
|
||||
reporting.py # new pure-logic module (mirrors data_processing.py's role)
|
||||
nightscout_client.py # + fetch_entries_range / fetch_treatments_range
|
||||
```
|
||||
|
||||
Each report page has its **own** period control per the user's edit to FR1
|
||||
(90-day default for statistics/TIR pages, 14-day default for the daily-log page),
|
||||
rather than one global selector — consistent with "each page separately."
|
||||
|
||||
## 2. Auth (NFR6 — new requirement from the user's edit)
|
||||
|
||||
Streamlit has no built-in auth. Given this is a single-user-plus-a-few-clinicians
|
||||
homelab app (not a multi-tenant product), a lightweight session-based password
|
||||
gate is proportionate — consistent with how `nightscout_client._auth()` already
|
||||
hashes the API secret rather than storing/sending it raw:
|
||||
|
||||
```python
|
||||
# auth.py
|
||||
import hashlib, os
|
||||
import streamlit as st
|
||||
|
||||
def require_auth() -> None:
|
||||
if st.session_state.get("authenticated"):
|
||||
return
|
||||
st.title("Glucose dashboard")
|
||||
password = st.text_input("Wachtwoord", type="password")
|
||||
if not password:
|
||||
st.stop()
|
||||
expected = os.environ.get("DASHBOARD_PASSWORD_HASH", "")
|
||||
if hashlib.sha256(password.encode()).hexdigest() == expected:
|
||||
st.session_state["authenticated"] = True
|
||||
st.rerun()
|
||||
st.error("Onjuist wachtwoord")
|
||||
st.stop()
|
||||
```
|
||||
|
||||
Called as the first line of `app.py` and every `pages/*.py` file (Streamlit runs
|
||||
each page as an independent script, so the guard must repeat per-file — no
|
||||
shared middleware). New env var `DASHBOARD_PASSWORD_HASH` (sha256 hex digest,
|
||||
not the plaintext password) added to `.env.example`/README with instructions to
|
||||
generate it (`python -c "import hashlib; print(hashlib.sha256(b'yourpassword').hexdigest())"`).
|
||||
Session-based means each browser session re-enters the password once; this is a
|
||||
basic gate against casual URL sharing, not a hardened auth system — acceptable
|
||||
for this context but worth knowing if the diabetes team needs individual
|
||||
accounts later.
|
||||
|
||||
## 3. `nightscout_client.py` additions
|
||||
|
||||
`fetch_entries`/`fetch_treatments` scale `count` linearly with `hours` — fine for
|
||||
the existing rolling 6-48h view, not for a 90-day report (~26,000 5-min
|
||||
readings). New functions follow the exact same "overfetch by count, then filter
|
||||
locally by timestamp" pattern already used (rather than introducing Mongo
|
||||
`find[]` date-query params, which the existing code's own comments flag as
|
||||
inconsistent across Nightscout versions):
|
||||
|
||||
```python
|
||||
def fetch_entries_range(start_date: date, end_date: date) -> pd.DataFrame:
|
||||
"""Haal glucosemetingen op tussen start_date en end_date (inclusief)."""
|
||||
days = (end_date - start_date).days + 1
|
||||
count = min(days * 300, 100_000) # ruime marge boven 5-min-interval
|
||||
... # zelfde _get/parse-logica als fetch_entries, lokaal gefilterd op [start_date, end_date]
|
||||
|
||||
def fetch_treatments_range(start_date: date, end_date: date) -> pd.DataFrame:
|
||||
... # analoog, count = min(days * 50, 50_000)
|
||||
```
|
||||
|
||||
## 4. `reporting.py` (new module)
|
||||
|
||||
Pure functions only (no Streamlit imports), mirroring `data_processing.py`'s
|
||||
existing style — testable with the same fixture-based pytest pattern.
|
||||
|
||||
- `gmi_percent(mean_mmol: float) -> float` — `3.31 + 0.4311 * mean_mmol`
|
||||
- `percent_cv(glucose: pd.DataFrame) -> float` — `stdev/mean * 100`
|
||||
- `tir_breakdown_5band(glucose) -> dict` — extends `data_processing.summarize_time_in_range`
|
||||
(which only has 3 bands) to the 5 clinical bands (<3.0, 3.0–3.9, 3.9–10.0,
|
||||
10.0–13.9, >13.9), each as % and as minutes/day
|
||||
- `detect_events(glucose, threshold, direction, min_duration_minutes=15) -> pd.DataFrame`
|
||||
— generic engine for FR5/FR6 (direction="below" for hypo, "above" for hyper);
|
||||
finds runs of consecutive out-of-threshold readings ≥15 min, merging across
|
||||
brief in-range blips per the persistence rule (a return above/below threshold
|
||||
only "counts" once it has itself persisted ≥15 min)
|
||||
- `summarize_events(events: pd.DataFrame, period_days: int) -> dict` — count, avg/longest
|
||||
duration, total time, events/week
|
||||
- `overnight_daytime_split(glucose, overnight_start=0, overnight_end=6) -> dict`
|
||||
— TIR breakdown for each segment
|
||||
- `day_of_week_pattern(glucose) -> pd.DataFrame` — per-weekday summary stats
|
||||
- `bolus_stats(treatments) -> dict` — meal vs. correction bolus split, using the
|
||||
existing `INSULIN_EVENT_TYPES` set from `nightscout_client.py` (currently
|
||||
defined but unused — this is its first real consumer)
|
||||
- `site_change_glucose_impact(glucose, site_changes, window_hours=6) -> pd.DataFrame`
|
||||
— TIR in the N hours after each site change vs. the period baseline
|
||||
|
||||
**Scoping caveat surfaced during design** (flagging rather than silently
|
||||
under-delivering): a true basal:bolus insulin ratio and total-daily-dose trend
|
||||
need the *scheduled* basal rate (from the Nightscout profile) reconciled against
|
||||
any temp-basal overrides — neither the profile nor "Temp Basal" treatments are
|
||||
fetched anywhere in this codebase today, and profile access is explicitly the
|
||||
other feature's territory ([requirements-profile-tuning.md](requirements-profile-tuning.md)).
|
||||
Implementing this now would mean quietly duplicating profile-fetch logic between
|
||||
two features that are supposed to stay separate. First pass ships **bolus-only**
|
||||
stats (which are fully derivable from `treatments` alone) with an explicit label
|
||||
that basal totals aren't included; true basal:bolus/TDD becomes a fast-follow
|
||||
once the profile-tuning feature's Nightscout profile client exists and can be
|
||||
reused read-only from here.
|
||||
|
||||
## 5. PDF export (FR10)
|
||||
|
||||
`kaleido` (Plotly's static-image renderer) bundles its own Chromium build —
|
||||
meaningfully bigger image and more Trivy surface for not much benefit here.
|
||||
Instead: `matplotlib` (already a transitive pandas dependency's peer, tiny
|
||||
incremental footprint) renders simplified static charts specifically for the
|
||||
PDF, and `fpdf2` (pure-Python, no system deps) assembles the PDF — same
|
||||
approach, smaller/simpler Docker image, consistent with how carefully the
|
||||
Trivy/registry-hostname issues were chased down in this project's CI history.
|
||||
`pages/8_Export.py` lets the user check which sections to include, renders the
|
||||
selected stats/charts via a new `pdf_export.py` (`build_report_pdf(sections: list[str], ...) -> bytes`),
|
||||
and offers it via `st.download_button`.
|
||||
|
||||
## 6. Testing strategy
|
||||
|
||||
- `reporting.py` functions: unit tests in `tests/test_reporting.py`, same
|
||||
fixture style as `tests/test_data_processing.py`, explicitly covering the
|
||||
edge cases named in the requirements doc (14:59 vs 15:00 min events, two dips
|
||||
separated by a brief recovery, empty input, 100%-in-range).
|
||||
- `nightscout_client.py` additions: tests in the existing
|
||||
`tests/test_nightscout_client.py`, same `monkeypatch`-on-`requests` pattern.
|
||||
- `auth.py`: the hashing/compare logic is trivial enough to test directly
|
||||
(`hashlib.sha256(...).hexdigest() == expected`) without needing to drive
|
||||
Streamlit's UI.
|
||||
- Page files (`pages/*.py`) are UI wiring only (like the existing `app.py`) —
|
||||
consistent with this project's existing convention of not unit-testing the
|
||||
Streamlit UI layer itself, only the logic it calls into.
|
||||
- `pdf_export.py`: unit test that a given set of sections produces a non-empty,
|
||||
valid PDF (`fpdf2`'s output starts with `%PDF`), without asserting on exact
|
||||
rendered pixels.
|
||||
|
||||
## 7. Implementation order
|
||||
|
||||
1. Foundation: `auth.py`, multipage restructure (move current `app.py` →
|
||||
`pages/1_Live.py`, new minimal `app.py`), `nightscout_client.py` range-fetch
|
||||
additions — all independently testable, no visible feature yet.
|
||||
2. `reporting.py` core: GMI, %CV, 5-band TIR, event detection + summaries —
|
||||
the clinically load-bearing pieces, fully unit tested.
|
||||
3. Wire pages 2–3–4–5–6 (summary, AGP, daily, hypo/hyper events) to `reporting.py`.
|
||||
4. `reporting.py` extras: overnight/daytime split, day-of-week, bolus stats,
|
||||
site-change impact; wire page 7 (trends).
|
||||
5. PDF export (page 8) — last, since it's the most novel dependency and lowest
|
||||
clinical risk if delayed a step.
|
||||
|
||||
Each step lands as its own commit(s) behind `ruff check .` + `pytest tests/ -v`,
|
||||
pushed through the existing `dev` → PR → `main` flow so the CI/CD pipeline
|
||||
(test → SonarQube → build+Trivy+push → deploy) validates every increment, per
|
||||
the user's original request to implement requirement-by-requirement with as
|
||||
much automated testing as possible.
|
||||
@@ -0,0 +1,202 @@
|
||||
# Requirements: oref0-autotune profile tuning
|
||||
|
||||
Status: **draft — for review and editing**
|
||||
Owner: willemjan
|
||||
Related: [requirements-reporting.md](requirements-reporting.md) (separate feature, same Nightscout data source)
|
||||
|
||||
## 1. Purpose & background
|
||||
|
||||
AndroidAPS (AAPS) only gained built-in Autotune in version 3.4, and even that
|
||||
version tunes only a single ISF/carb-ratio value plus the basal schedule from
|
||||
on-device history — it doesn't use Nightscout at all, and the user is still on
|
||||
AAPS 3.3 regardless. Previously, the user ran a hosted tool (inspired by
|
||||
`MarkMpn/AutotuneWeb`, now archived/discontinued) that analyzed Nightscout data
|
||||
with `oref0-autotune` and proposed a new pump profile, which was then manually
|
||||
entered into AAPS — a process the user describes as "a lot of trial and error."
|
||||
|
||||
This feature builds a guided, human-in-the-loop workflow: trigger an
|
||||
oref0-autotune analysis from a dashboard, review the proposed profile against
|
||||
the current one with the underlying estimations visible, selectively
|
||||
accept/adjust values, and stage the result in Nightscout for manual activation.
|
||||
|
||||
The user has two LXC containers on Proxmox from earlier experimentation —
|
||||
`autotune` (VMID 102, 192.168.2.13) and `oref0` (VMID 107, 192.168.2.14) —
|
||||
currently stopped, of unknown/half-remembered working state.
|
||||
|
||||
## 2. Scope
|
||||
|
||||
**In scope:**
|
||||
- Booting and inspecting the two existing LXCs to determine (or establish) a
|
||||
working oref0-autotune setup.
|
||||
- A small HTTP microservice on that LXC that runs oref0-autotune against the
|
||||
user's Nightscout data on demand.
|
||||
- A **separate** Streamlit app/service (own deployment, own CI/CD pipeline) that
|
||||
triggers the analysis, displays old-vs-new profile comparisons with
|
||||
estimation detail, lets the user selectively adjust proposed values, and
|
||||
stages the result in Nightscout.
|
||||
|
||||
**Out of scope (this iteration):**
|
||||
- Automatically activating the new profile in AndroidAPS (creating a
|
||||
`"Profile Switch"` treatment) — the dashboard stages the profile only; the
|
||||
user activates it manually. This may become an opt-in "phase 2" once trust is
|
||||
established (see §8).
|
||||
- Any change to AAPS settings or on-device configuration.
|
||||
- Continuous/scheduled autotune runs — this is a manually triggered, on-demand
|
||||
analysis, not a nightly automated loop.
|
||||
|
||||
## 3. Architecture decisions (made during spec review)
|
||||
|
||||
1. **Execution**: an always-on HTTP microservice on whichever LXC has (or gets)
|
||||
a working oref0-autotune setup, rather than SSH-triggering a one-off Docker
|
||||
run or bundling autotune into the dashboard's own container. The dashboard
|
||||
calls it over HTTP and polls for job completion.
|
||||
2. **Profile push safety**: **stage-only**. The service writes a new named
|
||||
profile into the Nightscout profile store but does **not** create a
|
||||
`"Profile Switch"` treatment — final activation stays a deliberate manual
|
||||
step in AAPS or the Nightscout UI. This matters because AndroidAPS applies
|
||||
profiles via Profile Switch treatments (gated behind an explicit "accept
|
||||
profile switch from Nightscout" AAPS setting), not by simply reading
|
||||
`profile.json` — so "stage-only" is a real, meaningful safety boundary, not
|
||||
just a formality.
|
||||
3. **Deployment**: a separate app/service from `glucose-dashboard`, with its own
|
||||
repo-or-directory, Dockerfile, and Gitea CI/CD pipeline — isolating
|
||||
write-capable, dosing-adjacent code from the read-only dashboard.
|
||||
|
||||
## 4. Functional requirements
|
||||
|
||||
### FR1 — Infra investigation (first task, needs explicit go-ahead)
|
||||
Boot LXC 102 (`autotune`) and LXC 107 (`oref0`) and inspect what's installed
|
||||
(Docker containers, cron jobs, scripts, config, any README/notes left behind)
|
||||
to determine which one — if either — has a working oref0-autotune setup.
|
||||
Document findings (what's there, what's missing, what needs rebuilding).
|
||||
|
||||
*Acceptance:* a clear decision on which LXC hosts the microservice, and what
|
||||
(if anything) needs to be installed/fixed there before FR2 can start.
|
||||
|
||||
### FR2 — Autotune HTTP microservice
|
||||
A small HTTP API (exact framework TBD during design — e.g. Flask/FastAPI
|
||||
wrapping a Docker-based oref0-autotune image) exposing at minimum:
|
||||
- `POST /analyze` — body: Nightscout URL/token, date range, (optionally) the
|
||||
starting profile to tune from. Returns a job ID.
|
||||
- `GET /analyze/{job_id}` — status (`pending`/`running`/`done`/`failed`) and,
|
||||
once done, the recommended profile JSON plus the estimation/recommendation
|
||||
detail (per-hour deviations, the autotune log).
|
||||
|
||||
*Acceptance:* a manual `curl` against the running service, pointed at the
|
||||
user's real Nightscout instance, returns a plausible recommended profile within
|
||||
a reasonable time (minutes, not seconds — oref0-autotune is not instant).
|
||||
|
||||
### FR3 — Trigger analysis from the dashboard
|
||||
A new Streamlit app page where the user picks a date range (and confirms which
|
||||
current profile to tune from, fetched live from Nightscout), starts the
|
||||
analysis, and sees progress while the microservice job runs (via polling).
|
||||
|
||||
*Acceptance:* starting an analysis with bad/unreachable Nightscout credentials
|
||||
fails with a clear error rather than hanging silently.
|
||||
|
||||
### FR4 — Old-vs-new profile comparison view
|
||||
Side-by-side display of the current and proposed profile: hourly basal rates,
|
||||
ISF schedule, carb ratio schedule, DIA — plus the underlying estimation detail
|
||||
(per-hour deviations, confidence notes) surfaced from the autotune log, so the
|
||||
user can judge *why* a change is being proposed, not just see the new number.
|
||||
|
||||
*Acceptance:* every changed value is visually distinguishable from unchanged
|
||||
values; the estimation detail for a given time slot is viewable alongside that
|
||||
slot's proposed change.
|
||||
|
||||
### FR5 — Selective adjustment
|
||||
Per time-slot controls (accept/reject each proposed basal/ISF/CR change
|
||||
individually) plus editable numeric fields so the user can fine-tune any
|
||||
accepted value before staging — matching the user's explicit request to
|
||||
"select parts of the proposed profile and still adjust the values a bit."
|
||||
|
||||
*Acceptance:* rejecting a slot keeps the current value unchanged in the
|
||||
staged profile; editing an accepted value's number is reflected in what
|
||||
actually gets sent to Nightscout.
|
||||
|
||||
### FR6 — Stage to Nightscout
|
||||
Write the resulting merged profile (accepted + manually adjusted values) as a
|
||||
new named profile in the Nightscout profile store (e.g. `"Autotune
|
||||
YYYY-MM-DD"`) via a new POST to `/api/v1/profile`, without creating a Profile
|
||||
Switch treatment. After staging, show clear instructions for manual activation
|
||||
in AAPS (or the Nightscout UI).
|
||||
|
||||
*Acceptance:* the new profile appears in Nightscout's profile list/editor with
|
||||
exactly the merged values; the previously active profile is untouched and
|
||||
still active until the user manually switches.
|
||||
|
||||
### FR7 — Audit / history
|
||||
Keep a record of past analysis runs and what was accepted vs. rejected vs.
|
||||
manually adjusted, for traceability (useful both for the user's own reference
|
||||
and potentially for the hospital team). Could reuse Nightscout's own profile
|
||||
history (each staged profile is already a new dated document) rather than a
|
||||
separate store — **decide during design**.
|
||||
|
||||
### FR8 — Safety guardrails
|
||||
- Surface oref0-autotune's own `autosens_min`/`autosens_max` bounds so the user
|
||||
can see the algorithm's own safety rails, not just the raw numbers.
|
||||
- Flag any accepted value that represents an unusually large jump from the
|
||||
current profile.
|
||||
- Require an explicit confirmation step (e.g. a review summary + confirmation
|
||||
checkbox) before the FR6 write happens — no silent/implicit staging.
|
||||
|
||||
## 5. Non-functional requirements
|
||||
|
||||
- **NFR1 — Isolation**: separate repository-or-directory, Dockerfile, and Gitea
|
||||
CI/CD pipeline from `glucose-dashboard`, modeled on the existing pipeline
|
||||
(test/lint → build+Trivy+push → deploy) but with its own secrets.
|
||||
- **NFR2 — Nightscout token scope**: the existing dashboard token is documented
|
||||
read-only by convention (see `glucose-dashboard/README.md`). This feature
|
||||
needs a **separate, write-scoped** Nightscout token (Nightscout supports
|
||||
role-based tokens — e.g. a `careportal`-level role) rather than upgrading the
|
||||
shared read-only token, so the read-only reporting dashboard's blast radius
|
||||
doesn't change.
|
||||
- **NFR3 — Test safety**: automated tests must never call a real Nightscout
|
||||
instance or the real autotune microservice — mock both, following the
|
||||
existing `monkeypatch`-on-`requests` pattern from `glucose-dashboard`'s test
|
||||
suite.
|
||||
- **NFR4 — CI/CD gating**: given this pipeline can end in a write to a real
|
||||
medical data store, consider (during design) adding a `pull_request` trigger
|
||||
for the test/lint job so changes are validated *before* merge, not only
|
||||
after — the current `glucose-dashboard` pipeline only triggers on `push` to
|
||||
`main`.
|
||||
|
||||
## 6. Data requirements
|
||||
|
||||
- `GET /api/v1/profile.json` — read the current active profile store
|
||||
(net-new capability; nothing in `nightscout_client.py` handles profiles today).
|
||||
- `POST /api/v1/profile.json` — write the new named profile (net-new).
|
||||
- The autotune microservice's own job API (FR2).
|
||||
- Read access to entries/treatments over the analysis date range (can reuse the
|
||||
reporting feature's date-range fetch capability once built, or the existing
|
||||
rolling-window fetch for shorter tuning windows).
|
||||
|
||||
## 7. Testing & acceptance criteria
|
||||
|
||||
- Unit tests for the profile merge/validation logic (accept/reject/adjust
|
||||
producing the correct final profile document) — pure functions, no I/O,
|
||||
fully testable without any live service.
|
||||
- Unit tests for the new Nightscout profile read/write client functions, using
|
||||
the existing `FakeResponse`/`monkeypatch.setattr(requests, ...)` pattern.
|
||||
- Integration-style tests against the autotune microservice using a mocked
|
||||
HTTP layer (never the real LXC service in CI).
|
||||
- A UAT checklist (produced during the design phase) exercising the full flow
|
||||
end-to-end against the user's real Nightscout instance and the real
|
||||
microservice: trigger → review → adjust → stage → manually verify the new
|
||||
profile appears correctly in Nightscout, with no unintended Profile Switch.
|
||||
|
||||
## 8. Open questions for the user's edit pass
|
||||
|
||||
1. Which LXC (102 or 107) ends up hosting the service, once FR1 is done?
|
||||
2. Which autotune Docker image to standardize on —
|
||||
`p5nbTgip0r/autotune-docker`, `houthacker42/nightscout-autotune`, or
|
||||
building directly from the `oref0` npm package? (Design-phase decision,
|
||||
informed by FR1's findings.)
|
||||
3. Should automatic Profile Switch activation become an opt-in "phase 2" later,
|
||||
once you've built trust in the recommendations — or should manual
|
||||
activation remain permanent policy regardless of trust?
|
||||
4. FR7 (audit/history) — is reusing Nightscout's own profile-document history
|
||||
sufficient, or do you want a dedicated log outside Nightscout?
|
||||
5. Confirm the separate write-scoped Nightscout token is something you're
|
||||
comfortable creating/managing (NFR2) — this is a manual one-time setup step
|
||||
in the Nightscout admin UI, not something the app can do for you.
|
||||
@@ -0,0 +1,198 @@
|
||||
# Requirements: Clinical reporting page
|
||||
|
||||
Status: **draft — for review and editing**
|
||||
Owner: willemjan
|
||||
Related: [requirements-profile-tuning.md](requirements-profile-tuning.md) (separate feature, separate app)
|
||||
|
||||
## 1. Purpose & background
|
||||
|
||||
The previous FreeStyle Libre 2 setup (via LibreView) produced clinical reports
|
||||
that the hospital/diabetes team wants to keep seeing: a multi-month time-in-range
|
||||
(TIR) overview, day-by-day glucose graphs, hypoglycemia event analysis, and an
|
||||
estimated HbA1c. The current DIY closed-loop setup (Medtrum TouchCare Nano 200U +
|
||||
Lyumjev + Dexcom One+ via xDrip+ + AndroidAPS) stores everything in Nightscout,
|
||||
which has no equivalent built-in report.
|
||||
|
||||
This feature adds a **reporting page to the existing `glucose-dashboard` Streamlit
|
||||
app**, generating the closest practical equivalent from Nightscout data — read-only,
|
||||
no new infrastructure, deployed through the existing CI/CD pipeline.
|
||||
|
||||
## 2. Scope
|
||||
|
||||
**In scope:**
|
||||
- A new set of pages in the existing Streamlit app, reading `entries` (glucose) and
|
||||
`treatments` (insulin, carbs) from Nightscout.
|
||||
- Multi-day statistical summaries, an AGP-style percentile chart, day-by-day
|
||||
glucose graphs, hypo/hyper event analysis, and an estimated HbA1c (GMI) on separate pages where not closely related.
|
||||
- Generating PDF output of user selected pages, so the user can send the report to the hospital.
|
||||
|
||||
**Out of scope (this feature):**
|
||||
- Anything that writes to Nightscout (see the separate profile-tuning spec).
|
||||
- Non-Nightscout data sources.
|
||||
- The oref0-autotune integration.
|
||||
|
||||
## 3. Clinical definitions & standards used
|
||||
|
||||
These are the standard definitions this report is built against (international
|
||||
consensus, mainly based on dutch requirements, not invented for this project) the ranges are described in the table below.
|
||||
|
||||
| Term | Definition | Source |
|
||||
|---|---|---|
|
||||
| Hypoglycemia Level 1 | < 3.9 mmol/L (<70 mg/dL) | Battelino et al. 2019, *Diabetes Care* 42(8):1593–1603 |
|
||||
| Hypoglycemia Level 2 | < 3.0 mmol/L (<54 mg/dL) | same |
|
||||
| Hyperglycemia Level 1 | > 10.0 mmol/L (>180 mg/dL) | same |
|
||||
| Hyperglycemia Level 2 | > 13.9 mmol/L (>250 mg/dL) | same |
|
||||
| Event persistence rule | A hypo/hyper **event** starts once glucose has been continuously past the threshold for **≥15 minutes**, and ends once it has recovered past the threshold for **≥15 minutes**. A brief single reading crossing back doesn't end the event. | Battelino et al. 2019 |
|
||||
| Time-in-range targets (adult, general) | TIR (3.9–10.0) >70% · TBR<3.9 <4% · TBR<3.0 <1% · TAR>10.0 <25% · TAR>13.9 <5% | Battelino et al. 2019 |
|
||||
| GMI (estimated HbA1c) | `GMI% = 3.31 + 0.02392 × mean(mg/dL)` = `3.31 + 0.4311 × mean(mmol/L)` | Bergenstal et al. 2018, *Diabetes Care* 41(11):2275–2280 |
|
||||
| Minimum valid report | ≥14 days of data with ≥70% sensor-active time | consensus standard |
|
||||
|
||||
Note: this dashboard already uses `TARGET_LOW = 3.9` / `TARGET_HIGH = 10.0` in
|
||||
[`data_processing.py`](../data_processing.py) — that matches Level 1 thresholds
|
||||
above, so the existing constants carry over directly; Level 2 thresholds are new.
|
||||
|
||||
GMI is explicitly **not** a substitute for a lab HbA1c (can differ by ~0.5
|
||||
percentage points due to red blood cell/hemoglobin factors) — the report should
|
||||
label it "estimated HbA1c (GMI)", not "HbA1c", to avoid confusing the hospital
|
||||
team.
|
||||
|
||||
## 4. Functional requirements
|
||||
|
||||
Each FR below is meant to be independently implementable and testable, so they
|
||||
can be delivered one at a time through the CI/CD pipeline.
|
||||
|
||||
### FR1 — Report period selector
|
||||
The user can choose the reporting period: **14 days** (clinical default), 30
|
||||
days, **90 days** (matches the old Libre 2 quarterly view), or a custom date
|
||||
range. The period should be settable for each page separately since 90 days are relevant for A1C calculations and time in range but not so much for daily overview where 14 days is more relavant to the clinic. The selection drives every other requirement below.
|
||||
|
||||
*Acceptance:* selecting a period reloads the specific report page for that exact date
|
||||
range; the 90-day option must not time out or hang (see NFR1 on data volume).
|
||||
|
||||
### FR2 — AGP-style summary statistics
|
||||
A summary panel showing, for the selected period:
|
||||
- % of time CGM was active (data completeness)
|
||||
- Average glucose (mmol/L)
|
||||
- Estimated HbA1c (GMI)
|
||||
- Glucose variability (%CV = stdev/mean × 100)
|
||||
- Time-in-range breakdown across the 5 standard bands (<3.0, 3.0–3.9, 3.9–10.0,
|
||||
10.0–13.9, >13.9), each shown as a percentage and as hours:minutes/day
|
||||
- Each stat shown against its consensus target (pass/fail or color-coded)
|
||||
|
||||
*Acceptance:* stats match a hand-calculated reference on a small fixture dataset
|
||||
(see §7); TIR/TBR/TAR percentages sum to 100%.
|
||||
|
||||
### FR3 — AGP percentile chart
|
||||
A single chart showing the 5th/25th/50th(median)/75th/95th percentile glucose
|
||||
bands, aggregated by time-of-day across the selected period onto one 24-hour
|
||||
composite axis (the standard AGP visualization).
|
||||
|
||||
*Acceptance:* median line at a given hour matches the median of all readings at
|
||||
that hour-of-day across the period, computed independently in a test.
|
||||
|
||||
### FR4 — Day-by-day glucose graphs
|
||||
One glucose trace per calendar day in the selected period (reusing the existing
|
||||
target-range-band chart style from [`app.py`](../app.py)'s `build_chart`, adapted
|
||||
for a single day per panel), so the hospital team can review specific days the
|
||||
way the Libre 2 report's daily log did.
|
||||
|
||||
*Acceptance:* each day's panel shows exactly that day's readings; days with no
|
||||
data are shown as empty/skipped rather than causing an error.
|
||||
|
||||
### FR5 — Hypoglycemia event analysis
|
||||
Detect discrete hypo events per the persistence rule in §3, and report:
|
||||
- Total event count (Level 1 and Level 2 separately)
|
||||
- Duration of each event, and summary stats (average, longest, total time low)
|
||||
- Events per week
|
||||
- Time-of-day distribution (e.g., overnight vs. daytime split)
|
||||
|
||||
*Acceptance:* event count/duration matches a hand-traced fixture (a synthetic
|
||||
glucose series with known dips of known duration, including edge cases: a dip
|
||||
just under 15 minutes that should NOT count, and two dips separated by a brief
|
||||
recovery that should count as one continuous event per the persistence rule).
|
||||
|
||||
### FR6 — Hyperglycemia event analysis
|
||||
Same as FR5, symmetric, for Level 1/Level 2 highs. (Not explicitly requested by
|
||||
the user but a natural complement and confirmed to implement by the user.)
|
||||
|
||||
### FR7 — Estimated HbA1c trend
|
||||
GMI computed per rolling period (e.g., per month within a 90-day report),
|
||||
plotted as a small trend, so the hospital team can see whether control is
|
||||
improving/worsening over the quarter, not just the endpoint value.
|
||||
|
||||
### FR8 — Candidate additional statistics
|
||||
The user mentioned having seen "other interesting stats" in other reports
|
||||
without specifying which. Candidates to consider:
|
||||
- Day-of-week pattern (e.g., weekends vs. weekdays)
|
||||
- Overnight (00:00–06:00) vs. daytime TIR split
|
||||
- Basal:bolus insulin ratio
|
||||
- Insulin pump site-change and sensor-change adherence (interval between changes,
|
||||
reusing existing `classify_treatments` output)
|
||||
- Glucose trends after site-change
|
||||
- Pump site age vs insuline usage vs glucose levels. Corrected for carb intake to see if age of the pump-site and/or insuline age influence insulin usage
|
||||
- Total daily insulin dose trend
|
||||
|
||||
### FR9 — Data completeness / quality indicator
|
||||
Flag periods with low sensor-active time (<70%) or visible data gaps, since a
|
||||
report built on incomplete data is clinically unreliable — the report should
|
||||
surface this rather than silently computing misleading stats.
|
||||
|
||||
### FR10 — Export / print-friendly view (open question)
|
||||
The Libre 2 reports were shown to or printed for the hospital team.
|
||||
The pages needs a print-optimized layout, a PDF export, and a
|
||||
"share the dashboard URL".
|
||||
|
||||
## 5. Non-functional requirements
|
||||
|
||||
- **NFR1 — Data volume**: the existing `fetch_entries`/`fetch_treatments` in
|
||||
[`nightscout_client.py`](../nightscout_client.py) scale their `count` parameter
|
||||
linearly with `hours`, which doesn't scale cleanly to 90 days (~26,000 5-minute
|
||||
readings). This feature needs either a proper date-range fetch (`find[date]`
|
||||
query params) or paginated fetching — a new capability, not a reuse of the
|
||||
existing rolling-window functions as-is.
|
||||
- **NFR2 — Caching**: historical data for a fixed past period doesn't change; cache
|
||||
TTL should be much longer than the existing 60-second rolling-view cache
|
||||
(`app.py`'s `load_nightscout_data`), e.g. cached per exact date-range until
|
||||
explicitly refreshed.
|
||||
- **NFR3 — Consistency**: reuse the existing `PALETTE` dict and `st.context.theme`
|
||||
detection pattern from `app.py` rather than introducing new colors; reuse
|
||||
`TARGET_LOW`/`TARGET_HIGH` from `data_processing.py`.
|
||||
- **NFR4 — Language**: Dutch UI strings and docstrings, consistent with the rest
|
||||
of the app.
|
||||
- **NFR5 — Multipage navigation**: this is the first feature needing more than one
|
||||
page — introduces `st.Page`/`st.navigation` (anticipated in the project's own
|
||||
`homelab-cicd-werkinstructie.md`).
|
||||
- **NFR6 - Password protection on the dashboard pages**: as dashboard urls will be sharable and reachable by the diabetes team, access should be restricted to authorized users.
|
||||
|
||||
|
||||
## 6. Data requirements
|
||||
|
||||
- Glucose entries over an arbitrary date range (new: `fetch_entries` needs a
|
||||
date-range mode, not just "last N hours").
|
||||
- Treatments over the same range (same scaling concern).
|
||||
- No new Nightscout permissions needed — this feature only reads, so the
|
||||
existing read-only token is sufficient.
|
||||
|
||||
## 7. Testing & acceptance criteria
|
||||
|
||||
- Each new pure function (event detection, GMI calculation, percentile
|
||||
aggregation, %CV) gets unit tests in `data_processing.py` or a new
|
||||
`reporting.py` module, following the existing `tests/test_data_processing.py`
|
||||
fixture-based pattern — including edge cases called out per FR above (empty
|
||||
data, exactly-15-minute boundary events, 100% in-range, etc.).
|
||||
- Any new Nightscout fetch logic gets tests following the existing
|
||||
`tests/test_nightscout_client.py` `monkeypatch`-on-`requests` pattern.
|
||||
- A UAT checklist (produced during the design phase) will map each FR's
|
||||
"Acceptance" line above to a concrete scenario the user can manually verify
|
||||
against their own real Nightscout data.
|
||||
|
||||
## 8. Open questions for the user's edit pass
|
||||
|
||||
1. Confirm the clinical thresholds/targets in §3 match what your hospital
|
||||
actually uses, or note any local adaptations. - Use the suggested clinical thresholds and targets from §3
|
||||
2. FR6 (hyperglycemia analysis) — keep
|
||||
3. FR8 — which candidate stats actually matter to you / your team? All that are in FR8 now
|
||||
4. FR10 — is a print/export view needed, or is viewing the dashboard directly
|
||||
enough? - pdf export needed
|
||||
5. Default report period: 14-day (clinical standard) or 90-day (matches your
|
||||
old Libre 2 habit) as the page's default view? Default 90 days for statistics, but 14 days for recent glucose charts.
|
||||
Reference in New Issue
Block a user