glucose-dashboard/docs/requirements-profile-tuning.md

# Requirements: oref0-autotune profile tuning

Status: **draft — for review and editing**
Owner: willemjan
Related: [requirements-reporting.md](requirements-reporting.md) (separate feature, same Nightscout data source)

## 1. Purpose & background

AndroidAPS (AAPS) only gained built-in Autotune in version 3.4, and even that
version tunes only a single ISF/carb-ratio value plus the basal schedule from
on-device history — it doesn't use Nightscout at all, and the user is still on
AAPS 3.3 regardless. Previously, the user ran a hosted tool (inspired by
`MarkMpn/AutotuneWeb`, now archived/discontinued) that analyzed Nightscout data
with `oref0-autotune` and proposed a new pump profile, which was then manually
entered into AAPS — a process the user describes as "a lot of trial and error."

This feature builds a guided, human-in-the-loop workflow: trigger an
oref0-autotune analysis from a dashboard, review the proposed profile against
the current one with the underlying estimations visible, selectively
accept/adjust values, and stage the result in Nightscout for manual activation.

The user has two LXC containers on Proxmox from earlier experimentation —
`autotune` (VMID 102, 192.168.2.13) and `oref0` (VMID 107, 192.168.2.14) —
currently stopped, of unknown/half-remembered working state.

## 2. Scope

**In scope:**
- Booting and inspecting the two existing LXCs to determine (or establish) a
  working oref0-autotune setup.
- A small HTTP microservice on that LXC that runs oref0-autotune against the
  user's Nightscout data on demand.
- A **separate** Streamlit app/service (own deployment, own CI/CD pipeline) that
  triggers the analysis, displays old-vs-new profile comparisons with
  estimation detail, lets the user selectively adjust proposed values, and
  stages the result in Nightscout.

**Out of scope (this iteration):**
- Automatically activating the new profile in AndroidAPS (creating a
  `"Profile Switch"` treatment) — the dashboard stages the profile only; the
  user activates it manually. This may become an opt-in "phase 2" once trust is
  established (see §8).
- Any change to AAPS settings or on-device configuration.
- Continuous/scheduled autotune runs — this is a manually triggered, on-demand
  analysis, not a nightly automated loop.

## 3. Architecture decisions (made during spec review)

1. **Execution**: an always-on HTTP microservice on whichever LXC has (or gets)
   a working oref0-autotune setup, rather than SSH-triggering a one-off Docker
   run or bundling autotune into the dashboard's own container. The dashboard
   calls it over HTTP and polls for job completion.
2. **Profile push safety**: **stage-only**. The service writes a new named
   profile into the Nightscout profile store but does **not** create a
   `"Profile Switch"` treatment — final activation stays a deliberate manual
   step in AAPS or the Nightscout UI. This matters because AndroidAPS applies
   profiles via Profile Switch treatments (gated behind an explicit "accept
   profile switch from Nightscout" AAPS setting), not by simply reading
   `profile.json` — so "stage-only" is a real, meaningful safety boundary, not
   just a formality.
3. **Deployment**: a separate app/service from `glucose-dashboard`, with its own
   repo-or-directory, Dockerfile, and Gitea CI/CD pipeline — isolating
   write-capable, dosing-adjacent code from the read-only dashboard.

## 4. Functional requirements

### FR1 — Infra investigation (first task, needs explicit go-ahead)
Boot LXC 102 (`autotune`) and LXC 107 (`oref0`) and inspect what's installed
(Docker containers, cron jobs, scripts, config, any README/notes left behind)
to determine which one — if either — has a working oref0-autotune setup.
Document findings (what's there, what's missing, what needs rebuilding).

*Acceptance:* a clear decision on which LXC hosts the microservice, and what
(if anything) needs to be installed/fixed there before FR2 can start.

### FR2 — Autotune HTTP microservice
A small HTTP API (exact framework TBD during design — e.g. Flask/FastAPI
wrapping a Docker-based oref0-autotune image) exposing at minimum:
- `POST /analyze` — body: Nightscout URL/token, date range, (optionally) the
  starting profile to tune from. Returns a job ID.
- `GET /analyze/{job_id}` — status (`pending`/`running`/`done`/`failed`) and,
  once done, the recommended profile JSON plus the estimation/recommendation
  detail (per-hour deviations, the autotune log).

*Acceptance:* a manual `curl` against the running service, pointed at the
user's real Nightscout instance, returns a plausible recommended profile within
a reasonable time (minutes, not seconds — oref0-autotune is not instant).

### FR3 — Trigger analysis from the dashboard
A new Streamlit app page where the user picks a date range (and confirms which
current profile to tune from, fetched live from Nightscout), starts the
analysis, and sees progress while the microservice job runs (via polling).

*Acceptance:* starting an analysis with bad/unreachable Nightscout credentials
fails with a clear error rather than hanging silently.

### FR4 — Old-vs-new profile comparison view
Side-by-side display of the current and proposed profile: hourly basal rates,
ISF schedule, carb ratio schedule, DIA — plus the underlying estimation detail
(per-hour deviations, confidence notes) surfaced from the autotune log, so the
user can judge *why* a change is being proposed, not just see the new number.

*Acceptance:* every changed value is visually distinguishable from unchanged
values; the estimation detail for a given time slot is viewable alongside that
slot's proposed change.

### FR5 — Selective adjustment
Per time-slot controls (accept/reject each proposed basal/ISF/CR change
individually) plus editable numeric fields so the user can fine-tune any
accepted value before staging — matching the user's explicit request to
"select parts of the proposed profile and still adjust the values a bit."

*Acceptance:* rejecting a slot keeps the current value unchanged in the
staged profile; editing an accepted value's number is reflected in what
actually gets sent to Nightscout.

### FR6 — Stage to Nightscout
Write the resulting merged profile (accepted + manually adjusted values) as a
new named profile in the Nightscout profile store (e.g. `"Autotune
YYYY-MM-DD"`) via a new POST to `/api/v1/profile`, without creating a Profile
Switch treatment. After staging, show clear instructions for manual activation
in AAPS (or the Nightscout UI).

*Acceptance:* the new profile appears in Nightscout's profile list/editor with
exactly the merged values; the previously active profile is untouched and
still active until the user manually switches.

### FR7 — Audit / history
Keep a record of past analysis runs and what was accepted vs. rejected vs.
manually adjusted, for traceability (useful both for the user's own reference
and potentially for the hospital team). Could reuse Nightscout's own profile
history (each staged profile is already a new dated document) rather than a
separate store — **decide during design**.

### FR8 — Safety guardrails
- Surface oref0-autotune's own `autosens_min`/`autosens_max` bounds so the user
  can see the algorithm's own safety rails, not just the raw numbers.
- Flag any accepted value that represents an unusually large jump from the
  current profile.
- Require an explicit confirmation step (e.g. a review summary + confirmation
  checkbox) before the FR6 write happens — no silent/implicit staging.

## 5. Non-functional requirements

- **NFR1 — Isolation**: separate repository-or-directory, Dockerfile, and Gitea
  CI/CD pipeline from `glucose-dashboard`, modeled on the existing pipeline
  (test/lint → build+Trivy+push → deploy) but with its own secrets.
- **NFR2 — Nightscout token scope**: the existing dashboard token is documented
  read-only by convention (see `glucose-dashboard/README.md`). This feature
  needs a **separate, write-scoped** Nightscout token (Nightscout supports
  role-based tokens — e.g. a `careportal`-level role) rather than upgrading the
  shared read-only token, so the read-only reporting dashboard's blast radius
  doesn't change.
- **NFR3 — Test safety**: automated tests must never call a real Nightscout
  instance or the real autotune microservice — mock both, following the
  existing `monkeypatch`-on-`requests` pattern from `glucose-dashboard`'s test
  suite.
- **NFR4 — CI/CD gating**: given this pipeline can end in a write to a real
  medical data store, consider (during design) adding a `pull_request` trigger
  for the test/lint job so changes are validated *before* merge, not only
  after — the current `glucose-dashboard` pipeline only triggers on `push` to
  `main`.

## 6. Data requirements

- `GET /api/v1/profile.json` — read the current active profile store
  (net-new capability; nothing in `nightscout_client.py` handles profiles today).
- `POST /api/v1/profile.json` — write the new named profile (net-new).
- The autotune microservice's own job API (FR2).
- Read access to entries/treatments over the analysis date range (can reuse the
  reporting feature's date-range fetch capability once built, or the existing
  rolling-window fetch for shorter tuning windows).

## 7. Testing & acceptance criteria

- Unit tests for the profile merge/validation logic (accept/reject/adjust
  producing the correct final profile document) — pure functions, no I/O,
  fully testable without any live service.
- Unit tests for the new Nightscout profile read/write client functions, using
  the existing `FakeResponse`/`monkeypatch.setattr(requests, ...)` pattern.
- Integration-style tests against the autotune microservice using a mocked
  HTTP layer (never the real LXC service in CI).
- A UAT checklist (produced during the design phase) exercising the full flow
  end-to-end against the user's real Nightscout instance and the real
  microservice: trigger → review → adjust → stage → manually verify the new
  profile appears correctly in Nightscout, with no unintended Profile Switch.

## 8. Open questions for the user's edit pass

1. Which LXC (102 or 107) ends up hosting the service, once FR1 is done?
2. Which autotune Docker image to standardize on —
   `p5nbTgip0r/autotune-docker`, `houthacker42/nightscout-autotune`, or
   building directly from the `oref0` npm package? (Design-phase decision,
   informed by FR1's findings.)
3. Should automatic Profile Switch activation become an opt-in "phase 2" later,
   once you've built trust in the recommendations — or should manual
   activation remain permanent policy regardless of trust?
4. FR7 (audit/history) — is reusing Nightscout's own profile-document history
   sufficient, or do you want a dedicated log outside Nightscout?
5. Confirm the separate write-scoped Nightscout token is something you're
   comfortable creating/managing (NFR2) — this is a manual one-time setup step
   in the Nightscout admin UI, not something the app can do for you.