fix: Nightscout API silently truncated multi-day fetches to ~3-4 days

_entries_since/_treatments_since requested a single page via `count`,
but Nightscout's /api/v1/entries and /api/v1/treatments ignore a large
`count` and cap the response around ~1150/~700 documents regardless of
how far back the requested period goes. Every report page relying on
fetch_entries_range/fetch_treatments_range (7/14/30/90-day selectors)
was silently only getting the most recent ~3-4 days, no matter the
selected period.

Fixed by paginating with `find[date][$lte]` cursors in a new
_paginated_get(), walking back until the requested `since` boundary is
reached or the server returns a short/empty page (= genuinely no more
data, confirmed by manually paginating against the real instance:
entries go back 50+ days, treatments only ~4 days on this install).

Also added reporting.completeness_warning(), shared by the interactive
pages (via ui_common.render_completeness_warning) and the PDF export,
that distinguishes "a structural gap at the start of the period" (data
wasn't fetched/available) from "scattered gaps below the clinical AGP
coverage threshold" (Danne et al. 2017) — the previous single generic
"X% sensor coverage" message read as a CGM/sensor problem even when
the real cause was this fetch bug.
This commit is contained in:
2026-07-31 14:21:35 +02:00
parent fe6a471f17
commit 29d20ca441
13 changed files with 322 additions and 30 deletions
+39
View File
@@ -1,9 +1,12 @@
from datetime import date
import pandas as pd
import pytest
from reporting import (
agp_percentile_bands,
bolus_stats,
completeness_warning,
day_of_week_pattern,
detect_events,
gmi_percent,
@@ -93,6 +96,42 @@ def test_sensor_active_percent_handles_gaps():
assert result == pytest.approx(50.0, abs=1.0)
def test_completeness_warning_none_for_empty_glucose():
empty = pd.DataFrame(columns=["timestamp", "mmol"])
assert completeness_warning(empty, "2026-01-01", "2026-01-14") == "Geen glucosedata gevonden in deze periode."
def test_completeness_warning_none_when_period_is_fully_covered():
# 14 volle dagen aan 5-minuten-metingen, precies de gekozen periode
timestamps = pd.date_range("2026-01-01 00:00", "2026-01-14 23:55", freq="5min")
glucose = pd.DataFrame({"timestamp": timestamps, "mmol": [6.0] * len(timestamps)})
assert completeness_warning(glucose, date(2026, 1, 1), date(2026, 1, 14)) is None
def test_completeness_warning_flags_missing_days_at_start_of_period():
# gevraagd: 1 t/m 14 januari, maar data begint pas op 11 januari (de
# exacte bug: alleen de laatste ~4 dagen van een langere periode zijn
# opgehaald)
timestamps = pd.date_range("2026-01-11 00:00", "2026-01-14 23:55", freq="5min")
glucose = pd.DataFrame({"timestamp": timestamps, "mmol": [6.0] * len(timestamps)})
message = completeness_warning(glucose, date(2026, 1, 1), date(2026, 1, 14))
assert message is not None
assert "11-01-2026" in message
assert "10 dag" in message # 1 t/m 10 januari ontbreken volledig = 10 dagen
assert "sensorhiaten" in message
def test_completeness_warning_flags_low_coverage_without_start_gap():
# data start op tijd, maar met veel verspreide hiaten -> lage dekking
# zonder dat er een aaneengesloten gat aan het begin is
timestamps = pd.date_range("2026-01-01 00:00", periods=20, freq="5min")
glucose = pd.DataFrame({"timestamp": timestamps, "mmol": [6.0] * 20})
message = completeness_warning(glucose, date(2026, 1, 1), date(2026, 1, 14))
assert message is not None
assert "sensordekking" in message
assert "sensorhiaten" not in message # dit is de generieke, niet de "ontbreekt volledig"-boodschap
def test_detect_events_ignores_dip_shorter_than_minimum_duration():
# dip van 10 minuten (< 15 min drempel) mag niet meetellen
glucose = _series(