Graph returned 404 for /users/google:<email>/mailFolders: Google-
Workspace-only accounts (id "google:<email>") reached the M365 scan.
- getSelectedUsers() fell back to "every ticked checkbox in the DOM"
whenever no M365 account was ticked — including Google-only rows. The
fallback is now only used before the user list has loaded, and never
includes google: rows.
- startScan() skips the M365 part (with a log line) when M365 sources are
ticked but no M365 accounts are: the server treats an empty account
list as "every user in the tenant".
- run_scan: _resolve_manual_users -> _resolve_scan_users, which now also
drops google: ids (reported as a phase message). Covers scheduled scans
and profiles that already contain Google-only accounts. Never widens to
all users.
Tests: 3 new in tests/test_manual_users.py; the run_scan one fails on the
old engine with exactly the reported Graph call.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Pressing Stop could look like the scan restarting, and for scheduled jobs
it really did restart the Google stage.
- Scheduler: check the abort flag between stages and never clear the
Google abort after a Stop (that started a fresh all-users Google scan).
A stopped job is recorded as "cancelled" and skips email/retention;
scheduler_done carries cancelled=true.
- SSE: scan_cancelled and file_scan_done now end the current scan like
scan_done, and sse.replay_messages() never replays a finished scan.
Previously a stopped M365 scan (and every file scan) stayed "current",
so each reconnect replayed it and re-showed Stop + progress.
- Frontend: scan_cancelled clears the M365 running flag and only tears
down the UI when no Google/file scan is still running; scheduler_done /
scheduler_error clear all running flags.
- run_scan: Stop during collection now ends the scan (checkpoint, finish,
scan_cancelled) instead of announcing scan_start / "Scanning…".
- Stop button: switches to a disabled "Stopping…" state on click; all
show/hide goes through _setStopBtnDisplay(), which resets it when hidden
or when a new scan starts.
Adds tests/test_stop_scan.py (10 tests).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Manual accounts were never scanned:
- Sidebar "add user" entries had no platform, and getSelectedUsers()
required platform === 'm365', so they were dropped from every scan.
- Profile-editor manual accounts were stored as "manual:<email>", which
is not a Graph user id, and applying the profile never put them in the
sidebar list, so they could not be selected.
Now the editor looks the email up via /api/users/lookup and stores the
directory id plus name/email (falling back to manual:<email> only when the
lookup fails). Applying a profile adds its manual accounts to the sidebar
and resolves legacy manual:<email> entries, logging any that can't be
found. The scan engine resolves remaining manual:<email> ids (e.g. from
scheduled scans) via the new M365Connector.lookup_user(), skipping and
reporting failures without widening to all users.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
get_session_items / get_open_items / latest_scan_id all require
finished_at IS NOT NULL, but the M365 and Google engines return early
on abort (skipping finish_scan) and a process kill mid-scan (deploy,
OOM, crash) never reaches it either. Result on prod: 41/42 scans had
finished_at NULL, so 291 already-saved flagged items were invisible —
the grid showed nothing.
- finalize_orphan_scans(): finalises every finished_at-NULL scan; runs
once at startup before the scheduler (nothing is scanning at boot, so
any unfinished scan is dead). Recovers existing stranded items and
guards against future mid-scan restarts.
- run_scan: finalise the DB scan on the abort early-return too, so a
stopped scan's items stay visible without waiting for a restart.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Operators can now choose Tesseract language pack(s) per profile via a
sidebar select (#optOcrLang) and profile editor (#peOptOcrLang). Presets:
dan+eng (default), dan, eng, dan+eng+deu, dan+eng+swe, dan+eng+fra. The
ocr_lang option flows from the UI through all three scan engines (M365
files/attachments, Google Drive, Gmail) down to document_scanner.scan_pdf
and scan_image — including the spawned PDF-OCR subprocess worker.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
feat: video/audio metadata scanning, profile rename fix, route tests
- Scan .mp4/.mov/.avi/.mkv and .mp3/.flac/.ogg/.m4a/.wma (+ 7 more)
for GPS coordinates, artist/author, title, comment — metadata only,
no frame or audio analysis. Uses mutagen (added to requirements.txt).
GPS-tagged phone recordings now flag with gps_location like photos.
- Fix _extract_audio_metadata silently returning empty results:
mutagen.File() first positional arg is `filename`, not `fileobj` —
was passing BytesIO as the filename. Fixed to keyword args.
- Fix profile copy rename not reflected in left column until modal
reopen: _pmgmtSaveFullEdit called loadProfiles() but never
_renderProfileMgmt(). Added re-render and active-row highlight.
- Add TestProfileRoutes (10 tests) covering all profile API endpoints
including a rename regression test. Total: 182 tests.
- generate_fixtures.py now produces 6 audio/video fixtures (14–19):
2 MP3, 2 FLAC, 2 MP4 — 4 flagged, 2 negative cases.
- Scan history browser (history.js, GET /api/db/sessions, get_sessions(),
get_session_items(ref_scan_id)) — review any past session without rescanning
- User-scoped viewer tokens (#34) — scope by individual employee across M365
and GWS; autocomplete from Accounts list; dual-email support
- Fix: GWS scan never marked finished (end_scan → finish_scan) and emitted
wrong SSE event (scan_done → google_scan_done), excluding GWS items from all
exports
- Fix: file scan begin_scan called with wrong keyword args (TypeError swallowed),
so local/SMB items were never written to DB
- Fix: Graph sendMail reported failure on success — _post() now returns {} on
empty 202 response instead of raising JSONDecodeError
- Fix: Graph error hidden behind generic "No SMTP host" message when both Graph
and SMTP were unavailable
- Fix: Gmail vs Google Workspace SMTP error messages distinguished by username
domain; Workspace errors point to admin console, not personal security settings
- Docs: update README, MANUAL-EN, MANUAL-DA, CLAUDE.md, TODO.md, CHANGELOG.md
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add M365DriveNotFound(M365Error) exception raised by _get() on HTTP 404.
Catch it explicitly in _scan_user_onedrive before the generic handler,
broadcasting a scan_phase ("not provisioned — skipped") instead of a red
scan_error card. Full-scan path is unaffected (bare except Exception: return
in _iter_drive_folder_for already silenced the same 404).
Root cause: _get() fell through to raise_for_status() on 404, caught by
the generic except Exception handler and broadcast as scan_error. The
asymmetry with full scans (which silently skipped 404s) was confusing.
Common causes of OneDrive 404: no licence assigned, service plan disabled,
drive never provisioned (account never signed in), account suspended.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
New profile options to reduce noise when scanning student accounts:
- skip_gps_images: images flagged solely by GPS coordinates are suppressed.
GPS data is still extracted and shown in the detail card when the item
is flagged by another signal (faces, EXIF author/comment).
- min_cpr_count (default 1): only flag a file if it contains at least N
distinct CPR numbers. Deduplication is by value. Faces and EXIF PII
still trigger flags regardless of CPR count.
Both options apply to M365, Google, and file scan paths. Saved in profiles
and editable in the Profile Manager editor. Docs, manuals, i18n (DA/EN/DE),
CHANGELOG, and VERSION (1.6.14 → 1.6.15) updated.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
