Date-range scoping for viewer tokens
This commit is contained in:
1 parent
c820d6f6db
commit
4ef2dfb352
9 files changed
+69
-6
No files matched your search
@@ -72,6 +72,7 @@ Read-only access for DPOs and reviewers. Key invariants:
|
||||
- **`GET /api/db/flagged`** — returns `get_session_items()` (last completed scan session, joined with dispositions), filtered by `session["viewer_scope"].role` when set. Used exclusively by `_loadViewerResults()` in `results.js`. Do not confuse with `get_flagged_items()` (single scan_id, no disposition join).
|
||||
- **Rate-limit state** (`_pin_attempts` dict in `routes/viewer.py`) — in-memory only, resets on server restart. Intentional — a restart clears lockouts without a persistent store.
|
||||
- **User-scoped tokens (#34)** — scope `{"user": ["alice@m365.dk", "alice@gws.dk"], "display_name": "Alice Smith"}` filters `GET /api/db/flagged` by `account_id IN (list)`, covering both M365 and GWS items for the same person. `scope.user` is always stored as a list; a legacy single-string value is coerced to `[string]` on read. `scope.display_name` is used for UI only (badge, viewer header) — not for filtering. File-scan items (`account_id = ""`) never appear in user-scoped views. `POST /api/viewer/tokens` rejects combined `role`+`user` scope with 400. Share modal: scope-type `<select>` (`#shareScopeType`) reveals either the role dropdown (`#shareScopeRoleWrap`) or a name-search autocomplete (`#shareScopeUserWrap`). Autocomplete reads `S._allUsers`; selecting a row stores `{ emails, display_name }` in module-level `_selectedScopeUser`; editing the input manually clears it (free-text email fallback). In viewer mode, `auth.js` shows `#viewerIdentityBadge` with `VIEWER_SCOPE.display_name`.
|
||||
- **Date-range scoping** — tokens can carry `valid_from` and/or `valid_to` fields (YYYY-MM-DD) in their scope dict. `GET /api/db/flagged` filters items whose `modified` date falls outside the range using lexicographic string comparison (ISO dates sort correctly without parsing). `POST /api/viewer/tokens` validates format and enforces `valid_from ≤ valid_to`. The share modal shows `#shareValidFrom` / `#shareValidTo` date inputs (apply to any scope type). The token list shows a green date-range badge when a range is stored. All three scope dimensions (role, user, date-range) are independent and combinable.
|
||||
- **Token onclick attributes** — Copy/Revoke buttons in `_renderTokenList()` pass the token as a single-quoted JS string literal (`'\'' + tok.token + '\''`), never via `JSON.stringify`. `JSON.stringify` produces double-quoted strings that break the surrounding `onclick="…"` HTML attribute.
|
||||
- **Settings Security pane** — Admin PIN and Viewer PIN groups live in `stPaneSecurity`, not `stPaneGeneral`. `switchSettingsTab('security')` in `sources.js` triggers both `stLoadPinStatus()` and `stLoadViewerPinStatus()`. The Share modal Configure button opens `openSettings('security')`.
|
||||
- **`stClearViewerPin` guard** — validates that the current-PIN field is non-empty client-side before sending the DELETE request; shows an inline error and focuses the field if empty.
|
||||
@@ -87,12 +88,14 @@ Read-only access for DPOs and reviewers. Key invariants:
|
||||
|
||||
## Scan filter options — scan_engine.py
|
||||
|
||||
Both options live in the profile `options` dict and apply to **all three scan engines** (M365, Google, file scan).
|
||||
All options live in the profile `options` dict and apply to **all three scan engines** (M365, Google, file scan).
|
||||
|
||||
- **`skip_gps_images` (bool, default `false`)** — When enabled, images whose only PII is GPS coordinates are not flagged. GPS data is still extracted and stored in the card `exif` field if the item is flagged by another signal (faces, EXIF author/comment). The `gps_location` special category is also suppressed. Evaluated via `_exif_has_pii` which rechecks `pii_fields` and `author` when GPS is skipped.
|
||||
- **`min_cpr_count` (int, default `1`)** — Minimum number of **distinct** CPR numbers in a file before it is flagged. Deduplication uses `list(dict.fromkeys(c["formatted"] for c in cprs))` — `cprs` is a list of dicts from `extract_matches`, not strings. Do not revert to `dict.fromkeys(cprs)` — that raises `TypeError: unhashable type: 'dict'` on every file with CPR hits. Files with faces or EXIF PII are still flagged regardless of CPR count — the threshold gates only CPR-based hits.
|
||||
- **File scan** reads both from `source` dict keys (passed directly from the `/api/file_scan/start` payload). **M365 scan** reads both from `scan_opts = options.get("options", {})`. Both paths apply the same `_cpr_qualifies` / `_exif_has_pii` logic before the flagging gate.
|
||||
- **UI:** sidebar controls `#optSkipGps` (toggle) and `#optMinCpr` (number); profile editor controls `#peOptSkipGps` and `#peOptMinCpr`. Both are saved/loaded by `profiles.js`.
|
||||
- **`cpr_only` (bool, default `false`)** — When enabled, items whose only hits are email addresses, phone numbers, detected faces, or EXIF/GPS metadata are skipped; only items with at least one qualifying CPR number are flagged. Implemented as a compact short-circuit at each engine's flagging gate: `if not (_cpr_qualifies and cprs) and (cpr_only or (<other PII absent>)): continue`. This preserves existing behavior when `cpr_only=False`. Sidebar toggle `#optCprOnly`; profile editor `#peOptCprOnly`.
|
||||
- **`ocr_lang` (str, default `"dan+eng"`)** — Tesseract language pack(s) used when scanning scanned PDFs and images. Presets: `dan+eng`, `dan`, `eng`, `dan+eng+deu`, `dan+eng+swe`, `dan+eng+fra`. Threaded through `_scan_bytes`/`_scan_bytes_timeout` → `document_scanner.scan_pdf`/`scan_image` and the spawned PDF-OCR subprocess worker (`_worker_scan_pdf`). The OCR result cache key already included `lang`, so per-language results are cached independently. Sidebar select `#optOcrLang`; profile editor `#peOptOcrLang`.
|
||||
- **File scan** reads all options from `source` dict keys (passed directly from the `/api/file_scan/start` payload). **M365 scan** reads them from `scan_opts = options.get("options", {})`. Both paths apply the same `_cpr_qualifies` / `_exif_has_pii` logic before the flagging gate.
|
||||
- **UI:** sidebar controls `#optSkipGps`, `#optMinCpr`, `#optCprOnly`, `#optOcrLang`; profile editor controls `#peOptSkipGps`, `#peOptMinCpr`, `#peOptCprOnly`, `#peOptOcrLang`. All are saved/loaded by `profiles.js`.
|
||||
|
||||
## M365 connector exceptions — m365_connector.py
|
||||
|
||||
@@ -174,6 +177,7 @@ Allows reviewing results from any past scan session without running a new scan.
|
||||
- **Rule:** close `S.es` (and reset `S._userStartedScan`) only inside the branch where *all* concurrent scans have finished: `scan_done` checks `!S._googleScanRunning && !S._fileScanRunning`; `google_scan_done` checks `!S._m365ScanRunning && !S._fileScanRunning`; `file_scan_done` checks `!S._m365ScanRunning && !S._googleScanRunning`.
|
||||
- **Scheduled scans** — `S._userStartedScan` is false for scheduler-triggered runs, so the SSE connection is never closed and future scheduler events continue to arrive.
|
||||
- **`scan_start` is M365-only** — `run_scan()` broadcasts `scan_start`; `run_file_scan()` and `routes/google_scan.py` must NOT. The `scan_start` handler in `_attachSchedulerListeners` unconditionally sets `S._m365ScanRunning = true`. If a file scan emits `scan_start`, the flag is set without a matching `scan_done` to clear it, and `file_scan_done` refuses to re-enable the scan button because `!S._m365ScanRunning` is false. Use `scan_phase` (file) and `google_scan_phase` (google) instead — these are routed correctly by the phase-source detection logic in `_attachScanListeners`.
|
||||
- **Two separate abort events** — `state._scan_abort` (M365 + file) and `state._google_scan_abort` (Google). `POST /api/scan/stop` sets **both**. `_check_abort()` inside `_run_google_scan` must use the module-level `_scan_abort` alias (`= state._google_scan_abort`), not `gdpr_scanner._scan_abort` (which is the M365 event). Do not conflate them — a Google-only scan must react to Stop, and `gdpr_scanner._scan_abort` is not the right event for that path.
|
||||
|
||||
## Email sending — routes/email.py + m365_connector.py
|
||||
|
||||
|
||||
Reference in new issue
Block a user