Recover unfinished scans so their items aren't stranded
get_session_items / get_open_items / latest_scan_id all require finished_at IS NOT NULL, but the M365 and Google engines return early on abort (skipping finish_scan) and a process kill mid-scan (deploy, OOM, crash) never reaches it either. Result on prod: 41/42 scans had finished_at NULL, so 291 already-saved flagged items were invisible — the grid showed nothing. - finalize_orphan_scans(): finalises every finished_at-NULL scan; runs once at startup before the scheduler (nothing is scanning at boot, so any unfinished scan is dead). Recovers existing stranded items and guards against future mid-scan restarts. - run_scan: finalise the DB scan on the abort early-return too, so a stopped scan's items stay visible without waiting for a restart. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
1 parent
7bf589bf7a
commit
29d9168643
5 files changed
+100
No files matched your search
+30
@@ -29,11 +29,14 @@ Usage (from gdpr_scanner.py)
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import logging
|
||||
import sqlite3
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import Iterator
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
from pathlib import Path as _P
|
||||
_DATA_DIR = _P.home() / ".gdprscanner"
|
||||
_DATA_DIR.mkdir(exist_ok=True)
|
||||
@@ -432,6 +435,33 @@ class ScanDB:
|
||||
|
||||
c.commit()
|
||||
|
||||
def finalize_orphan_scans(self) -> int:
|
||||
"""Finalise scans left unfinished by a crash, kill, or mid-scan restart.
|
||||
|
||||
After a fresh process start nothing is scanning, so any scan still
|
||||
carrying finished_at IS NULL is dead — the process that owned it is gone.
|
||||
Its already-saved flagged_items were stranded: both get_session_items
|
||||
and get_open_items require finished_at, so those items are invisible and
|
||||
effectively lost. Finalising the orphans on startup makes them show up
|
||||
and prevents permanent data loss from interrupted scans (the M365 and
|
||||
Google engines return early on abort and never reach finish_scan; only
|
||||
the file scan finalises in a finally block).
|
||||
|
||||
Safe to call only when no scan is running (i.e. at startup). Returns the
|
||||
number of scans finalised.
|
||||
"""
|
||||
rows = self._connect().execute(
|
||||
"SELECT id, total_scanned FROM scans WHERE finished_at IS NULL"
|
||||
).fetchall()
|
||||
count = 0
|
||||
for sid, total in rows:
|
||||
try:
|
||||
self.finish_scan(sid, total or 0)
|
||||
count += 1
|
||||
except Exception as e:
|
||||
logger.warning("[db] finalize_orphan_scans: scan %s failed: %s", sid, e)
|
||||
return count
|
||||
|
||||
# ── Query helpers ─────────────────────────────────────────────────────────
|
||||
|
||||
def latest_scan_id(self) -> int | None:
|
||||
|
||||
Reference in new issue
Block a user