6 Commits
Author SHA1 Message Date
StyxX65 b77bf70ff5 Fix deprecated smbprotocol.query_info import
Build — Windows, Linux & macOS / GDPRScanner / linux (push) Has been cancelled
Build — Windows, Linux & macOS / GDPRScanner / macos (push) Has been cancelled
Build — Windows, Linux & macOS / GDPRScanner / windows (push) Has been cancelled
Build — Windows, Linux & macOS / Create GitHub Release (push) Has been cancelled
smbprotocol.query_info is a thin deprecation shim (from smbprotocol.file_info
import * + a DeprecationWarning) scheduled for removal in the library's next
major release. Switched to importing FileDirectoryInformation directly from
smbprotocol.file_info, the actual current location - confirmed present there
in the installed package. Pure rename, no behavior change: all 227 tests
still pass, and the warning is gone from the test output.
2026-09-03 22:59:38 +02:00
StyxX65andClaude Sonnet 5 edc820051e Fix classification/ directory never matching where data files lived
Build — Windows, Linux & macOS / GDPRScanner / linux (push) Has been cancelled
Build — Windows, Linux & macOS / GDPRScanner / macos (push) Has been cancelled
Build — Windows, Linux & macOS / GDPRScanner / windows (push) Has been cancelled
Build — Windows, Linux & macOS / Create GitHub Release (push) Has been cancelled
The code has always expected classification/m365_skus.json and
classification/google_ou_roles.json (present since the initial commit
in m365_connector.py, google_connector.py, and build_gdpr.py's own
skus_dir variable) - but the actual data files lived in skus/ under a
different name (education.json) or just the wrong directory. Both
loaders degrade to a fallback on failure, so this was never fatal, but
it meant:

- M365 SKU-based role classification silently ran on fragment-matching
  only, never touching the real 13 student + 11 staff SKU list.
- Google Workspace OU-based role classification silently used a
  hardcoded 4-prefix fallback missing real categories the actual
  config has - /Elev, /Pupils (student), /Teachers, /Admin (staff).
  _load_ou_roles() had a bare `except Exception: return <fallback>`
  with no logging at all, so this had zero visibility anywhere.

Fix: moved both files to classification/ (git mv, so history is
preserved), updated .gitignore's exception rule from !skus/*.json to
!classification/*.json, fixed the stale "skus/google_ou_roles.json"
docstring, and added a module logger + proper warning/error logging to
_load_ou_roles() matching the pattern _load_sku_data() already used -
so if this class of bug recurs, it won't be silent again.

Verified: both loaders now log successful loads with the real counts
(13+11 SKUs, 4+6 OU prefixes including the previously-missing
categories), all 227 tests still pass, and a full app boot confirms
the startup warning is gone and the app still serves normally.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-03 22:58:24 +02:00
StyxX65 2260a451a5 Fix deprecated fitz import, log unhandled scan-thread crashes
Build — Windows, Linux & macOS / GDPRScanner / linux (push) Has been cancelled
Build — Windows, Linux & macOS / GDPRScanner / macos (push) Has been cancelled
Build — Windows, Linux & macOS / GDPRScanner / windows (push) Has been cancelled
Build — Windows, Linux & macOS / Create GitHub Release (push) Has been cancelled
document_scanner.py: `import fitz` is PyMuPDF's deprecated import
alias, scheduled for removal - switched both sites to `import pymupdf
as _fitz`. Pure rename, no behavior change (all 227 tests still pass).

routes/scan.py: the background scan thread caught no exceptions, so
an unexpected crash in run_scan() escaped into Python's default
unhandled-thread-exception hook instead of the app's own logging/audit
trail. The scan lock was already released correctly via `finally` -
now the crash itself is also logged (`_log.exception`, full traceback)
and audit-recorded before that. Confirmed by test_lock_released_when_
run_scan_raises: the PytestUnhandledThreadExceptionWarning it
triggered is now gone, with no change to the test's own assertions.
2026-09-03 22:55:07 +02:00
StyxX65andClaude Opus 4.8 efbbeb7306 Restore M365Connector.delete_message (was an orphaned method body)
Build — Windows, Linux & macOS / GDPRScanner / linux (push) Has been cancelled
Build — Windows, Linux & macOS / GDPRScanner / macos (push) Has been cancelled
Build — Windows, Linux & macOS / GDPRScanner / windows (push) Has been cancelled
Build — Windows, Linux & macOS / Create GitHub Release (push) Has been cancelled
The def line for delete_message had been lost, leaving its body as
unreachable dead code at the end of _delete() and no delete_message
attribute on the connector. Deleting an Outlook message therefore failed
with "'M365Connector' object has no attribute 'delete_message'". Restored
the method (soft-delete: move to Deleted Items, fall back to DELETE).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 15:43:46 +02:00
StyxX65andClaude Opus 4.8 54f8848e30 Document renderGrid landing-card hiding in static/js/CLAUDE.md
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 14:49:43 +02:00
StyxX65andClaude Opus 4.8 8a446509c6 Hide landing/last-scan card whenever results render
The live scan_file_flagged handler showed the grid but never hid
#emptyState / #lastScanSummary, so when a scan ran with the landing
card visible, results appeared underneath it until a manual refresh
(which re-ran loadOpenItems and cleared it). Hide both panels in
renderGrid whenever files are present, covering every render path
(live SSE, open-items load, history, filters).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-22 14:45:50 +02:00
10 changed files with 31 additions and 6 deletions

No files matched your search

+1 -1
View File
@@ -5,7 +5,7 @@
*.json *.json
!lang/*.json !lang/*.json
!keywords/*.json !keywords/*.json
!skus/*.json !classification/*.json
!package*.json !package*.json
# Be explicit about the most sensitive files # Be explicit about the most sensitive files
File renamed without changes.
File renamed without changes.
+2 -2
View File
@@ -106,7 +106,7 @@ except ImportError:
MASK_AVAILABLE = False MASK_AVAILABLE = False
try: try:
import fitz as _fitz # PyMuPDF — for secure (sanitised) PDF redaction import pymupdf as _fitz # PyMuPDF — for secure (sanitised) PDF redaction
PYMUPDF_AVAILABLE = True PYMUPDF_AVAILABLE = True
except ImportError: except ImportError:
PYMUPDF_AVAILABLE = False PYMUPDF_AVAILABLE = False
@@ -2727,7 +2727,7 @@ def count_faces_in_file(path, poppler_path=None, neighbors: int = 4) -> int:
elif ext == ".pdf": elif ext == ".pdf":
if PYMUPDF_AVAILABLE: if PYMUPDF_AVAILABLE:
import fitz as _fitz import pymupdf as _fitz
doc = _fitz.open(str(path)) doc = _fitz.open(str(path))
for page_idx in range(min(5, len(doc))): for page_idx in range(min(5, len(doc))):
pix = doc[page_idx].get_pixmap(dpi=100) pix = doc[page_idx].get_pixmap(dpi=100)
+1 -1
View File
@@ -38,7 +38,7 @@ try:
FileAttributes, FilePipePrinterAccessMask, ShareAccess, FileAttributes, FilePipePrinterAccessMask, ShareAccess,
ImpersonationLevel, ImpersonationLevel,
) )
from smbprotocol.query_info import FileDirectoryInformation from smbprotocol.file_info import FileDirectoryInformation
SMB_OK = True SMB_OK = True
except ImportError: except ImportError:
SMB_OK = False SMB_OK = False
+10 -2
View File
@@ -31,6 +31,8 @@ import threading
from pathlib import Path from pathlib import Path
from typing import Iterator, Optional from typing import Iterator, Optional
logger = logging.getLogger(__name__)
# ── google-auth / google-api-python-client ──────────────────────────────────── # ── google-auth / google-api-python-client ────────────────────────────────────
try: try:
from google.oauth2 import service_account from google.oauth2 import service_account
@@ -93,15 +95,21 @@ _MAX_EXPORT_BYTES = 20 * 1024 * 1024 # 20 MB
_OU_ROLES_PATH = Path(__file__).parent / "classification" / "google_ou_roles.json" _OU_ROLES_PATH = Path(__file__).parent / "classification" / "google_ou_roles.json"
def _load_ou_roles() -> tuple[list, list]: def _load_ou_roles() -> tuple[list, list]:
"""Load student/staff OU prefix lists from skus/google_ou_roles.json. """Load student/staff OU prefix lists from classification/google_ou_roles.json.
Returns (student_prefixes, staff_prefixes) — both lowercased.""" Returns (student_prefixes, staff_prefixes) — both lowercased."""
try: try:
import json as _j import json as _j
data = _j.loads(_OU_ROLES_PATH.read_text(encoding="utf-8")) data = _j.loads(_OU_ROLES_PATH.read_text(encoding="utf-8"))
students = [p.lower() for p in data.get("student_ou_prefixes", [])] students = [p.lower() for p in data.get("student_ou_prefixes", [])]
staff = [p.lower() for p in data.get("staff_ou_prefixes", [])] staff = [p.lower() for p in data.get("staff_ou_prefixes", [])]
logger.info("[ou-roles] Loaded %d student + %d staff OU prefixes from classification/google_ou_roles.json",
len(students), len(staff))
return students, staff return students, staff
except Exception: except FileNotFoundError:
logger.warning("[ou-roles] classification/google_ou_roles.json not found — using hardcoded fallback prefixes only")
return ["/elever", "/students"], ["/personale", "/staff", "/lærere", "/ansatte"]
except Exception as e:
logger.error("[ou-roles] Failed to load classification/google_ou_roles.json: %s — using hardcoded fallback prefixes only", e)
return ["/elever", "/students"], ["/personale", "/staff", "/lærere", "/ansatte"] return ["/elever", "/students"], ["/personale", "/staff", "/lærere", "/ansatte"]
def classify_ou_role(org_unit_path: str) -> str: def classify_ou_role(org_unit_path: str) -> str:
+2
View File
@@ -552,6 +552,8 @@ class M365Connector:
r.raise_for_status() r.raise_for_status()
return True # 204 No Content = success return True # 204 No Content = success
raise _requests.exceptions.RetryError(f"Gave up after {self._MAX_RETRIES} attempts: {url}") raise _requests.exceptions.RetryError(f"Gave up after {self._MAX_RETRIES} attempts: {url}")
def delete_message(self, user_id: str, message_id: str) -> bool:
"""Move an email to Deleted Items (soft delete).""" """Move an email to Deleted Items (soft delete)."""
base = "/me" if (not user_id or user_id == "me") else f"/users/{user_id}" base = "/me" if (not user_id or user_id == "me") else f"/users/{user_id}"
try: try:
+3
View File
@@ -114,6 +114,9 @@ def scan_start():
try: try:
run_scan(options) run_scan(options)
_maybe_send_auto_email() _maybe_send_auto_email()
except Exception:
_log.exception("[scan] background scan thread crashed")
_audit("scan_error", "background scan thread crashed", ip="")
finally: finally:
state._scan_lock.release() state._scan_lock.release()
threading.Thread(target=_run, daemon=True).start() threading.Thread(target=_run, daemon=True).start()
+1
View File
@@ -48,6 +48,7 @@ Never revert to `!!window._googleConnected` / `_fileSources.length > 0` — thos
- **Cache invalidation** — `invalidateHistoryCache()` clears `_sessions` and `_latestRefScanId`. All three `*_done` SSE handlers call `window.invalidateHistoryCache?.()`. - **Cache invalidation** — `invalidateHistoryCache()` clears `_sessions` and `_latestRefScanId`. All three `*_done` SSE handlers call `window.invalidateHistoryCache?.()`.
- **Re-scan diff** — items present in the previous session but absent from the current one are tagged `_resolved: true`, rendered with `.card-resolved` and a green ✓ badge, and NOT added to `S.flaggedData` (grid-only, cannot be bulk-selected or exported). - **Re-scan diff** — items present in the previous session but absent from the current one are tagged `_resolved: true`, rendered with `.card-resolved` and a green ✓ badge, and NOT added to `S.flaggedData` (grid-only, cannot be bulk-selected or exported).
- **Mode transitions** — `startScan()` calls `window.exitHistoryMode?.()` before clearing the grid. - **Mode transitions** — `startScan()` calls `window.exitHistoryMode?.()` before clearing the grid.
- **`renderGrid(files)` hides the landing cards** — whenever `files.length > 0` it hides `#emptyState` and `#lastScanSummary` and shows `#grid`. This is centralised here because the live `scan_file_flagged` handler (`scan.js`) shows the grid but does NOT clear those panels, so results would render *underneath* a still-visible landing/last-scan card until a manual refresh. Do not move this hiding back into individual callers — every render path (live SSE, `loadOpenItems`, history, filters) must clear the landing. The empty case (`files.length === 0`) is left untouched so callers still control the empty/landing state.
## Card user/group badge — results.js ## Card user/group badge — results.js
+11
View File
@@ -122,6 +122,17 @@ function renderGrid(files) {
const grid = document.getElementById('grid'); const grid = document.getElementById('grid');
grid.innerHTML = ''; grid.innerHTML = '';
files.forEach(f => appendCard(f)); files.forEach(f => appendCard(f));
// Whenever results are rendered, the landing/last-scan cards must be hidden —
// the live scan_file_flagged path shows the grid but does not clear them, so
// results would otherwise appear underneath the still-visible landing page
// until a manual refresh. Centralised here so every render path is covered.
if (files && files.length) {
const es = document.getElementById('emptyState');
if (es) es.style.display = 'none';
const ls = document.getElementById('lastScanSummary');
if (ls) ls.style.display = 'none';
if (grid) grid.style.display = S.isListView ? 'block' : 'grid';
}
_updateBulkBar(); _updateBulkBar();
updateDispositionStats(); updateDispositionStats();
} }