Initial commit
This commit is contained in:
Henrik Højmark committed
commit
9c7df76fbd
91 files changed
+36572
No files matched your search
@@ -0,0 +1,144 @@
|
||||
# Google Workspace Setup
|
||||
|
||||
Step-by-step guide for connecting GDPRScanner to Google Workspace via a service account.
|
||||
|
||||
GDPRScanner connects using a **service account** with **domain-wide delegation** — this allows it to scan all users' Gmail and Drive without requiring each user to sign in individually.
|
||||
|
||||
---
|
||||
|
||||
## 1. Create a Google Cloud project
|
||||
|
||||
Go to [console.cloud.google.com](https://console.cloud.google.com) and create a new project (or use an existing one).
|
||||
|
||||
---
|
||||
|
||||
## 2. Enable the required APIs
|
||||
|
||||
In your project: **APIs & Services → Enable APIs and Services**. Enable:
|
||||
|
||||
- **Gmail API**
|
||||
- **Google Drive API**
|
||||
- **Admin SDK API**
|
||||
|
||||
---
|
||||
|
||||
## 3. Create a service account
|
||||
|
||||
Go to **IAM & Admin → Service accounts → Create service account**.
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Name | gdprscanner (or any name) |
|
||||
| Description | GDPRScanner service account |
|
||||
|
||||
Click **Create and continue**. Skip the optional role and user access steps. Click **Done**.
|
||||
|
||||
### Create a key
|
||||
|
||||
Click on the service account → **Keys → Add key → Create new key → JSON**.
|
||||
|
||||
Download the JSON file. This is your service account key — treat it like a password.
|
||||
|
||||
---
|
||||
|
||||
## 4. Enable domain-wide delegation
|
||||
|
||||
Back on the service account page: **Show advanced settings → Domain-wide delegation → Enable**.
|
||||
|
||||
Note the **Client ID** (a long number) — you'll need it in the next step.
|
||||
|
||||
---
|
||||
|
||||
## 5. Authorise scopes in Google Admin Console
|
||||
|
||||
Go to [admin.google.com](https://admin.google.com) →
|
||||
**Security → Access and data control → API controls → Manage domain-wide delegation → Add new**.
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Client ID | The numeric Client ID from the service account |
|
||||
| OAuth scopes | See below |
|
||||
|
||||
Add all of these scopes (paste as a comma-separated list):
|
||||
|
||||
```
|
||||
https://www.googleapis.com/auth/admin.directory.user.readonly,
|
||||
https://www.googleapis.com/auth/gmail.readonly,
|
||||
https://www.googleapis.com/auth/drive.readonly
|
||||
```
|
||||
|
||||
Click **Authorise**. Changes can take a few minutes to propagate.
|
||||
|
||||
---
|
||||
|
||||
## 6. Connect in GDPRScanner
|
||||
|
||||
Open GDPRScanner → **Source Management → Google Workspace** tab.
|
||||
|
||||
1. **Upload service account key** — select the JSON file you downloaded in step 3
|
||||
2. **Admin email** — enter the email address of a Google Workspace admin user in your domain (e.g. `admin@skolen.dk`). The service account impersonates this user to call the Admin Directory API.
|
||||
|
||||
Click **Connect**. If successful, the status dot turns green and shows the service account email.
|
||||
|
||||
---
|
||||
|
||||
## 7. User role classification
|
||||
|
||||
GDPRScanner classifies Google Workspace users as **staff** or **student** based on their **Organisational Unit (OU) path** in Google Admin.
|
||||
|
||||
The mapping is in `classification/google_ou_roles.json`. Edit it to match your school's OU structure — no code change required.
|
||||
|
||||
Default mapping:
|
||||
|
||||
| OU prefix | Role |
|
||||
|---|---|
|
||||
| `/Elever` | student |
|
||||
| `/Personale` | staff |
|
||||
| `/Admin` | staff |
|
||||
|
||||
To see your OU structure: **Google Admin → Directory → Administrer organisationsenheder**.
|
||||
|
||||
Example `classification/google_ou_roles.json` for a typical Danish school (Gudenaaskolen.dk structure):
|
||||
|
||||
```json
|
||||
{
|
||||
"student_ou_prefixes": ["/Elever"],
|
||||
"staff_ou_prefixes": ["/Personale", "/Admin"]
|
||||
}
|
||||
```
|
||||
|
||||
After editing the file, restart GDPRScanner — no rebuild required.
|
||||
|
||||
---
|
||||
|
||||
## 8. Verify
|
||||
|
||||
After connecting:
|
||||
|
||||
- **Sources panel** shows Gmail and Google Drive checkboxes
|
||||
- **Accounts panel** shows all Google Workspace users with `GWS` badges
|
||||
- Users are classified as Elev / Ansat based on their OU
|
||||
|
||||
Select one or more accounts, check Gmail and/or Google Drive, and click Scan.
|
||||
|
||||
---
|
||||
|
||||
## Notes on what is scanned
|
||||
|
||||
| Source | What is scanned |
|
||||
|---|---|
|
||||
| Gmail | Email bodies and attachments for all mail folders |
|
||||
| Google Drive | My Drive files — Docs, Sheets, Slides are auto-exported to text for scanning |
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Symptom | Likely cause |
|
||||
|---|---|
|
||||
| `unauthorized_client` on connect | Domain-wide delegation not enabled, or scopes not authorised in Admin Console |
|
||||
| 0 users listed | `admin.directory.user.readonly` scope missing, or wrong admin email |
|
||||
| Users show as "Anden" (other) | OU path not matched in `classification/google_ou_roles.json` — check OU paths in Google Admin and compare with the file |
|
||||
| Gmail scan finds nothing | `gmail.readonly` scope not authorised |
|
||||
| Drive scan finds nothing | `drive.readonly` scope not authorised |
|
||||
| `RefreshError` on scan | Service account key expired or revoked — generate a new key |
|
||||
@@ -0,0 +1,160 @@
|
||||
# Microsoft 365 Setup
|
||||
|
||||
Step-by-step guide for connecting GDPRScanner to Microsoft 365.
|
||||
|
||||
---
|
||||
|
||||
## 1. Register an app in Azure
|
||||
|
||||
Go to **Azure Portal → Microsoft Entra ID → App registrations → New registration**.
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Name | GDPRScanner (or any name) |
|
||||
| Supported account types | Accounts in this organisational directory only |
|
||||
| Redirect URI | Leave blank |
|
||||
|
||||
Click **Register**. Note the **Application (client) ID** and **Directory (tenant) ID** — you'll need both.
|
||||
|
||||
---
|
||||
|
||||
## 2. Choose an authentication mode
|
||||
|
||||
| Mode | How it works | When to use |
|
||||
|---|---|---|
|
||||
| **Application** | Client credentials — client ID + tenant ID + client secret. No user interaction. | Automated / scheduled scans, all-user scans |
|
||||
| **Delegated** | OAuth device code flow — user signs in interactively. | Single-user scans, testing |
|
||||
|
||||
### Application mode — create a client secret
|
||||
|
||||
In your app registration: **Certificates & secrets → New client secret**.
|
||||
|
||||
Set an expiry (24 months recommended) and copy the **Value** immediately — it is only shown once.
|
||||
|
||||
### Delegated mode — no secret needed
|
||||
|
||||
The scanner will show a device code URL. Open it in a browser, sign in, and the scanner authenticates as that user.
|
||||
|
||||
---
|
||||
|
||||
## 3. Add API permissions
|
||||
|
||||
Go to **API permissions → Add a permission → Microsoft Graph**.
|
||||
|
||||
### Scan only
|
||||
|
||||
| Permission | Type |
|
||||
|---|---|
|
||||
| `Mail.Read` | Application or Delegated |
|
||||
| `Files.Read.All` | Application or Delegated |
|
||||
| `Sites.Read.All` | Application or Delegated |
|
||||
| `ChannelMessage.Read.All` | Application |
|
||||
| `Team.ReadBasic.All` | Application |
|
||||
| `User.Read.All` | Application |
|
||||
|
||||
### Scan + Delete
|
||||
|
||||
Add these in addition to the read permissions above:
|
||||
|
||||
| Permission | Type |
|
||||
|---|---|
|
||||
| `Mail.ReadWrite` | Application or Delegated |
|
||||
| `Files.ReadWrite.All` | Application or Delegated |
|
||||
| `Sites.ReadWrite.All` | Application or Delegated |
|
||||
|
||||
### Email reports via Graph
|
||||
|
||||
If you want the scanner to send email reports via Microsoft 365 (not SMTP):
|
||||
|
||||
| Permission | Type |
|
||||
|---|---|
|
||||
| `Mail.Send` | Application or Delegated |
|
||||
|
||||
### Grant admin consent
|
||||
|
||||
All **Application** permissions require admin consent. Click **Grant admin consent for [your tenant]** at the top of the API permissions page. Without this, scans will fail with 403 errors.
|
||||
|
||||
---
|
||||
|
||||
## 4. Connect in GDPRScanner
|
||||
|
||||
Open GDPRScanner → **Source Management → Microsoft 365** tab.
|
||||
|
||||
| Field | Where to find it |
|
||||
|---|---|
|
||||
| Client ID | App registration → Overview → Application (client) ID |
|
||||
| Tenant ID | App registration → Overview → Directory (tenant) ID |
|
||||
| Client Secret | The value you copied in step 2 (Application mode only) |
|
||||
|
||||
Click **Connect**. In Application mode, the connection is immediate. In Delegated mode, a browser window opens for sign-in.
|
||||
|
||||
---
|
||||
|
||||
## 5. Verify
|
||||
|
||||
After connecting, the Sources panel shows:
|
||||
|
||||
- **Email** — Exchange mailboxes
|
||||
- **OneDrive** — personal drives
|
||||
- **SharePoint** — site file libraries
|
||||
- **Teams** — Teams channel files
|
||||
|
||||
The Accounts panel lists all users in the tenant (Application mode) or just the signed-in user (Delegated mode).
|
||||
|
||||
---
|
||||
|
||||
## Notes on deletion
|
||||
|
||||
Emails deleted via the scanner are moved to **Deleted Items** — recoverable for 14–30 days depending on admin configuration. Files are sent to the **OneDrive/SharePoint recycle bin** — retained for 93 days across both recycle bin stages before permanent deletion. Nothing is permanently destroyed without a second manual step.
|
||||
|
||||
---
|
||||
|
||||
## Headless / scheduled mode
|
||||
|
||||
Headless mode uses Application auth only. Credentials are read in priority order:
|
||||
|
||||
1. `--settings FILE` — a JSON file you provide
|
||||
2. Environment variables: `M365_CLIENT_ID`, `M365_TENANT_ID`, `M365_CLIENT_SECRET`
|
||||
|
||||
Example settings file:
|
||||
|
||||
```json
|
||||
{
|
||||
"client_id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
|
||||
"tenant_id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
|
||||
"client_secret": "your-secret",
|
||||
"sources": ["email", "onedrive"],
|
||||
"options": {
|
||||
"older_than_days": 365,
|
||||
"email_body": true,
|
||||
"attachments": true,
|
||||
"delta": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
python gdpr_scanner.py --headless --output ~/Reports/ --settings settings.json
|
||||
```
|
||||
|
||||
See the full CLI flag reference in `README.md`.
|
||||
|
||||
---
|
||||
|
||||
## Role classification (staff / student)
|
||||
|
||||
GDPRScanner classifies users as **staff** or **student** based on their Microsoft 365 licence SKU. The mapping is in `classification/m365_skus.json`. If users appear as "other", open **Settings → SKU debug** to see which SKU IDs are assigned in your tenant and add any missing ones to `m365_skus.json`.
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Symptom | Likely cause |
|
||||
|---|---|
|
||||
| 403 on scan start | Admin consent not granted, or wrong permissions added |
|
||||
| `AADSTS7000215` | Invalid client secret — check it was copied correctly |
|
||||
| No users listed | `User.Read.All` permission missing or not consented |
|
||||
| Teams files not appearing | `ChannelMessage.Read.All` or `Team.ReadBasic.All` missing |
|
||||
| Delta scan not working | Delta tokens require at least one full scan first |
|
||||
Reference in new issue
Block a user