# Course / program material files — protection, end to end

Covers **both repos**:

| Side | Repo | Path |
|---|---|---|
| Backend (Laravel) | `bateel` | `/home/karim/projects/bateel` |
| Frontend (Next.js) | `Bateel` monorepo, app `bateel-web` | `/home/karim/projects/bateel-platform/Bateel/apps/bateel-web` |

> A copy of this file lives in both repos (`docs/material-file-protection.md`).
> Keep them in sync.

---

## 1. Why this exists

Material files (PDFs, images, office files, …) attached to courses and training
programs were:

1. **Returned to guests.** The public course detail returned every lesson's
   materials with the full storage URL; the public program detail returned the
   program-level materials. Only session materials were gated.
2. **Stored on the public disk** at `/storage/uploads/file/<random-name>` —
   served directly by Apache, no check, valid forever, shareable.
3. **Visible in the browser.** The web app's `/api/client/...` proxy forwarded
   the backend JSON as-is, so the real storage URL appeared in the Network tab,
   and links pointed straight at the backend host.

The change has three independent layers. Each works on its own; together they
close the gap.

| Layer | Where | Effect | Needs rollout step? |
|---|---|---|---|
| **A. Enrolled-only visibility** | backend resources | Guests and non-enrolled users no longer receive material file URLs. | No — active on deploy. |
| **B. Same-origin links + server-side fetch** | frontend | The browser only ever sees `/materials/...`; the backend URL is resolved and fetched on the server. | No — active on deploy. |
| **C. Private storage + signed, access-checked stream** | backend | Files leave the public web root; every read is authorised, links expire, access is revocable. | **Yes** — opt-in via `MEDIA_MATERIAL_DISK` and a migration command. |

---

## 2. Vocabulary

| Term | Meaning |
|---|---|
| **Material** | A `CourseMaterial` or `ProgramMaterial` row. Its file is a `Media` row in collection `file`. Program *session* materials are `ProgramMaterial` rows with a `session_id`. |
| **Owner** | The course or program a material belongs to. |
| **Public disk** | Laravel disk `public` → `storage/app/public`, exposed via the `public/storage` symlink. Anyone with the URL can fetch the file. |
| **Private disk** | Laravel disk `local` → `storage/app/private`. Not web-reachable. |
| **Signed link** | `/api/v1/user/materials/{kind}/{uuid}/file?user=…&expires=…&signature=…` — a relative, time-limited, tamper-proof URL. |
| **Proxy link** | `/materials/{kind}/{ownerId}/{materialId}/{name}.{ext}` — the same-origin URL the web app shows to the user. |
| **Flag** | `MEDIA_MATERIAL_DISK` (backend env). Unset = public disk (legacy). `local` = private. |

---

## 3. Architecture at a glance

```
                    BROWSER (learner)
                         │  only ever sees  /materials/course/<id>/<id>/Guide.pdf
                         ▼
   ┌───────────────── Next.js (bateel-web) ─────────────────┐
   │  /api/client/[...path]   masks `file` in JSON          │  (Layer B)
   │  /materials/...          session + enrolment lookup,   │
   │                          fetches the file server-side  │
   └───────────────────────────┬────────────────────────────┘
                               │  Bearer token (JSON calls only)
                               ▼
   ┌───────────────── Laravel (bateel) ─────────────────────┐
   │  user/courses/{id}[/materials], user/programs/{id}     │  (Layer A)
   │     → `file` only for enrolled users                   │
   │  user/materials/{kind}/{uuid}/file  (signed:relative)  │  (Layer C)
   │     → re-checks access, streams from the private disk  │
   └───────────────────────────┬────────────────────────────┘
                               ▼
                 storage/app/private/uploads/file/…   (not web-reachable)
```

Other clients (mobile app, trainer dashboard) talk to the backend directly and
receive the **same `file` field**, which is now a signed link when the file is
private.

---

## 4. Backend (Laravel)

### 4.1 What was added / changed

| File | Change |
|---|---|
| `modules/Media/config/media.php` | `collection_disks` is now `array_filter([...])` with a new entry `'file' => env('MEDIA_MATERIAL_DISK')`. Unset ⇒ filtered out ⇒ no behaviour change. |
| `config/materials.php` (new) | `file_url_minutes` ← `MATERIAL_FILE_URL_MINUTES` (default **15**). |
| `.env.example` | Documents `MEDIA_MATERIAL_DISK` and `MATERIAL_FILE_URL_MINUTES` (commented). |
| `app/Providers/AppServiceProvider.php` | Gate `upload-private-media` now receives the collection. Admins/supervisors: any private collection. **Trainers: only `file`.** |
| `modules/Media/src/Http/Requests/Media/MediaUploadRequest.php` | Passes the collection to the gate. |
| `modules/Media/src/Services/FileUploadService.php` | Private collections skip `ImageProcessor` (it writes to the public disk and would re-publish a protected image). |
| `app/Models/CourseMaterial.php`, `ProgramMaterial.php` | `getAttribute('file')` → `MaterialFileService::url()`. |
| `app/Services/Education/MaterialFileService.php` (new) | `url()`, `canRead()`, `stream()`, `isPrivate()`. |
| `app/Http/Controllers/Api/V1/Client/MaterialFileController.php` (new) | Thin controller for the stream. |
| `routes/api/v1/user.php` | New route `user.materials.file`. |
| `app/Http/Resources/Client/TrainingProgram/TrainingProgramDetailsResource.php` | `materials` only when `is_enrolled`. |
| `app/Http/Resources/User/Course/CourseCardResource.php` | Lesson `materials` is `[]` unless `is_enrolled`. |
| `app/Console/Commands/PrivatizeMaterialFiles.php` (new) | `media:privatize-materials` migration command. |
| `tests/Feature/Education/MaterialFileProtectionTest.php` (new) | 16 tests. |
| `tests/Feature/Education/CourseAttachmentTest.php` | Guest test updated to the new rule. |

### 4.2 The `file` field — four states

`MaterialFileService::url($material)` decides what `$material->file` returns
(this is what every resource — learner, trainer, admin — outputs):

| Media state | Caller | `file` value |
|---|---|---|
| No media attached | anyone | placeholder, as before |
| Stored on **public** disk | anyone | plain `…/storage/uploads/file/…` URL, as before |
| Stored on a **private** disk | signed-in (any role) | **signed link**, valid `MATERIAL_FILE_URL_MINUTES` |
| Stored on a **private** disk | guest | `null` |

> The signed link is issued without an access check (that keeps list endpoints
> to zero extra queries). It is harmless to hand out because the stream
> authorises again — see 4.4.

### 4.3 Layer A — enrolled-only visibility

| Endpoint | Guest / not enrolled | Enrolled |
|---|---|---|
| `GET user/courses/{course}` | `lessons[].materials = []` | materials with `file` |
| `GET user/programs/{program}` | no `materials` key | `materials[]` with `file` |
| `GET user/programs/{program}` → `sessions[].materials` | omitted (unchanged) | present |
| `GET user/courses/{course}/materials` | 403 (unchanged) | list |

"Enrolled" = the existing `is_enrolled` flag the controllers already set.

### 4.4 Layer C — the signed stream

`GET /api/v1/user/materials/{kind}/{material}/file`

* `kind` ∈ `course | program` (route constraint), `material` = material UUID.
* Middleware: `signed:relative`, `throttle:60,1`. **No bearer token** — the
  signature carries the material and the viewer (`user=<uuid>`).
* Invalid or expired signature → **404** (the app's exception handler maps
  `InvalidSignatureException` to 404 for every signed route, same as the book
  reader).

Per-request checks, in order:

1. The `user` UUID must exist and be active (`isInactiveUser`, role not inactive) → else **403**.
2. Material must exist for that `kind` → else **403**.
3. `canRead(user, material)` → else **403**:

   | Viewer | Allowed when |
   |---|---|
   | Super admin / supervisor | always |
   | Trainer | owns the course/program (`owner.trainer_id` = their `TrainerProfile`) |
   | Learner, course material | enrollment with status `active` or `completed` |
   | Learner, program material | any enrollment row for the program |

4. Media row and file must exist → else **404**.

Response: `BinaryFileResponse` (answers `Range` → PDF viewers can seek).

| Header | Value |
|---|---|
| `Content-Type` | the stored mime type if it is `application/pdf`, `image/png`, `image/jpeg` or `image/webp`; otherwise `application/octet-stream` |
| `Content-Disposition` | `inline` for those four types, `attachment` otherwise; filename = sanitised original name |
| `X-Content-Type-Options` | `nosniff` |
| `Cache-Control` | `private, no-store` |

Because access is re-checked on **every** request, removing/cancelling an
enrollment stops an already-issued link immediately.

### 4.5 Upload flow (trainer)

```
Trainer UI ──POST /media (multipart: file, collection=file)──► MediaUploadController
   MediaUploadRequest::authorize()
      collection in media.collection_disks ?   (only true when flag is set)
         no  → allowed (legacy, public disk)
         yes → Gate upload-private-media($user, 'file')
                 admin/supervisor → ok · trainer → ok (collection 'file') · learner → 403
   FileUploadService::upload()
      disk = media.collection_disks.file ?? media.disk ('public')
      private collection → skip ImageProcessor (stored as-is)
      Media row: disk, path, hash, mime, size, collection='file'
Trainer UI ──POST …/materials { file: <hash>, … }──► attaches the Media by hash
```

A file's disk is fixed at upload time; attaching later never moves it. That is
why existing files need the migration command (4.6).

### 4.6 Migration command

```bash
php artisan media:privatize-materials             # dry run (default)
php artisan media:privatize-materials --force     # apply
php artisan media:privatize-materials --force --to=local
```

* Considers `Media` rows with `collection = 'file'`, `disk = public`, owned by a
  `CourseMaterial` / `ProgramMaterial`.
* **Dry run** prints `would move  #id path` and a summary; touches nothing.
* **`--force`**, per file: verify every path (main file, variants, thumbnail)
  exists → copy to the private disk → verify size → set `media.disk` → delete
  the public copy. A failed copy is rolled back and the row is left unchanged.
* **Skips videos** (their chunked upload path stays public) and files missing on
  the source.
* **Idempotent / resumable** — only files still on the public disk are
  selected; iterates with `lazyById` so rows leaving the result set don't cause
  skips.
* Refuses a target that is the public disk or a non-`local` driver
  (`--to=public` → failure).
* Exit code `0` on success, non-zero if any file failed.

### 4.7 Backend configuration reference

| Variable | Default | Purpose |
|---|---|---|
| `MEDIA_MATERIAL_DISK` | unset | `local` ⇒ new `file` uploads go private. Unset ⇒ public (legacy). |
| `MATERIAL_FILE_URL_MINUTES` | `15` | Lifetime of a signed link. |
| `APP_URL` | — | Host used inside signed links. **Must be the same host as the web app's `API_BASE_URL`** (the web route only fetches from that host). |

---

## 5. Frontend (Next.js `bateel-web`)

### 5.1 What was added / changed

| File | Change |
|---|---|
| `lib/material-url.ts` (new) | `materialProxyUrl(owner, material)`, `materialFileName()`, `MaterialOwner` type. Client-safe. |
| `lib/server/material-links.ts` (new) | `materialOwnerFromPath()`, `maskMaterialFiles()`. Server-side logic used by the client proxy. |
| `app/api/client/[...path]/route.ts` | After the backend responds, rewrites material `file` fields for logged-in learners. |
| `app/materials/[kind]/[ownerId]/[materialId]/[filename]/route.ts` (new) | The `/materials/...` handler: auth, enrollment lookup, server-side fetch, streaming. |
| `components/profile/reference-file-row.tsx` | New required `owner` prop; link uses `materialProxyUrl`. |
| `components/profile/training-page.tsx`, `education-page.tsx` | Pass `owner` to rows, session chips and sidebar rows. `LiveCoursePathSidebar` takes `courseId`. |
| `turbo.json` (repo root) | Declares `MATERIAL_ALLOWED_HOSTS`. |
| Tests | `lib/material-url.test.ts`, `lib/server/material-links.test.ts`, `app/materials/.../route.test.ts`, "material file masking" in `app/api/client/[...path]/route.test.ts`. |

### 5.2 Proxy link format

```
/materials/{course|program}/{ownerId}/{materialId}/{title}.{ext}
```

* `ownerId` is required because the backend authorises per course/program;
  there is no per-material endpoint.
* `{title}.{ext}` is **cosmetic** (what the user sees / saves as). Authorisation
  never depends on it.
* Extension: from the backend file path, else the declared `material_type`,
  else `pdf`. Titles keep Arabic characters; path separators and control
  characters are replaced with `-`; max 80 characters.
* The route lives outside `[locale]`. `proxy.ts` excludes paths containing a
  `.`, so the i18n middleware never touches it.

### 5.3 JSON masking (`/api/client/...`)

`maskMaterialFiles` replaces `file` with the proxy link, only when **all** of:

* the request is a `GET`, the backend answered `ok`, and the body is JSON;
* the caller has a valid token and role `user`;
* the path matches `courses/{id}`, `courses/{id}/materials` or `programs/{id}`
  (optional `user/` prefix);
* the object looks like a material: has `id`, `material_type` (string) and a
  non-empty `file` string. Other `file`/URL fields (cover images, …) are left
  alone.

If the backend body is not valid JSON, the proxy returns 502 instead of leaking
the raw body.

### 5.4 The `/materials/...` route, step by step

```
1  Validate   kind ∈ {course, program}; ownerId/materialId match ^[\w-]{1,64}$  → else 404
2  Auth       next-auth JWT: role "user", has accessToken, no error, not expired → else 401
3  Lookup     course : GET user/courses/{id}/materials   (403/404 from backend pass through)
                        not found there → GET user/courses/{id} (session handouts)
              program: GET user/programs/{id}; require data.is_enrolled === true → else 403
              The material is located by `id` anywhere in the payload → else 404
4  Validate   file URL must be http(s) and its host ∈ { API_BASE_URL host } ∪ MATERIAL_ALLOWED_HOSTS → else 502
5  Fetch      GET file URL, forwarding only `Range`; NO Authorization header;
              cache: no-store; redirects: error
              upstream 416 → 416, other non-2xx → 502
6  Respond    status 200/206; body streamed
```

Response headers:

| Header | Value |
|---|---|
| `Content-Type` | upstream type if inline-safe (`application/pdf`, `image/png|jpeg|webp`, `video/mp4|webm|quicktime`) or download (`text/csv`, `application/vnd.ms-excel`, `…spreadsheetml.sheet`); anything else → `application/octet-stream` |
| `Content-Disposition` | `inline` for inline-safe types, else `attachment`; `filename*=UTF-8''<name from the URL>` |
| `Accept-Ranges`, `Content-Length`, `Content-Range` | passed through from upstream |
| `Cache-Control` | `private, no-store` |
| `X-Content-Type-Options` / `X-Robots-Tag` | `nosniff` / `noindex, nofollow` |

HTML/SVG/anything unexpected is **never** rendered inline from our origin.

When the backend file is private, step 3 returns the freshly signed link, and
step 5 fetches it (the signature is the credential — which is why no
`Authorization` header is sent). The browser never receives that signed link.

### 5.5 Where the links are used

| Screen | Component | Owner |
|---|---|---|
| Course "References" tab | `reference-file-row.tsx` via `EducationReferences` | course |
| Live course "References" tab | `reference-file-row.tsx` | course |
| Program "References" tab | `ApiReferencesPane` → `reference-file-row.tsx` | program |
| Session chips (program sessions pane, sidebar rows) | `ApiSessionsPane`, `CourseLessonRow` | program |
| Session chips (live-course sidebar/sessions) | `CourseLessonRow` | course |

### 5.6 Frontend configuration reference

| Variable | Purpose |
|---|---|
| `API_BASE_URL` | Backend API base. Its host is the only host `/materials` will fetch from. |
| `MATERIAL_ALLOWED_HOSTS` | Optional, comma-separated extra hosts (e.g. a CDN). |

---

## 6. Lifecycles

### 6.1 Trainer uploads a material

```mermaid
sequenceDiagram
  participant T as Trainer UI
  participant N as Next (/api/client)
  participant B as Laravel
  T->>N: POST media (file, collection=file)
  N->>B: POST /media (trainer token)
  B->>B: Gate upload-private-media($user,'file') — only if flag set
  B->>B: store on media.collection_disks.file ?? public
  B-->>T: { hash, … }
  T->>B: POST …/materials { file: hash }
  B-->>T: material (file = plain URL or signed link, by disk)
```

* Flag **off** → stored public, exactly as before.
* Flag **on** → stored on `local`; images are not re-encoded.

### 6.2 Learner opens the references tab

```mermaid
sequenceDiagram
  participant U as Learner browser
  participant N as Next /api/client
  participant B as Laravel
  U->>N: GET /api/client/courses/{id}/materials
  N->>B: GET user/courses/{id}/materials (Bearer)
  B-->>N: [{ id, title, material_type, file: <url or signed link> }]
  N->>N: maskMaterialFiles → file = /materials/course/{id}/{mid}/Guide.pdf
  N-->>U: JSON with same-origin links only
```

### 6.3 Learner clicks a PDF

```mermaid
sequenceDiagram
  participant U as Learner browser
  participant M as Next /materials/...
  participant B as Laravel
  U->>M: GET /materials/course/{id}/{mid}/Guide.pdf (session cookie)
  M->>M: check session (role user)
  M->>B: GET user/courses/{id}/materials (Bearer)  → enrolment gate
  B-->>M: list incl. file = signed link (private) or public URL
  M->>M: find material, validate host
  M->>B: GET <file url> (Range if sent, no auth header)
  B->>B: signature + canRead(user, material)  [private files]
  B-->>M: 200/206 bytes
  M-->>U: bytes, inline, no-store
```

### 6.4 Guest visits a public course / program page

Backend returns `materials: []` (course lessons) / omits `materials` (program).
The public Next pages already tolerate both, so the materials list simply does
not render download links. Titles/counts for guests must come from other fields.

### 6.5 Trainer / admin / mobile client

* Call the backend directly; `file` is a signed link (private) or plain URL
  (public). The signed link works for the **owning trainer** and admins.
* **Do not cache or store** the link — it expires. Re-fetch the resource to get
  a fresh one.
* The web `/materials/...` route is for role `user` only; the trainer dashboard
  still opens `file` directly (works until the link expires — a page left open
  longer than `MATERIAL_FILE_URL_MINUTES` needs a reload).

### 6.6 Enrollment ends

| Moment | Result |
|---|---|
| Next `/materials` request | Backend list endpoint returns 403 → 403 |
| Direct hit on an already issued signed link | `canRead` fails → 403 |
| URL a guest copied **before** this change, files not yet migrated | **Still works** (file still public) — only moving the files kills it |
| Same URL after files are moved | 404 |

### 6.7 Migrating existing files

```
backup storage/app/public/uploads/file
   → php artisan media:privatize-materials          (read the list)
   → php artisan media:privatize-materials --force  (apply)
   → verify: learner opens PDFs · non-enrolled gets 403 · old /storage/... URL 404s
```

---

## 7. State matrix

| `MEDIA_MATERIAL_DISK` | Files | Guests get URL? | Browser sees backend URL? | Old direct URLs | Revocable? |
|---|---|---|---|---|---|
| unset | public | No (Layer A) | No on web profile pages (Layer B); yes for mobile/trainer | **valid** | No |
| `local` | still public (not migrated) | No | same | **valid** | No |
| `local` | **migrated** | No | No on web; mobile/trainer get a 15-min signed link | **dead** | **Yes** |

---

## 8. Rollout

Both repos work with either version of the other, so deploy order is flexible.

**Backend**

1. Back up the database and `storage/app/public/uploads/file`.
2. Deploy (no new packages, no migrations).
3. `php artisan config:clear` → `config:cache` / `route:cache` if used; restart PHP-FPM and workers.
4. `php artisan route:list --name=materials.file` shows the route.
5. Guest gating check: public course → lesson `materials: []`; public program → no `materials`.
6. `APP_URL` host == web `API_BASE_URL` host.

**Frontend**

7. Build + deploy with `API_BASE_URL` set (and `MATERIAL_ALLOWED_HOSTS` if needed).
8. Smoke test as an enrolled learner: address bar `/materials/...`, PDF opens, Network tab shows no backend storage URL.

**Private storage (separate, later)**

9. `MEDIA_MATERIAL_DISK=local` → `config:cache`.
10. `media:privatize-materials` (dry run) → backup confirmed → `--force`.
11. Re-test: enrolled learner ✓, non-enrolled 403, trainer upload ✓, owner trainer opens file ✓.
12. Optional hard block at the web server (see 10) — **only after** step 10.

### Rollback

* Layers A/B: redeploy the previous version.
* Layer C: unsetting `MEDIA_MATERIAL_DISK` affects **new** uploads only. For
  already moved files, copy them back to `storage/app/public/<path>` and set
  `media.disk = 'public'` on those rows, or restore the backup.

---

## 9. Verification commands

```bash
# Backend tests (in-memory SQLite — safe)
php artisan test tests/Feature/Education/MaterialFileProtectionTest.php
php artisan test tests/Feature/Education

# Frontend tests / types (from apps/bateel-web)
npx vitest run
npx tsc --noEmit

# Guest cannot see materials
curl -s https://API/api/v1/user/programs/<uuid> | jq '.data | has("materials")'   # false

# Unauthenticated web route
curl -s -o /dev/null -w "%{http_code}\n" https://WEB/materials/course/x/y/z.pdf     # 401
```

Known failures that exist **without** this change (do not block the rollout):
backend — top sellers, admin destroy/bulk-delete program with reviews, address
snapshot, search suggestions, program-details language; frontend — layout CSS
layer test, consultation join window.

---

## 10. Optional hard block of the old public path

Middleware cannot do this: `public/storage` is a symlink and Apache serves
existing files **before** PHP runs, so Laravel never sees those requests. Use
`public/.htaccess` (versioned with the code) **after** the migration:

```apache
RewriteRule ^storage/uploads/file/.*\.(pdf|jpe?g|png|webp|docx?|pptx?|xlsx?|zip)$ - [NC,F,L]
```

* The extension list leaves video materials reachable (still public).
* Enabling this before the migration breaks every PDF.
* After migration the block is redundant but harmless: Laravel's built-in
  `storage/{path}` route already 404s unsigned requests for private disks.

---

## 11. Troubleshooting

| Symptom | Likely cause |
|---|---|
| `/materials/...` → **401** | Not logged in, expired token, or logged in as a trainer (route is learner-only). |
| → **403** | Not enrolled (course: backend 403; program: `is_enrolled` false), or signed link issued to a user who is no longer enrolled. |
| → **404** | Bad `kind`/id, material id not in that owner's payload, or invalid/expired signature (the app maps those to 404). |
| → **502** | File host not allowed (`APP_URL` host ≠ `API_BASE_URL` host and not in `MATERIAL_ALLOWED_HOSTS`), upstream error, or redirect. |
| PDF downloads instead of opening | Stored mime type is not `application/pdf`/image (check `media.mime_type`). |
| Trainer "open" link 404s | Signed link older than `MATERIAL_FILE_URL_MINUTES`; reload the page. |
| Trainer upload 403 after enabling the flag | Gate not deployed, or the upload is not in collection `file`. |
| `file` is `null` for a signed-in user | Material is private and the request carried no valid bearer token. |
| Guests see no materials on public pages | Intended (Layer A). |

---

## 12. Security properties and honest limits

**Gained**

* No material URL for guests or non-enrolled users.
* Web users never see a storage or signed URL.
* After migration: files unreachable without a valid, unexpired signature **and** a passing access check on every request; revocation is immediate; old URLs are dead.
* Served types are allow-listed; active content (HTML/SVG) is never rendered from our origin.
* Server-side fetch is host-allow-listed and does not follow redirects or forward credentials.

**Not prevented**

* An **enrolled learner can still download and re-share** the file. Mitigation is outside this feature (e.g. per-buyer watermarking).
* A signed link can be passed on and used by anyone for up to `MATERIAL_FILE_URL_MINUTES` — access is still evaluated for the learner it was issued to, so it stops when their access does.
* URLs a guest **already obtained** stay valid until the files are migrated.

**Not covered**

* **Video materials** (chunked upload ignores collection rules; stay public).
* Remote private disks (S3): the stream needs a local disk, as for books.
* Public catalog pages and the trainer dashboard still read `file` directly on the web side (guests get nothing sensitive after Layer A; trainers get a signed link).
* Session videos play from their direct URL (`VideoAttachment`).
* **Unverified:** whether live-course (non-recorded) sessions have their own attachment model. Program sessions use `ProgramMaterial` (covered).
* Digital books / research papers (`digital_file`) were **already private** and use the book reader's signed stream — untouched.
