#+TITLE: youdis v2 architecture
#+DATE: [2026-03-31 Tue]
#+VERSION: 3

* Purpose
Youdis v2 splits the current monolithic Discord bot into:
- a private backend worker service that owns yt-dlp execution
- thin chat adapters that translate user actions into backend requests

The goal is to make downloader behavior inspectable, reusable, and easier to debug across multiple frontends without introducing queueing infrastructure or complex deployment.

** Goals
- Keep the backend transport-agnostic.
- Preserve archive-first duplicate prevention via `archive.txt`.
- Keep single-job semantics explicit.
- Treat requester and origin metadata as informational, not authorization.
- Make yt-dlp configuration understandable and debuggable.

** Non-Goals
- No backend auth or user membership logic.
- No database, Redis, or external queue.
- No multi-worker scheduling in v2.
- No frontend-owned downloader logic.

* Proposed Components
** Backend worker service
Owns:
- request validation
- single active job state
- yt-dlp executable invocation
- yt-dlp configuration selection and runtime argument construction
- archive behavior
- output path behavior
- progress and final result events

Does not own:
- Discord slash-command parsing
- chat formatting decisions
- access control policy

** Frontend adapters
1. Discord
2. Zulip
3. XMPP

Own:
- command parsing
- user-facing message formatting
- transport-specific reply behavior
- optional frontend-side gating if the deployment wants it

Do not own:
- yt-dlp option construction
- archive checks
- output path logic
- job lifecycle rules

* Initial Deployment Shape

V2 should start as a single private process boundary with two layers:
1. backend service process
2. one Discord adapter process

The backend is intended for private/local deployment. Trust comes from deployment topology and network placement, not backend auth.

** Execution model
The backend is a thin FastAPI wrapper around the `yt-dlp` executable, not a deep Python integration, unless proven necessary.

Why this is the default:
- keeps behavior closer to native `yt-dlp`
- makes container debugging easier because commands can be reproduced manually
- avoids reimplementing `yt-dlp` option parsing unless we truly need tighter integration

This means the backend is primarily responsible for:
- deciding when `yt-dlp` may run
- constructing a safe effective command
- supervising the subprocess while it is active
- translating process outcomes into a small API contract

** Backend Contract
*** Request model
Minimal fields:
- `url`
- `requester_id` optional
- `requester_name` optional
- `origin` optional
- `requested_at` optional
Only `url` affects downloader behavior in v2. The rest is passthrough metadata for logs and frontend context.

*** Result states
The backend should keep its external state model deliberately small:
- `accepted`
- `busy`
- `running`
- `completed`
- `failed`
- `cancelled`

These states should be transport-agnostic, human-debuggable, and only distinct when a frontend would behave differently.

Internal phases and details may be richer, but they should usually be carried as metadata rather than promoted to top-level states.

Examples of detail fields that can travel alongside the small state model:
- `phase=validating`
- `phase=downloading`
- `disposition=archive_hit`
- `message=already in archive`
- `result_path=/downloads/...`

This keeps the API simple while still leaving room for useful operator-facing detail.

*** Minimal endpoints
The first backend seam should stay small:
- `POST /jobs`
- `GET /jobs/current`
- `POST /jobs/current/cancel`
- `GET /health`
- `GET /version`

Polling is the default integration model for v2. Streaming or push delivery is deferred unless a real frontend need appears.

** Single-Job Semantics
V2 explicitly supports one active job at a time.
- If idle, the backend accepts a new job and marks it active.
- If busy, the backend rejects the new request with a `busy` result.
- Cancel is coarse and best-effort.
- The backend clears active state when the job reaches `completed`, `failed`, or `cancelled`.

This is a feature, not a limitation, for the initial service.

* yt-dlp Configuration Ownership

This is the most important v2 cleanup.

The backend owns all yt-dlp invocation behavior. Frontends must not build `yt-dlp` commands or option dictionaries.

** Config split

Static settings belong in `default-yt-dlp.conf`:
- archive path
- output template
- embedding and metadata preferences
- stable format defaults
- retry defaults

Runtime settings belong in backend code:
- target URL
- config-file path, if explicitly set at launch
- request-scoped flags or overrides
- subprocess lifecycle and cancellation behavior
- any values that must vary per request

*** Merge rule
The backend should invoke `yt-dlp` with the default config first, then apply a small set of explicit runtime overrides.

If a runtime override conflicts with file-backed config, the override wins and the backend should log the effective value used.

*** Debuggability requirement
For each job, the backend should make it easy to inspect:

- config file path used
- effective `yt-dlp` command or key override arguments
- final normalized job result

This exists specifically because the current config behavior has been difficult to reason about.

*** Config hygiene expectations

The default config should be safe for real downloads in production-like use.

That means test-only settings such as `--simulate` should not remain enabled in the default runtime path unless the backend intentionally supports an explicit dry-run mode.

** Recommended Repo Shape
One reasonable first cut:

#+begin_quote
youdis/
  README.md
  default-yt-dlp.conf
  youdis/
    __init__.py
    main.py
    models.py
    adapters/
      __init__.py
      discord.py
  docs/
    architecture-v2.org
#+end_quote

Notes:

- `main.py` owns the FastAPI app, active-job state, and `yt-dlp` subprocess lifecycle.
- `models.py` owns request and response models.
- `adapters/discord.py` becomes a thin client of the backend.

** Example response shape

One likely response shape for v1:

#+begin_src json
{
  "state": "completed",
  "disposition": "archive_hit",
  "message": "already in archive",
  "job_id": "abc123"
}
#+end_src

And for an active job:

#+begin_src json
{
  "state": "running",
  "phase": "downloading",
  "message": "downloading 3 of 9",
  "job_id": "abc123"
}
#+end_src

** Framework Recommendation

FastAPI is the current recommendation for the backend seam.

Why it fits:

- typed request and response models help define the contract cleanly
- built-in docs make local inspection easier during early iterations
- it stays small enough for this service if we keep the surface area disciplined

Why this is not a strong ideological choice:

- Flask would also work if we decide the type/model ergonomics are not worth it
- the important decision is keeping the seam small, not choosing a fashionable framework

** Immediate Next Task

`2.0.1` should implement the backend skeleton with just enough real behavior to prove the seam:

- package/module layout
- health and version endpoint
- single active-job state holder
- one submit-job path
- one current-job status path
- `yt-dlp` subprocess invocation using `default-yt-dlp.conf`
- explicit runtime overrides for request URL and cancel behavior

The Discord adapter should remain unchanged until that backend skeleton can accept a job and report status coherently.