ben/youdis
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
309ce87abbc6cf7ae7fed21c2854a137a63ea9ac
youdis/pm/tasks-v2.org

17 KiB
Raw Blame History

Task Log

youdis v2 goals:

  1. Separate backend from frontend
  2. Offload auth
  3. Ensure auto nightly builds
  4. Default output format supports plex browsing youtube channels as "tv shows"
  5. Facilitate multiple GUI inbounds: Discord, Zulip, XMPP

[X] 2.0.0: define architecture (2-4)

define the target architecture for a private backend yt-dlp worker with thin chat frontends

pm notes:

  • keep this iterative. the point is to choose the shape and seam, not prematurely implement infra. likely decisions include backend framework, request/status model, and how thin the discord shim should be.
  • goal is simple, private, maintainable deployment
  • avoid auth, queues, or persistence beyond clear & immediate needs

Acceptance Criteria

  1. document the target architecture at a high level

    • backend owns yt-dlp execution and job state
    • frontends own chat-specific UX

  2. identify key decisions still open

    • backend choice
    • service seam/endpoints
    • status/progress model

  3. capture enough structure to begin implementation

    • repo/component layout is sketched
    • next implementation task is unblocked

evidence

  • commit: 0ed16ec
  • tests: n/a
  • datetime: [2026-03-31 Tue 18:48]

notes

  • first architecture draft captured in `docs/architecture-v2.org`

[X] 2.0.1: build backend yt-dlp worker (3)

create the minimal backend/service skeleton and establish a working yt-dlp baseline with clean hooks for future frontends

pm notes

  • foundation; don't need the full finished service here, just the basic shape plus enough real yt-dlp execution to validate the seam and build on it.
  • keep single-job semantics
  • prioritize inspectable behavior over polish

Acceptance Criteria

  1. create the backend/service skeleton

    • app/module layout exists
    • core request path is stubbed or minimally working

  2. establish a working yt-dlp baseline

    • archive behavior is preserved
    • output path behavior is preserved or intentionally updated
    • use yt-dlp .conf and set reasonable default

  3. expose basic hooks/interfaces for future frontends

    • submit/request path exists
    • status/progress hook exists
    • basic health/version visibility exists

  4. make local testing practical without breaking container defaults

    • backend can run when `/config` or `/downloads` are not writable in local dev
    • env vars may override executable/config/download paths
    • status makes the effective runtime command/paths inspectable

evidence

{"status":"ok"} user@paladin:~/proj/youdis$ curl http://127.0.0.1:8000/version {"version":"20250829-ec72c56","active_job":false} user@paladin:~/proj/youdis$ curl -X POST http://127.0.0.1:8000/jobs -H 'content-type: application/json' -d '{"url":"https://www.youtube.com/watch?v=3i72yY_LaW4"}' {"job_id":"cc85165e-d906-4eee-864f-1a398b6de2e0","state":"accepted","url":"https://www.youtube.com/watch?v=3i72yY_LaW4","message":"accepted","phase":"queued","disposition":null,"requester_id":null,"requester_name":null,"origin":null,"result_path":null,"command":[],"returncode":null,"created_at":"2026-04-01T00:44:35.657196","updated_at":"2026-04-01T00:44:35.657198"} user@paladin:~/proj/youdis$ curl http://127.0.0.1:8000/jobs/current {"active":false,"job":{"job_id":"cc85165e-d906-4eee-864f-1a398b6de2e0","state":"failed","url":"https://www.youtube.com/watch?v=3i72yY_LaW4","message":"ERROR: unable to create directory [Errno 13] Permission denied: '/downloads'","phase":null,"disposition":null,"requester_id":null,"requester_name":null,"origin":null,"result_path":null,"command":["yt-dlp","--config-locations","/home/user/proj/youdis/default-yt-dlp.conf","https://www.youtube.com/watch?v=3i72yY_LaW4"],"returncode":1,"created_at":"2026-04-01T00:44:35.657196","updated_at":"2026-03-31T20:44:36.653353"}}user@paladin:~/proj/youdis$

:END:

  • datetime: [2026-03-31 Tue 20:45]

notes

  • validate `yt-dlp` subprocess invocation in-container; not verifiable in the current shell because `yt-dlp` is not installed here
  • confirm `–config-locations` behavior against the installed `yt-dlp` version during integration testing
  • current backend scaffold is not yet wired into `dockerfile` or `run-youdis.sh`
  • archive-hit and result-path parsing currently depend on `yt-dlp` stdout text patterns, so treat them as provisional until integration-tested
  • local dev now falls back to repo-local `.runtime/{config,downloads}` when `/config` or `/downloads` are not writable
  • had to uninstall yt-dlp python pkg from the venv, which resulted in a '403 Forbidden'

[X] 2.0.2: update discord bot to use new backend (3)

update the discord bot into a thin frontend that talks to the backend and verify the flow end to end

pm notes

  • this is the first real frontend proof. once this works cleanly, zulip/xmpp should mostly be adapter work rather than downloader rewrites.
  • keep discord logic thin; no auth
  • do not duplicate yt-dlp behavior in the bot

Acceptance Criteria

  1. discord bot submits requests to backend

    • command/input handling works
    • acceptance/busy/failure responses are clear

  2. discord bot relays useful backend status

    • progress reporting works at a basic level
    • completion/failure/skipped outcomes are surfaced

  3. backend-discord flow is tested end to end

    • valid request path tested
    • busy or conflict behavior tested
    • failure path tested

  4. add dotenv support to ease dev

    • os.getenv methods remain standard for prod/docker build
    • populate .env with dev env defaults

evidence

  • commit: 5210d2c, 043cb4

  • tests: https://youtu.be/20HxMMSqRyg?si=3v7mN2L88c_FxpQR 18m

    1. start backend: `python3 -m uvicorn youdis.main:app –host 127.0.0.1 –port 8000`
    2. create local env file: `cp .env.example .env`
    3. add `api_token` to `.env`
    4. start adapter: `python3 ./youdis.py`

    5. in discord, run `/youtube url:https://www.youtube.com/watch?v=dQw4w9WgXcQ`

      (venv) user@paladin:~/proj/youdis$ python ./youdis.py
Task exception was never retrieved
future: <Task finished name='Task-43' coro=<Client._dispatch_interaction() done, defined at /home/user/proj/youdis/venv/lib/python3.10/site-packages/interactions/client/client.py:1773> exception=KeyError('youtube')>
Traceback (most recent call last):
File "/home/user/proj/youdis/venv/lib/python3.10/site-packages/interactions/client/client.py", line 1798, in _dispatch_interaction
if ctx.command:
File "/home/user/proj/youdis/venv/lib/python3.10/site-packages/interactions/models/internal/context.py", line 329, in command
return self.client._interaction_lookup[self._command_name]
KeyError: 'youtube'
    6. confirm channel response says the job was submitted to backend

    7. confirm requester receives DM updates for accepted/running/completed or failed

      accepted job 4ef6fde5-57cc-478c-b0e2-18458c693fa4 for https://www.youtube.com/watch?v=dQw4w9WgXcQ

state=running | phase=running | [youtube] dQw4w9WgXcQ: Downloading webpage | path=/home/user/proj/youdis/downloads state=running | phase=downloading | [download] Destination: /home/user/proj/youdis/downloads/Rick_Astley/NA/NANARickAstley-_Never_Gonna_Give_You_Up_Official_Video_4K_Remaster.f401.mp4 | path=/home/user/proj/youdis/downloads/Rick_Astley/NA/NANARickAstley-_Never_Gonna_Give_You_Up_Official_Video_4K_Remaster.f401.mp4 state=completed | [Metadata] There isn't any metadata to add | path=/home/user/proj/youdis/downloads/Rick_Astley/NA/NANARickAstley-_Never_Gonna_Give_You_Up_Official_Video_4K_Remaster.mp4

🔚

  1. while first job is active, submit another `/youtube` and confirm busy behavior

    Error: AttributeError
Traceback (most recent call last):
File "/home/user/proj/youdis/venv/lib/python3.10/site-packages/interactions/client/client.py", line 1900, in __dispatch_interaction
response = await callback
File "/home/user/proj/youdis/venv/lib/python3.10/site-packages/interactions/client/client.py", line 1771, in _run_slash_command
return await command(ctx, **ctx.kwargs)
File "/home/user/proj/youdis/venv/lib/python3.10/site-packages/interactions/models/internal/command.py", line 132, in __call__
await self.call_callback(self.callback, context)
File "/home/user/proj/youdis/venv/lib/python3.10/site-packages/interactions/models/internal/application_commands.py", line 833, in call_callback
return await self.call_with_binding(callback, ctx, *new_args, **new_kwargs)
File "/home/user/proj/youdis/venv/lib/python3.10/site-packages/interactions/models/internal/callback.py", line 43, in call_with_binding
return await callback(*args, **kwargs)
File "/home/user/proj/youdis/youdis/adapters/discord.py", line 167, in youtube
await ctx.channel.send(f"Submitted <{url}> to the backend. Status updates via DM.")
AttributeError: 'NoneType' object has no attribute 'send'

    busy: busy with https://youtu.be/20HxMMSqRyg?si=3v7mN2L88c_FxpQR

state=running | phase=running | cancel requested | path=/home/user/proj/youdis/downloads/James_Hoffmann/NA/NANAThe_Beginner_s_Guide_To_Latte_Art.f401.mp4

🔚

  1. run `/status` and confirm it reflects current or last backend job

    last job: state=completed | [Metadata] There isn't any metadata to add | path=/home/user/proj/youdis/downloads/Rick_Astley/NA/NANARickAstley-_Never_Gonna_Give_You_Up_Official_Video_4K_Remaster.mp4

  2. run `/interrupt` as owner and confirm cancellation is surfaced via DM

last job: state=cancelled | cancelled | path=/home/user/proj/youdis/downloads/James_Hoffmann/NA/NANAThe_Beginner_s_Guide_To_Latte_Art.f401.mp4

  • datetime:[2026-04-02 Thu 11:52]

org-block tests 

  source ./venv/bin/activate
  python3 -m uvicorn youdis.main:app --host 127.0.0.1 --port 8000
echo ok
ok

python ./youdis.py

notes

  • discord adapter is now a thin HTTP client of the backend; it no longer imports or configures yt-dlp
  • `YOUDIS_BACKEND_URL` controls which backend the adapter targets
  • progress updates are currently implemented by polling `/jobs/current` and DMing only when the summary changes
  • legacy auth/user-management commands were removed from the active adapter path
  • `.env` is now supported for local/dev convenience, while real environment variables still override it in prod/docker
  • command registration required explicit binding plus `@bot.listen()` listeners in this adapter structure

[X] 2.0.3: remove deprecated discord-bot functionality (2)

delete or retire legacy bot behaviors that no longer fit once the backend split is in place

pm notes

  • only remove this after the new path works. this is cleanup, not pioneering work.
  • favor deletion over compatibility shims
  • keep operator controls only if still useful

Acceptance Criteria

  1. remove obsolete auth/user-management behavior

    • old user persistence and commands are removed
    • backend-facing flow no longer depends on them

  2. remove obsolete downloader/runtime logic from bot

    • bot no longer owns yt-dlp execution
    • dead code paths are deleted

  3. leave the bot in a coherent state

    • remaining commands reflect actual supported behavior
    • deprecated artifacts are clearly removed or marked

evidence

  • commit: 0aa9950

  • tests:

    1. `python3 -m py_compile ./youdis.py ./youdis/adapters/discord.py`
    2. `rg -n "users.json|api_token" README.md unraid-ca-template.xml youdis.py youdis/adapters/discord.py`
    3. start backend: `python3 -m uvicorn youdis.main:app –host 127.0.0.1 –port 8000`
    4. start adapter: `python3 ./youdis.py`
    5. in discord, run `/youtube`, `/status`, and `/interrupt` and confirm the adapter still works after cleanup
  • datetime: [2026-04-02 Thu 12:09]

notes

  • active bot path no longer includes local auth or user-management behavior
  • top-level `youdis.py` remains as a thin launcher so existing operator habits and scripts do not break during the refactor
  • cleanup updated user-facing deployment artifacts to match the v2 naming and architecture, including `DISCORD_BOT_TOKEN` and removal of `users.json` references
  • archived planning docs were intentionally left untouched: `pm/tasks.org` is historical and `pm/notes.org` is personal working notes

[ ] 2.0.4: make youdis.py app runner for backend + adapters (3)

Build a simple Python orchestration layer in youdis.py so the standard app stack can be launched from one entrypoint while backend and adapters remain independently runnable for testing.

pm notes

  • keep components independently runnable
  • youdis.py as app-level orchestrator; let Docker just run python3 /app/youdis.py
  • no plugin discovery or hot-loading
  • Docker should package and launch the app, not decide internal process topology
  • preserve the ability to run backend or adapters directly for debugging
  • optimize for one obvious default run path

Acceptance Criteria

  1. define the default app startup path

    • youdis.py launches the standard stack for v2
    • backend and Discord adapter startup order is explicit
    • shutdown behavior is coherent enough for local/dev use

  2. preserve modular run paths

    • backend can still be run directly
    • Discord adapter can still be run directly
    • orchestration layer does not bury component-level testing

  3. keep orchestration simple

    • no dynamic adapter discovery
    • no hot reloading framework
    • configured components are started explicitly

  4. document runtime ownership

    • clarify what Python orchestrates vs what Docker orchestrates
    • identify env vars or flags that control which components start
    • leave room for future Zulip/XMPP adapters without redesigning the runner

evidence

  • commit:

  • tests:

    1. `python3 -m py_compile ./youdis.py ./youdis/adapters/discord.py`
    2. backend direct run still works: `python3 -m uvicorn youdis.main:app –host 127.0.0.1 –port 8000`
    3. discord direct run still works: `python3 -m youdis.adapters.discord`
    4. app runner backend-only smoke test: `YOUDIS_RUN_DISCORD=0 timeout 5s python3 ./youdis.py`
    5. app runner default path: `python3 ./youdis.py`
  • date:

notes

  • `youdis.py` is now the default v2 app runner and starts the standard stack explicitly rather than dynamically discovering adapters
  • backend starts first and must pass a health check before the Discord adapter is launched
  • backend and Discord adapter remain directly runnable for debugging and tests
  • Docker is intended to invoke `python3 /app/youdis.py`; Python owns app orchestration while Docker owns packaging and runtime environment
  • runner flags currently include `YOUDIS_RUN_BACKEND`, `YOUDIS_RUN_DISCORD`, `YOUDIS_BACKEND_HOST`, and `YOUDIS_BACKEND_PORT`

== BACKLOG ==

Tasks below this line are inactive and should not be touched.

[ ] 2.0.X: fix automation and build pipeline (3)

repair and simplify the build/update/deploy path so it matches the new backend-plus-frontend structure

pm notes

  • this should come after architecture and discord integration stabilize. no point polishing the pipeline for the wrong shape.
  • optimize for simple manual ops first
  • stop here after pipeline is sane

Acceptance Criteria

  1. align build artifacts with the new structure

    • docker/build scripts reflect current components
    • runtime assumptions are consistent

  2. review old automation artifacts

    • stale runner/update/restart logic is removed or updated
    • manual update/rebuild flow is clear

  3. confirm deployment path works

    • local or unraid deployment is validated
    • pipeline is understandable enough to maintain

evidence

  • commit:
  • tests:
  • datetime:

notes

[ ] X.x.x: clean up discord adapter UI

acceptance criteria

  1. fix interaction pattern so it doesnt time out - prefer "command accepted" or somehting

  2. remove all intermediate messages between "accepted/running" and "complete" - /status handles this!

    • discord can also output a "busy" signal, research this
  3. fix output syntax, we dont need to get crazy with discord cards

evidence

  • commit:
  • tests:
  • date:

notes

[ ] X.x.x: fix youtube -> plex default output

acceptance criteria

evidence

  • commit:
  • tests:
  • date:

notes

/

Reference in New Issue View Git Blame Copy Permalink