Compare commits

..

4 Commits

Author SHA1 Message Date
ben
cbe891c973 0.1.2 zulip api constraints 2026-04-14 16:04:16 -04:00
ben
e706294cc5 xmpp stack decision 2026-04-14 15:33:18 -04:00
ben
c72b1d8aba added basic envelope data model and message handling 2026-04-14 15:16:01 -04:00
ben
5ad2b6b9b7 0.1.0 base architecture 2026-04-14 14:36:18 -04:00
5 changed files with 482 additions and 15 deletions

106
pm/arch.org Normal file
View File

@@ -0,0 +1,106 @@
#+title: 0.1.0 v0 Bridge Contract
#+created: [2026-04-14 Tue 14:28]
* summary
v0 is a tiny Python relay bridging one XMPP MUC room to one Zulip stream/topic.
The bridge is append-only. It relays new human-written messages in both directions. It does not attempt to sync edits, reactions, threads beyond a fixed Zulip topic, presence, history, or rich metadata.
The design goal is to make transport behavior obvious. If something breaks, it should be clear whether the problem is Zulip ingress, XMPP ingress, relay logic, or configuration.
* first-run assumptions
- single bridge process
- single XMPP MUC room
- single Zulip stream + fixed topic
- trusted private deployment
- Python implementation
- one bridge identity on each side
- human-readable sender attribution is sufficient for v0
* system actors
** Zulip bot/client
- connects to one Zulip server
- subscribes to events for one stream/topic target
- receives new Zulip messages
- sends relay messages into the configured stream/topic
** XMPP client
- connects as one XMPP account
- joins one MUC room with one nick
- receives new groupchat messages
- sends relay messages into that room
** relay core
- owns config, logging, message normalization, and loop prevention
- accepts inbound events from either side
- converts them into one minimal internal message shape
- republishes them to the opposite side
* message flow
** Zulip -> XMPP
1. a new Zulip message arrives in the configured stream/topic
2. Zulip bot/client emits an inbound event to the relay core
3. relay core validates scope
- correct stream
- correct fixed topic
- not authored by the bridge itself
4. relay core normalizes sender name + body
5. relay core formats a plain relay message
6. XMPP client posts that message into the configured MUC room
7. relay core logs success/failure
** XMPP -> Zulip
1. a new XMPP groupchat message arrives in the configured MUC room
2. XMPP client emits an inbound event to the relay core
3. relay core validates scope
- correct room
- not authored by the bridge nick/account
- ignore non-message noise
4. relay core normalizes sender name + body
5. relay core formats a plain relay message
6. Zulip bot/client posts that message into the configured stream/topic
7. relay core logs success/failure
* v0 relay format
Plain text only.
Recommended shape:
- Zulip -> XMPP: "[zulip] <sender>: <body>"
- XMPP -> Zulip: "[xmpp] <sender>: <body>"
This is intentionally crude. The point is attribution and debuggability, not elegance.
* non-goals
- no message edit sync
- no delete sync
- no emoji/reaction sync
- no attachment mirroring in v0
- no presence mirroring
- no roster/contact sync
- no multi-room routing
- no per-thread/topic mapping beyond one fixed Zulip topic
- no history backfill
- no Hermes behavior in the transport process
- no Retcon writeback in the transport process
* operational boundaries
- the bridge should ignore service noise such as joins, parts, presence, and subscription events
- the bridge should only handle newly observed messages after startup
- the bridge should fail loudly in logs and stay simple in behavior
- the bridge should not try to be clever about replay, dedupe, or recovery yet beyond basic self-loop avoidance
* why this scope
This gives the smallest useful system:
- phone-friendly Zulip UI
- XMPP room remains the low-friction bus
- one boring relay to test whether the transport idea is worth keeping
If this small version is unreliable, adding Hermes or Retcon would only hide the real problems.
* exit condition for 0.1.0
Task 0.1.0 is complete when:
- this note exists
- the actor model is frozen
- both message directions are described
- v0 non-goals are explicit
- first-run assumptions are frozen

89
pm/relay-model.org Normal file
View File

@@ -0,0 +1,89 @@
#+title: 0.1.1 Relay Envelope and Anti-Loop Rules
#+created: [2026-04-14 Tue 14:40]
#+updated: [2026-04-14 Tue 14:55]
* envelope
#+begin_src python
from dataclasses import dataclass
from typing import Literal, Optional
Origin = Literal["zulip", "xmpp"]
Direction = Literal["zulip_to_xmpp", "xmpp_to_zulip"]
@dataclass(frozen=True)
class RelayEnvelope:
envelope_id: str
origin: Origin
source_message_id: str
sender_display_name: str
target_mapping: str
timestamp: str
body_text: str
sender_platform_id: Optional[str] = None
raw_direction: Optional[Direction] = None
receipt_time: Optional[str] = None
#+end_src
Required for v0:
| field | purpose |
|---------------------+-----------------------|
| origin | source platform |
| source_message_id | duplicate detection |
| sender_display_name | relay attribution |
| target_mapping | fixed bridge route |
| timestamp | event ordering / logs |
* relay output
| direction | plain-text shape |
|---------------+--------------------------|
| Zulip -> XMPP | [zulip] <sender>: <body> |
| XMPP -> Zulip | [xmpp] <sender>: <body> |
Optional machine stamp:
- bridge_id
- relayed_from
- relayed_source_id
* loop guards
1. self-authored
- Zulip sender == bridge account
- XMPP sender == bridge nick/jid
2. stamped bridge message
- inbound carries this bridge instance stamp
3. recent-seen duplicate
- key = (origin, source_message_id)
Order:
1. scope check
2. self/stamp check
3. duplicate check
4. normalize + relay once
5. record seen key after success
* failure policy
| case | action |
|--------------------------------+--------------------------------|
| empty body after normalization | drop + log |
| unknown origin | drop + log |
| unmapped target | drop + log |
| no usable id | synthesize fallback id, log it |
| non-chat event | drop + log |
| duplicate upstream event | drop, info/debug log |
* reconnect policy
- no history backfill
- only newly delivered events
- in-memory TTL cache suppresses short replay bursts
- restart may re-emit old events; acceptable for v0
* non-goals
- edits / deletes / reactions
- attachment mirroring
- presence / roster sync
- multi-room routing
- Hermes / Retcon integration
* exit condition
- envelope shape is frozen
- loop guards are explicit
- duplicate / malformed / reconnect behavior is explicit

View File

@@ -1,22 +1,191 @@
#+title: Task Log #+title: Task Log
#+updated: [2026-03-18 Wed 14:19] #+updated: [2026-04-14 Tue 13:26]
#+startup: overview
Use the template below, which should be a top-level org-mode header. * [X] 0.1.0: Define v0 bridge contract (estimate 1 commit)
Write a one-page spec for the first bridge: one XMPP MUC room <-> one Zulip stream/topic, append-only relay, no edits/reactions/history sync.
* [ ] M.m.m: Task Title (estimate # commits) keep scope painfully small so transport bugs are obvious.
replace the old observed/canonical workflow with a review-first pipeline that groups normalized rows only during review/combine and links them to catalog items my preference is for python-based systems.
** Acceptance Criteria ** Acceptance Criteria
1. Criterion 1. A short design note exists under pm/ or docs/
- expanded data - includes message flow in both directions
2. Criterion - identifies system actors: Zulip bot, XMPP client, relay core
2. Explicit non-goals are listed
- pm note: amplifying information - no thread mapping beyond fixed topic
- no rich sync
- no presence mirroring
3. First-run assumptions are frozen
- single room
- single stream/topic
- trusted private deployment
** evidence ** evidence
- commit: abc123, bcd234 - commit: 5ad2b6b
- tests: - tests: n/a (design note only)
- datetime: [2026-03-18 Wed 14:15] - datetime: [2026-04-14 Tue 14:28]
- artifact: pm/0.1.0-v0-bridge-contract.org
** notes ** notes
- explanation of work done, decisions made, reasoning - This is the boundary-setting task; everything else depends on it.
* [X] 0.1.1: Define canonical relay envelope + anti-loop rules (estimate 1 commit)
Specify the minimal message metadata the bridge must preserve and the exact loop-prevention mechanism.
** Acceptance Criteria
1. Canonical fields are listed
- origin
- source message id
- sender display name
- target room mapping
- timestamp
2. Loop prevention rule is explicit
- bridge stamps outgoing messages
- bridge ignores messages carrying its own origin marker
3. Failure handling is defined
- duplicate delivery behavior
- malformed message behavior
- reconnect replay policy
- pm note: this is the real core of v0; if this is fuzzy the bridge will thrash.
** evidence
- commit: c72b1d8
- tests: n/a (design note only)
- datetime: [2026-04-14 Tue 14:40]
- artifact: pm/0.1.1-relay-envelope-and-loop-rules.org
** notes
- Prefer boring explicit metadata over clever heuristics.
* [X] 0.1.2: Survey Zulip bot constraints + auth model (estimate 1 commit)
Confirm what kind of bot/client should be used on the Zulip side and document any constraints that change architecture.
** Acceptance Criteria
1. Zulip integration mode is chosen
- bot user vs normal user account
- event queue / polling approach
2. Constraints are captured
- message formatting limitations
- topic requirements
- rate limits / event semantics if relevant
3. Setup checklist is written
- credentials needed
- permissions needed
- minimal stream access needed
- pm note: Zulip quirks may force the shape of the relay more than XMPP will.
** evidence
- commit:
- tests: n/a (design note only)
- datetime: [2026-04-14 Tue 15:25]
- artifact: pm/zulip-bot-constraints.org
** notes
- Keep this empirical; document only constraints that affect v0.
* [X] 0.1.3: Survey XMPP MUC event model + library options (estimate 1 commit)
Pick the first XMPP client library/runtime and document the exact events needed for room relay.
** Acceptance Criteria
1. First implementation language/runtime is chosen
- likely Python unless a strong reason says otherwise
2. Library candidates are compared briefly
- connection support
- MUC message receive/send
- reconnect behavior
3. Required event subset is documented
- groupchat message receive
- send message to MUC
- ignore join/leave/presence noise
- pm note: avoid solving the Emacs client here; this is just relay transport.
** evidence
- commit: e706294
- tests: n/a (design note only)
- datetime: [2026-04-14 Tue 15:05]
- artifact: pm/0.1.3-xmpp-event-model-and-library-choice.org
** notes
- Goal is “good enough to prototype,” not perfect XMPP abstraction.
* [ ] 0.1.4: Write config schema for single-room bridge (estimate 1 commit)
Design the smallest possible config file/environment contract for running one bridge instance.
** Acceptance Criteria
1. Config fields are listed
- Zulip site/email/api key
- Zulip stream/topic
- XMPP jid/password/muc room/nick
2. One-room mapping is represented simply
- avoid generalized many-room config unless needed
3. Secrets vs non-secrets are separated
- env vars or secrets file for credentials
- checked-in example config for everything else
- pm note: optimize for easy local bring-up, not future elegance.
** evidence
- commit:
- tests:
- datetime:
** notes
- If the config shape feels overdesigned, shrink it again.
* [ ] 0.1.5: Define observability + manual test plan for v0 (estimate 1 commit)
Specify how to tell whether the bridge is healthy before adding Hermes.
** Acceptance Criteria
1. Logs are defined
- startup config summary without secrets
- inbound message event
- outbound relay result
- duplicate/loop drop reason
2. Manual test cases are listed
- Zulip -> XMPP happy path
- XMPP -> Zulip happy path
- duplicate suppression
- reconnect after disconnect
3. Success criteria are concrete
- messages appear once on each side
- sender attribution preserved enough for humans
- no infinite echo loops
- pm note: make the bridge boring before attaching Hermes or Retcon.
** evidence
- commit:
- tests:
- datetime:
** notes
- This should become the operator checklist for first deploy.
* [ ] 0.1.6: Sketch Hermes/Retcon follow-on integration points (estimate 1 commit)
Document where Hermes and Retcon could attach after transport is stable, without implementing them yet.
** Acceptance Criteria
1. Hermes interaction modes are listed
- passive listener
- command-triggered actor
- summarizer
2. Retcon integration points are listed
- append raw inbox
- synthesize summaries
- extract durable notes
3. Trust boundaries are explicit
- what can be written automatically
- what should remain user-confirmed
- pm note: this keeps the long-term vision visible without contaminating v0 scope.
** evidence
- commit:
- tests:
- datetime:
** notes
- Hermes should arrive after transport and logging are stable.

62
pm/xmpp-lib-decision.org Normal file
View File

@@ -0,0 +1,62 @@
#+title: 0.1.3 XMPP Event Model and Library Choice
#+created: [2026-04-14 Tue 15:05]
#+updated: [2026-04-14 Tue 15:20]
* initial research and recommendation
Slixmpp is an MIT-licensed XMPP library for Python 3.11+.
It is a fork of SleekXMPP and uses Rust for some performance-sensitive modules. In practice that means:
- Python app, not Rust app
- usually installable via PyPI wheels
- if wheels are unavailable, cargo may be needed to build the extension module
- maintained enough to be a sane v0 choice
For this bridge, slixmpp is the best MVP pick.
* minimum viable raw xmpp
If we do not use a library, we would need to implement the XMPP client plumbing ourselves:
- TCP + XML stream handling
- SASL auth
- TLS
- stanza parsing/serialization
- MUC join/leave semantics
- keepalive/reconnect handling
- duplicate/self-message suppression rules on top
Pros:
- maximum control
- fewer dependencies
- easier to trim down to exactly what we need
Cons:
- you are writing a mini XMPP client anyway
- more edge cases
- slower to reach a reliable relay
- easier to get auth/reconnect/MUC details subtly wrong
For v0, that trade is not worth it.
* event model we actually need
| event | use |
|----------------------+----------------------------------|
| connect/auth | establish session |
| self-presence | confirm MUC join before relaying |
| groupchat message | relay input |
| presence join/part | ignore or log only |
| subject/topic change | optional log only |
| disconnect/error | reconnect handling |
* recommendation
Use slixmpp for the XMPP side.
Why:
- it covers the protocol plumbing we do not want to hand-roll
- it is a straightforward bot/daemon fit
- it lets us focus on bridge behavior, not XML socket mechanics
- it is good enough for a single-room relay MVP
* summary
Use slixmpp unless we deliberately choose to prototype a raw XMPP client first.
The bridge only needs a narrow slice of XMPP. Slixmpp gives us that slice without forcing us to reinvent session management, MUC join semantics, or reconnect logic.

View File

@@ -0,0 +1,41 @@
#+title: Zulip Bot Constraints and Auth Model
#+created: [2026-04-14 Tue 15:25]
#+updated: [2026-04-14 Tue 15:25]
* initial research and recommendation
Use a Zulip bot user for the bridge.
Why:
- smaller permission surface than a human account
- API access is explicit and automatable
- easier to treat as infrastructure than as a person
Zulip messages can be sent as stream messages with a fixed topic using the API.
Inbound updates can be read through Zulip's real-time events API via an event queue.
Relevant API shape:
- send message: type = stream, to = channel, topic = fixed topic
- receive events: register an event queue, then long-poll get-events
Bot API key access is restricted to the bot owner and organization administrators.
* minimal zulip constraints
| area | constraint |
|------+------------|
| identity | use a bot user, not a normal user account |
| auth | bot API key needed; owner/admin can access it |
| outbound | stream message + fixed topic |
| inbound | event queue + long-poll events API |
| scope | one stream/topic pair for v0 |
| formatting | plain text is enough for the bridge |
* what this means for v0
- the Zulip side is not a general client UI problem
- it is a bot that reads events and sends stream messages
- no need for fancy topic routing yet
- one fixed topic is enough to keep the bridge simple
* summary
Bot user + event queue + fixed stream/topic is the clean v0 path.
It keeps Zulip acting like a relay endpoint, not a second full chat client.