usajobs/tasks.org

#+title: USAJobs Tasks
#+startup: overview
#+date: [2026-05-18 Mon 14:36]

* Template
create new tasks in this format:
title number is miletone.task with est. commits in parens
#+begin_src org
 * [] 1.1: task title (1)
 instructions

 ** acceptance criteria
 1.
    1.
    2.
 2.

 ** notes
 - document what you did
 - include decisions and instructions
 - when done,

 ** evidence
 - commit: like so: beb5cf4 (AC1-2), e7df0b2 (AC3-6)
 - tests: describe tests here so another user can run and validate
 - datetime: include timestamp eg [2026-05-18 Mon 14:37]
#+end_src

* open questions
** clearance param shape
- api param name guess: ~SecurityClearances~ (unconfirmed, passed through but not tested)
- response field: ~UserArea.Details.SecurityClearance~ is a plain text string e.g. "Sensitive Compartmented Information"
- numeric values (3, 4) mapping to api codes is still unknown — no local filtering yet
- action: test a live call with ~--clearance~ values set to confirm param name and accepted values

** DONE series api param name
- param name is ~JobCategoryCode~, semicolon-delimited values confirmed working
- e.g. ~JobCategoryCode=2210;0340~

** DONE location filtering — decided
- api returns full state names e.g. "Washington, District of Columbia", not abbreviations
- filter matches on city part only: split user input on "," and check first token
- "Washington, DC" → "washington" in "Washington, District of Columbia" ✓

* milestone 1 — setup and scaffolding

* [X] 1.1: project scaffold (1)
create usajobs.py, requirements.txt, .env.example, .gitignore

** acceptance criteria
1. running ~python usajobs.py --help~ prints top-level help without error
2. requirements.txt installs cleanly with ~pip install -r requirements.txt~
3. .env.example documents USAJOBS_EMAIL and USAJOBS_KEY
4. .gitignore covers .cache/, exports/, .env

** notes
- entrypoint: usajobs.py with a click group ~cli~ and subcommand ~search~
- all functions implemented (not stubbed) — milestones 1-6 done in one pass

** evidence
- commit: see initial usajobs commit
- tests: ~python usajobs.py --help~ and ~python usajobs.py search --help~ both pass
- datetime: [2026-05-18 Sun 15:00]

* [X] 1.2: env validation (1)
implement get_credentials() and wire startup check into search command

** acceptance criteria
1. running ~search~ without USAJOBS_EMAIL set prints a clear error and exits nonzero
2. running ~search~ without USAJOBS_KEY set prints a clear error and exits nonzero
3. both vars present → no error, continues to api call

** notes
- get_credentials() -> tuple[str, str]
- uses click.echo to stderr + sys.exit(1)
- load_dotenv() called at module level

** evidence
- commit: see initial usajobs commit
- datetime: [2026-05-18 Sun 15:00]

* milestone 2 — api and data layer

* [X] 2.1: build_params() (1)
construct api query dict from cli args

** acceptance criteria
1. --series 2210 --series 0340 produces correct semicolon param
2. --clearance 3 --clearance 4 produces correct semicolon param (placeholder value ok)
3. --pay-plan gs --pay-plan gg produces correct semicolon param
4. None/empty args are omitted from returned dict
5. always includes: fields=full, resultsperpage=500, sortfield=opendate, sortdirection=desc

** notes
- series param confirmed as ~JobCategoryCode~ (verified via live call)
- pay plan param: ~PayPlanCode~ (best guess, not confirmed by api — filtering is local anyway)
- clearance param: ~SecurityClearances~ (best guess, unconfirmed — see open questions)

** evidence
- commit: see initial usajobs commit
- tests: ~--debug~ flag prints params; verified ~JobCategoryCode=2210~ returns correct results
- datetime: [2026-05-18 Sun 15:00]

* [X] 2.2: fetch_page() with caching (1)
fetch one page from the api with disk cache

** acceptance criteria
1. first call hits network, writes json to .cache/usajobs/<hash>_p<n>.json
2. second call with same params reads from cache, does not hit network
3. --offline with no cache raises a clear error
4. --offline with cache returns cached data
5. response is returned as parsed dict, cache file is never mutated

** notes
- cache key: sha256 of sorted(params.items()) + page string, first 16 hex chars
- cache file written with json.dumps then read back with json.loads — never mutated

** evidence
- commit: see initial usajobs commit
- tests: ran twice; second run served from cache (no network calls). ~--offline~ with cache returns data.
- datetime: [2026-05-18 Sun 15:00]

* [X] 2.3: fetch_all() (1)
page through api results up to --limit

** acceptance criteria
1. stops fetching when total collected >= limit
2. stops fetching when api returns no more results
3. returns flat list of raw job dicts
4. --debug prints total fetched before returning

** notes
- totalcount field confirmed: ~SearchResult.SearchResultCountAll~
- debug output shows per-page count, running total, and api-reported total

** evidence
- commit: see initial usajobs commit
- tests: ~--debug~ shows "page 1: got 132, running total 132, api reports 132 total"
- datetime: [2026-05-18 Sun 15:00]

* [X] 2.4: normalize_job() (1)
flatten raw api shape into a stable dict

** acceptance criteria
1. all required fields present in output (None if absent): document_id, title, agency, department, pay_plan, low_grade, high_grade, salary_min, salary_max, location, close_date, travel, clearance, clearance_text_match, url, raw_posting_text
2. handles MatchedObjectDescriptor wrapper correctly
3. handles UserArea.Details for extended fields
4. raw_posting_text concatenates: Summary, Duties, Requirements, Qualifications, Evaluations, Other Information, Key Requirements
5. strips html tags from raw_posting_text

** notes
- pay_plan from ~JobGrade[0].Code~ (e.g. "GS") — NOT PositionSchedule (that's work schedule)
- grades from ~UserArea.Details.LowGrade~ / ~HighGrade~
- salary from ~PositionRemuneration[0].MinimumRange~ / ~MaximumRange~ (strings, cast to int)
- clearance from ~UserArea.Details.SecurityClearance~ (plain text string)
- url from ~ApplyURI[0]~ with ~PositionURI~ as fallback

** evidence
- commit: see initial usajobs commit
- tests: org output shows correct grade, salary, clearance, posting text for real jobs
- datetime: [2026-05-18 Sun 15:00]

* milestone 3 — filtering

* [X] 3.1: passes_filters() (1)
local filter predicate applied after api fetch

** acceptance criteria
1. job with pay_plan not in allowed list → excluded
2. job with low_grade < grade_min → excluded
3. job with high_grade > grade_max → excluded
4. job with salary_max < salary_min (and salary_min present) → excluded
5. job with salary_max absent but salary_min >= salary_min threshold → included
6. job whose location does not contain the --location substring (case-insensitive) → excluded
7. --debug prints count before and after filtering

** notes
- clearance filter skipped (open question)
- salary_min_k * 1000 before comparison
- location: match city part only (before first comma) due to api returning full state names

** evidence
- commit: see initial usajobs commit
- tests: --grade-min 15 --grade-max 15 → only GS-15 results; --salary-min 150 → all jobs >= $150k
- datetime: [2026-05-18 Sun 15:13]

* milestone 4 — display

* [X] 4.1: render_table() (1)
print filtered results as a rich table

** acceptance criteria
1. columns: idx, title, agency, grade, salary, location, close date, clearance, url
2. title truncated to ~50 chars in table
3. salary formatted as "$Xk–$Yk" (or "$Xk" if max absent)
4. grade formatted as "GS-15" or "GG-14/15" if low != high
5. empty results prints a message and exits cleanly

** notes
- using ASCII dash (-) not en-dash for salary range (Windows cp1252 compat)
- ellipsis uses "..." not unicode "…" for same reason

** evidence
- commit: see initial usajobs commit
- tests: table renders correctly in Windows terminal; "No jobs matched" shown when filters exclude all
- datetime: [2026-05-18 Sun 15:00]

* [X] 4.2: compact_job_label() (1)
one-line label for questionary checkbox rows

** acceptance criteria
1. format: "[{idx:>3}] {agency:<20} | {grade:<8} | {salary:<14} | {location:<18} | {title}"
2. title truncated to ~55 chars
3. total width fits within 120 cols on typical input

** notes
- no url in label; url stays in rich table and org output

** evidence
- commit: see initial usajobs commit
- datetime: [2026-05-18 Sun 15:00]

* milestone 5 — selection and export

* [X] 5.1: choose_jobs() (1)
questionary checkbox prompt for export selection

** acceptance criteria
1. each checkbox row uses compact_job_label(), value is document_id
2. arrows and j/k navigate, space toggles, enter confirms, ctrl-c cancels
3. --select-all preselects all rows
4. empty selection or ctrl-c returns [] without writing
5. instruction text reads: "space=mark/unmark, enter=export, ctrl-c=cancel"

** notes
- questionary.checkbox(...).ask() returns None on ctrl-c; treated as empty → no write
- use_jk_keys=True, use_emacs_keys=True per readme

** evidence
- commit: see initial usajobs commit
- tests: interactive flow needs manual terminal test — covered in 6.2
- datetime: [2026-05-18 Sun 15:00]

* [X] 5.2: make_output_path() (1)
generate timestamped export filename or use --out

** acceptance criteria
1. --out set → use that path directly
2. --out absent → exports/usajobs_<location-slug>_<filters-slug>_<yyyymmdd-hhmm>.org
3. location slug: lowercase, spaces→hyphens, punctuation stripped
4. filters slug includes: series, pay_plan, grade, salary (only what is set)
5. exports/ dir created if it does not exist

** notes
- example output: ~usajobs_washington-dc_2210_gsgg15_salary150_20260518-1513.org~
- exports/ created via mkdir(parents=True, exist_ok=True)

** evidence
- commit: see initial usajobs commit
- tests: verified filename format from two live runs with different filter combos
- datetime: [2026-05-18 Sun 15:13]

* [X] 5.3: export_org() (1)
write selected jobs to org-mode file

** acceptance criteria
1. each job entry matches the org format in readme exactly
2. shortened title strips all-caps runs where reasonable, max 80 chars
3. properties drawer contains agency, grade, close_date
4. body contains salary, location, travel, clearance (each "unknown" if absent)
5. posting block contains raw_posting_text
6. blank line between job entries
7. --dry-run prints would-export list, does not write

** notes
- _shorten_title: regex lowercases runs of 3+ consecutive all-caps words
- org link: [[url][link]]
- travel and clearance fall back to "unknown" if empty

** evidence
- commit: see initial usajobs commit
- tests: spot-checked org output for first job; format matches readme spec exactly
- datetime: [2026-05-18 Sun 15:00]

* milestone 6 — cli wiring and polish

* [X] 6.1: wire search command (1)
connect all options to all functions end-to-end

** acceptance criteria
1. full example command from readme runs without error
2. --no-interactive exports all filtered jobs without questionary prompt
3. --dry-run shows selection output, writes nothing
4. --debug prints params dict + before/after filter counts
5. --offline works with populated cache

** notes
- --series / --clearance / --pay-plan all use multiple=True
- --salary-min is int in thousands; multiplied by 1000 inside passes_filters

** evidence
- commit: see initial usajobs commit
- tests: all five ACs verified via live runs with --debug and --no-interactive
- datetime: [2026-05-18 Sun 15:13]

* [] 6.2: acceptance tests (manual) (1)
validate filter correctness and edge cases

** acceptance criteria
1. --grade-min 15 --grade-max 15 → no GS/GG-13 or GS/GG-14 jobs in output
2. --salary-min 150 → all displayed jobs have max salary >= $150,000
3. ctrl-c or empty selection → no file written, clean exit
4. --offline with cache → same results as online run, no network

** notes
- run against real api with valid credentials
- document results in evidence below

** evidence
- commit:
- datetime:
* === Backlog ===