milestones 1-6 complete: fetch/cache from data.usajobs.gov, local filters for pay plan/grade/salary/location, rich table output, questionary selection prompt, and org-mode export. key field mappings resolved from live api inspection (JobGrade[0].Code for pay plan, UserArea.Details for grades and clearance, city-part location matching due to api returning full state names). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
12 KiB
12 KiB
USAJobs Tasks
- Template
- open questions
- milestone 1 — setup and scaffolding
- [x] 1.1: project scaffold (1)
- [x] 1.2: env validation (1)
- milestone 2 — api and data layer
- [x] 2.1: build_params() (1)
- [x] 2.2: fetch_page() with caching (1)
- [x] 2.3: fetch_all() (1)
- [x] 2.4: normalize_job() (1)
- milestone 3 — filtering
- [x] 3.1: passes_filters() (1)
- milestone 4 — display
- [x] 4.1: render_table() (1)
- [x] 4.2: compact_job_label() (1)
- milestone 5 — selection and export
- [x] 5.1: choose_jobs() (1)
- [x] 5.2: make_output_path() (1)
- [x] 5.3: export_org() (1)
- milestone 6 — cli wiring and polish
- [x] 6.1: wire search command (1)
- [] 6.2: acceptance tests (manual) (1)
Template
create new tasks in this format: title number is miletone.task with est. commits in parens
* [] 1.1: task title (1)
instructions
** acceptance criteria
1.
1.
2.
2.
** notes
- document what you did
- include decisions and instructions
- when done,
** evidence
- commit: like so: beb5cf4 (AC1-2), e7df0b2 (AC3-6)
- tests: describe tests here so another user can run and validate
- datetime: include timestamp eg [2026-05-18 Mon 14:37]
open questions
clearance param shape
- api param name guess:
SecurityClearances(unconfirmed, passed through but not tested) - response field:
UserArea.Details.SecurityClearanceis a plain text string e.g. "Sensitive Compartmented Information" - numeric values (3, 4) mapping to api codes is still unknown — no local filtering yet
- action: test a live call with
--clearancevalues set to confirm param name and accepted values
DONE series api param name
- param name is
JobCategoryCode, semicolon-delimited values confirmed working - e.g.
JobCategoryCode=2210;0340
DONE location filtering — decided
- api returns full state names e.g. "Washington, District of Columbia", not abbreviations
- filter matches on city part only: split user input on "," and check first token
- "Washington, DC" → "washington" in "Washington, District of Columbia" ✓
milestone 1 — setup and scaffolding
[x] 1.1: project scaffold (1)
create usajobs.py, requirements.txt, .env.example, .gitignore
acceptance criteria
- running
python usajobs.py --helpprints top-level help without error - requirements.txt installs cleanly with
pip install -r requirements.txt - .env.example documents USAJOBS_EMAIL and USAJOBS_KEY
- .gitignore covers .cache/, exports/, .env
notes
- entrypoint: usajobs.py with a click group
cliand subcommandsearch - all functions implemented (not stubbed) — milestones 1-6 done in one pass
evidence
- commit: see initial usajobs commit
- tests:
python usajobs.py --helpandpython usajobs.py search --helpboth pass - datetime: [2026-05-18 Sun 15:00]
[x] 1.2: env validation (1)
implement get_credentials() and wire startup check into search command
acceptance criteria
- running
searchwithout USAJOBS_EMAIL set prints a clear error and exits nonzero - running
searchwithout USAJOBS_KEY set prints a clear error and exits nonzero - both vars present → no error, continues to api call
notes
- get_credentials() -> tuple[str, str]
- uses click.echo to stderr + sys.exit(1)
- load_dotenv() called at module level
evidence
- commit: see initial usajobs commit
- datetime: [2026-05-18 Sun 15:00]
milestone 2 — api and data layer
[x] 2.1: build_params() (1)
construct api query dict from cli args
acceptance criteria
- –series 2210 –series 0340 produces correct semicolon param
- –clearance 3 –clearance 4 produces correct semicolon param (placeholder value ok)
- –pay-plan gs –pay-plan gg produces correct semicolon param
- None/empty args are omitted from returned dict
- always includes: fields=full, resultsperpage=500, sortfield=opendate, sortdirection=desc
notes
- series param confirmed as
JobCategoryCode(verified via live call) - pay plan param:
PayPlanCode(best guess, not confirmed by api — filtering is local anyway) - clearance param:
SecurityClearances(best guess, unconfirmed — see open questions)
evidence
- commit: see initial usajobs commit
- tests:
--debugflag prints params; verifiedJobCategoryCode=2210returns correct results - datetime: [2026-05-18 Sun 15:00]
[x] 2.2: fetch_page() with caching (1)
fetch one page from the api with disk cache
acceptance criteria
- first call hits network, writes json to .cache/usajobs/<hash>_p<n>.json
- second call with same params reads from cache, does not hit network
- –offline with no cache raises a clear error
- –offline with cache returns cached data
- response is returned as parsed dict, cache file is never mutated
notes
- cache key: sha256 of sorted(params.items()) + page string, first 16 hex chars
- cache file written with json.dumps then read back with json.loads — never mutated
evidence
- commit: see initial usajobs commit
- tests: ran twice; second run served from cache (no network calls).
--offlinewith cache returns data. - datetime: [2026-05-18 Sun 15:00]
[x] 2.3: fetch_all() (1)
page through api results up to –limit
acceptance criteria
- stops fetching when total collected >= limit
- stops fetching when api returns no more results
- returns flat list of raw job dicts
- –debug prints total fetched before returning
notes
- totalcount field confirmed:
SearchResult.SearchResultCountAll - debug output shows per-page count, running total, and api-reported total
evidence
- commit: see initial usajobs commit
- tests:
--debugshows "page 1: got 132, running total 132, api reports 132 total" - datetime: [2026-05-18 Sun 15:00]
[x] 2.4: normalize_job() (1)
flatten raw api shape into a stable dict
acceptance criteria
- all required fields present in output (None if absent): document_id, title, agency, department, pay_plan, low_grade, high_grade, salary_min, salary_max, location, close_date, travel, clearance, clearance_text_match, url, raw_posting_text
- handles MatchedObjectDescriptor wrapper correctly
- handles UserArea.Details for extended fields
- raw_posting_text concatenates: Summary, Duties, Requirements, Qualifications, Evaluations, Other Information, Key Requirements
- strips html tags from raw_posting_text
notes
- pay_plan from
JobGrade[0].Code(e.g. "GS") — NOT PositionSchedule (that's work schedule) - grades from
UserArea.Details.LowGrade/HighGrade - salary from
PositionRemuneration[0].MinimumRange/MaximumRange(strings, cast to int) - clearance from
UserArea.Details.SecurityClearance(plain text string) - url from
ApplyURI[0]withPositionURIas fallback
evidence
- commit: see initial usajobs commit
- tests: org output shows correct grade, salary, clearance, posting text for real jobs
- datetime: [2026-05-18 Sun 15:00]
milestone 3 — filtering
[x] 3.1: passes_filters() (1)
local filter predicate applied after api fetch
acceptance criteria
- job with pay_plan not in allowed list → excluded
- job with low_grade < grade_min → excluded
- job with high_grade > grade_max → excluded
- job with salary_max < salary_min (and salary_min present) → excluded
- job with salary_max absent but salary_min >= salary_min threshold → included
- job whose location does not contain the –location substring (case-insensitive) → excluded
- –debug prints count before and after filtering
notes
- clearance filter skipped (open question)
- salary_min_k * 1000 before comparison
- location: match city part only (before first comma) due to api returning full state names
evidence
- commit: see initial usajobs commit
- tests: –grade-min 15 –grade-max 15 → only GS-15 results; –salary-min 150 → all jobs >= $150k
- datetime: [2026-05-18 Sun 15:13]
milestone 4 — display
[x] 4.1: render_table() (1)
print filtered results as a rich table
acceptance criteria
- columns: idx, title, agency, grade, salary, location, close date, clearance, url
- title truncated to ~50 chars in table
- salary formatted as "$Xk–$Yk" (or "$Xk" if max absent)
- grade formatted as "GS-15" or "GG-14/15" if low != high
- empty results prints a message and exits cleanly
notes
- using ASCII dash (-) not en-dash for salary range (Windows cp1252 compat)
- ellipsis uses "…" not unicode "…" for same reason
evidence
- commit: see initial usajobs commit
- tests: table renders correctly in Windows terminal; "No jobs matched" shown when filters exclude all
- datetime: [2026-05-18 Sun 15:00]
[x] 4.2: compact_job_label() (1)
one-line label for questionary checkbox rows
acceptance criteria
- format: "[{idx:>3}] {agency:<20} | {grade:<8} | {salary:<14} | {location:<18} | {title}"
- title truncated to ~55 chars
- total width fits within 120 cols on typical input
notes
- no url in label; url stays in rich table and org output
evidence
- commit: see initial usajobs commit
- datetime: [2026-05-18 Sun 15:00]
milestone 5 — selection and export
[x] 5.1: choose_jobs() (1)
questionary checkbox prompt for export selection
acceptance criteria
- each checkbox row uses compact_job_label(), value is document_id
- arrows and j/k navigate, space toggles, enter confirms, ctrl-c cancels
- –select-all preselects all rows
- empty selection or ctrl-c returns [] without writing
- instruction text reads: "space=mark/unmark, enter=export, ctrl-c=cancel"
notes
- questionary.checkbox(…).ask() returns None on ctrl-c; treated as empty → no write
- use_jk_keys=True, use_emacs_keys=True per readme
evidence
- commit: see initial usajobs commit
- tests: interactive flow needs manual terminal test — covered in 6.2
- datetime: [2026-05-18 Sun 15:00]
[x] 5.2: make_output_path() (1)
generate timestamped export filename or use –out
acceptance criteria
- –out set → use that path directly
- –out absent → exports/usajobs_<location-slug>_<filters-slug>_<yyyymmdd-hhmm>.org
- location slug: lowercase, spaces→hyphens, punctuation stripped
- filters slug includes: series, pay_plan, grade, salary (only what is set)
- exports/ dir created if it does not exist
notes
- example output:
usajobs_washington-dc_2210_gsgg15_salary150_20260518-1513.org - exports/ created via mkdir(parents=True, exist_ok=True)
evidence
- commit: see initial usajobs commit
- tests: verified filename format from two live runs with different filter combos
- datetime: [2026-05-18 Sun 15:13]
[x] 5.3: export_org() (1)
write selected jobs to org-mode file
acceptance criteria
- each job entry matches the org format in readme exactly
- shortened title strips all-caps runs where reasonable, max 80 chars
- properties drawer contains agency, grade, close_date
- body contains salary, location, travel, clearance (each "unknown" if absent)
- posting block contains raw_posting_text
- blank line between job entries
- –dry-run prints would-export list, does not write
notes
- _shorten_title: regex lowercases runs of 3+ consecutive all-caps words
- org link: link
- travel and clearance fall back to "unknown" if empty
evidence
- commit: see initial usajobs commit
- tests: spot-checked org output for first job; format matches readme spec exactly
- datetime: [2026-05-18 Sun 15:00]
milestone 6 — cli wiring and polish
[x] 6.1: wire search command (1)
connect all options to all functions end-to-end
acceptance criteria
- full example command from readme runs without error
- –no-interactive exports all filtered jobs without questionary prompt
- –dry-run shows selection output, writes nothing
- –debug prints params dict + before/after filter counts
- –offline works with populated cache
notes
- –series / –clearance / –pay-plan all use multiple=True
- –salary-min is int in thousands; multiplied by 1000 inside passes_filters
evidence
- commit: see initial usajobs commit
- tests: all five ACs verified via live runs with –debug and –no-interactive
- datetime: [2026-05-18 Sun 15:13]
[] 6.2: acceptance tests (manual) (1)
validate filter correctness and edge cases
acceptance criteria
- –grade-min 15 –grade-max 15 → no GS/GG-13 or GS/GG-14 jobs in output
- –salary-min 150 → all displayed jobs have max salary >= $150,000
- ctrl-c or empty selection → no file written, clean exit
- –offline with cache → same results as online run, no network
notes
- run against real api with valid credentials
- document results in evidence below
evidence
- commit:
- datetime: