Files
usajobs/tasks.org
2026-05-18 15:35:12 -04:00

12 KiB
Raw Blame History

USAJobs Tasks

Template

create new tasks in this format: title number is miletone.task with est. commits in parens

 * [] 1.1: task title (1)
 instructions

 ** acceptance criteria
 1.
    1.
    2.
 2.

 ** notes
 - document what you did
 - include decisions and instructions
 - when done,

 ** evidence
 - commit: like so: beb5cf4 (AC1-2), e7df0b2 (AC3-6)
 - tests: describe tests here so another user can run and validate
 - datetime: include timestamp eg [2026-05-18 Mon 14:37]

open questions

clearance param shape

  • api param name guess: SecurityClearances (unconfirmed, passed through but not tested)
  • response field: UserArea.Details.SecurityClearance is a plain text string e.g. "Sensitive Compartmented Information"
  • numeric values (3, 4) mapping to api codes is still unknown — no local filtering yet
  • action: test a live call with --clearance values set to confirm param name and accepted values

DONE series api param name

  • param name is JobCategoryCode, semicolon-delimited values confirmed working
  • e.g. JobCategoryCode=2210;0340

DONE location filtering — decided

  • api returns full state names e.g. "Washington, District of Columbia", not abbreviations
  • filter matches on city part only: split user input on "," and check first token
  • "Washington, DC" → "washington" in "Washington, District of Columbia" ✓

milestone 1 — setup and scaffolding

[X] 1.1: project scaffold (1)

create usajobs.py, requirements.txt, .env.example, .gitignore

acceptance criteria

  1. running python usajobs.py --help prints top-level help without error
  2. requirements.txt installs cleanly with pip install -r requirements.txt
  3. .env.example documents USAJOBS_EMAIL and USAJOBS_KEY
  4. .gitignore covers .cache/, exports/, .env

notes

  • entrypoint: usajobs.py with a click group cli and subcommand search
  • all functions implemented (not stubbed) — milestones 1-6 done in one pass

evidence

  • commit: see initial usajobs commit
  • tests: python usajobs.py --help and python usajobs.py search --help both pass
  • datetime: [2026-05-18 Sun 15:00]

[X] 1.2: env validation (1)

implement get_credentials() and wire startup check into search command

acceptance criteria

  1. running search without USAJOBS_EMAIL set prints a clear error and exits nonzero
  2. running search without USAJOBS_KEY set prints a clear error and exits nonzero
  3. both vars present → no error, continues to api call

notes

  • get_credentials() -> tuple[str, str]
  • uses click.echo to stderr + sys.exit(1)
  • load_dotenv() called at module level

evidence

  • commit: see initial usajobs commit
  • datetime: [2026-05-18 Sun 15:00]

milestone 2 — api and data layer

[X] 2.1: build_params() (1)

construct api query dict from cli args

acceptance criteria

  1. series 2210 series 0340 produces correct semicolon param
  2. clearance 3 clearance 4 produces correct semicolon param (placeholder value ok)
  3. pay-plan gs pay-plan gg produces correct semicolon param
  4. None/empty args are omitted from returned dict
  5. always includes: fields=full, resultsperpage=500, sortfield=opendate, sortdirection=desc

notes

  • series param confirmed as JobCategoryCode (verified via live call)
  • pay plan param: PayPlanCode (best guess, not confirmed by api — filtering is local anyway)
  • clearance param: SecurityClearances (best guess, unconfirmed — see open questions)

evidence

  • commit: see initial usajobs commit
  • tests: --debug flag prints params; verified JobCategoryCode=2210 returns correct results
  • datetime: [2026-05-18 Sun 15:00]

[X] 2.2: fetch_page() with caching (1)

fetch one page from the api with disk cache

acceptance criteria

  1. first call hits network, writes json to .cache/usajobs/<hash>_p<n>.json
  2. second call with same params reads from cache, does not hit network
  3. offline with no cache raises a clear error
  4. offline with cache returns cached data
  5. response is returned as parsed dict, cache file is never mutated

notes

  • cache key: sha256 of sorted(params.items()) + page string, first 16 hex chars
  • cache file written with json.dumps then read back with json.loads — never mutated

evidence

  • commit: see initial usajobs commit
  • tests: ran twice; second run served from cache (no network calls). --offline with cache returns data.
  • datetime: [2026-05-18 Sun 15:00]

[X] 2.3: fetch_all() (1)

page through api results up to limit

acceptance criteria

  1. stops fetching when total collected >= limit
  2. stops fetching when api returns no more results
  3. returns flat list of raw job dicts
  4. debug prints total fetched before returning

notes

  • totalcount field confirmed: SearchResult.SearchResultCountAll
  • debug output shows per-page count, running total, and api-reported total

evidence

  • commit: see initial usajobs commit
  • tests: --debug shows "page 1: got 132, running total 132, api reports 132 total"
  • datetime: [2026-05-18 Sun 15:00]

[X] 2.4: normalize_job() (1)

flatten raw api shape into a stable dict

acceptance criteria

  1. all required fields present in output (None if absent): document_id, title, agency, department, pay_plan, low_grade, high_grade, salary_min, salary_max, location, close_date, travel, clearance, clearance_text_match, url, raw_posting_text
  2. handles MatchedObjectDescriptor wrapper correctly
  3. handles UserArea.Details for extended fields
  4. raw_posting_text concatenates: Summary, Duties, Requirements, Qualifications, Evaluations, Other Information, Key Requirements
  5. strips html tags from raw_posting_text

notes

  • pay_plan from JobGrade[0].Code (e.g. "GS") — NOT PositionSchedule (that's work schedule)
  • grades from UserArea.Details.LowGrade / HighGrade
  • salary from PositionRemuneration[0].MinimumRange / MaximumRange (strings, cast to int)
  • clearance from UserArea.Details.SecurityClearance (plain text string)
  • url from ApplyURI[0] with PositionURI as fallback

evidence

  • commit: see initial usajobs commit
  • tests: org output shows correct grade, salary, clearance, posting text for real jobs
  • datetime: [2026-05-18 Sun 15:00]

milestone 3 — filtering

[X] 3.1: passes_filters() (1)

local filter predicate applied after api fetch

acceptance criteria

  1. job with pay_plan not in allowed list → excluded
  2. job with low_grade < grade_min → excluded
  3. job with high_grade > grade_max → excluded
  4. job with salary_max < salary_min (and salary_min present) → excluded
  5. job with salary_max absent but salary_min >= salary_min threshold → included
  6. job whose location does not contain the location substring (case-insensitive) → excluded
  7. debug prints count before and after filtering

notes

  • clearance filter skipped (open question)
  • salary_min_k * 1000 before comparison
  • location: match city part only (before first comma) due to api returning full state names

evidence

  • commit: see initial usajobs commit
  • tests: grade-min 15 grade-max 15 → only GS-15 results; salary-min 150 → all jobs >= $150k
  • datetime: [2026-05-18 Sun 15:13]

milestone 4 — display

[X] 4.1: render_table() (1)

print filtered results as a rich table

acceptance criteria

  1. columns: idx, title, agency, grade, salary, location, close date, clearance, url
  2. title truncated to ~50 chars in table
  3. salary formatted as "$Xk$Yk" (or "$Xk" if max absent)
  4. grade formatted as "GS-15" or "GG-14/15" if low != high
  5. empty results prints a message and exits cleanly

notes

  • using ASCII dash (-) not en-dash for salary range (Windows cp1252 compat)
  • ellipsis uses "…" not unicode "…" for same reason

evidence

  • commit: see initial usajobs commit
  • tests: table renders correctly in Windows terminal; "No jobs matched" shown when filters exclude all
  • datetime: [2026-05-18 Sun 15:00]

[X] 4.2: compact_job_label() (1)

one-line label for questionary checkbox rows

acceptance criteria

  1. format: "[{idx:>3}] {agency:<20} | {grade:<8} | {salary:<14} | {location:<18} | {title}"
  2. title truncated to ~55 chars
  3. total width fits within 120 cols on typical input

notes

  • no url in label; url stays in rich table and org output

evidence

  • commit: see initial usajobs commit
  • datetime: [2026-05-18 Sun 15:00]

milestone 5 — selection and export

[X] 5.1: choose_jobs() (1)

questionary checkbox prompt for export selection

acceptance criteria

  1. each checkbox row uses compact_job_label(), value is document_id
  2. arrows and j/k navigate, space toggles, enter confirms, ctrl-c cancels
  3. select-all preselects all rows
  4. empty selection or ctrl-c returns [] without writing
  5. instruction text reads: "space=mark/unmark, enter=export, ctrl-c=cancel"

notes

  • questionary.checkbox(…).ask() returns None on ctrl-c; treated as empty → no write
  • use_jk_keys=True, use_emacs_keys=True per readme

evidence

  • commit: see initial usajobs commit
  • tests: interactive flow needs manual terminal test — covered in 6.2
  • datetime: [2026-05-18 Sun 15:00]

[X] 5.2: make_output_path() (1)

generate timestamped export filename or use out

acceptance criteria

  1. out set → use that path directly
  2. out absent → exports/usajobs_<location-slug>_<filters-slug>_<yyyymmdd-hhmm>.org
  3. location slug: lowercase, spaces→hyphens, punctuation stripped
  4. filters slug includes: series, pay_plan, grade, salary (only what is set)
  5. exports/ dir created if it does not exist

notes

  • example output: usajobs_washington-dc_2210_gsgg15_salary150_20260518-1513.org
  • exports/ created via mkdir(parents=True, exist_ok=True)

evidence

  • commit: see initial usajobs commit
  • tests: verified filename format from two live runs with different filter combos
  • datetime: [2026-05-18 Sun 15:13]

[X] 5.3: export_org() (1)

write selected jobs to org-mode file

acceptance criteria

  1. each job entry matches the org format in readme exactly
  2. shortened title strips all-caps runs where reasonable, max 80 chars
  3. properties drawer contains agency, grade, close_date
  4. body contains salary, location, travel, clearance (each "unknown" if absent)
  5. posting block contains raw_posting_text
  6. blank line between job entries
  7. dry-run prints would-export list, does not write

notes

  • _shorten_title: regex lowercases runs of 3+ consecutive all-caps words
  • org link: link
  • travel and clearance fall back to "unknown" if empty

evidence

  • commit: see initial usajobs commit
  • tests: spot-checked org output for first job; format matches readme spec exactly
  • datetime: [2026-05-18 Sun 15:00]

milestone 6 — cli wiring and polish

[X] 6.1: wire search command (1)

connect all options to all functions end-to-end

acceptance criteria

  1. full example command from readme runs without error
  2. no-interactive exports all filtered jobs without questionary prompt
  3. dry-run shows selection output, writes nothing
  4. debug prints params dict + before/after filter counts
  5. offline works with populated cache

notes

  • series / clearance / pay-plan all use multiple=True
  • salary-min is int in thousands; multiplied by 1000 inside passes_filters

evidence

  • commit: see initial usajobs commit
  • tests: all five ACs verified via live runs with debug and no-interactive
  • datetime: [2026-05-18 Sun 15:13]

[] 6.2: acceptance tests (manual) (1)

validate filter correctness and edge cases

acceptance criteria

  1. grade-min 15 grade-max 15 → no GS/GG-13 or GS/GG-14 jobs in output
  2. salary-min 150 → all displayed jobs have max salary >= $150,000
  3. ctrl-c or empty selection → no file written, clean exit
  4. offline with cache → same results as online run, no network

notes

  • run against real api with valid credentials
  • document results in evidence below

evidence

  • commit:
  • datetime:

= Backlog =