ben/scrape-giant
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Align refactor paths with data layout

Browse Source
This commit is contained in:
ben
2026-03-20 10:04:58 -04:00
parent 424a777dd0
commit d2e6f2afd3
5 changed files with 81 additions and 48 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
pm/tasks.org
View File

@@ -546,7 +546,7 @@ make Giant and Costco emit the shared normalized line-item schema without introd
- `normalized_item_id` is always present, but it only collapses repeated rows when the evidence is strong; otherwise it falls back to row-level identity via `normalized_row_id`.
- Added `normalize_*` entry points for the new data-model layout while leaving the legacy `enrich_*` commands available during the transition.
* [ ] t1.14.2: finalize filesystem and schema alignment for the refactor (2-4 commits)
* [X] t1.14.2: finalize filesystem and schema alignment for the refactor (2-4 commits)
bring on-disk outputs fully into the target `data/` structure without changing retailer behavior
** Acceptance Criteria
@@ -579,10 +579,13 @@ bring on-disk outputs fully into the target `data/` structure without changing r
** evidence
- commit:
- tests:
- datetime:
- tests: `./venv/bin/python -m unittest discover -s tests`; `./venv/bin/python build_purchases.py`; `./venv/bin/python review_products.py --refresh-only`; `./venv/bin/python report_pipeline_status.py`; `./venv/bin/python build_purchases.py --help`; `./venv/bin/python review_products.py --help`; `./venv/bin/python report_pipeline_status.py --help`; verified `data/giant-web/collected_orders.csv`, `data/giant-web/collected_items.csv`, `data/costco-web/collected_orders.csv`, `data/costco-web/collected_items.csv`, `data/catalog.csv`, and archived transitional review outputs under `data/review/archive/`
- datetime: 2026-03-20 10:04:15 EDT
** notes
- No recollection was needed; existing raw and collected exports were adapted in place and moved into the target names.
- Updated the active script defaults to point at `data/...` so the code and on-disk layout now agree.
- Kept obviously obsolete review artifacts, but moved them under `data/review/archive/` instead of deleting them outright.
* [ ] t1.14.3: retailer-specific Costco normalization cleanup (2-4 commits)
tighten Costco-specific normalization so normalized item names are cleaner and deterministic retailer grouping is less noisy