From eaaefb66f2be30af792f494d730ec7a12e5cb2f8 Mon Sep 17 00:00:00 2001 From: eulaly Date: Thu, 7 May 2026 18:00:51 -0400 Subject: [PATCH] adding image --- README.md | 34 ++++++++++++++++++---------------- docs/vatownhall.org | 2 ++ 2 files changed, 20 insertions(+), 16 deletions(-) diff --git a/README.md b/README.md index 7767f69..91fa8a5 100644 --- a/README.md +++ b/README.md @@ -1,18 +1,18 @@ # Table of Contents - 1. [Project Goals](#orgf37a106) - 1. [Research questions](#orgec50d46) - 2. [Architecture](#org7a5389e) - 1. [Scraper](#org7771df2) - 2. [Analysis](#org16a9e36) - 3. [Storage](#org7341391) - 3. [Instructions](#org692b2f6) -1. [Roadmap](#org9f21934) + 1. [Project Goals](#org2da6874) + 1. [Research questions](#org1a2b8b3) + 2. [Architecture](#orgfabfcd9) + 1. [Scraper](#org2c5c7a2) + 2. [Analysis](#org72990f4) + 3. [Storage](#org58a5b72) + 3. [Instructions](#org24fe465) +1. [Roadmap](#org5739d49) - + ## Project Goals @@ -23,7 +23,7 @@ 3. Generalize to other public comment tools. - + ### Research questions @@ -38,7 +38,7 @@ (I anticipate this will not be measurable from currently available data) - + ## Architecture @@ -47,8 +47,10 @@ 3. Display: streamlit 4. Storage: jsonl, csv, parquet +![img](//pipeline-v1.2.3.svg) - + + ### Scraper @@ -59,7 +61,7 @@ Scrapy provides a simple mechanism for retrieving, parsing, and saving content f 3. Individual comment page: \`viewcomments.cfm?commentid=X\` - shows regulation title + brief description at the top, plus the comment - + ### Analysis @@ -101,7 +103,7 @@ We selected gpt-5.4-mini for a good balance of quality, cost, and time. \`\`\` - + ### Storage @@ -120,7 +122,7 @@ We selected gpt-5.4-mini for a good balance of quality, cost, and time. - Once complete, the cleanup script saves \`review.csv\`, \`review.pqt\`, and \`review.sqlite\` in this folder. - + ## Instructions @@ -144,7 +146,7 @@ We selected gpt-5.4-mini for a good balance of quality, cost, and time. \`python analysis/openaibatch.py submit\` - + # Roadmap diff --git a/docs/vatownhall.org b/docs/vatownhall.org index 0c12b41..d8a616f 100644 --- a/docs/vatownhall.org +++ b/docs/vatownhall.org @@ -26,6 +26,8 @@ 3. Display: streamlit 4. Storage: jsonl, csv, parquet +[[file://./pipeline-v1.2.3.svg]] + *** Scraper Scrapy provides a simple mechanism for retrieving, parsing, and saving content form the forums. 1. Forums listing page: `Forums.cfm` - lists all open forums with agency, reg title, action type, brief description, closing date, comment count