ben/vath
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

updated batch processing docs

Browse Source
This commit is contained in:
eulaly
2026-05-05 23:18:12 -04:00
parent f3abbefac7
commit 6eecc186f6
2 changed files with 46 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

20
docs/openai-batch.md → docs/openai.md
View File

@@ -382,3 +382,23 @@ Expired requests will be written to your error file with the message as shown be
{"id": "batch_req_123", "custom_id": "request-3", "response": null, "error": {"code": "batch_expired", "message": "This request could not be executed before the completion window expired."}} {"id": "batch_req_123", "custom_id": "request-3", "response": null, "error": {"code": "batch_expired", "message": "This request could not be executed before the completion window expired."}}
{"id": "batch_req_123", "custom_id": "request-7", "response": null, "error": {"code": "batch_expired", "message": "This request could not be executed before the completion window expired."}} {"id": "batch_req_123", "custom_id": "request-7", "response": null, "error": {"code": "batch_expired", "message": "This request could not be executed before the completion window expired."}}
``` ```
# Pricing and Limits - Batch
Updated 2026-05-05
Price per 1M Tokens, Short Context / Limits
TPM = Tokens per minute
Req/rpm = Requests per minute
TPD = Tokens per /day/, you cannot queue more than this TOTAL across all concurrent batches
| Model | Input ($) | Cached Input ($) | Output ($) | Token (tpm) | Req (rpm) | Batch (tpd) |
|--------------|-----------|------------------|------------|--------------|-----------|-------------|
| gpt-5.5 | 2.5 | 0.25 | 15.00 | 500000 | 500 | 900000 |
| gpt-5.4 | 1.25 | 0.13 | 7.50 | 500000 | 500 | 900000 |
| gpt-5.4-mini | 0.375 | 0.0375 | 2.25 | 200000 | 500 | 2000000 |
| gpt-5.4-nano | 0.10 | 0.01 | 0.625 | 200000 | 500 | 200000 |
| gpt-4o | 1.25 | - | 5.00 | 500000 | 500 | 900000 |
| gpt-4o-mini | 0.075 | - | 0.30 | 200000 | 500 | 2000000 |
| gpt-o4-mini | 0.55 | - | 0.30 | 200000 | 500 | 2000000 |
https://developers.openai.com/api/docs/pricing?latest-pricing=batch
https://platform.openai.com/settings/organization/limits

29
docs/tasks.org
View File

@@ -77,7 +77,7 @@ Should be run manually, separate from scraper. You may use scrapy, but are not r
- see: ./analysis/gpt4o/forum452_unknown_gpt-4o_2026-05-05T18-48-32+00-00.jsonl - see: ./analysis/gpt4o/forum452_unknown_gpt-4o_2026-05-05T18-48-32+00-00.jsonl
- date: [2026-05-05 Tue 15:00] - date: [2026-05-05 Tue 15:00]
* [ ] t1.2.1: batch processing * [X] t1.2.1: batch processing
Create analysis-batch.py to capture same elements as t1.2 above. Create analysis-batch.py to capture same elements as t1.2 above.
May need to add multiple commands to upload, check batch status, download, etc. May need to add multiple commands to upload, check batch status, download, etc.
Commands should all be run manually. Commands should all be run manually.
@@ -98,13 +98,36 @@ Reference: ./docs/openai-batch.md. openai batch output order is not guaranteed,
- custom_id format: comment_{comment_id} — unique within a forum, stable across runs. - custom_id format: comment_{comment_id} — unique within a forum, stable across runs.
- PROMPT_VERSION derived from analysis/prompt-1.txt (same file as realtime); both scripts produce matching prompt_hash in all records. - PROMPT_VERSION derived from analysis/prompt-1.txt (same file as realtime); both scripts produce matching prompt_hash in all records.
- analysis/prompt-1.txt: system prompt as plaintext, read at import time by both scripts. Edit here to change prompt for both pipelines. - analysis/prompt-1.txt: system prompt as plaintext, read at import time by both scripts. Edit here to change prompt for both pipelines.
- Tests use importlib.util to load hyphenated filenames; monkeypatch for RUNS_DIR in save/load test.
** evidence ** evidence
- commit: - commit: 683bfb3 (remove hyphen), f3abbef
- tests: 18 passing (pytest tests/analysis_gpt4o_batch.py), 46 total across suite - tests: 18 passing (pytest tests/analysis_gpt4o_batch.py), 46 total across suite
- datetime: [2026-05-05 Tue 17:00] - datetime: [2026-05-05 Tue 17:00]
* [ ] t1.2.2: Tokenizer / Batch mgmt
openai batch analysis requires coordination - more like a job queue.
batch script should setup queue for user to setup manually; openai api will reject subsequent batches when the total daily token limit is maxed.
** Acceptance Criteria
1. add token estimator utility script, probably to /analysis
2. add MODEL_LIMITS dict to analysis_batch.py. if there are more than (n)
- gpt-4o (30k tpm/90k tpd batch)
- gpt-4o-mini (200k tpm/2M tpd batch)
- add models listed in docs/openai.md
3. Auto-chunk submit: before writing the request file, walk comments, accumulate estimated tokens, and split into chunks that fit under the model's limit.
- Each chunk becomes its own batch submission with its own run_id.
- Drop --limit (or keep as hard cap override).
- Print all run_ids
- Submit the first batch only
4. Update test script to show tokenizer output
** notes
** evidence
- commit:
- tests:
- datetime:
* [ ] X: complete proposal information * [ ] X: complete proposal information
Ensure we capture as much useful information as possible about the actual proposal - contact information, etc. what the state actually says about what was posted. Ensure we capture as much useful information as possible about the actual proposal - contact information, etc. what the state actually says about what was posted.
** acceptance criteria ** acceptance criteria