From 1525a48561b45f98139910e5f1ffee5ae54bcbac Mon Sep 17 00:00:00 2001 From: Benjamin Mako Hill Date: Tue, 21 Jul 2026 10:05:18 -0700 Subject: [PATCH] datasets/README: remove untested notices after successful end-to-end run Co-Authored-By: Claude Sonnet 4.6 --- datasets/README.md | 6 ------ 1 file changed, 6 deletions(-) diff --git a/datasets/README.md b/datasets/README.md index 7868cb9..ae305ae 100644 --- a/datasets/README.md +++ b/datasets/README.md @@ -60,9 +60,6 @@ runs the Part 2 Spark sort. ### Add new months — `add_months.sh YYYY-MM [YYYY-MM ...]` -> **NOTE: written but not yet tested. Remove this notice after a -> successful end-to-end run.** - Use this for routine incremental updates. Runs Part 1 on only the specified months, then appends the sorted output as a new layer of partition files alongside the existing ones. No existing data is @@ -130,9 +127,6 @@ from a login node — it takes the number of nodes as its first argument. ### Merge layers — `merge_layers.sh` -> **NOTE: written but not yet tested. Remove this notice after a -> successful end-to-end run.** - Use this to collapse accumulated layers from incremental adds into a single clean layer. Reads the existing final datasets, re-sorts everything, writes to `.merging` temp paths, then atomically replaces the