Benjamin Mako Hill 7c9db519ac datasets/add_months.sh: set PYSPARK_PYTHON so spark workers use the venv ...

spark-submit defaults to the container Python, which doesn't have the
project's dependencies. Export PYSPARK_PYTHON=$PYTHON before starting
the cluster so workers use the same interpreter as Part 1.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-05-25 22:16:50 -07:00

6.2 KiB

Executable File

Raw Blame History

View Raw