GNU parallel spawns fresh shells that don't inherit the active venv.
Using an explicit PYTHON path ensures the right interpreter is used in
parallel tasks. Defaults to python3 but can be overridden:
PYTHON=/path/to/venv/bin/python3 ./add_months.sh ...
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
add_months.sh now targets a single fat node directly: starts a local
Spark cluster via start_spark_cluster.sh, submits jobs, stops the
cluster. No salloc needed.
add_months_multinode.sh is a new script for the multi-node case using
start_spark_and_run.sh from a login node. Usage takes NODES as first arg.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
