13
0
cdsc_reddit/density/job_script.sh
Nate E TeBlunthuis 4dc949de5f Changes from hyak.
2021-02-22 16:03:48 -08:00

5 lines
311 B
Bash
Executable File

#!/usr/bin/bash
start_spark_cluster.sh
spark-submit --master spark://$(hostname):18899 overlap_density.py authors --inpath=/gscratch/comdata/output/reddit_similarity/comment_authors_10000.feather --outpath=/gscratch/comdata/output/reddit_density/comment_authors_10000.feather --agg=pd.DataFrame.sum
stop-all.sh