collective/cdsc_reddit
13
0
Fork 0
Code

Script to run both parts of submissions_2_parquet.sh

Browse Source
This commit is contained in:
Nate E TeBlunthuis 2020-07-06 23:27:18 -07:00
parent fd3b615544
commit 33e088492c
2 changed files with 10 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
.gitignore vendored
View File

@ -1,2 +1,3 @@
/#*# /#*#
/.#* /.#*
/job_script.sh

9
submissions_2_parquet.sh Normal file
View File

@ -0,0 +1,9 @@
#!/usr/bin/env bash
# part2 should be run on one ore more spark nodes
./submissions_2_parquet_part1.py
start_spark_and_run.sh 1 $(pwd)/submissions_2_parquet_part2.py