collective/cdsc_reddit
18
0
Fork 0
Code Activity

Refactor and reorganze.

Browse Source
This commit is contained in:
Nate E TeBlunthuis
2020-12-08 17:32:20 -08:00
parent a60747292e
commit e6294b5b90
47 changed files with 731 additions and 313 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
datasets/submissions_2_parquet.sh Normal file
View File

@@ -0,0 +1,9 @@
## this should be run manually since we don't have a nice way to wait on parallel_sql jobs
#!/usr/bin/env bash
./parse_submissions.sh
start_spark_and_run.sh 1 $(pwd)/submissions_2_parquet_part2.py