collective/cdsc_reddit
20
0
Fork 0
Code Activity

Update reddit comments data with daily dumps.

Browse Source
This commit is contained in:
Nate E TeBlunthuis
2020-10-03 16:42:22 -07:00
parent 2740f55915
commit 4ced659d19
4 changed files with 12 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
comments_2_parquet.sh
View File

@@ -1,9 +1,6 @@
## needs to be run by hand since i don't have a nice way of waiting on a parallel-sql job to complete
#!/usr/bin/env bash
echo "#!/usr/bin/bash" > job_script.sh
echo "source $(pwd)/../bin/activate" >> job_script.sh
echo "python3 $(pwd)/comments_2_parquet_part1.py" >> job_script.sh