collective/cdsc_reddit
20
0
Fork 0
Code Activity
40 Commits 9 Branches 0 Tags
4ced659d1961630c20a1ef817422f242f723af7f
Go to file
Clone
Open with VS Code Open with VSCodium Open with Intellij IDEA
Download ZIP Download TAR.GZ Download BUNDLE
Nate E TeBlunthuis 4ced659d19 Update reddit comments data with daily dumps.
2020-10-03 16:42:22 -07:00
examples
clean up comments in streaming example.
2020-07-07 12:28:57 -07:00
.gitignore
update .gitignore
2020-07-07 12:28:44 -07:00
check_comments_shas.py
Update reddit comments data with daily dumps.
2020-10-03 16:42:22 -07:00
check_submission_shas.py
Script for checking shas for submissions.
2020-07-03 13:35:46 -07:00
checkpoint_parallelsql.sbatch
Compute IDF for terms and authors.
2020-08-23 11:57:55 -07:00
comments_2_parquet_part1.py
Build comments dataset similarly to submissions and improve partitioning scheme
2020-07-07 11:45:43 -07:00
comments_2_parquet_part2.py
Compute IDF for terms and authors.
2020-08-23 11:57:55 -07:00
comments_2_parquet.sh
Update reddit comments data with daily dumps.
2020-10-03 16:42:22 -07:00
helper.py
Update reddit comments data with daily dumps.
2020-10-03 16:42:22 -07:00
idf_authors.py
Compute IDF for terms and authors.
2020-08-23 11:57:55 -07:00
idf_comments.py
Compute IDF for terms and authors.
2020-08-23 11:57:55 -07:00
pull_pushshift_comments.sh
Update reddit comments data with daily dumps.
2020-10-03 16:42:22 -07:00
pull_pushshift_submissions.sh
bugfix in checking submission shas
2020-08-11 14:21:54 -07:00
run_tf_jobs.sh
Compute IDF for terms and authors.
2020-08-23 11:57:55 -07:00
sort_tf_comments.py
code to sort tf
2020-08-03 17:56:36 -07:00
submissions_2_parquet_part1.py
Update submissions to parse using the backfill queue.
2020-08-11 22:37:36 -07:00
submissions_2_parquet_part2.py
Update submissions to parse using the backfill queue.
2020-08-11 22:37:36 -07:00
submissions_2_parquet.sh
Update submissions to parse using the backfill queue.
2020-08-11 22:37:36 -07:00
tf_comments.py
Compute IDF for terms and authors.
2020-08-23 11:57:55 -07:00
top_comment_phrases.py
Finish generating multiword expressions.
2020-08-09 22:43:48 -07:00
Description
No description provided
2.9 MiB
Languages
Python 74.9%
Shell 14.4%
Makefile 10.7%