collective/cdsc_reddit
20
0
Fork 0
Code Activity
42 Commits 9 Branches 0 Tags
9075a8153c6a900cce0ffb28c95da09211956ff9
Go to file
Clone
Open with VS Code Open with VSCodium Open with Intellij IDEA
Download ZIP Download TAR.GZ Download BUNDLE
Nathan TeBlunthuis 9075a8153c Merge branch 'master' of code:cdsc_reddit into master
2020-11-01 21:50:44 -08:00
examples
clean up comments in streaming example.
2020-07-07 12:28:57 -07:00
.gitignore
update .gitignore
2020-07-07 12:28:44 -07:00
check_comments_shas.py
Update reddit comments data with daily dumps.
2020-10-03 16:42:22 -07:00
check_submission_shas.py
Script for checking shas for submissions.
2020-07-03 13:35:46 -07:00
checkpoint_parallelsql.sbatch
Compute IDF for terms and authors.
2020-08-23 11:57:55 -07:00
comments_2_parquet_part1.py
Build comments dataset similarly to submissions and improve partitioning scheme
2020-07-07 11:45:43 -07:00
comments_2_parquet_part2.py
Compute IDF for terms and authors.
2020-08-23 11:57:55 -07:00
comments_2_parquet.sh
Update reddit comments data with daily dumps.
2020-10-03 16:42:22 -07:00
helper.py
Update reddit comments data with daily dumps.
2020-10-03 16:42:22 -07:00
idf_authors.py
Compute IDF for terms and authors.
2020-08-23 11:57:55 -07:00
idf_comments.py
Compute IDF for terms and authors.
2020-08-23 11:57:55 -07:00
pull_pushshift_comments.sh
Update reddit comments data with daily dumps.
2020-10-03 16:42:22 -07:00
pull_pushshift_submissions.sh
bugfix in checking submission shas
2020-08-11 14:21:54 -07:00
README.md
Create README.md
2020-11-01 21:50:27 -08:00
run_tf_jobs.sh
Compute IDF for terms and authors.
2020-08-23 11:57:55 -07:00
sort_tf_comments.py
code to sort tf
2020-08-03 17:56:36 -07:00
submissions_2_parquet_part1.py
Update submissions to parse using the backfill queue.
2020-08-11 22:37:36 -07:00
submissions_2_parquet_part2.py
Update submissions to parse using the backfill queue.
2020-08-11 22:37:36 -07:00
submissions_2_parquet.sh
Update submissions to parse using the backfill queue.
2020-08-11 22:37:36 -07:00
tf_comments.py
Compute IDF for terms and authors.
2020-08-23 11:57:55 -07:00
top_comment_phrases.py
Finish generating multiword expressions.
2020-08-09 22:43:48 -07:00

README.md

Utilities for Reddit Data Science

Pulling data from Pushshift

View Git Blame Copy Permalink
Description
No description provided
Readme 2.9 MiB
Languages
Python 74.9%
Shell 14.4%
Makefile 10.7%