Nate E TeBlunthuis 40d4563770 Build comments dataset similarly to submissions and improve partitioning scheme

2020-07-07 11:45:43 -07:00

examples

Script for example of streaming pyarrow.

2020-07-07 00:57:05 -07:00

.gitignore

update .gitignore

2020-07-07 00:58:26 -07:00

check_comments_shas.py

Check the shas when we download dumps

2020-07-06 23:31:52 -07:00

check_submission_shas.py

Script for checking shas for submissions.

2020-07-03 13:35:46 -07:00

comments_2_parquet_part1.py

Build comments dataset similarly to submissions and improve partitioning scheme

2020-07-07 11:45:43 -07:00

comments_2_parquet_part2.py

Build comments dataset similarly to submissions and improve partitioning scheme

2020-07-07 11:45:43 -07:00

comments_2_parquet.sh

Build comments dataset similarly to submissions and improve partitioning scheme

2020-07-07 11:45:43 -07:00

helper.py

Build comments dataset similarly to submissions and improve partitioning scheme

2020-07-07 11:45:43 -07:00

pull_pushshift_comments.sh

Check the shas when we download dumps

2020-07-06 23:31:52 -07:00

pull_pushshift_submissions.sh

Check the shas when we download dumps

2020-07-06 23:31:52 -07:00

submissions_2_parquet_part1.py

Build comments dataset similarly to submissions and improve partitioning scheme

2020-07-07 11:45:43 -07:00

submissions_2_parquet_part2.py

Build comments dataset similarly to submissions and improve partitioning scheme

2020-07-07 11:45:43 -07:00

submissions_2_parquet.sh

Build comments dataset similarly to submissions and improve partitioning scheme

2020-07-07 11:45:43 -07:00

Description

No description provided

3.2 MiB

Languages

Python 84.5%

Makefile 11.4%

Shell 4.1%