Benjamin Mako Hill
1cec120dfa
changes to allow historical view data collection
...
- fix bug where it would fail if the first essay had no view data
- add ability to override dates in the cron script
2020-04-02 13:28:34 -05:00
Benjamin Mako Hill
b26a2b5a86
fix bug in previous commit
...
forgot to import digobs module in the scraper script
2020-04-01 18:22:36 -05:00
Benjamin Mako Hill
b457cd726b
use the type= feature in argparse
...
- integrated the type= feature in argparse in all three scripts
- removed some redundant code from the third file
2020-04-01 18:13:02 -05:00
Benjamin Mako Hill
070d23f718
changes in response to code review by nate
...
- moved some common functions into files
- other smaller changes
2020-04-01 17:16:34 -05:00
Benjamin Mako Hill
d655e1ce93
tweaks to revision export code
...
- flags were not being exported (e.g., minor, anon)
- broke with hidden/deleted user names
2020-04-01 16:39:53 -05:00
Benjamin Mako Hill
3f19805d36
fix bug in rev scraper script
...
Bug was a break, added for debugging, that caused the script to only
work for the first article.
2020-04-01 15:49:28 -05:00
Benjamin Mako Hill
5739d1c404
Merge branch 'master' of github.com:makoshark/COVID-19_Digital_Observatory
2020-04-01 15:18:50 -05:00
Benjamin Mako Hill
04e00f363b
address confusion with date
...
The timestamps in files should be the day that the exports are done. For
the view data, the query date needs to be the day before but this
shouldn't be the timestamp we use in files, etc.
2020-04-01 15:14:05 -05:00
Benjamin Mako Hill
06d2fd1563
fix bugs with the date stamps
2020-04-01 10:47:33 -05:00
Benjamin Mako Hill
4e1b7fbdfe
fixed typo in debug message
2020-04-01 08:18:05 -07:00
Benjamin Mako Hill
784458f206
renamed the wikipedia_views module to wikipedia
2020-04-01 07:51:20 -07:00
