3f94144b1b
Changed logic for handling anonymous edits so that wikiq handles the type for editor ids consistently. Parquet can mix int64 and None, but not int64 and strings - previously the code used the empty string to denote anonymous editors. Tests failing. Don't merge yet. Signed-off-by: Will Beason <willbeason@gmail.com>
When you install this from git, you will need to first clone the repository:: git clone git://projects.mako.cc/mediawiki_dump_tools From within the repository working directory, initiatlize and set up the submodule like:: git submodule init git submodule update Wikimedia dumps are usually in a compressed format such as 7z (most common), gz, or bz2. Wikiq uses your computer's compression software to read these files. Therefore wikiq depends on `7za`, `gzcat`, and `zcat`. Dependencies ---------------- These non-Python dependencies must be installed on your system for wikiq and its associated tests to work. - 7zip - ffmpeg Tests ---- To run tests:: python -m unittest test.Wikiq_Unit_Test TODO: _______________ 1. [] Output metadata about the run. What parameters were used? What versions of deltas? 2. [] Url encoding by default