mediawiki_dump_tools

Go to file

Will Beason 9c5bf577e6 Remove unused dependencies and fix spacing The "mw" and "numpy" dependencies were unneeded. Spaces and tabs were inconsistently used. They are now used consistently, changes via auto-formatter. Signed-off-by: Will Beason <willbeason@gmail.com>		2025-05-26 14:15:01 -05:00
test	Add additional test dependencies	2025-05-26 12:29:49 -05:00
.gitignore	Add additional test dependencies	2025-05-26 12:29:49 -05:00
.gitmodules	migrate to mwpersistence. this fixes many issues. We preserve legacy persistence behavior using the --persistence-legacy.	2018-07-04 19:06:07 -07:00
README.rst	Add additional test dependencies	2025-05-26 12:29:49 -05:00
requirements.txt	Remove unused dependencies and fix spacing	2025-05-26 14:15:01 -05:00
wikiq	Remove unused dependencies and fix spacing	2025-05-26 14:15:01 -05:00

README.rst

When you install this from git, you will need to first clone the repository::

  git clone git://projects.mako.cc/mediawiki_dump_tools

From within the repository working directory, initiatlize and set up the
submodule like::

  git submodule init
  git submodule update


Wikimedia dumps are usually in a compressed format such as 7z (most common), gz, or bz2. Wikiq uses your computer's compression software to read these files. Therefore wikiq depends on
`7za`, `gzcat`, and `zcat`. 

Dependencies
----------------
These non-Python dependencies must be installed on your system for wikiq and its
associated tests to work.

- 7zip
- ffmpeg


TODO:
_______________
1. [] Output metadata about the run. What parameters were used? What versions of deltas?
2. [] Url encoding by default