Benjamin Mako Hill
14886106ed
Drop the [tool.uv.sources] pins of deltas, mwxml, and yamlconf to forks under a personal github account: - The deltas fork was byte-identical to upstream. - The mwxml fork carried one patch to mwxml.map(), which wikiq does not call, and was missing eleven upstream commits of fixes. Require mwxml >= 0.3.8 for those fixes; note that 0.3.7 changed namespace handling to trust the dump's embedded <ns> tag rather than overriding it based on the title prefix. - The yamlconf fork loosened an old pyyaml pin in requirements.txt. yamlconf is not imported by wikiq and is only needed transitively by deltas, so drop the direct dependency; the PyPI release installs fine, though its pyyaml pin is old (worth fixing upstream). With pywikidiff2 already declared as a PEP 508 direct https reference, no uv-specific configuration remains and the package installs with pip. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
When you install this from git, you will need to first clone the repository:
git clone git://projects.mako.cc/mediawiki_dump_tools
From within the repository working directory, initiatlize and set up the submodule like:
git submodule init
git submodule update
Wikimedia dumps are usually in a compressed format such as 7z (most common), gz, or bz2. Wikiq uses your computer's compression software to read these files. Therefore wikiq depends on [7za]{.title-ref}, [gzcat]{.title-ref}, and [zcat]{.title-ref}.
Dependencies
These non-Python dependencies must be installed on your system for wikiq and its associated tests to work.
- 7zip
- ffmpeg
A new diff engine based on [_wikidiff2]{.title-ref} can be used for word-persistence. Wikiq can also output the diffs between each page revision. This requires installing Wikidiff 2 on your system. On Debian or Ubuntu Linux this can be done via.
apt-get install php-wikidiff2
You may have to also run. sudo phpenmod wikidiff2.
Tests ----To run tests:
python -m unittest test.Wikiq_Unit_Test
TODO:
- [] Output metadata about the run. What parameters were used? What versions of deltas?
- [] Url encoding by default