The package declared no authors, no URLs, no classifiers and no
keywords, so a PyPI page for it would be blank and it would not turn up
in a search. Fill those in.
Contributors are listed by name only. They are the same people the
README credits, so this discloses nothing new, and publishing other
people's email addresses to PyPI is not ours to decide.
Also mirror the dev dependencies into [project.optional-dependencies].
[dependency-groups] is PEP 735, which only uv reads, so there was no way
for a pip user to install what the tests need; `pip install -e '.[dev]'`
now works. The two lists have to be kept in step by hand.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
(cherry picked from commit 7b3c0f9a80e39990df90124a7c592a80d93e0d91)
With no sdist target configured, hatchling walked the whole working tree
and included whatever it found. The resulting 8.9MB sdist carried 448
files from a virtualenv left in the repository root, plus .claude/ and
.gitmodules, while omitting the test dumps -- those are gitignored, so
the tests it did ship could not have run.
List the sdist contents explicitly. It is now 40KB of source, README and
COPYING, and both it and the wheel pass twine check. Tests run from a
clone rather than from the sdist, so test/ is deliberately excluded.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
(cherry picked from commit 28406563bcdb7b2f31412e511ea30f66eb3620b5)
Add the full license text as COPYING (the same text Debian ships in
common-licenses), declare the license and license file in
pyproject.toml, add copyright and permission notices to the source
files, and describe the license in the README. Also replace the
placeholder package description in pyproject.toml.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
(cherry picked from commit 90a14cf999)
Python 3.9 reached end of life in October 2025, and 3.11 is where the
current ecosystem sits: it unlocks pyarrow 25 (we were held at 21, the
last release supporting 3.9), more-itertools 11, and pandas 3 for the
test suite, along with the large CPython 3.11 interpreter speedups on
exactly the kind of CPU-bound work wikiq does. Nothing in the code
needed changes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Drop the [tool.uv.sources] pins of deltas, mwxml, and yamlconf to forks
under a personal github account:
- The deltas fork was byte-identical to upstream.
- The mwxml fork carried one patch to mwxml.map(), which wikiq does not
call, and was missing eleven upstream commits of fixes. Require
mwxml >= 0.3.8 for those fixes; note that 0.3.7 changed namespace
handling to trust the dump's embedded <ns> tag rather than overriding
it based on the title prefix.
- The yamlconf fork loosened an old pyyaml pin. yamlconf is not imported
by wikiq and is only needed transitively via deltas, so drop the direct
dependency. PyPI's 0.2.6 pinned pyyaml == 5.4.1, which has no wheels
for 3.11+ and no longer builds from source; that was fixed upstream in
0.2.7, and the project has since moved to the mediawiki-utilities
organization.
With pywikidiff2 no longer a dependency at all, no uv-specific
configuration remains and the package installs with plain pip.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
-p legacy is the only thing that uses mediawiki-utilities, and it exists
solely to reproduce numbers from research predating the mwxml and
mwtypes split. The package is a 2015 monolith that also contains
mw.database and mw.api, so it declares requests and pymysql
unconditionally: every wikiq install pulled a MySQL client and six other
packages to support one flag, building them from an sdist that has no
wheel.
Move it to a `legacy` extra. mw.lib.persistence imports nothing from
mw.database or mw.api, so nothing about -p legacy changes for people who
install the extra, and everyone else stops paying for it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
(cherry picked from commit 66dcc6d8cea620a619fc19121b5c27a085b11991)
pywikidiff2 was imported at module scope, so every invocation of wikiq
required it -- including `wikiq --help`. It is not on PyPI, compiles a
C++ extension, and needs libthai, which made a compiler a hard
requirement for installing a tool that mostly does not need one.
Only --diff and -p wikidiff2 actually use it. Import it inside those two
code paths instead, and report what to install when it is missing rather
than failing with an ImportError traceback. The check also runs once at
startup, since both use sites sit deep in the per-revision loop and a run
can stream for hours before reaching them.
Dropping the dependency also removes the PEP 508 direct reference from
the package metadata, and with it the need for hatchling's
allow-direct-references. PyPI rejects uploads whose metadata contains a
direct URL, so this is a prerequisite for publishing wikiq there.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
(cherry picked from commit 12325afd46670e949abf1c0e14799e212d7ff6ce)
The spark indexing pass is moving out of this repository, and
src/wikiq_spark was never committed here: pyproject.toml declared a
wikiq-spark console script and wheel package that do not exist in the
tree, and pulled in pyspark for every install of a tool that never
imports it. The --print-schema flag and its Spark-format schema
converters remain, since they are pure Python and produce the schema
the external indexing pass consumes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
(cherry picked from commit 392211af17)
Before, only fandom wikis dumps were compressed with .7z.
These archives can have several .xml files in the .7z; not just one.
So we need to have a flag for the fandom-2020 dumps.
This fixes the bug so .7z archives work in either case.
