yamlconf 0.2.7 removes the pyyaml == 5.4.1 pin that made 0.2.6
unbuildable on Python 3.11+, so the interim reference to the fork
carrying that fix is no longer needed. yamlconf is not a direct
dependency of wikiq—it returns to being resolved transitively via
deltas. The upstream repository has also moved to the
mediawiki-utilities organization.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Resolve the package version from installed metadata, report it via a
--version flag, and print it at the start of each run so output logs
record which wikiq produced them. Bump the version to 0.2.0 for the
changes on this branch: regex match counting, the removal of parquet
output, the Python 3.11 floor, and the dependency cleanup.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add the full license text as COPYING (the same text Debian ships in
common-licenses), declare the license and license file in
pyproject.toml, add copyright and permission notices to the source
files, and describe the license in the README. Also replace the
placeholder package description in pyproject.toml.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Python 3.9 reached end of life in October 2025, and 3.11 is where the
current ecosystem sits: it unlocks pyarrow 25 (we were held at 21, the
last release supporting 3.9), more-itertools 11, and pandas 3 for the
test suite, along with the large CPython 3.11 interpreter speedups on
exactly the kind of CPU-bound work wikiq does. Nothing in the code
needed changes.
Raising the floor exposed that PyPI's yamlconf 0.2.6 (a transitive
dependency via deltas) pins pyyaml == 5.4.1, which has no wheels for
3.11+ and no longer builds from source under current setuptools. Add a
PEP 508 reference to the fork that loosens the pin; it can be dropped
when https://github.com/halfak/yamlconf/pull/10 is merged and released.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Drop the [tool.uv.sources] pins of deltas, mwxml, and yamlconf to forks
under a personal github account:
- The deltas fork was byte-identical to upstream.
- The mwxml fork carried one patch to mwxml.map(), which wikiq does not
call, and was missing eleven upstream commits of fixes. Require
mwxml >= 0.3.8 for those fixes; note that 0.3.7 changed namespace
handling to trust the dump's embedded <ns> tag rather than overriding
it based on the title prefix.
- The yamlconf fork loosened an old pyyaml pin in requirements.txt.
yamlconf is not imported by wikiq and is only needed transitively by
deltas, so drop the direct dependency; the PyPI release installs fine,
though its pyyaml pin is old (worth fixing upstream).
With pywikidiff2 already declared as a PEP 508 direct https reference,
no uv-specific configuration remains and the package installs with pip.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Move pywikidiff2 from a [tool.uv.sources] entry to a direct git URL in
the dependency list, and switch it from ssh to anonymous https. The ssh
URL required an account on the gitea instance just to install the
package. A PEP 508 direct reference is understood by pip and every other
standards-compliant installer, not just uv, so this is a step toward
making uv an optional convenience rather than a requirement for working
with this repository.
Hatchling rejects direct references by default because they are not
allowed in packages published to PyPI (this one is not); set
tool.hatch.metadata.allow-direct-references to permit them.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The spark indexing pass is moving out of this repository, and
src/wikiq_spark was never committed here: pyproject.toml declared a
wikiq-spark console script and wheel package that do not exist in the
tree, and pulled in pyspark for every install of a tool that never
imports it. The --print-schema flag and its Spark-format schema
converters remain, since they are pure Python and produce the schema
the external indexing pass consumes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Before, only fandom wikis dumps were compressed with .7z.
These archives can have several .xml files in the .7z; not just one.
So we need to have a flag for the fandom-2020 dumps.
This fixes the bug so .7z archives work in either case.
