diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..d5aae3a --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,73 @@ +# Changelog + +## 0.3.0 + +Changes relative to what people have been running. Earlier version tags +in this repository were internal markers rather than releases, so there +is no previous release to compare against. + +### Incompatible changes + +- **Redirect detection is now per revision.** The page-level + `redirect_target` column has been replaced by `revision_is_redirect` + and `revision_redirect_target`, which classify each revision from its + own text. The old column reported a page's redirect status as of the + time the dump was exported and applied that single value to every + revision in the page's history, which is wrong for any page that was + turned into a redirect, or turned back, at some point in its life. + Revisions with deleted text report null in both columns rather than + false. Wikis using a localized redirect keyword can add it with + `--redirect-aliases`; `#REDIRECT` is always recognized. + + Code reading `redirect_target` needs updating. + +- **Parquet output has been removed**, along with the + `--partition-namespaces` and `--max-revisions-per-file` options that + only applied to it. Use JSONL, which supports `--resume`. + +- **pyspark is no longer a dependency** and the `wikiq-spark` entry point + is gone. The second pass of that pipeline was never in this + repository. + +- **Python 3.11 or later is required.** Python 3.9 reached end of life in + October 2025, and the floor unlocks current pyarrow, more-itertools and + pandas, along with the CPython 3.11 speedups on the CPU-bound work + wikiq does. + +- **The project is now called wikiq**, rather than mediawiki_dump_tools. + +### Installation + +- **wikiq installs with plain `pip` and needs no C++ compiler.** + Previously every install required pywikidiff2, built from an ssh-only + git URL, so anyone outside the collective could not install the tool at + all. It is now optional and needed only for `--diff` and + `-p wikidiff2`. + +- **`-p legacy` needs the new `legacy` extra.** It was the only user of + `mediawiki-utilities`, a 2015 package that also pulls in a MySQL client + and six other packages. Everyone was paying for that to support one + flag. + +- **Dependencies resolve from PyPI** rather than from pinned forks under + personal accounts. Requires mwxml >= 0.3.8, which fixes eleven upstream + issues; note that mwxml 0.3.7 changed namespace handling to trust the + dump's embedded `` tag. + +### Added + +- `--version`, and the version is printed to stderr on every run so it + lands in job logs beside the output. +- Licensing: wikiq is GPL-3.0-or-later. See `COPYING`. + +### Fixed + +- Regex matching (`-RP`/`-CP`) crashed with a `TypeError` on revisions + whose text or comment was deleted or suppressed. Deleted content now + reports null. +- Page titles in non-main namespaces were prefixed twice under mwxml + 0.3.7 and later, producing titles like `Category:Category:Alaska`. +- The test suite had 16 failing tests, and `pytest test/` silently + collected only half of it because the main test file matched neither of + pytest's discovery patterns. It is now `test_wikiq.py` and the suite + runs green. diff --git a/pyproject.toml b/pyproject.toml index 8f16667..22924cc 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -81,6 +81,7 @@ packages = ["src/wikiq"] include = [ "src/wikiq", "README.md", + "CHANGELOG.md", "COPYING", "pyproject.toml", ]