# Changelog

## 0.3.0

Changes relative to what people have been running. Earlier version tags
in this repository were internal markers rather than releases, so there
is no previous release to compare against.

### Incompatible changes

- **Redirect detection is now per revision.** The page-level
  `redirect_target` column has been replaced by `revision_is_redirect`
  and `revision_redirect_target`, which classify each revision from its
  own text. The old column reported a page's redirect status as of the
  time the dump was exported and applied that single value to every
  revision in the page's history, which is wrong for any page that was
  turned into a redirect, or turned back, at some point in its life.
  Revisions with deleted text report null in both columns rather than
  false. Wikis using a localized redirect keyword can add it with
  `--redirect-aliases`; `#REDIRECT` is always recognized.

  Code reading `redirect_target` needs updating.

- **Parquet output has been removed**, along with the
  `--partition-namespaces` and `--max-revisions-per-file` options that
  only applied to it. Use JSONL, which supports `--resume`.

- **pyspark is no longer a dependency** and the `wikiq-spark` entry point
  is gone. The second pass of that pipeline was never in this
  repository.

- **Python 3.11 or later is required.** Python 3.9 reached end of life in
  October 2025, and the floor unlocks current pyarrow, more-itertools and
  pandas, along with the CPython 3.11 speedups on the CPU-bound work
  wikiq does.

- **The project is now called wikiq**, rather than mediawiki_dump_tools.

### Installation

- **wikiq installs with plain `pip` and needs no C++ compiler.**
  Previously every install required pywikidiff2, built from an ssh-only
  git URL, so anyone outside the collective could not install the tool at
  all. It is now optional and needed only for `--diff` and
  `-p wikidiff2`.

- **`-p legacy` needs the new `legacy` extra.** It was the only user of
  `mediawiki-utilities`, a 2015 package that also pulls in a MySQL client
  and six other packages. Everyone was paying for that to support one
  flag.

- **Dependencies resolve from PyPI** rather than from pinned forks under
  personal accounts. Requires mwxml >= 0.3.8, which fixes eleven upstream
  issues; note that mwxml 0.3.7 changed namespace handling to trust the
  dump's embedded `<ns>` tag.

### Added

- `--version`, and the version is printed to stderr on every run so it
  lands in job logs beside the output.
- Licensing: wikiq is GPL-3.0-or-later. See `COPYING`.

### Fixed

- Regex matching (`-RP`/`-CP`) crashed with a `TypeError` on revisions
  whose text or comment was deleted or suppressed. Deleted content now
  reports null.
- Page titles in non-main namespaces were prefixed twice under mwxml
  0.3.7 and later, producing titles like `Category:Category:Alaska`.
- The test suite had 16 failing tests, and `pytest test/` silently
  collected only half of it because the main test file matched neither of
  pytest's discovery patterns. It is now `test_wikiq.py` and the suite
  runs green.
