make redirect detection revision-level

The redirect_target column added in 59fea19 was page-level: the page's
state at dump time, stamped onto every revision row of the page. That
invited the wrong inference that a given revision was a redirect, and
the column was empty for dumps whose export never wrote <redirect>
elements, such as the 2023 wikitravel.org scrapes; a Swedish Wikitravel
scrape with 732 in-text redirect revisions produced no redirect signal
at all.

Remove that column and detect redirects from each revision's own text
instead. revision_is_redirect records whether the text begins with a
redirect directive and revision_redirect_target records the directive's
link target with any fragment and label stripped. #REDIRECT is
recognized on every wiki; localized keywords (e.g. OMDIRIGERING on
Swedish wikis) can be added with --redirect-aliases. Revisions with
deleted or unavailable text get nulls in both columns. The redirect-map
use case behind 59fea19 survives: the last revision's
revision_redirect_target per page reconstructs the page-level map, now
also on dumps without <redirect> elements.

Document that title and namespace are page-level identity values as of
the time of export, not historical facts about each revision.

Regenerate the test baselines for the column change. Every regenerated
file was verified to differ from its predecessor only by removing
redirect_target and adding the two new columns, with identical values
in all shared columns.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
(cherry picked from commit 40132f04dd)

test/Wikiq_Unit_Test.py

@@ -427,6 +427,63 @@ def test_regex_deleted_revisions():
     assert deleted["npov_npov"].isna().all()
     assert deleted["talk_talk"].isna().all()
 
+def test_redirect_detection():
+    from wikiq.tables import RedirectDetector
+
+    detector = RedirectDetector()
+
+    # a plain redirect directive
+    assert detector.detect("#REDIRECT [[Target]]") == (True, "Target")
+
+    # localized aliases are only recognized when configured
+    assert detector.detect("#OMDIRIGERING [[Mål]]") == (False, None)
+    swedish = RedirectDetector(["OMDIRIGERING"])
+    assert swedish.detect("#OMDIRIGERING [[Mål]]") == (True, "Mål")
+    assert swedish.detect("#REDIRECT [[Target]]") == (True, "Target")
+
+    # fragment and label are stripped from the target
+    assert detector.detect("#REDIRECT [[Target#Section|label]]") == (True, "Target")
+
+    # leading whitespace and lowercase are accepted
+    assert detector.detect("  \n#redirect [[Target]]") == (True, "Target")
+
+    # each revision is classified by its own text
+    history = ["#REDIRECT [[A]]", "An article now.", "#REDIRECT [[B]]"]
+    assert [detector.detect(t) for t in history] == [
+        (True, "A"), (False, None), (True, "B"),
+    ]
+
+    # a directive that is not at the start of the text is not a redirect
+    assert detector.detect("Some text. #REDIRECT [[Target]]") == (False, None)
+
+def test_redirect_columns_e2e():
+    # revision-level redirect columns on a real dump: the ikwiki dump
+    # contains both redirect revisions and revisions with deleted text
+    tester = WikiqTester(IKWIKI, "redirect_columns")
+
+    try:
+        tester.call_wikiq()
+    except subprocess.CalledProcessError as exc:
+        pytest.fail(exc.stderr.decode("utf8"))
+
+    test = pd.read_table(tester.output)
+
+    assert "revision_is_redirect" in test.columns
+    assert "revision_redirect_target" in test.columns
+    assert "redirect_target" not in test.columns
+
+    # revisions with deleted text have null values in both columns
+    deleted = test[test["deleted"]]
+    assert len(deleted) > 0
+    assert deleted["revision_is_redirect"].isna().all()
+    assert deleted["revision_redirect_target"].isna().all()
+
+    # the dump contains real redirect revisions, and every detected
+    # redirect has a target
+    redirects = test[test["revision_is_redirect"] == True]
+    assert len(redirects) > 0
+    assert redirects["revision_redirect_target"].notna().all()
+
 def test_external_links_only():
     """Test that --external-links extracts external links correctly."""
     import mwparserfromhell