collective/mediawiki_dump_tools
16
0
Fork 0
Code Issues Pull Requests Activity

Compare commits

merge into: collective:master
Branches Tags
collective:master
collective:compute-diffs
collective:parquet_support
collective:will-setup
collective:mako_changes-20230429
collective:redirects
collective:regex_scanner
collective:tests
collective:advanced_persistence
collective:user_level_wikiq
collective:mediawiki-utils-migration
collective:legacy
...
pull from: collective:will-setup
Branches Tags
collective:compute-diffs
collective:parquet_support
collective:will-setup
collective:mako_changes-20230429
collective:redirects
collective:regex_scanner
collective:master
collective:tests
collective:advanced_persistence
collective:user_level_wikiq
collective:mediawiki-utils-migration
collective:legacy

21 Commits
master ... will-setup

Author SHA1 Message Date
Will Beason
df0ad1de63 Finish test standardization 
Test logic is executed within the WikiqTestCase, while WikiqTester
handles creating and managing the variables tests need.

Signed-off-by: Will Beason <willbeason@gmail.com>
 2025-05-28 10:11:58 -05:00
Will Beason
f3e6cc9392 Begin refactor of tests to make new tests easier to write 
Handle file naming logic centrally rather than requiring a dedicated
class per input file.

Signed-off-by: Will Beason <willbeason@gmail.com>
 2025-05-28 09:11:36 -05:00
Will Beason
c8b14c3303 Refactor test temporary file logic and wikiq call pattern 
Test file refreshing and path computation is now handled by a helper.

The wikiq command is now constructed and handled by a single method
rather than in several ad-hoc ways.

The last places relying on the working directory are now removed.

Signed-off-by: Will Beason <willbeason@gmail.com>
 2025-05-27 16:24:07 -05:00
Will Beason
4d3900b541 Standardize calling for wikiq in tests 
This way failures show the output of stderr/etc.

Also create path constant strings for use in tests to avoid repetition
and make changes easier.

Signed-off-by: Will Beason <willbeason@gmail.com>
 2025-05-27 14:27:49 -05:00
Will Beason
ebc57864f2 Make tests runnable from anywhere 
Tests no longer implicitly require that the caller be in
a specific working directory.

Signed-off-by: Will Beason <willbeason@gmail.com>
 2025-05-27 13:40:57 -05:00
Will Beason
3d0bf89938 Move main logic to main() 
This avoids:
1) the main function running when sourcing the file
2) Creating many globally-scoped variables in the main logic

Also begin refactor of test output file logic

Signed-off-by: Will Beason <willbeason@gmail.com>
 2025-05-27 11:10:42 -05:00
Will Beason
6d133575c7 Remove resource leaks from tests 
Close subprocesses within tests to fix resource leak warning.

Signed-off-by: Will Beason <willbeason@gmail.com>
 2025-05-26 15:08:47 -05:00
Will Beason
09a84e7d11 Reformat Wikiq_Unit_Test.py 
Separate out reformatting from editing.

Signed-off-by: Will Beason <willbeason@gmail.com>
 2025-05-26 15:07:39 -05:00
Will Beason
9c5bf577e6 Remove unused dependencies and fix spacing 
The "mw" and "numpy" dependencies were unneeded.

Spaces and tabs were inconsistently used.
They are now used consistently, changes via auto-formatter.

Signed-off-by: Will Beason <willbeason@gmail.com>
 2025-05-26 14:15:01 -05:00
Will Beason
4804ecc4b3 Add additional test dependencies 
These are now noted in requirements.txt

Also make dependency on 7zip and ffmpeg explicit in README

Signed-off-by: Will Beason <willbeason@gmail.com>
 2025-05-26 12:29:49 -05:00
Will Beason
7a4c41159c Exclude JetBrains config folder in .gitignore 
Signed-off-by: Will Beason <willbeason@gmail.com>
 2025-05-26 10:48:17 -05:00
Nathan TeBlunthuis
1aea601a30 [Bugfix] Call the correct matchmake function. 2021-11-16 16:53:21 -08:00
Nathan TeBlunthuis
c437b357db rename matchmake functions 2021-11-11 19:09:41 -08:00
Nathan TeBlunthuis
bb83d62b74 Add some descriptive comments. 2021-10-19 16:55:24 -07:00
Nathan TeBlunthuis
c285402683 add todos to readme 2021-10-18 14:14:11 -07:00
Nathan TeBlunthuis
b1bea09ad6 fix bugs and unit tests 2021-10-18 13:33:05 -07:00
Nathan TeBlunthuis
9a0c157ebb bugfix 2021-10-18 10:15:03 -07:00
Nathan TeBlunthuis
ae870fed0b parquet path is code-complete 2021-10-17 21:46:31 -07:00
Nathan TeBlunthuis
26f6d8f984 remove dependency on pandas. 2021-10-17 20:24:33 -07:00
Nathan TeBlunthuis
ae9a241747 use dataclasses and pyarrow for types. 2021-10-17 20:21:22 -07:00
Nathan TeBlunthuis
d8d20f670b initial work on parquet support 2021-10-17 13:22:22 -07:00
22 changed files with 10276 additions and 9995 deletions
Show Stats Expand all files Collapse all files

7
.gitignore vendored
View File

@ -3,3 +3,10 @@
*.xml.bz2
*.xml.xz
*.swp
# JetBrains
/.idea
# Python build and test output
__pycache__/
test_output/

19
README.rst
View File

@ -11,3 +11,22 @@ submodule like::
Wikimedia dumps are usually in a compressed format such as 7z (most common), gz, or bz2. Wikiq uses your computer's compression software to read these files. Therefore wikiq depends on
`7za`, `gzcat`, and `zcat`.
Dependencies
----------------
These non-Python dependencies must be installed on your system for wikiq and its
associated tests to work.
- 7zip
- ffmpeg
Tests
----
To run tests::
python -m unittest test.Wikiq_Unit_Test
TODO:
_______________
1. [] Output metadata about the run. What parameters were used? What versions of deltas?
2. [] Url encoding by default

39
requirements.txt Normal file
View File

@ -0,0 +1,39 @@
attrs==25.3.0
certifi==2025.4.26
charset-normalizer==3.4.2
Cython==0.29.37
deltas==0.7.0
docopt==0.6.2
gnureadline==8.1.2
idna==3.10
jsonable==0.3.1
jsonschema==4.23.0
jsonschema-specifications==2025.4.1
mediawiki-utilities==0.4.18
mwcli==0.0.3
mwdiffs==0.0.2
mwpersistence==0.2.4
mwreverts==0.1.5
mwtypes==0.4.0
mwxml==0.3.6
pandas==2.2.3
para==0.0.8
parsimonious==0.10.0
pyarrow==20.0.0
pydub==0.25.1
PyMySQL==1.1.1
python-dateutil==2.9.0.post0
pytz==2025.2
PyYAML==5.4.1
referencing==0.36.2
regex==2024.11.6
requests==2.32.3
rpds-py==0.25.1
setuptools==80.8.0
six==1.17.0
stopit==1.1.2
typing_extensions==4.13.2
tzdata==2025.2
urllib3==2.4.0
wheel==0.45.1
yamlconf==0.2.6

530
test/Wikiq_Unit_Test.py
View File

@ -3,396 +3,316 @@ import os
import subprocess
from shutil import copyfile
import pandas as pd
from pandas.util.testing import assert_frame_equal
from pandas.testing import assert_frame_equal
from io import StringIO
import tracemalloc
from typing import Final
# Make references to files and wikiq relative to this file, not to the current working directory.
TEST_DIR: Final[str] = os.path.dirname(os.path.realpath(__file__))
WIKIQ: Final[str] = os.path.join(os.path.dirname(TEST_DIR), "wikiq")
TEST_OUTPUT_DIR: Final[str] = os.path.join(TEST_DIR, "test_output")
BASELINE_DIR: Final[str] = os.path.join(TEST_DIR, "baseline_output")
IKWIKI: Final[str] = "ikwiki-20180301-pages-meta-history"
SAILORMOON: Final[str] = "sailormoon"
TWINPEAKS: Final[str] = "twinpeaks"
REGEXTEST: Final[str] = "regextest"
def setup():
tracemalloc.start()
# Perform directory check and reset here as this is a one-time setup step as opposed to per-test setup.
if not os.path.exists(TEST_OUTPUT_DIR):
os.mkdir(TEST_OUTPUT_DIR)
# Always run setup, even if this is executed via "python -m unittest" rather
# than as __main__.
setup()
class WikiqTester:
def __init__(self,
wiki: str,
case_name: str | None = None,
suffix: str | None = None,
in_compression: str = "bz2",
out_format: str = "tsv",
):
self.input_file = os.path.join(TEST_DIR, "dumps", "{0}.xml.{1}".format(wiki, in_compression))
if suffix is None:
self.wikiq_out_name = "{0}.{1}".format(wiki, out_format)
else:
self.wikiq_out_name = "{0}_{1}.{2}".format(wiki, suffix, out_format)
self.call_output = os.path.join(TEST_OUTPUT_DIR, "{0}.{1}".format(wiki, out_format))
# If case_name is unset, there are no relevant baseline or test files.
if case_name is not None:
self.baseline_file = os.path.join(BASELINE_DIR, "{0}_{1}".format(case_name, self.wikiq_out_name))
self.test_file = os.path.join(TEST_OUTPUT_DIR, "{0}_{1}".format(case_name, self.wikiq_out_name))
if os.path.exists(self.test_file):
os.remove(self.test_file)
def call_wikiq(self, *args: str, out: bool = True):
"""
Calls wikiq with the passed arguments on the input file relevant to the test.
:param args: The command line arguments to pass to wikiq.
:param out: Whether to pass an output argument to wikiq.
:return: The output of the wikiq call.
"""
if out:
call = ' '.join([WIKIQ, self.input_file, "-o", TEST_OUTPUT_DIR, *args])
else:
call = ' '.join([WIKIQ, self.input_file, *args])
print(call)
return subprocess.check_output(call, stderr=subprocess.PIPE, shell=True)
# with / without pwr DONE
# with / without url encode DONE
# with / without collapse user DONE
# with output to sdtout DONE
# with output to stdout DONE
# note that the persistence radius is 7 by default
# reading various file formats including
# 7z, gz, bz2, xml DONE
# wikia and wikipedia data DONE
# malformed xmls DONE
class Test_Wikipedia(unittest.TestCase):
def setUp(self):
if not os.path.exists("test_output"):
os.mkdir("test_output")
self.wiki = 'ikwiki-20180301-pages-meta-history'
self.wikiq_out_name = self.wiki + ".tsv"
self.test_output_dir = os.path.join(".", "test_output")
self.call_output = os.path.join(self.test_output_dir, self.wikiq_out_name)
self.infile = "{0}.xml.bz2".format(self.wiki)
self.base_call = "../wikiq {0} -o {1}"
self.input_dir = "dumps"
self.input_file = os.path.join(".", self.input_dir,self.infile)
self.baseline_output_dir = "baseline_output"
class WikiqTestCase(unittest.TestCase):
def test_WP_url_encode(self):
test_filename = "url-encode_" + self.wikiq_out_name
test_file = os.path.join(self.test_output_dir, test_filename)
if os.path.exists(test_file):
os.remove(test_file)
call = self.base_call.format(self.input_file, self.test_output_dir)
call = call + " --url-encode"
proc = subprocess.Popen(call,stdout=subprocess.PIPE,shell=True)
proc.wait()
tester = WikiqTester(IKWIKI, "url-encode")
copyfile(self.call_output, test_file)
baseline_file = os.path.join(".", self.baseline_output_dir, test_filename)
try:
tester.call_wikiq("--url-encode")
except subprocess.CalledProcessError as exc:
self.fail(exc.stderr.decode("utf8"))
copyfile(tester.call_output, tester.test_file)
# as a test let's make sure that we get equal data frames
test = pd.read_table(test_file)
baseline = pd.read_table(baseline_file)
assert_frame_equal(test,baseline)
test = pd.read_table(tester.test_file)
baseline = pd.read_table(tester.baseline_file)
assert_frame_equal(test, baseline, check_like=True)
def test_WP_namespaces(self):
print(os.path.abspath('.'))
test_filename = "namespaces_" + self.wikiq_out_name
test_file = os.path.join(self.test_output_dir, test_filename)
if os.path.exists(test_file):
os.remove(test_file)
call = self.base_call.format(self.input_file, self.test_output_dir)
call = call + " -n 0 -n 1"
print(call)
proc = subprocess.Popen(call,stdout=subprocess.PIPE,shell=True)
proc.wait()
copyfile(self.call_output, test_file)
baseline_file = os.path.join(os.path.abspath("."), self.baseline_output_dir, test_filename)
tester = WikiqTester(IKWIKI, "namespaces")
try:
tester.call_wikiq("-n 0", "-n 1")
except subprocess.CalledProcessError as exc:
self.fail(exc.stderr.decode("utf8"))
copyfile(tester.call_output, tester.test_file)
# as a test let's make sure that we get equal data frames
test = pd.read_table(test_file)
num_wrong_ns = sum(~ test.namespace.isin({0,1}))
test = pd.read_table(tester.test_file)
num_wrong_ns = sum(~ test.namespace.isin({0, 1}))
self.assertEqual(num_wrong_ns, 0)
baseline = pd.read_table(baseline_file)
assert_frame_equal(test,baseline)
baseline = pd.read_table(tester.baseline_file)
assert_frame_equal(test, baseline, check_like=True)
def test_WP_revert_radius(self):
print(os.path.abspath('.'))
test_filename = "revert_radius_" + self.wikiq_out_name
test_file = os.path.join(self.test_output_dir, test_filename)
if os.path.exists(test_file):
os.remove(test_file)
call = self.base_call.format(self.input_file, self.test_output_dir)
call = call + " -n 0 -n 1 -rr 1"
print(call)
proc = subprocess.Popen(call,stdout=subprocess.PIPE,shell=True)
proc.wait()
copyfile(self.call_output, test_file)
baseline_file = os.path.join(os.path.abspath("."), self.baseline_output_dir, test_filename)
tester = WikiqTester(IKWIKI, "revert_radius")
try:
tester.call_wikiq("-n 0", "-n 1", "-rr 1")
except subprocess.CalledProcessError as exc:
self.fail(exc.stderr.decode("utf8"))
copyfile(tester.call_output, tester.test_file)
# as a test let's make sure that we get equal data frames
test = pd.read_table(test_file)
num_wrong_ns = sum(~ test.namespace.isin({0,1}))
test = pd.read_table(tester.test_file)
num_wrong_ns = sum(~ test.namespace.isin({0, 1}))
self.assertEqual(num_wrong_ns, 0)
baseline = pd.read_table(baseline_file)
assert_frame_equal(test,baseline)
class Test_Basic(unittest.TestCase):
def setUp(self):
if not os.path.exists("test_output"):
os.mkdir("test_output")
self.wiki = 'sailormoon'
self.wikiq_out_name = self.wiki + ".tsv"
self.test_output_dir = os.path.join(".", "test_output")
self.call_output = os.path.join(self.test_output_dir, self.wikiq_out_name)
self.infile = "{0}.xml.7z".format(self.wiki)
self.base_call = "../wikiq {0} -o {1}"
self.input_dir = "dumps"
self.input_file = os.path.join(".", self.input_dir,self.infile)
self.baseline_output_dir = "baseline_output"
baseline = pd.read_table(tester.baseline_file)
assert_frame_equal(test, baseline, check_like=True)
def test_noargs(self):
tester = WikiqTester(SAILORMOON, "noargs", in_compression="7z")
test_filename = "noargs_" + self.wikiq_out_name
test_file = os.path.join(self.test_output_dir, test_filename)
if os.path.exists(test_file):
os.remove(test_file)
call = self.base_call.format(self.input_file, self.test_output_dir)
proc = subprocess.Popen(call,stdout=subprocess.PIPE,shell=True)
proc.wait()
try:
tester.call_wikiq()
except subprocess.CalledProcessError as exc:
self.fail(exc.stderr.decode("utf8"))
copyfile(self.call_output, test_file)
baseline_file = os.path.join(".", self.baseline_output_dir, test_filename)
test = pd.read_table(test_file)
baseline = pd.read_table(baseline_file)
assert_frame_equal(test,baseline)
copyfile(tester.call_output, tester.test_file)
test = pd.read_table(tester.test_file)
baseline = pd.read_table(tester.baseline_file)
assert_frame_equal(test, baseline, check_like=True)
def test_collapse_user(self):
test_filename = "collapse-user_" + self.wikiq_out_name
test_file = os.path.join(self.test_output_dir, test_filename)
if os.path.exists(test_file):
os.remove(test_file)
call = self.base_call.format(self.input_file, self.test_output_dir)
call = call + " --collapse-user"
tester = WikiqTester(SAILORMOON, "collapse-user", in_compression="7z")
proc = subprocess.Popen(call,stdout=subprocess.PIPE,shell=True)
proc.wait()
try:
tester.call_wikiq("--collapse-user")
except subprocess.CalledProcessError as exc:
self.fail(exc.stderr.decode("utf8"))
copyfile(self.call_output, test_file)
copyfile(tester.call_output, tester.test_file)
baseline_file = os.path.join(".", self.baseline_output_dir, test_filename)
test = pd.read_table(test_file)
baseline = pd.read_table(baseline_file)
assert_frame_equal(test,baseline)
test = pd.read_table(tester.test_file)
baseline = pd.read_table(tester.baseline_file)
assert_frame_equal(test, baseline, check_like=True)
def test_pwr_segment(self):
test_filename = "persistence_segment_" + self.wikiq_out_name
test_file = os.path.join(self.test_output_dir, test_filename)
if os.path.exists(test_file):
os.remove(test_file)
call = self.base_call.format(self.input_file, self.test_output_dir)
call = call + " --persistence segment"
proc = subprocess.Popen(call,stdout=subprocess.PIPE,shell=True)
proc.wait()
tester = WikiqTester(SAILORMOON, "persistence_segment", in_compression="7z")
try:
tester.call_wikiq("--persistence segment")
except subprocess.CalledProcessError as exc:
self.fail(exc.stderr.decode("utf8"))
copyfile(self.call_output, test_file)
copyfile(tester.call_output, tester.test_file)
baseline_file = os.path.join(".", self.baseline_output_dir, test_filename)
test = pd.read_table(test_file)
baseline = pd.read_table(baseline_file)
assert_frame_equal(test,baseline)
test = pd.read_table(tester.test_file)
baseline = pd.read_table(tester.baseline_file)
assert_frame_equal(test, baseline, check_like=True)
def test_pwr_legacy(self):
test_filename = "persistence_legacy_" + self.wikiq_out_name
test_file = os.path.join(self.test_output_dir, test_filename)
if os.path.exists(test_file):
os.remove(test_file)
call = self.base_call.format(self.input_file, self.test_output_dir)
call = call + " --persistence legacy"
proc = subprocess.Popen(call,stdout=subprocess.PIPE,shell=True)
proc.wait()
tester = WikiqTester(SAILORMOON, "persistence_legacy", in_compression="7z")
try:
tester.call_wikiq("--persistence legacy")
except subprocess.CalledProcessError as exc:
self.fail(exc.stderr.decode("utf8"))
copyfile(self.call_output, test_file)
copyfile(tester.call_output, tester.test_file)
baseline_file = os.path.join(".", self.baseline_output_dir, test_filename)
test = pd.read_table(test_file)
baseline = pd.read_table(baseline_file)
assert_frame_equal(test,baseline)
test = pd.read_table(tester.test_file)
baseline = pd.read_table(tester.baseline_file)
assert_frame_equal(test, baseline, check_like=True)
def test_pwr(self):
test_filename = "persistence_" + self.wikiq_out_name
test_file = os.path.join(self.test_output_dir, test_filename)
if os.path.exists(test_file):
os.remove(test_file)
call = self.base_call.format(self.input_file, self.test_output_dir)
call = call + " --persistence"
proc = subprocess.Popen(call,stdout=subprocess.PIPE,shell=True)
proc.wait()
tester = WikiqTester(SAILORMOON, "persistence", in_compression="7z")
try:
tester.call_wikiq("--persistence")
except subprocess.CalledProcessError as exc:
self.fail(exc.stderr.decode("utf8"))
copyfile(self.call_output, test_file)
copyfile(tester.call_output, tester.test_file)
baseline_file = os.path.join(".", self.baseline_output_dir, test_filename)
test = pd.read_table(test_file)
baseline = pd.read_table(baseline_file)
assert_frame_equal(test,baseline)
test = pd.read_table(tester.test_file)
baseline = pd.read_table(tester.baseline_file)
test = test.reindex(columns=sorted(test.columns))
assert_frame_equal(test, baseline, check_like=True)
def test_url_encode(self):
test_filename = "url-encode_" + self.wikiq_out_name
tester = WikiqTester(SAILORMOON, "url-encode", in_compression="7z")
test_file = os.path.join(self.test_output_dir, test_filename)
if os.path.exists(test_file):
os.remove(test_file)
call = self.base_call.format(self.input_file, self.test_output_dir)
call = call + " --url-encode"
proc = subprocess.Popen(call,stdout=subprocess.PIPE,shell=True)
proc.wait()
try:
tester.call_wikiq("--url-encode")
except subprocess.CalledProcessError as exc:
self.fail(exc.stderr.decode("utf8"))
copyfile(self.call_output, test_file)
baseline_file = os.path.join(".", self.baseline_output_dir, test_filename)
test = pd.read_table(test_file)
baseline = pd.read_table(baseline_file)
assert_frame_equal(test,baseline)
class Test_Malformed(unittest.TestCase):
def setUp(self):
if not os.path.exists("test_output"):
os.mkdir("test_output")
self.wiki = 'twinpeaks'
self.wikiq_out_name = self.wiki + ".tsv"
self.test_output_dir = os.path.join(".", "test_output")
self.call_output = os.path.join(self.test_output_dir, self.wikiq_out_name)
self.infile = "{0}.xml.7z".format(self.wiki)
self.base_call = "../wikiq {0} -o {1}"
self.input_dir = "dumps"
self.input_file = os.path.join(".", self.input_dir,self.infile)
copyfile(tester.call_output, tester.test_file)
test = pd.read_table(tester.test_file)
baseline = pd.read_table(tester.baseline_file)
test = test.reindex(columns=sorted(test.columns))
assert_frame_equal(test, baseline, check_like=True)
def test_malformed_noargs(self):
tester = WikiqTester(wiki=TWINPEAKS, in_compression="7z")
want_exception = 'xml.etree.ElementTree.ParseError: no element found: line 1369, column 0'
call = self.base_call.format(self.input_file, self.test_output_dir)
proc = subprocess.Popen(call,stdout=subprocess.PIPE,stderr=subprocess.PIPE, shell=True)
proc.wait()
outs, errs = proc.communicate()
errlines = str(errs).split("\\n")
self.assertEqual(errlines[-2],'xml.etree.ElementTree.ParseError: no element found: line 1369, column 0')
try:
tester.call_wikiq()
except subprocess.CalledProcessError as exc:
errlines = exc.stderr.decode("utf8").splitlines()
self.assertEqual(errlines[-1], want_exception)
else:
self.fail("No exception raised, want: {}".format(want_exception))
class Test_Stdout(unittest.TestCase):
def test_stdout_noargs(self):
tester = WikiqTester(wiki=SAILORMOON, case_name="noargs", in_compression="7z")
def setUp(self):
self.wiki = 'sailormoon'
self.wikiq_out_name = self.wiki + ".tsv"
try:
outs = tester.call_wikiq( "--stdout", out=False).decode("utf8")
except subprocess.CalledProcessError as exc:
self.fail(exc.stderr.decode("utf8"))
self.infile = "{0}.xml.7z".format(self.wiki)
self.base_call = "../wikiq {0} --stdout"
self.input_dir = "dumps"
self.input_file = os.path.join(".", self.input_dir,self.infile)
self.baseline_output_dir = "baseline_output"
copyfile(tester.call_output, tester.test_file)
def test_noargs(self):
call = self.base_call.format(self.input_file)
proc = subprocess.run(call,stdout=subprocess.PIPE,shell=True)
outs = proc.stdout.decode("utf8")
test_file = "noargs_" + self.wikiq_out_name
baseline_file = os.path.join(".", self.baseline_output_dir, test_file)
print(baseline_file)
test = pd.read_table(StringIO(outs))
baseline = pd.read_table(baseline_file)
assert_frame_equal(test,baseline)
baseline = pd.read_table(tester.baseline_file)
assert_frame_equal(test, baseline, check_like=True)
class Test_Regex(unittest.TestCase):
def test_bad_regex(self):
tester = WikiqTester(wiki=REGEXTEST)
def setUp(self):
self.wiki = 'regextest'
self.wikiq_out_name = self.wiki + '.tsv'
self.infile = "{0}.xml.bz2".format(self.wiki)
self.input_dir = "dumps"
self.input_file = os.path.join(".", self.input_dir,self.infile)
if not os.path.exists("test_output"):
os.mkdir("test_output")
self.test_output_dir = os.path.join(".", "test_output")
self.call_output = os.path.join(self.test_output_dir, self.wikiq_out_name)
# we have two base calls, one for checking inputs and the other for checking outputs
self.base_call = "../wikiq {0}"
self.base_call_outs = "../wikiq {0} -o {1}"
self.baseline_output_dir = "baseline_output"
# sample inputs for checking that bad inputs get terminated / test_regex_inputs
self.bad_inputs_list = [
#label is missing
"-RP '\\b\\d+\\b'",
#number of reg and number of labels do not match
# sample arguments for checking that bad arguments get terminated / test_regex_arguments
bad_arguments_list = [
# label is missing
"-RP '\\b\\d+\\b'",
# number of reg and number of labels do not match
"-RP 'NPO V' -RP THE -RPl testlabel",
#cp but rp label
# cp but rp label
"-CP '(Tamil|Li)' -RPl testlabel",
#regex is missing
# regex is missing
"-CPl testlabel",
"-RP '\\b\\w{3}\\b' -RPl threeletters -CP '\\b\\w{3}\\b'"
]
# sample inputs for checking the outcomes of good inputs / test_basic_regex
self.good_inputs_list = [
for arguments in bad_arguments_list:
try:
tester.call_wikiq("--stdout", arguments, out=False)
except subprocess.CalledProcessError as exc:
# we want to check that the bad arguments were caught and sys.exit is stopping the code
print(exc.stderr.decode("utf-8"))
else:
self.fail("No exception raised, want Exception")
def test_good_regex(self):
# sample arguments for checking the outcomes of good arguments / test_basic_regex
good_arguments_list = [
"-RP '\\b\\d{3}\\b' -RPl threedigits",
"-RP 'TestCase' -RP 'page' -RPl testcases -RPl page_word",
"-CP 'Chevalier' -CPl chev_com -RP 'welcome to Wikipedia' -RPl wiki_welcome -CP 'Warning' -CPl warning",
"-CP 'WP:EVADE' -CPl wp_evade"
"-CP 'WP:EVADE' -CPl wp_evade"
]
self.cap_inputs_list = [
for i, arguments in enumerate(good_arguments_list):
tester = WikiqTester(wiki=REGEXTEST, case_name="basic", suffix=str(i))
try:
tester.call_wikiq( arguments)
except subprocess.CalledProcessError as exc:
self.fail(exc.stderr.decode("utf8"))
copyfile(tester.call_output, tester.test_file)
test = pd.read_table(tester.test_file)
baseline = pd.read_table(tester.baseline_file)
assert_frame_equal(test, baseline, check_like=True)
print(i)
def test_capturegroup_regex(self):
cap_arguments_list = [
"-RP 'Li Chevalier' -RPl li_cheval -CP '(?P<letter>\\b[a-zA-Z]{3}\\b)|(?P<number>\\b\\d+\\b)|(?P<cat>\\bcat\\b)' -CPl three",
"-CP '(?P<a>\\bTestCaseA\\b)|(?P<b>\\bTestCaseB\\b)|(?P<c>\\bTestCaseC\\b)|(?P<d>\\bTestCaseD\\b)' -CPl testcase -RP '(?P<npov>npov|NPOV)|(?P<neutral>neutral point of view)' -RPl npov"
]
for i, arguments in enumerate(cap_arguments_list):
tester = WikiqTester(wiki=REGEXTEST, case_name="capturegroup", suffix=str(i))
try:
tester.call_wikiq(arguments)
except subprocess.CalledProcessError as exc:
self.fail(exc.stderr.decode("utf8"))
def test_regex_inputs(self):
for input in self.bad_inputs_list:
call = self.base_call.format(self.input_file)
call = call + " --stdout " + input
print(call)
proc = subprocess.Popen(call,stdout=subprocess.PIPE,stderr=subprocess.PIPE,shell=True)
stdout,stderr = proc.communicate()
#print(proc.returncode)
# we want to check that the bad inputs were caught and sys.exit is stopping the code
print(stderr.decode("utf-8"))
self.assertNotEqual(proc.returncode,0)
copyfile(tester.call_output, tester.test_file)
def test_basic_regex(self):
for i, input in enumerate(self.good_inputs_list):
test = pd.read_table(tester.test_file)
test_filename = "basic_{0}_{1}.tsv".format(self.wikiq_out_name[:-4], str(i))
#print(test_filename)
test_file = os.path.join(self.test_output_dir, test_filename)
if os.path.exists(test_file):
os.remove(test_file)
call = self.base_call_outs.format(self.input_file, self.test_output_dir)
call = call + " " + input
print(call)
proc = subprocess.Popen(call,stdout=subprocess.PIPE,stderr=subprocess.PIPE,shell=True)
proc.wait()
copyfile(self.call_output, test_file)
test = pd.read_table(test_file)
baseline_file = os.path.join(".", self.baseline_output_dir, test_filename)
baseline = pd.read_table(baseline_file)
assert_frame_equal(test, baseline)
print(i)
def test_capturegroup_regex(self):
for i, input in enumerate(self.cap_inputs_list):
test_filename = "capturegroup_{0}_{1}.tsv".format(self.wikiq_out_name[:-4], str(i))
print(test_filename)
test_file = os.path.join(self.test_output_dir, test_filename)
if os.path.exists(test_file):
os.remove(test_file)
call = self.base_call_outs.format(self.input_file, self.test_output_dir)
call = call + " " + input
print(call)
proc = subprocess.Popen(call,stdout=subprocess.PIPE,stderr=subprocess.PIPE,shell=True)
proc.wait()
copyfile(self.call_output, test_file)
test = pd.read_table(test_file)
baseline_file = os.path.join(".", self.baseline_output_dir, test_filename)
baseline = pd.read_table(baseline_file)
assert_frame_equal(test, baseline)
baseline = pd.read_table(tester.baseline_file)
assert_frame_equal(test, baseline, check_like=True)
if __name__ == '__main__':

0
test/__init__.py Normal file
View File

26
test/baseline_output/basic_regextest_0.tsv
View File

@ -1,8 +1,8 @@
anon articleid date_time deleted editor editor_id minor namespace revert reverteds revid sha1 text_chars threedigits title
anon articleid date_time deleted editor editorid minor namespace revert reverteds revid sha1 text_chars threedigits title
FALSE 56237363 2018-01-07 10:40:58 FALSE "NinjaRobotPirate" 3742946 FALSE 3 FALSE 819091731 135nz8q6lfam6cojla7azb7k5alx3t3 1141 126, 126, 126, 126 "User talk:86.139.142.254"
FALSE 56237364 2018-01-07 10:41:10 FALSE "Kavin kavitha" 32792125 FALSE 3 FALSE 819091755 0pwezjc6yopz0smc8al6ogc4fax5bwo 663 None "User talk:Kavin kavitha"
FALSE 56237365 2018-01-07 10:41:26 FALSE "Amicable always" 32621254 FALSE 3 FALSE 819091788 sz3t2ap7z8bpkdvdvi195f3i35949bv 399 None "User talk:Dr.vivek163"
FALSE 56237366 2018-01-07 10:41:31 FALSE "ClueBot NG" 13286072 FALSE 3 FALSE 819091796 r6s5j8j3iykenrhuhpnkpsmmd71vubf 1260 None "User talk:Twistorl"
FALSE 56237364 2018-01-07 10:41:10 FALSE "Kavin kavitha" 32792125 FALSE 3 FALSE 819091755 0pwezjc6yopz0smc8al6ogc4fax5bwo 663 "User talk:Kavin kavitha"
FALSE 56237365 2018-01-07 10:41:26 FALSE "Amicable always" 32621254 FALSE 3 FALSE 819091788 sz3t2ap7z8bpkdvdvi195f3i35949bv 399 "User talk:Dr.vivek163"
FALSE 56237366 2018-01-07 10:41:31 FALSE "ClueBot NG" 13286072 FALSE 3 FALSE 819091796 r6s5j8j3iykenrhuhpnkpsmmd71vubf 1260 "User talk:Twistorl"
FALSE 56237368 2018-01-07 10:41:51 FALSE "Khruner" 8409334 FALSE 0 FALSE 819091825 tf5qz2yaswx61zrlm9ovxzuhl7r2dc4 2249 119, 978, 500, 292, 225, 199, 292 "Kom Firin"
FALSE 56237368 2018-01-27 12:16:02 FALSE "Khruner" 8409334 TRUE 0 FALSE 822610647 e6oa4g0qv64icdaq26uu1zzbyr5hcbh 2230 119, 978, 500, 292, 225, 199, 292 "Kom Firin"
FALSE 56237369 2018-01-07 10:42:05 FALSE "Editingaccount1994" 32794215 FALSE 2 FALSE 819091844 0fyvyh2a8xu41gt8obr34oba0bfixj6 27840 798, 150, 150, 150, 621, 137, 137, 150, 150, 350, 195, 350, 195, 180, 180, 350, 195, 300, 150, 150, 150, 180, 180, 621 "User:Editingaccount1994/sandbox"
@ -12,16 +12,16 @@ FALSE 56237369 2018-01-12 23:28:11 FALSE "SporkBot" 12406635 TRUE 2 FALSE 82007
FALSE 56237369 2018-01-12 23:28:39 FALSE "SporkBot" 12406635 TRUE 2 FALSE 820078733 531dizmmloyxffbkdr5vph7owh921eg 27782 798, 150, 150, 150, 621, 137, 137, 150, 150, 350, 195, 350, 195, 180, 180, 350, 195, 300, 150, 150, 150, 180, 180, 621 "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-13 13:45:33 FALSE "Frietjes" 13791031 FALSE 2 FALSE 820177382 nik9p2u2fuk4yazjxt8ymbicxv5qid9 27757 798, 150, 150, 150, 621, 100, 621 "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-24 01:35:22 FALSE "CommonsDelinker" 2304267 FALSE 2 FALSE 822038928 gwk6pampl8si1v5pv3kwgteg710sfw3 27667 798, 150, 150, 150, 621, 100, 621 "User:Editingaccount1994/sandbox"
FALSE 56237370 2018-01-07 10:42:20 FALSE "PamD" 1368779 FALSE 0 FALSE 819091874 n4ozbsgle13p9yywtfrz982ccj8woc9 25 None "Anita del Rey"
FALSE 56237370 2018-01-07 10:42:20 FALSE "PamD" 1368779 FALSE 0 FALSE 819091874 n4ozbsgle13p9yywtfrz982ccj8woc9 25 "Anita del Rey"
FALSE 56237371 2018-01-07 10:42:27 FALSE "ClueBot NG" 13286072 FALSE 3 FALSE 819091883 ksohnvsbeuzwpl5vb8a3v8m18hva0a7 1274 119, 157, 119, 157, 119, 157, 119, 157 "User talk:119.94.96.157"
FALSE 56237372 2018-01-07 10:42:50 FALSE "Underbar dk" 677153 FALSE 14 FALSE 819091914 je7aw21fedbwyqsyofpisdrynsu7olr 113 None "Category:Ohmi Railway"
FALSE 56237375 2018-01-07 10:43:32 FALSE "TastyPoutine" 882433 FALSE 3 FALSE 819091968 cpm4tkzcx4hc6irr9ukbi06ogud8dtq 199 None "User talk:92.226.219.222"
FALSE 56237375 2018-01-07 11:10:24 FALSE "AnomieBOT" 7611264 TRUE 3 FALSE 819094036 artmfz8b2gxhb3pp8a5p4ksplxqfkpg 1840 None "User talk:92.226.219.222"
FALSE 56237372 2018-01-07 10:42:50 FALSE "Underbar dk" 677153 FALSE 14 FALSE 819091914 je7aw21fedbwyqsyofpisdrynsu7olr 113 "Category:Ohmi Railway"
FALSE 56237375 2018-01-07 10:43:32 FALSE "TastyPoutine" 882433 FALSE 3 FALSE 819091968 cpm4tkzcx4hc6irr9ukbi06ogud8dtq 199 "User talk:92.226.219.222"
FALSE 56237375 2018-01-07 11:10:24 FALSE "AnomieBOT" 7611264 TRUE 3 FALSE 819094036 artmfz8b2gxhb3pp8a5p4ksplxqfkpg 1840 "User talk:92.226.219.222"
FALSE 56237375 2018-01-07 14:33:36 FALSE "Only" 702940 FALSE 3 FALSE 819112363 dn9wj0n8d8pdd5lqe56uw5xamupowr1 2949 126, 126, 126, 126 "User talk:92.226.219.222"
FALSE 56237376 2018-01-07 10:44:01 FALSE "Dipayanacharya" 32794237 FALSE 2 FALSE 819092004 ofueugwatmmn7u73isw732neuza57gk 28 None "User:Dipayanacharya"
FALSE 56237376 2018-01-07 10:49:08 FALSE "Dipayanacharya" 32794237 FALSE 2 FALSE 819092390 dsz55xv96ec2uv6w9c1z7c52ipfovbw 38 None "User:Dipayanacharya"
FALSE 56237378 2018-01-07 10:44:56 FALSE "Vinegarymass911" 21516552 FALSE 0 FALSE 819092066 9ma38hak0ef1ew4fpiutxpnzd8oz1wd 65 None "BSCIC"
FALSE 56237379 2018-01-07 10:45:21 FALSE "BrownHairedGirl" 754619 FALSE 14 FALSE 819092102 4dvakoat58bzyf5hmtthxukt29hip6n 285 None "Category:Women government ministers of Yemen"
FALSE 56237381 2018-01-07 10:45:54 FALSE "PRehse" 410898 FALSE 1 FALSE 819092135 2sjrxsc7os9k9pg4su2t4rk2j8nn0h7 103 None "Talk:List of Morning Glories Characters"
FALSE 56237376 2018-01-07 10:44:01 FALSE "Dipayanacharya" 32794237 FALSE 2 FALSE 819092004 ofueugwatmmn7u73isw732neuza57gk 28 "User:Dipayanacharya"
FALSE 56237376 2018-01-07 10:49:08 FALSE "Dipayanacharya" 32794237 FALSE 2 FALSE 819092390 dsz55xv96ec2uv6w9c1z7c52ipfovbw 38 "User:Dipayanacharya"
FALSE 56237378 2018-01-07 10:44:56 FALSE "Vinegarymass911" 21516552 FALSE 0 FALSE 819092066 9ma38hak0ef1ew4fpiutxpnzd8oz1wd 65 "BSCIC"
FALSE 56237379 2018-01-07 10:45:21 FALSE "BrownHairedGirl" 754619 FALSE 14 FALSE 819092102 4dvakoat58bzyf5hmtthxukt29hip6n 285 "Category:Women government ministers of Yemen"
FALSE 56237381 2018-01-07 10:45:54 FALSE "PRehse" 410898 FALSE 1 FALSE 819092135 2sjrxsc7os9k9pg4su2t4rk2j8nn0h7 103 "Talk:List of Morning Glories Characters"
FALSE 56237382 2018-01-07 10:45:56 FALSE "ClueBot NG" 13286072 FALSE 3 FALSE 819092138 3y9t5wpk6ur5jhone75rhm4wjf01fgi 1330 106, 207, 126, 114, 106, 207, 126, 114, 106, 207, 126, 114, 106, 207, 126, 114 "User talk:106.207.126.114"
FALSE 56237382 2018-01-07 10:50:22 FALSE "HindWIKI" 31190506 FALSE 3 FALSE 819092495 8wvn6vh3isyt0dorpe89lztrburgupe 2355 106, 207, 126, 114, 106, 207, 126, 114, 106, 207, 126, 114, 106, 207, 126, 114 "User talk:106.207.126.114"

1 anon articleid date_time deleted editor editor_id editorid minor namespace revert reverteds revid sha1 text_chars threedigits title
2 FALSE 56237363 2018-01-07 10:40:58 FALSE NinjaRobotPirate 3742946 FALSE 3 FALSE 819091731 135nz8q6lfam6cojla7azb7k5alx3t3 1141 126, 126, 126, 126 User talk:86.139.142.254
3 FALSE 56237364 2018-01-07 10:41:10 FALSE Kavin kavitha 32792125 FALSE 3 FALSE 819091755 0pwezjc6yopz0smc8al6ogc4fax5bwo 663 None User talk:Kavin kavitha
4 FALSE 56237365 2018-01-07 10:41:26 FALSE Amicable always 32621254 FALSE 3 FALSE 819091788 sz3t2ap7z8bpkdvdvi195f3i35949bv 399 None User talk:Dr.vivek163
5 FALSE 56237366 2018-01-07 10:41:31 FALSE ClueBot NG 13286072 FALSE 3 FALSE 819091796 r6s5j8j3iykenrhuhpnkpsmmd71vubf 1260 None User talk:Twistorl
6 FALSE 56237368 2018-01-07 10:41:51 FALSE Khruner 8409334 FALSE 0 FALSE 819091825 tf5qz2yaswx61zrlm9ovxzuhl7r2dc4 2249 119, 978, 500, 292, 225, 199, 292 Kom Firin
7 FALSE 56237368 2018-01-27 12:16:02 FALSE Khruner 8409334 TRUE 0 FALSE 822610647 e6oa4g0qv64icdaq26uu1zzbyr5hcbh 2230 119, 978, 500, 292, 225, 199, 292 Kom Firin
8 FALSE 56237369 2018-01-07 10:42:05 FALSE Editingaccount1994 32794215 FALSE 2 FALSE 819091844 0fyvyh2a8xu41gt8obr34oba0bfixj6 27840 798, 150, 150, 150, 621, 137, 137, 150, 150, 350, 195, 350, 195, 180, 180, 350, 195, 300, 150, 150, 150, 180, 180, 621 User:Editingaccount1994/sandbox
12 FALSE 56237369 2018-01-12 23:28:39 FALSE SporkBot 12406635 TRUE 2 FALSE 820078733 531dizmmloyxffbkdr5vph7owh921eg 27782 798, 150, 150, 150, 621, 137, 137, 150, 150, 350, 195, 350, 195, 180, 180, 350, 195, 300, 150, 150, 150, 180, 180, 621 User:Editingaccount1994/sandbox
13 FALSE 56237369 2018-01-13 13:45:33 FALSE Frietjes 13791031 FALSE 2 FALSE 820177382 nik9p2u2fuk4yazjxt8ymbicxv5qid9 27757 798, 150, 150, 150, 621, 100, 621 User:Editingaccount1994/sandbox
14 FALSE 56237369 2018-01-24 01:35:22 FALSE CommonsDelinker 2304267 FALSE 2 FALSE 822038928 gwk6pampl8si1v5pv3kwgteg710sfw3 27667 798, 150, 150, 150, 621, 100, 621 User:Editingaccount1994/sandbox
15 FALSE 56237370 2018-01-07 10:42:20 FALSE PamD 1368779 FALSE 0 FALSE 819091874 n4ozbsgle13p9yywtfrz982ccj8woc9 25 None Anita del Rey
16 FALSE 56237371 2018-01-07 10:42:27 FALSE ClueBot NG 13286072 FALSE 3 FALSE 819091883 ksohnvsbeuzwpl5vb8a3v8m18hva0a7 1274 119, 157, 119, 157, 119, 157, 119, 157 User talk:119.94.96.157
17 FALSE 56237372 2018-01-07 10:42:50 FALSE Underbar dk 677153 FALSE 14 FALSE 819091914 je7aw21fedbwyqsyofpisdrynsu7olr 113 None Category:Ohmi Railway
18 FALSE 56237375 2018-01-07 10:43:32 FALSE TastyPoutine 882433 FALSE 3 FALSE 819091968 cpm4tkzcx4hc6irr9ukbi06ogud8dtq 199 None User talk:92.226.219.222
19 FALSE 56237375 2018-01-07 11:10:24 FALSE AnomieBOT 7611264 TRUE 3 FALSE 819094036 artmfz8b2gxhb3pp8a5p4ksplxqfkpg 1840 None User talk:92.226.219.222
20 FALSE 56237375 2018-01-07 14:33:36 FALSE Only 702940 FALSE 3 FALSE 819112363 dn9wj0n8d8pdd5lqe56uw5xamupowr1 2949 126, 126, 126, 126 User talk:92.226.219.222
21 FALSE 56237376 2018-01-07 10:44:01 FALSE Dipayanacharya 32794237 FALSE 2 FALSE 819092004 ofueugwatmmn7u73isw732neuza57gk 28 None User:Dipayanacharya
22 FALSE 56237376 2018-01-07 10:49:08 FALSE Dipayanacharya 32794237 FALSE 2 FALSE 819092390 dsz55xv96ec2uv6w9c1z7c52ipfovbw 38 None User:Dipayanacharya
23 FALSE 56237378 2018-01-07 10:44:56 FALSE Vinegarymass911 21516552 FALSE 0 FALSE 819092066 9ma38hak0ef1ew4fpiutxpnzd8oz1wd 65 None BSCIC
24 FALSE 56237379 2018-01-07 10:45:21 FALSE BrownHairedGirl 754619 FALSE 14 FALSE 819092102 4dvakoat58bzyf5hmtthxukt29hip6n 285 None Category:Women government ministers of Yemen
25 FALSE 56237381 2018-01-07 10:45:54 FALSE PRehse 410898 FALSE 1 FALSE 819092135 2sjrxsc7os9k9pg4su2t4rk2j8nn0h7 103 None Talk:List of Morning Glories Characters
26 FALSE 56237382 2018-01-07 10:45:56 FALSE ClueBot NG 13286072 FALSE 3 FALSE 819092138 3y9t5wpk6ur5jhone75rhm4wjf01fgi 1330 106, 207, 126, 114, 106, 207, 126, 114, 106, 207, 126, 114, 106, 207, 126, 114 User talk:106.207.126.114
27 FALSE 56237382 2018-01-07 10:50:22 FALSE HindWIKI 31190506 FALSE 3 FALSE 819092495 8wvn6vh3isyt0dorpe89lztrburgupe 2355 106, 207, 126, 114, 106, 207, 126, 114, 106, 207, 126, 114, 106, 207, 126, 114 User talk:106.207.126.114

52
test/baseline_output/basic_regextest_1.tsv
View File

@ -1,27 +1,27 @@
anon articleid date_time deleted editor editor_id minor namespace page_word revert reverteds revid sha1 testcases text_chars title
FALSE 56237363 2018-01-07 10:40:58 FALSE "NinjaRobotPirate" 3742946 FALSE 3 page, page FALSE 819091731 135nz8q6lfam6cojla7azb7k5alx3t3 None 1141 "User talk:86.139.142.254"
FALSE 56237364 2018-01-07 10:41:10 FALSE "Kavin kavitha" 32792125 FALSE 3 None FALSE 819091755 0pwezjc6yopz0smc8al6ogc4fax5bwo None 663 "User talk:Kavin kavitha"
FALSE 56237365 2018-01-07 10:41:26 FALSE "Amicable always" 32621254 FALSE 3 None FALSE 819091788 sz3t2ap7z8bpkdvdvi195f3i35949bv TestCase, TestCase 399 "User talk:Dr.vivek163"
FALSE 56237366 2018-01-07 10:41:31 FALSE "ClueBot NG" 13286072 FALSE 3 page FALSE 819091796 r6s5j8j3iykenrhuhpnkpsmmd71vubf None 1260 "User talk:Twistorl"
anon articleid date_time deleted editor editorid minor namespace page_word revert reverteds revid sha1 testcases text_chars title
FALSE 56237363 2018-01-07 10:40:58 FALSE "NinjaRobotPirate" 3742946 FALSE 3 page, page FALSE 819091731 135nz8q6lfam6cojla7azb7k5alx3t3 1141 "User talk:86.139.142.254"
FALSE 56237364 2018-01-07 10:41:10 FALSE "Kavin kavitha" 32792125 FALSE 3 FALSE 819091755 0pwezjc6yopz0smc8al6ogc4fax5bwo 663 "User talk:Kavin kavitha"
FALSE 56237365 2018-01-07 10:41:26 FALSE "Amicable always" 32621254 FALSE 3 FALSE 819091788 sz3t2ap7z8bpkdvdvi195f3i35949bv TestCase, TestCase 399 "User talk:Dr.vivek163"
FALSE 56237366 2018-01-07 10:41:31 FALSE "ClueBot NG" 13286072 FALSE 3 page FALSE 819091796 r6s5j8j3iykenrhuhpnkpsmmd71vubf 1260 "User talk:Twistorl"
FALSE 56237368 2018-01-07 10:41:51 FALSE "Khruner" 8409334 FALSE 0 page FALSE 819091825 tf5qz2yaswx61zrlm9ovxzuhl7r2dc4 TestCase 2249 "Kom Firin"
FALSE 56237368 2018-01-27 12:16:02 FALSE "Khruner" 8409334 TRUE 0 page FALSE 822610647 e6oa4g0qv64icdaq26uu1zzbyr5hcbh None 2230 "Kom Firin"
FALSE 56237369 2018-01-07 10:42:05 FALSE "Editingaccount1994" 32794215 FALSE 2 page, page FALSE 819091844 0fyvyh2a8xu41gt8obr34oba0bfixj6 None 27840 "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-07 11:09:52 FALSE "AnomieBOT" 7611264 TRUE 2 page, page FALSE 819093984 8gy52aolt5rg3eaketwj5v7eiw0apv2 None 27787 "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-12 21:45:50 FALSE "SporkBot" 12406635 TRUE 2 page, page FALSE 820064189 he8ydemaanxlrpftqxkez8jfpge1fsj None 27784 "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-12 23:28:11 FALSE "SporkBot" 12406635 TRUE 2 page, page FALSE 820078679 0to17w9rth3url8n7gvucdtobybdq5h None 27783 "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-12 23:28:39 FALSE "SporkBot" 12406635 TRUE 2 page, page FALSE 820078733 531dizmmloyxffbkdr5vph7owh921eg None 27782 "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-13 13:45:33 FALSE "Frietjes" 13791031 FALSE 2 page, page FALSE 820177382 nik9p2u2fuk4yazjxt8ymbicxv5qid9 None 27757 "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-24 01:35:22 FALSE "CommonsDelinker" 2304267 FALSE 2 page, page FALSE 822038928 gwk6pampl8si1v5pv3kwgteg710sfw3 None 27667 "User:Editingaccount1994/sandbox"
FALSE 56237370 2018-01-07 10:42:20 FALSE "PamD" 1368779 FALSE 0 None FALSE 819091874 n4ozbsgle13p9yywtfrz982ccj8woc9 None 25 "Anita del Rey"
FALSE 56237371 2018-01-07 10:42:27 FALSE "ClueBot NG" 13286072 FALSE 3 page FALSE 819091883 ksohnvsbeuzwpl5vb8a3v8m18hva0a7 None 1274 "User talk:119.94.96.157"
FALSE 56237372 2018-01-07 10:42:50 FALSE "Underbar dk" 677153 FALSE 14 None FALSE 819091914 je7aw21fedbwyqsyofpisdrynsu7olr None 113 "Category:Ohmi Railway"
FALSE 56237375 2018-01-07 10:43:32 FALSE "TastyPoutine" 882433 FALSE 3 None FALSE 819091968 cpm4tkzcx4hc6irr9ukbi06ogud8dtq None 199 "User talk:92.226.219.222"
FALSE 56237375 2018-01-07 11:10:24 FALSE "AnomieBOT" 7611264 TRUE 3 page, page, page, page FALSE 819094036 artmfz8b2gxhb3pp8a5p4ksplxqfkpg None 1840 "User talk:92.226.219.222"
FALSE 56237375 2018-01-07 14:33:36 FALSE "Only" 702940 FALSE 3 page, page, page, page, page, page FALSE 819112363 dn9wj0n8d8pdd5lqe56uw5xamupowr1 None 2949 "User talk:92.226.219.222"
FALSE 56237376 2018-01-07 10:44:01 FALSE "Dipayanacharya" 32794237 FALSE 2 None FALSE 819092004 ofueugwatmmn7u73isw732neuza57gk None 28 "User:Dipayanacharya"
FALSE 56237376 2018-01-07 10:49:08 FALSE "Dipayanacharya" 32794237 FALSE 2 None FALSE 819092390 dsz55xv96ec2uv6w9c1z7c52ipfovbw None 38 "User:Dipayanacharya"
FALSE 56237378 2018-01-07 10:44:56 FALSE "Vinegarymass911" 21516552 FALSE 0 None FALSE 819092066 9ma38hak0ef1ew4fpiutxpnzd8oz1wd None 65 "BSCIC"
FALSE 56237379 2018-01-07 10:45:21 FALSE "BrownHairedGirl" 754619 FALSE 14 None FALSE 819092102 4dvakoat58bzyf5hmtthxukt29hip6n None 285 "Category:Women government ministers of Yemen"
FALSE 56237381 2018-01-07 10:45:54 FALSE "PRehse" 410898 FALSE 1 None FALSE 819092135 2sjrxsc7os9k9pg4su2t4rk2j8nn0h7 None 103 "Talk:List of Morning Glories Characters"
FALSE 56237382 2018-01-07 10:45:56 FALSE "ClueBot NG" 13286072 FALSE 3 page FALSE 819092138 3y9t5wpk6ur5jhone75rhm4wjf01fgi None 1330 "User talk:106.207.126.114"
FALSE 56237382 2018-01-07 10:50:22 FALSE "HindWIKI" 31190506 FALSE 3 page FALSE 819092495 8wvn6vh3isyt0dorpe89lztrburgupe None 2355 "User talk:106.207.126.114"
FALSE 56237368 2018-01-27 12:16:02 FALSE "Khruner" 8409334 TRUE 0 page FALSE 822610647 e6oa4g0qv64icdaq26uu1zzbyr5hcbh 2230 "Kom Firin"
FALSE 56237369 2018-01-07 10:42:05 FALSE "Editingaccount1994" 32794215 FALSE 2 page, page FALSE 819091844 0fyvyh2a8xu41gt8obr34oba0bfixj6 27840 "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-07 11:09:52 FALSE "AnomieBOT" 7611264 TRUE 2 page, page FALSE 819093984 8gy52aolt5rg3eaketwj5v7eiw0apv2 27787 "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-12 21:45:50 FALSE "SporkBot" 12406635 TRUE 2 page, page FALSE 820064189 he8ydemaanxlrpftqxkez8jfpge1fsj 27784 "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-12 23:28:11 FALSE "SporkBot" 12406635 TRUE 2 page, page FALSE 820078679 0to17w9rth3url8n7gvucdtobybdq5h 27783 "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-12 23:28:39 FALSE "SporkBot" 12406635 TRUE 2 page, page FALSE 820078733 531dizmmloyxffbkdr5vph7owh921eg 27782 "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-13 13:45:33 FALSE "Frietjes" 13791031 FALSE 2 page, page FALSE 820177382 nik9p2u2fuk4yazjxt8ymbicxv5qid9 27757 "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-24 01:35:22 FALSE "CommonsDelinker" 2304267 FALSE 2 page, page FALSE 822038928 gwk6pampl8si1v5pv3kwgteg710sfw3 27667 "User:Editingaccount1994/sandbox"
FALSE 56237370 2018-01-07 10:42:20 FALSE "PamD" 1368779 FALSE 0 FALSE 819091874 n4ozbsgle13p9yywtfrz982ccj8woc9 25 "Anita del Rey"
FALSE 56237371 2018-01-07 10:42:27 FALSE "ClueBot NG" 13286072 FALSE 3 page FALSE 819091883 ksohnvsbeuzwpl5vb8a3v8m18hva0a7 1274 "User talk:119.94.96.157"
FALSE 56237372 2018-01-07 10:42:50 FALSE "Underbar dk" 677153 FALSE 14 FALSE 819091914 je7aw21fedbwyqsyofpisdrynsu7olr 113 "Category:Ohmi Railway"
FALSE 56237375 2018-01-07 10:43:32 FALSE "TastyPoutine" 882433 FALSE 3 FALSE 819091968 cpm4tkzcx4hc6irr9ukbi06ogud8dtq 199 "User talk:92.226.219.222"
FALSE 56237375 2018-01-07 11:10:24 FALSE "AnomieBOT" 7611264 TRUE 3 page, page, page, page FALSE 819094036 artmfz8b2gxhb3pp8a5p4ksplxqfkpg 1840 "User talk:92.226.219.222"
FALSE 56237375 2018-01-07 14:33:36 FALSE "Only" 702940 FALSE 3 page, page, page, page, page, page FALSE 819112363 dn9wj0n8d8pdd5lqe56uw5xamupowr1 2949 "User talk:92.226.219.222"
FALSE 56237376 2018-01-07 10:44:01 FALSE "Dipayanacharya" 32794237 FALSE 2 FALSE 819092004 ofueugwatmmn7u73isw732neuza57gk 28 "User:Dipayanacharya"
FALSE 56237376 2018-01-07 10:49:08 FALSE "Dipayanacharya" 32794237 FALSE 2 FALSE 819092390 dsz55xv96ec2uv6w9c1z7c52ipfovbw 38 "User:Dipayanacharya"
FALSE 56237378 2018-01-07 10:44:56 FALSE "Vinegarymass911" 21516552 FALSE 0 FALSE 819092066 9ma38hak0ef1ew4fpiutxpnzd8oz1wd 65 "BSCIC"
FALSE 56237379 2018-01-07 10:45:21 FALSE "BrownHairedGirl" 754619 FALSE 14 FALSE 819092102 4dvakoat58bzyf5hmtthxukt29hip6n 285 "Category:Women government ministers of Yemen"
FALSE 56237381 2018-01-07 10:45:54 FALSE "PRehse" 410898 FALSE 1 FALSE 819092135 2sjrxsc7os9k9pg4su2t4rk2j8nn0h7 103 "Talk:List of Morning Glories Characters"
FALSE 56237382 2018-01-07 10:45:56 FALSE "ClueBot NG" 13286072 FALSE 3 page FALSE 819092138 3y9t5wpk6ur5jhone75rhm4wjf01fgi 1330 "User talk:106.207.126.114"
FALSE 56237382 2018-01-07 10:50:22 FALSE "HindWIKI" 31190506 FALSE 3 page FALSE 819092495 8wvn6vh3isyt0dorpe89lztrburgupe 2355 "User talk:106.207.126.114"

1 anon articleid date_time deleted editor editor_id editorid minor namespace page_word revert reverteds revid sha1 testcases text_chars title
2 FALSE 56237363 2018-01-07 10:40:58 FALSE NinjaRobotPirate 3742946 FALSE 3 page, page FALSE 819091731 135nz8q6lfam6cojla7azb7k5alx3t3 None 1141 User talk:86.139.142.254
3 FALSE 56237364 2018-01-07 10:41:10 FALSE Kavin kavitha 32792125 FALSE 3 None FALSE 819091755 0pwezjc6yopz0smc8al6ogc4fax5bwo None 663 User talk:Kavin kavitha
4 FALSE 56237365 2018-01-07 10:41:26 FALSE Amicable always 32621254 FALSE 3 None FALSE 819091788 sz3t2ap7z8bpkdvdvi195f3i35949bv TestCase, TestCase 399 User talk:Dr.vivek163
5 FALSE 56237366 2018-01-07 10:41:31 FALSE ClueBot NG 13286072 FALSE 3 page FALSE 819091796 r6s5j8j3iykenrhuhpnkpsmmd71vubf None 1260 User talk:Twistorl
6 FALSE 56237368 2018-01-07 10:41:51 FALSE Khruner 8409334 FALSE 0 page FALSE 819091825 tf5qz2yaswx61zrlm9ovxzuhl7r2dc4 TestCase 2249 Kom Firin
7 FALSE 56237368 2018-01-27 12:16:02 FALSE Khruner 8409334 TRUE 0 page FALSE 822610647 e6oa4g0qv64icdaq26uu1zzbyr5hcbh None 2230 Kom Firin
8 FALSE 56237369 2018-01-07 10:42:05 FALSE Editingaccount1994 32794215 FALSE 2 page, page FALSE 819091844 0fyvyh2a8xu41gt8obr34oba0bfixj6 None 27840 User:Editingaccount1994/sandbox
9 FALSE 56237369 2018-01-07 11:09:52 FALSE AnomieBOT 7611264 TRUE 2 page, page FALSE 819093984 8gy52aolt5rg3eaketwj5v7eiw0apv2 None 27787 User:Editingaccount1994/sandbox
10 FALSE 56237369 2018-01-12 21:45:50 FALSE SporkBot 12406635 TRUE 2 page, page FALSE 820064189 he8ydemaanxlrpftqxkez8jfpge1fsj None 27784 User:Editingaccount1994/sandbox
11 FALSE 56237369 2018-01-12 23:28:11 FALSE SporkBot 12406635 TRUE 2 page, page FALSE 820078679 0to17w9rth3url8n7gvucdtobybdq5h None 27783 User:Editingaccount1994/sandbox
12 FALSE 56237369 2018-01-12 23:28:39 FALSE SporkBot 12406635 TRUE 2 page, page FALSE 820078733 531dizmmloyxffbkdr5vph7owh921eg None 27782 User:Editingaccount1994/sandbox
13 FALSE 56237369 2018-01-13 13:45:33 FALSE Frietjes 13791031 FALSE 2 page, page FALSE 820177382 nik9p2u2fuk4yazjxt8ymbicxv5qid9 None 27757 User:Editingaccount1994/sandbox
14 FALSE 56237369 2018-01-24 01:35:22 FALSE CommonsDelinker 2304267 FALSE 2 page, page FALSE 822038928 gwk6pampl8si1v5pv3kwgteg710sfw3 None 27667 User:Editingaccount1994/sandbox
15 FALSE 56237370 2018-01-07 10:42:20 FALSE PamD 1368779 FALSE 0 None FALSE 819091874 n4ozbsgle13p9yywtfrz982ccj8woc9 None 25 Anita del Rey
16 FALSE 56237371 2018-01-07 10:42:27 FALSE ClueBot NG 13286072 FALSE 3 page FALSE 819091883 ksohnvsbeuzwpl5vb8a3v8m18hva0a7 None 1274 User talk:119.94.96.157
17 FALSE 56237372 2018-01-07 10:42:50 FALSE Underbar dk 677153 FALSE 14 None FALSE 819091914 je7aw21fedbwyqsyofpisdrynsu7olr None 113 Category:Ohmi Railway
18 FALSE 56237375 2018-01-07 10:43:32 FALSE TastyPoutine 882433 FALSE 3 None FALSE 819091968 cpm4tkzcx4hc6irr9ukbi06ogud8dtq None 199 User talk:92.226.219.222
19 FALSE 56237375 2018-01-07 11:10:24 FALSE AnomieBOT 7611264 TRUE 3 page, page, page, page FALSE 819094036 artmfz8b2gxhb3pp8a5p4ksplxqfkpg None 1840 User talk:92.226.219.222
20 FALSE 56237375 2018-01-07 14:33:36 FALSE Only 702940 FALSE 3 page, page, page, page, page, page FALSE 819112363 dn9wj0n8d8pdd5lqe56uw5xamupowr1 None 2949 User talk:92.226.219.222
21 FALSE 56237376 2018-01-07 10:44:01 FALSE Dipayanacharya 32794237 FALSE 2 None FALSE 819092004 ofueugwatmmn7u73isw732neuza57gk None 28 User:Dipayanacharya
22 FALSE 56237376 2018-01-07 10:49:08 FALSE Dipayanacharya 32794237 FALSE 2 None FALSE 819092390 dsz55xv96ec2uv6w9c1z7c52ipfovbw None 38 User:Dipayanacharya
23 FALSE 56237378 2018-01-07 10:44:56 FALSE Vinegarymass911 21516552 FALSE 0 None FALSE 819092066 9ma38hak0ef1ew4fpiutxpnzd8oz1wd None 65 BSCIC
24 FALSE 56237379 2018-01-07 10:45:21 FALSE BrownHairedGirl 754619 FALSE 14 None FALSE 819092102 4dvakoat58bzyf5hmtthxukt29hip6n None 285 Category:Women government ministers of Yemen
25 FALSE 56237381 2018-01-07 10:45:54 FALSE PRehse 410898 FALSE 1 None FALSE 819092135 2sjrxsc7os9k9pg4su2t4rk2j8nn0h7 None 103 Talk:List of Morning Glories Characters
26 FALSE 56237382 2018-01-07 10:45:56 FALSE ClueBot NG 13286072 FALSE 3 page FALSE 819092138 3y9t5wpk6ur5jhone75rhm4wjf01fgi None 1330 User talk:106.207.126.114
27 FALSE 56237382 2018-01-07 10:50:22 FALSE HindWIKI 31190506 FALSE 3 page FALSE 819092495 8wvn6vh3isyt0dorpe89lztrburgupe None 2355 User talk:106.207.126.114

54
test/baseline_output/basic_regextest_2.tsv
View File

@ -1,27 +1,27 @@
anon articleid chev_com date_time deleted editor editor_id minor namespace revert reverteds revid sha1 text_chars title warning wiki_welcome
FALSE 56237363 None 2018-01-07 10:40:58 FALSE "NinjaRobotPirate" 3742946 FALSE 3 FALSE 819091731 135nz8q6lfam6cojla7azb7k5alx3t3 1141 "User talk:86.139.142.254" None None
FALSE 56237364 None 2018-01-07 10:41:10 FALSE "Kavin kavitha" 32792125 FALSE 3 FALSE 819091755 0pwezjc6yopz0smc8al6ogc4fax5bwo 663 "User talk:Kavin kavitha" None None
FALSE 56237365 None 2018-01-07 10:41:26 FALSE "Amicable always" 32621254 FALSE 3 FALSE 819091788 sz3t2ap7z8bpkdvdvi195f3i35949bv 399 "User talk:Dr.vivek163" None None
FALSE 56237366 None 2018-01-07 10:41:31 FALSE "ClueBot NG" 13286072 FALSE 3 FALSE 819091796 r6s5j8j3iykenrhuhpnkpsmmd71vubf 1260 "User talk:Twistorl" Warning welcome to Wikipedia
FALSE 56237368 None 2018-01-07 10:41:51 FALSE "Khruner" 8409334 FALSE 0 FALSE 819091825 tf5qz2yaswx61zrlm9ovxzuhl7r2dc4 2249 "Kom Firin" None None
FALSE 56237368 None 2018-01-27 12:16:02 FALSE "Khruner" 8409334 TRUE 0 FALSE 822610647 e6oa4g0qv64icdaq26uu1zzbyr5hcbh 2230 "Kom Firin" None None
FALSE 56237369 Chevalier, Chevalier 2018-01-07 10:42:05 FALSE "Editingaccount1994" 32794215 FALSE 2 FALSE 819091844 0fyvyh2a8xu41gt8obr34oba0bfixj6 27840 "User:Editingaccount1994/sandbox" None None
FALSE 56237369 None 2018-01-07 11:09:52 FALSE "AnomieBOT" 7611264 TRUE 2 FALSE 819093984 8gy52aolt5rg3eaketwj5v7eiw0apv2 27787 "User:Editingaccount1994/sandbox" None None
FALSE 56237369 None 2018-01-12 21:45:50 FALSE "SporkBot" 12406635 TRUE 2 FALSE 820064189 he8ydemaanxlrpftqxkez8jfpge1fsj 27784 "User:Editingaccount1994/sandbox" None None
FALSE 56237369 None 2018-01-12 23:28:11 FALSE "SporkBot" 12406635 TRUE 2 FALSE 820078679 0to17w9rth3url8n7gvucdtobybdq5h 27783 "User:Editingaccount1994/sandbox" None None
FALSE 56237369 None 2018-01-12 23:28:39 FALSE "SporkBot" 12406635 TRUE 2 FALSE 820078733 531dizmmloyxffbkdr5vph7owh921eg 27782 "User:Editingaccount1994/sandbox" None None
FALSE 56237369 None 2018-01-13 13:45:33 FALSE "Frietjes" 13791031 FALSE 2 FALSE 820177382 nik9p2u2fuk4yazjxt8ymbicxv5qid9 27757 "User:Editingaccount1994/sandbox" None None
FALSE 56237369 Chevalier, Chevalier 2018-01-24 01:35:22 FALSE "CommonsDelinker" 2304267 FALSE 2 FALSE 822038928 gwk6pampl8si1v5pv3kwgteg710sfw3 27667 "User:Editingaccount1994/sandbox" None None
FALSE 56237370 None 2018-01-07 10:42:20 FALSE "PamD" 1368779 FALSE 0 FALSE 819091874 n4ozbsgle13p9yywtfrz982ccj8woc9 25 "Anita del Rey" None None
FALSE 56237371 None 2018-01-07 10:42:27 FALSE "ClueBot NG" 13286072 FALSE 3 FALSE 819091883 ksohnvsbeuzwpl5vb8a3v8m18hva0a7 1274 "User talk:119.94.96.157" Warning welcome to Wikipedia
FALSE 56237372 None 2018-01-07 10:42:50 FALSE "Underbar dk" 677153 FALSE 14 FALSE 819091914 je7aw21fedbwyqsyofpisdrynsu7olr 113 "Category:Ohmi Railway" None None
FALSE 56237375 None 2018-01-07 10:43:32 FALSE "TastyPoutine" 882433 FALSE 3 FALSE 819091968 cpm4tkzcx4hc6irr9ukbi06ogud8dtq 199 "User talk:92.226.219.222" None None
FALSE 56237375 None 2018-01-07 11:10:24 FALSE "AnomieBOT" 7611264 TRUE 3 FALSE 819094036 artmfz8b2gxhb3pp8a5p4ksplxqfkpg 1840 "User talk:92.226.219.222" None None
FALSE 56237375 None 2018-01-07 14:33:36 FALSE "Only" 702940 FALSE 3 FALSE 819112363 dn9wj0n8d8pdd5lqe56uw5xamupowr1 2949 "User talk:92.226.219.222" None None
FALSE 56237376 None 2018-01-07 10:44:01 FALSE "Dipayanacharya" 32794237 FALSE 2 FALSE 819092004 ofueugwatmmn7u73isw732neuza57gk 28 "User:Dipayanacharya" None None
FALSE 56237376 None 2018-01-07 10:49:08 FALSE "Dipayanacharya" 32794237 FALSE 2 FALSE 819092390 dsz55xv96ec2uv6w9c1z7c52ipfovbw 38 "User:Dipayanacharya" None None
FALSE 56237378 None 2018-01-07 10:44:56 FALSE "Vinegarymass911" 21516552 FALSE 0 FALSE 819092066 9ma38hak0ef1ew4fpiutxpnzd8oz1wd 65 "BSCIC" None None
FALSE 56237379 None 2018-01-07 10:45:21 FALSE "BrownHairedGirl" 754619 FALSE 14 FALSE 819092102 4dvakoat58bzyf5hmtthxukt29hip6n 285 "Category:Women government ministers of Yemen" None None
FALSE 56237381 None 2018-01-07 10:45:54 FALSE "PRehse" 410898 FALSE 1 FALSE 819092135 2sjrxsc7os9k9pg4su2t4rk2j8nn0h7 103 "Talk:List of Morning Glories Characters" None None
FALSE 56237382 None 2018-01-07 10:45:56 FALSE "ClueBot NG" 13286072 FALSE 3 FALSE 819092138 3y9t5wpk6ur5jhone75rhm4wjf01fgi 1330 "User talk:106.207.126.114" Warning welcome to Wikipedia
FALSE 56237382 None 2018-01-07 10:50:22 FALSE "HindWIKI" 31190506 FALSE 3 FALSE 819092495 8wvn6vh3isyt0dorpe89lztrburgupe 2355 "User talk:106.207.126.114" None welcome to Wikipedia
anon articleid chev_com date_time deleted editor editorid minor namespace revert reverteds revid sha1 text_chars title warning wiki_welcome
FALSE 56237363 2018-01-07 10:40:58 FALSE "NinjaRobotPirate" 3742946 FALSE 3 FALSE 819091731 135nz8q6lfam6cojla7azb7k5alx3t3 1141 "User talk:86.139.142.254"
FALSE 56237364 2018-01-07 10:41:10 FALSE "Kavin kavitha" 32792125 FALSE 3 FALSE 819091755 0pwezjc6yopz0smc8al6ogc4fax5bwo 663 "User talk:Kavin kavitha"
FALSE 56237365 2018-01-07 10:41:26 FALSE "Amicable always" 32621254 FALSE 3 FALSE 819091788 sz3t2ap7z8bpkdvdvi195f3i35949bv 399 "User talk:Dr.vivek163"
FALSE 56237366 2018-01-07 10:41:31 FALSE "ClueBot NG" 13286072 FALSE 3 FALSE 819091796 r6s5j8j3iykenrhuhpnkpsmmd71vubf 1260 "User talk:Twistorl" Warning welcome to Wikipedia
FALSE 56237368 2018-01-07 10:41:51 FALSE "Khruner" 8409334 FALSE 0 FALSE 819091825 tf5qz2yaswx61zrlm9ovxzuhl7r2dc4 2249 "Kom Firin"
FALSE 56237368 2018-01-27 12:16:02 FALSE "Khruner" 8409334 TRUE 0 FALSE 822610647 e6oa4g0qv64icdaq26uu1zzbyr5hcbh 2230 "Kom Firin"
FALSE 56237369 Chevalier, Chevalier 2018-01-07 10:42:05 FALSE "Editingaccount1994" 32794215 FALSE 2 FALSE 819091844 0fyvyh2a8xu41gt8obr34oba0bfixj6 27840 "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-07 11:09:52 FALSE "AnomieBOT" 7611264 TRUE 2 FALSE 819093984 8gy52aolt5rg3eaketwj5v7eiw0apv2 27787 "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-12 21:45:50 FALSE "SporkBot" 12406635 TRUE 2 FALSE 820064189 he8ydemaanxlrpftqxkez8jfpge1fsj 27784 "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-12 23:28:11 FALSE "SporkBot" 12406635 TRUE 2 FALSE 820078679 0to17w9rth3url8n7gvucdtobybdq5h 27783 "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-12 23:28:39 FALSE "SporkBot" 12406635 TRUE 2 FALSE 820078733 531dizmmloyxffbkdr5vph7owh921eg 27782 "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-13 13:45:33 FALSE "Frietjes" 13791031 FALSE 2 FALSE 820177382 nik9p2u2fuk4yazjxt8ymbicxv5qid9 27757 "User:Editingaccount1994/sandbox"
FALSE 56237369 Chevalier, Chevalier 2018-01-24 01:35:22 FALSE "CommonsDelinker" 2304267 FALSE 2 FALSE 822038928 gwk6pampl8si1v5pv3kwgteg710sfw3 27667 "User:Editingaccount1994/sandbox"
FALSE 56237370 2018-01-07 10:42:20 FALSE "PamD" 1368779 FALSE 0 FALSE 819091874 n4ozbsgle13p9yywtfrz982ccj8woc9 25 "Anita del Rey"
FALSE 56237371 2018-01-07 10:42:27 FALSE "ClueBot NG" 13286072 FALSE 3 FALSE 819091883 ksohnvsbeuzwpl5vb8a3v8m18hva0a7 1274 "User talk:119.94.96.157" Warning welcome to Wikipedia
FALSE 56237372 2018-01-07 10:42:50 FALSE "Underbar dk" 677153 FALSE 14 FALSE 819091914 je7aw21fedbwyqsyofpisdrynsu7olr 113 "Category:Ohmi Railway"
FALSE 56237375 2018-01-07 10:43:32 FALSE "TastyPoutine" 882433 FALSE 3 FALSE 819091968 cpm4tkzcx4hc6irr9ukbi06ogud8dtq 199 "User talk:92.226.219.222"
FALSE 56237375 2018-01-07 11:10:24 FALSE "AnomieBOT" 7611264 TRUE 3 FALSE 819094036 artmfz8b2gxhb3pp8a5p4ksplxqfkpg 1840 "User talk:92.226.219.222"
FALSE 56237375 2018-01-07 14:33:36 FALSE "Only" 702940 FALSE 3 FALSE 819112363 dn9wj0n8d8pdd5lqe56uw5xamupowr1 2949 "User talk:92.226.219.222"
FALSE 56237376 2018-01-07 10:44:01 FALSE "Dipayanacharya" 32794237 FALSE 2 FALSE 819092004 ofueugwatmmn7u73isw732neuza57gk 28 "User:Dipayanacharya"
FALSE 56237376 2018-01-07 10:49:08 FALSE "Dipayanacharya" 32794237 FALSE 2 FALSE 819092390 dsz55xv96ec2uv6w9c1z7c52ipfovbw 38 "User:Dipayanacharya"
FALSE 56237378 2018-01-07 10:44:56 FALSE "Vinegarymass911" 21516552 FALSE 0 FALSE 819092066 9ma38hak0ef1ew4fpiutxpnzd8oz1wd 65 "BSCIC"
FALSE 56237379 2018-01-07 10:45:21 FALSE "BrownHairedGirl" 754619 FALSE 14 FALSE 819092102 4dvakoat58bzyf5hmtthxukt29hip6n 285 "Category:Women government ministers of Yemen"
FALSE 56237381 2018-01-07 10:45:54 FALSE "PRehse" 410898 FALSE 1 FALSE 819092135 2sjrxsc7os9k9pg4su2t4rk2j8nn0h7 103 "Talk:List of Morning Glories Characters"
FALSE 56237382 2018-01-07 10:45:56 FALSE "ClueBot NG" 13286072 FALSE 3 FALSE 819092138 3y9t5wpk6ur5jhone75rhm4wjf01fgi 1330 "User talk:106.207.126.114" Warning welcome to Wikipedia
FALSE 56237382 2018-01-07 10:50:22 FALSE "HindWIKI" 31190506 FALSE 3 FALSE 819092495 8wvn6vh3isyt0dorpe89lztrburgupe 2355 "User talk:106.207.126.114" welcome to Wikipedia

1 anon articleid chev_com date_time deleted editor editor_id editorid minor namespace revert reverteds revid sha1 text_chars title warning wiki_welcome
2 FALSE 56237363 None 2018-01-07 10:40:58 FALSE NinjaRobotPirate 3742946 FALSE 3 FALSE 819091731 135nz8q6lfam6cojla7azb7k5alx3t3 1141 User talk:86.139.142.254 None None
3 FALSE 56237364 None 2018-01-07 10:41:10 FALSE Kavin kavitha 32792125 FALSE 3 FALSE 819091755 0pwezjc6yopz0smc8al6ogc4fax5bwo 663 User talk:Kavin kavitha None None
4 FALSE 56237365 None 2018-01-07 10:41:26 FALSE Amicable always 32621254 FALSE 3 FALSE 819091788 sz3t2ap7z8bpkdvdvi195f3i35949bv 399 User talk:Dr.vivek163 None None
5 FALSE 56237366 None 2018-01-07 10:41:31 FALSE ClueBot NG 13286072 FALSE 3 FALSE 819091796 r6s5j8j3iykenrhuhpnkpsmmd71vubf 1260 User talk:Twistorl Warning welcome to Wikipedia
6 FALSE 56237368 None 2018-01-07 10:41:51 FALSE Khruner 8409334 FALSE 0 FALSE 819091825 tf5qz2yaswx61zrlm9ovxzuhl7r2dc4 2249 Kom Firin None None
7 FALSE 56237368 None 2018-01-27 12:16:02 FALSE Khruner 8409334 TRUE 0 FALSE 822610647 e6oa4g0qv64icdaq26uu1zzbyr5hcbh 2230 Kom Firin None None
8 FALSE 56237369 Chevalier, Chevalier 2018-01-07 10:42:05 FALSE Editingaccount1994 32794215 FALSE 2 FALSE 819091844 0fyvyh2a8xu41gt8obr34oba0bfixj6 27840 User:Editingaccount1994/sandbox None None
9 FALSE 56237369 None 2018-01-07 11:09:52 FALSE AnomieBOT 7611264 TRUE 2 FALSE 819093984 8gy52aolt5rg3eaketwj5v7eiw0apv2 27787 User:Editingaccount1994/sandbox None None
10 FALSE 56237369 None 2018-01-12 21:45:50 FALSE SporkBot 12406635 TRUE 2 FALSE 820064189 he8ydemaanxlrpftqxkez8jfpge1fsj 27784 User:Editingaccount1994/sandbox None None
11 FALSE 56237369 None 2018-01-12 23:28:11 FALSE SporkBot 12406635 TRUE 2 FALSE 820078679 0to17w9rth3url8n7gvucdtobybdq5h 27783 User:Editingaccount1994/sandbox None None
12 FALSE 56237369 None 2018-01-12 23:28:39 FALSE SporkBot 12406635 TRUE 2 FALSE 820078733 531dizmmloyxffbkdr5vph7owh921eg 27782 User:Editingaccount1994/sandbox None None
13 FALSE 56237369 None 2018-01-13 13:45:33 FALSE Frietjes 13791031 FALSE 2 FALSE 820177382 nik9p2u2fuk4yazjxt8ymbicxv5qid9 27757 User:Editingaccount1994/sandbox None None
14 FALSE 56237369 Chevalier, Chevalier 2018-01-24 01:35:22 FALSE CommonsDelinker 2304267 FALSE 2 FALSE 822038928 gwk6pampl8si1v5pv3kwgteg710sfw3 27667 User:Editingaccount1994/sandbox None None
15 FALSE 56237370 None 2018-01-07 10:42:20 FALSE PamD 1368779 FALSE 0 FALSE 819091874 n4ozbsgle13p9yywtfrz982ccj8woc9 25 Anita del Rey None None
16 FALSE 56237371 None 2018-01-07 10:42:27 FALSE ClueBot NG 13286072 FALSE 3 FALSE 819091883 ksohnvsbeuzwpl5vb8a3v8m18hva0a7 1274 User talk:119.94.96.157 Warning welcome to Wikipedia
17 FALSE 56237372 None 2018-01-07 10:42:50 FALSE Underbar dk 677153 FALSE 14 FALSE 819091914 je7aw21fedbwyqsyofpisdrynsu7olr 113 Category:Ohmi Railway None None
18 FALSE 56237375 None 2018-01-07 10:43:32 FALSE TastyPoutine 882433 FALSE 3 FALSE 819091968 cpm4tkzcx4hc6irr9ukbi06ogud8dtq 199 User talk:92.226.219.222 None None
19 FALSE 56237375 None 2018-01-07 11:10:24 FALSE AnomieBOT 7611264 TRUE 3 FALSE 819094036 artmfz8b2gxhb3pp8a5p4ksplxqfkpg 1840 User talk:92.226.219.222 None None
20 FALSE 56237375 None 2018-01-07 14:33:36 FALSE Only 702940 FALSE 3 FALSE 819112363 dn9wj0n8d8pdd5lqe56uw5xamupowr1 2949 User talk:92.226.219.222 None None
21 FALSE 56237376 None 2018-01-07 10:44:01 FALSE Dipayanacharya 32794237 FALSE 2 FALSE 819092004 ofueugwatmmn7u73isw732neuza57gk 28 User:Dipayanacharya None None
22 FALSE 56237376 None 2018-01-07 10:49:08 FALSE Dipayanacharya 32794237 FALSE 2 FALSE 819092390 dsz55xv96ec2uv6w9c1z7c52ipfovbw 38 User:Dipayanacharya None None
23 FALSE 56237378 None 2018-01-07 10:44:56 FALSE Vinegarymass911 21516552 FALSE 0 FALSE 819092066 9ma38hak0ef1ew4fpiutxpnzd8oz1wd 65 BSCIC None None
24 FALSE 56237379 None 2018-01-07 10:45:21 FALSE BrownHairedGirl 754619 FALSE 14 FALSE 819092102 4dvakoat58bzyf5hmtthxukt29hip6n 285 Category:Women government ministers of Yemen None None
25 FALSE 56237381 None 2018-01-07 10:45:54 FALSE PRehse 410898 FALSE 1 FALSE 819092135 2sjrxsc7os9k9pg4su2t4rk2j8nn0h7 103 Talk:List of Morning Glories Characters None None
26 FALSE 56237382 None 2018-01-07 10:45:56 FALSE ClueBot NG 13286072 FALSE 3 FALSE 819092138 3y9t5wpk6ur5jhone75rhm4wjf01fgi 1330 User talk:106.207.126.114 Warning welcome to Wikipedia
27 FALSE 56237382 None 2018-01-07 10:50:22 FALSE HindWIKI 31190506 FALSE 3 FALSE 819092495 8wvn6vh3isyt0dorpe89lztrburgupe 2355 User talk:106.207.126.114 None welcome to Wikipedia

50
test/baseline_output/basic_regextest_3.tsv
View File

@ -1,27 +1,27 @@
anon articleid date_time deleted editor editor_id minor namespace revert reverteds revid sha1 text_chars title wp_evade
anon articleid date_time deleted editor editorid minor namespace revert reverteds revid sha1 text_chars title wp_evade
FALSE 56237363 2018-01-07 10:40:58 FALSE "NinjaRobotPirate" 3742946 FALSE 3 FALSE 819091731 135nz8q6lfam6cojla7azb7k5alx3t3 1141 "User talk:86.139.142.254" WP:EVADE
FALSE 56237364 2018-01-07 10:41:10 FALSE "Kavin kavitha" 32792125 FALSE 3 FALSE 819091755 0pwezjc6yopz0smc8al6ogc4fax5bwo 663 "User talk:Kavin kavitha" None
FALSE 56237365 2018-01-07 10:41:26 FALSE "Amicable always" 32621254 FALSE 3 FALSE 819091788 sz3t2ap7z8bpkdvdvi195f3i35949bv 399 "User talk:Dr.vivek163" None
FALSE 56237366 2018-01-07 10:41:31 FALSE "ClueBot NG" 13286072 FALSE 3 FALSE 819091796 r6s5j8j3iykenrhuhpnkpsmmd71vubf 1260 "User talk:Twistorl" None
FALSE 56237368 2018-01-07 10:41:51 FALSE "Khruner" 8409334 FALSE 0 FALSE 819091825 tf5qz2yaswx61zrlm9ovxzuhl7r2dc4 2249 "Kom Firin" None
FALSE 56237368 2018-01-27 12:16:02 FALSE "Khruner" 8409334 TRUE 0 FALSE 822610647 e6oa4g0qv64icdaq26uu1zzbyr5hcbh 2230 "Kom Firin" None
FALSE 56237369 2018-01-07 10:42:05 FALSE "Editingaccount1994" 32794215 FALSE 2 FALSE 819091844 0fyvyh2a8xu41gt8obr34oba0bfixj6 27840 "User:Editingaccount1994/sandbox" None
FALSE 56237369 2018-01-07 11:09:52 FALSE "AnomieBOT" 7611264 TRUE 2 FALSE 819093984 8gy52aolt5rg3eaketwj5v7eiw0apv2 27787 "User:Editingaccount1994/sandbox" None
FALSE 56237369 2018-01-12 21:45:50 FALSE "SporkBot" 12406635 TRUE 2 FALSE 820064189 he8ydemaanxlrpftqxkez8jfpge1fsj 27784 "User:Editingaccount1994/sandbox" None
FALSE 56237369 2018-01-12 23:28:11 FALSE "SporkBot" 12406635 TRUE 2 FALSE 820078679 0to17w9rth3url8n7gvucdtobybdq5h 27783 "User:Editingaccount1994/sandbox" None
FALSE 56237369 2018-01-12 23:28:39 FALSE "SporkBot" 12406635 TRUE 2 FALSE 820078733 531dizmmloyxffbkdr5vph7owh921eg 27782 "User:Editingaccount1994/sandbox" None
FALSE 56237369 2018-01-13 13:45:33 FALSE "Frietjes" 13791031 FALSE 2 FALSE 820177382 nik9p2u2fuk4yazjxt8ymbicxv5qid9 27757 "User:Editingaccount1994/sandbox" None
FALSE 56237369 2018-01-24 01:35:22 FALSE "CommonsDelinker" 2304267 FALSE 2 FALSE 822038928 gwk6pampl8si1v5pv3kwgteg710sfw3 27667 "User:Editingaccount1994/sandbox" None
FALSE 56237370 2018-01-07 10:42:20 FALSE "PamD" 1368779 FALSE 0 FALSE 819091874 n4ozbsgle13p9yywtfrz982ccj8woc9 25 "Anita del Rey" None
FALSE 56237371 2018-01-07 10:42:27 FALSE "ClueBot NG" 13286072 FALSE 3 FALSE 819091883 ksohnvsbeuzwpl5vb8a3v8m18hva0a7 1274 "User talk:119.94.96.157" None
FALSE 56237372 2018-01-07 10:42:50 FALSE "Underbar dk" 677153 FALSE 14 FALSE 819091914 je7aw21fedbwyqsyofpisdrynsu7olr 113 "Category:Ohmi Railway" None
FALSE 56237375 2018-01-07 10:43:32 FALSE "TastyPoutine" 882433 FALSE 3 FALSE 819091968 cpm4tkzcx4hc6irr9ukbi06ogud8dtq 199 "User talk:92.226.219.222" None
FALSE 56237375 2018-01-07 11:10:24 FALSE "AnomieBOT" 7611264 TRUE 3 FALSE 819094036 artmfz8b2gxhb3pp8a5p4ksplxqfkpg 1840 "User talk:92.226.219.222" None
FALSE 56237364 2018-01-07 10:41:10 FALSE "Kavin kavitha" 32792125 FALSE 3 FALSE 819091755 0pwezjc6yopz0smc8al6ogc4fax5bwo 663 "User talk:Kavin kavitha"
FALSE 56237365 2018-01-07 10:41:26 FALSE "Amicable always" 32621254 FALSE 3 FALSE 819091788 sz3t2ap7z8bpkdvdvi195f3i35949bv 399 "User talk:Dr.vivek163"
FALSE 56237366 2018-01-07 10:41:31 FALSE "ClueBot NG" 13286072 FALSE 3 FALSE 819091796 r6s5j8j3iykenrhuhpnkpsmmd71vubf 1260 "User talk:Twistorl"
FALSE 56237368 2018-01-07 10:41:51 FALSE "Khruner" 8409334 FALSE 0 FALSE 819091825 tf5qz2yaswx61zrlm9ovxzuhl7r2dc4 2249 "Kom Firin"
FALSE 56237368 2018-01-27 12:16:02 FALSE "Khruner" 8409334 TRUE 0 FALSE 822610647 e6oa4g0qv64icdaq26uu1zzbyr5hcbh 2230 "Kom Firin"
FALSE 56237369 2018-01-07 10:42:05 FALSE "Editingaccount1994" 32794215 FALSE 2 FALSE 819091844 0fyvyh2a8xu41gt8obr34oba0bfixj6 27840 "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-07 11:09:52 FALSE "AnomieBOT" 7611264 TRUE 2 FALSE 819093984 8gy52aolt5rg3eaketwj5v7eiw0apv2 27787 "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-12 21:45:50 FALSE "SporkBot" 12406635 TRUE 2 FALSE 820064189 he8ydemaanxlrpftqxkez8jfpge1fsj 27784 "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-12 23:28:11 FALSE "SporkBot" 12406635 TRUE 2 FALSE 820078679 0to17w9rth3url8n7gvucdtobybdq5h 27783 "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-12 23:28:39 FALSE "SporkBot" 12406635 TRUE 2 FALSE 820078733 531dizmmloyxffbkdr5vph7owh921eg 27782 "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-13 13:45:33 FALSE "Frietjes" 13791031 FALSE 2 FALSE 820177382 nik9p2u2fuk4yazjxt8ymbicxv5qid9 27757 "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-24 01:35:22 FALSE "CommonsDelinker" 2304267 FALSE 2 FALSE 822038928 gwk6pampl8si1v5pv3kwgteg710sfw3 27667 "User:Editingaccount1994/sandbox"
FALSE 56237370 2018-01-07 10:42:20 FALSE "PamD" 1368779 FALSE 0 FALSE 819091874 n4ozbsgle13p9yywtfrz982ccj8woc9 25 "Anita del Rey"
FALSE 56237371 2018-01-07 10:42:27 FALSE "ClueBot NG" 13286072 FALSE 3 FALSE 819091883 ksohnvsbeuzwpl5vb8a3v8m18hva0a7 1274 "User talk:119.94.96.157"
FALSE 56237372 2018-01-07 10:42:50 FALSE "Underbar dk" 677153 FALSE 14 FALSE 819091914 je7aw21fedbwyqsyofpisdrynsu7olr 113 "Category:Ohmi Railway"
FALSE 56237375 2018-01-07 10:43:32 FALSE "TastyPoutine" 882433 FALSE 3 FALSE 819091968 cpm4tkzcx4hc6irr9ukbi06ogud8dtq 199 "User talk:92.226.219.222"
FALSE 56237375 2018-01-07 11:10:24 FALSE "AnomieBOT" 7611264 TRUE 3 FALSE 819094036 artmfz8b2gxhb3pp8a5p4ksplxqfkpg 1840 "User talk:92.226.219.222"
FALSE 56237375 2018-01-07 14:33:36 FALSE "Only" 702940 FALSE 3 FALSE 819112363 dn9wj0n8d8pdd5lqe56uw5xamupowr1 2949 "User talk:92.226.219.222" WP:EVADE
FALSE 56237376 2018-01-07 10:44:01 FALSE "Dipayanacharya" 32794237 FALSE 2 FALSE 819092004 ofueugwatmmn7u73isw732neuza57gk 28 "User:Dipayanacharya" None
FALSE 56237376 2018-01-07 10:49:08 FALSE "Dipayanacharya" 32794237 FALSE 2 FALSE 819092390 dsz55xv96ec2uv6w9c1z7c52ipfovbw 38 "User:Dipayanacharya" None
FALSE 56237378 2018-01-07 10:44:56 FALSE "Vinegarymass911" 21516552 FALSE 0 FALSE 819092066 9ma38hak0ef1ew4fpiutxpnzd8oz1wd 65 "BSCIC" None
FALSE 56237379 2018-01-07 10:45:21 FALSE "BrownHairedGirl" 754619 FALSE 14 FALSE 819092102 4dvakoat58bzyf5hmtthxukt29hip6n 285 "Category:Women government ministers of Yemen" None
FALSE 56237381 2018-01-07 10:45:54 FALSE "PRehse" 410898 FALSE 1 FALSE 819092135 2sjrxsc7os9k9pg4su2t4rk2j8nn0h7 103 "Talk:List of Morning Glories Characters" None
FALSE 56237382 2018-01-07 10:45:56 FALSE "ClueBot NG" 13286072 FALSE 3 FALSE 819092138 3y9t5wpk6ur5jhone75rhm4wjf01fgi 1330 "User talk:106.207.126.114" None
FALSE 56237382 2018-01-07 10:50:22 FALSE "HindWIKI" 31190506 FALSE 3 FALSE 819092495 8wvn6vh3isyt0dorpe89lztrburgupe 2355 "User talk:106.207.126.114" None
FALSE 56237376 2018-01-07 10:44:01 FALSE "Dipayanacharya" 32794237 FALSE 2 FALSE 819092004 ofueugwatmmn7u73isw732neuza57gk 28 "User:Dipayanacharya"
FALSE 56237376 2018-01-07 10:49:08 FALSE "Dipayanacharya" 32794237 FALSE 2 FALSE 819092390 dsz55xv96ec2uv6w9c1z7c52ipfovbw 38 "User:Dipayanacharya"
FALSE 56237378 2018-01-07 10:44:56 FALSE "Vinegarymass911" 21516552 FALSE 0 FALSE 819092066 9ma38hak0ef1ew4fpiutxpnzd8oz1wd 65 "BSCIC"
FALSE 56237379 2018-01-07 10:45:21 FALSE "BrownHairedGirl" 754619 FALSE 14 FALSE 819092102 4dvakoat58bzyf5hmtthxukt29hip6n 285 "Category:Women government ministers of Yemen"
FALSE 56237381 2018-01-07 10:45:54 FALSE "PRehse" 410898 FALSE 1 FALSE 819092135 2sjrxsc7os9k9pg4su2t4rk2j8nn0h7 103 "Talk:List of Morning Glories Characters"
FALSE 56237382 2018-01-07 10:45:56 FALSE "ClueBot NG" 13286072 FALSE 3 FALSE 819092138 3y9t5wpk6ur5jhone75rhm4wjf01fgi 1330 "User talk:106.207.126.114"
FALSE 56237382 2018-01-07 10:50:22 FALSE "HindWIKI" 31190506 FALSE 3 FALSE 819092495 8wvn6vh3isyt0dorpe89lztrburgupe 2355 "User talk:106.207.126.114"

1 anon articleid date_time deleted editor editor_id editorid minor namespace revert reverteds revid sha1 text_chars title wp_evade
2 FALSE 56237363 2018-01-07 10:40:58 FALSE NinjaRobotPirate 3742946 FALSE 3 FALSE 819091731 135nz8q6lfam6cojla7azb7k5alx3t3 1141 User talk:86.139.142.254 WP:EVADE
3 FALSE 56237364 2018-01-07 10:41:10 FALSE Kavin kavitha 32792125 FALSE 3 FALSE 819091755 0pwezjc6yopz0smc8al6ogc4fax5bwo 663 User talk:Kavin kavitha None
4 FALSE 56237365 2018-01-07 10:41:26 FALSE Amicable always 32621254 FALSE 3 FALSE 819091788 sz3t2ap7z8bpkdvdvi195f3i35949bv 399 User talk:Dr.vivek163 None
5 FALSE 56237366 2018-01-07 10:41:31 FALSE ClueBot NG 13286072 FALSE 3 FALSE 819091796 r6s5j8j3iykenrhuhpnkpsmmd71vubf 1260 User talk:Twistorl None
6 FALSE 56237368 2018-01-07 10:41:51 FALSE Khruner 8409334 FALSE 0 FALSE 819091825 tf5qz2yaswx61zrlm9ovxzuhl7r2dc4 2249 Kom Firin None
7 FALSE 56237368 2018-01-27 12:16:02 FALSE Khruner 8409334 TRUE 0 FALSE 822610647 e6oa4g0qv64icdaq26uu1zzbyr5hcbh 2230 Kom Firin None
8 FALSE 56237369 2018-01-07 10:42:05 FALSE Editingaccount1994 32794215 FALSE 2 FALSE 819091844 0fyvyh2a8xu41gt8obr34oba0bfixj6 27840 User:Editingaccount1994/sandbox None
9 FALSE 56237369 2018-01-07 11:09:52 FALSE AnomieBOT 7611264 TRUE 2 FALSE 819093984 8gy52aolt5rg3eaketwj5v7eiw0apv2 27787 User:Editingaccount1994/sandbox None
10 FALSE 56237369 2018-01-12 21:45:50 FALSE SporkBot 12406635 TRUE 2 FALSE 820064189 he8ydemaanxlrpftqxkez8jfpge1fsj 27784 User:Editingaccount1994/sandbox None
11 FALSE 56237369 2018-01-12 23:28:11 FALSE SporkBot 12406635 TRUE 2 FALSE 820078679 0to17w9rth3url8n7gvucdtobybdq5h 27783 User:Editingaccount1994/sandbox None
12 FALSE 56237369 2018-01-12 23:28:39 FALSE SporkBot 12406635 TRUE 2 FALSE 820078733 531dizmmloyxffbkdr5vph7owh921eg 27782 User:Editingaccount1994/sandbox None
13 FALSE 56237369 2018-01-13 13:45:33 FALSE Frietjes 13791031 FALSE 2 FALSE 820177382 nik9p2u2fuk4yazjxt8ymbicxv5qid9 27757 User:Editingaccount1994/sandbox None
14 FALSE 56237369 2018-01-24 01:35:22 FALSE CommonsDelinker 2304267 FALSE 2 FALSE 822038928 gwk6pampl8si1v5pv3kwgteg710sfw3 27667 User:Editingaccount1994/sandbox None
15 FALSE 56237370 2018-01-07 10:42:20 FALSE PamD 1368779 FALSE 0 FALSE 819091874 n4ozbsgle13p9yywtfrz982ccj8woc9 25 Anita del Rey None
16 FALSE 56237371 2018-01-07 10:42:27 FALSE ClueBot NG 13286072 FALSE 3 FALSE 819091883 ksohnvsbeuzwpl5vb8a3v8m18hva0a7 1274 User talk:119.94.96.157 None
17 FALSE 56237372 2018-01-07 10:42:50 FALSE Underbar dk 677153 FALSE 14 FALSE 819091914 je7aw21fedbwyqsyofpisdrynsu7olr 113 Category:Ohmi Railway None
18 FALSE 56237375 2018-01-07 10:43:32 FALSE TastyPoutine 882433 FALSE 3 FALSE 819091968 cpm4tkzcx4hc6irr9ukbi06ogud8dtq 199 User talk:92.226.219.222 None
19 FALSE 56237375 2018-01-07 11:10:24 FALSE AnomieBOT 7611264 TRUE 3 FALSE 819094036 artmfz8b2gxhb3pp8a5p4ksplxqfkpg 1840 User talk:92.226.219.222 None
20 FALSE 56237375 2018-01-07 14:33:36 FALSE Only 702940 FALSE 3 FALSE 819112363 dn9wj0n8d8pdd5lqe56uw5xamupowr1 2949 User talk:92.226.219.222 WP:EVADE
21 FALSE 56237376 2018-01-07 10:44:01 FALSE Dipayanacharya 32794237 FALSE 2 FALSE 819092004 ofueugwatmmn7u73isw732neuza57gk 28 User:Dipayanacharya None
22 FALSE 56237376 2018-01-07 10:49:08 FALSE Dipayanacharya 32794237 FALSE 2 FALSE 819092390 dsz55xv96ec2uv6w9c1z7c52ipfovbw 38 User:Dipayanacharya None
23 FALSE 56237378 2018-01-07 10:44:56 FALSE Vinegarymass911 21516552 FALSE 0 FALSE 819092066 9ma38hak0ef1ew4fpiutxpnzd8oz1wd 65 BSCIC None
24 FALSE 56237379 2018-01-07 10:45:21 FALSE BrownHairedGirl 754619 FALSE 14 FALSE 819092102 4dvakoat58bzyf5hmtthxukt29hip6n 285 Category:Women government ministers of Yemen None
25 FALSE 56237381 2018-01-07 10:45:54 FALSE PRehse 410898 FALSE 1 FALSE 819092135 2sjrxsc7os9k9pg4su2t4rk2j8nn0h7 103 Talk:List of Morning Glories Characters None
26 FALSE 56237382 2018-01-07 10:45:56 FALSE ClueBot NG 13286072 FALSE 3 FALSE 819092138 3y9t5wpk6ur5jhone75rhm4wjf01fgi 1330 User talk:106.207.126.114 None
27 FALSE 56237382 2018-01-07 10:50:22 FALSE HindWIKI 31190506 FALSE 3 FALSE 819092495 8wvn6vh3isyt0dorpe89lztrburgupe 2355 User talk:106.207.126.114 None

54
test/baseline_output/capturegroup_regextest_0.tsv
View File

@ -1,27 +1,27 @@
anon articleid date_time deleted editor editor_id li_cheval minor namespace revert reverteds revid sha1 text_chars three_cat three_letter three_number title
FALSE 56237363 2018-01-07 10:40:58 FALSE "NinjaRobotPirate" 3742946 None FALSE 3 FALSE 819091731 135nz8q6lfam6cojla7azb7k5alx3t3 1141 None has, has None "User talk:86.139.142.254"
FALSE 56237364 2018-01-07 10:41:10 FALSE "Kavin kavitha" 32792125 None FALSE 3 FALSE 819091755 0pwezjc6yopz0smc8al6ogc4fax5bwo 663 None AES, for 01, 12, 2001 "User talk:Kavin kavitha"
FALSE 56237365 2018-01-07 10:41:26 FALSE "Amicable always" 32621254 None FALSE 3 FALSE 819091788 sz3t2ap7z8bpkdvdvi195f3i35949bv 399 None new None "User talk:Dr.vivek163"
FALSE 56237366 2018-01-07 10:41:31 FALSE "ClueBot NG" 13286072 None FALSE 3 FALSE 819091796 r6s5j8j3iykenrhuhpnkpsmmd71vubf 1260 None None 1 "User talk:Twistorl"
FALSE 56237368 2018-01-07 10:41:51 FALSE "Khruner" 8409334 None FALSE 0 FALSE 819091825 tf5qz2yaswx61zrlm9ovxzuhl7r2dc4 2249 None AES, jpg, the, the, the, the, and, you, Tor 67, 119 "Kom Firin"
FALSE 56237368 2018-01-27 12:16:02 FALSE "Khruner" 8409334 None TRUE 0 FALSE 822610647 e6oa4g0qv64icdaq26uu1zzbyr5hcbh 2230 None None None "Kom Firin"
FALSE 56237369 2018-01-07 10:42:05 FALSE "Editingaccount1994" 32794215 Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier FALSE 2 FALSE 819091844 0fyvyh2a8xu41gt8obr34oba0bfixj6 27840 None AES, nom None "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-07 11:09:52 FALSE "AnomieBOT" 7611264 Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier TRUE 2 FALSE 819093984 8gy52aolt5rg3eaketwj5v7eiw0apv2 27787 None web, See, for None "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-12 21:45:50 FALSE "SporkBot" 12406635 Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier TRUE 2 FALSE 820064189 he8ydemaanxlrpftqxkez8jfpge1fsj 27784 None per, TFD, TFD None "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-12 23:28:11 FALSE "SporkBot" 12406635 Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier TRUE 2 FALSE 820078679 0to17w9rth3url8n7gvucdtobybdq5h 27783 None per, for, Log, TFD 2010, 13 "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-12 23:28:39 FALSE "SporkBot" 12406635 Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier TRUE 2 FALSE 820078733 531dizmmloyxffbkdr5vph7owh921eg 27782 None per, for, Log, TFD 2011, 17 "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-13 13:45:33 FALSE "Frietjes" 13791031 Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier FALSE 2 FALSE 820177382 nik9p2u2fuk4yazjxt8ymbicxv5qid9 27757 None you, are, tor, you None "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-24 01:35:22 FALSE "CommonsDelinker" 2304267 Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier FALSE 2 FALSE 822038928 gwk6pampl8si1v5pv3kwgteg710sfw3 27667 None jpg, jpg, has, COM 16, 2018 "User:Editingaccount1994/sandbox"
FALSE 56237370 2018-01-07 10:42:20 FALSE "PamD" 1368779 None FALSE 0 FALSE 819091874 n4ozbsgle13p9yywtfrz982ccj8woc9 25 None alt None "Anita del Rey"
FALSE 56237371 2018-01-07 10:42:27 FALSE "ClueBot NG" 13286072 None FALSE 3 FALSE 819091883 ksohnvsbeuzwpl5vb8a3v8m18hva0a7 1274 None None 119, 94, 96, 157, 119, 94, 96, 157, 1 "User talk:119.94.96.157"
FALSE 56237372 2018-01-07 10:42:50 FALSE "Underbar dk" 677153 None FALSE 14 FALSE 819091914 je7aw21fedbwyqsyofpisdrynsu7olr 113 None AES None "Category:Ohmi Railway"
FALSE 56237375 2018-01-07 10:43:32 FALSE "TastyPoutine" 882433 None FALSE 3 FALSE 819091968 cpm4tkzcx4hc6irr9ukbi06ogud8dtq 199 None AES None "User talk:92.226.219.222"
FALSE 56237375 2018-01-07 11:10:24 FALSE "AnomieBOT" 7611264 None TRUE 3 FALSE 819094036 artmfz8b2gxhb3pp8a5p4ksplxqfkpg 1840 None See, for None "User talk:92.226.219.222"
FALSE 56237375 2018-01-07 14:33:36 FALSE "Only" 702940 None FALSE 3 FALSE 819112363 dn9wj0n8d8pdd5lqe56uw5xamupowr1 2949 None has, has None "User talk:92.226.219.222"
FALSE 56237376 2018-01-07 10:44:01 FALSE "Dipayanacharya" 32794237 None FALSE 2 FALSE 819092004 ofueugwatmmn7u73isw732neuza57gk 28 None None None "User:Dipayanacharya"
FALSE 56237376 2018-01-07 10:49:08 FALSE "Dipayanacharya" 32794237 None FALSE 2 FALSE 819092390 dsz55xv96ec2uv6w9c1z7c52ipfovbw 38 None None None "User:Dipayanacharya"
FALSE 56237378 2018-01-07 10:44:56 FALSE "Vinegarymass911" 21516552 None FALSE 0 FALSE 819092066 9ma38hak0ef1ew4fpiutxpnzd8oz1wd 65 None AES, and None "BSCIC"
FALSE 56237379 2018-01-07 10:45:21 FALSE "BrownHairedGirl" 754619 None FALSE 14 FALSE 819092102 4dvakoat58bzyf5hmtthxukt29hip6n 285 None AES, Non None "Category:Women government ministers of Yemen"
FALSE 56237381 2018-01-07 10:45:54 FALSE "PRehse" 410898 None FALSE 1 FALSE 819092135 2sjrxsc7os9k9pg4su2t4rk2j8nn0h7 103 None AES, low, low None "Talk:List of Morning Glories Characters"
FALSE 56237382 2018-01-07 10:45:56 FALSE "ClueBot NG" 13286072 None FALSE 3 FALSE 819092138 3y9t5wpk6ur5jhone75rhm4wjf01fgi 1330 None None 106, 207, 126, 114, 106, 207, 126, 114, 1 "User talk:106.207.126.114"
FALSE 56237382 2018-01-07 10:50:22 FALSE "HindWIKI" 31190506 None FALSE 3 FALSE 819092495 8wvn6vh3isyt0dorpe89lztrburgupe 2355 None None None "User talk:106.207.126.114"
anon articleid date_time deleted editor editorid li_cheval minor namespace revert reverteds revid sha1 text_chars three_cat three_letter three_number title
FALSE 56237363 2018-01-07 10:40:58 FALSE "NinjaRobotPirate" 3742946 FALSE 3 FALSE 819091731 135nz8q6lfam6cojla7azb7k5alx3t3 1141 has, has "User talk:86.139.142.254"
FALSE 56237364 2018-01-07 10:41:10 FALSE "Kavin kavitha" 32792125 FALSE 3 FALSE 819091755 0pwezjc6yopz0smc8al6ogc4fax5bwo 663 AES, for 01, 12, 2001 "User talk:Kavin kavitha"
FALSE 56237365 2018-01-07 10:41:26 FALSE "Amicable always" 32621254 FALSE 3 FALSE 819091788 sz3t2ap7z8bpkdvdvi195f3i35949bv 399 new "User talk:Dr.vivek163"
FALSE 56237366 2018-01-07 10:41:31 FALSE "ClueBot NG" 13286072 FALSE 3 FALSE 819091796 r6s5j8j3iykenrhuhpnkpsmmd71vubf 1260 1 "User talk:Twistorl"
FALSE 56237368 2018-01-07 10:41:51 FALSE "Khruner" 8409334 FALSE 0 FALSE 819091825 tf5qz2yaswx61zrlm9ovxzuhl7r2dc4 2249 AES, jpg, the, the, the, the, and, you, Tor 67, 119 "Kom Firin"
FALSE 56237368 2018-01-27 12:16:02 FALSE "Khruner" 8409334 TRUE 0 FALSE 822610647 e6oa4g0qv64icdaq26uu1zzbyr5hcbh 2230 "Kom Firin"
FALSE 56237369 2018-01-07 10:42:05 FALSE "Editingaccount1994" 32794215 Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier FALSE 2 FALSE 819091844 0fyvyh2a8xu41gt8obr34oba0bfixj6 27840 AES, nom "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-07 11:09:52 FALSE "AnomieBOT" 7611264 Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier TRUE 2 FALSE 819093984 8gy52aolt5rg3eaketwj5v7eiw0apv2 27787 web, See, for "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-12 21:45:50 FALSE "SporkBot" 12406635 Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier TRUE 2 FALSE 820064189 he8ydemaanxlrpftqxkez8jfpge1fsj 27784 per, TFD, TFD "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-12 23:28:11 FALSE "SporkBot" 12406635 Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier TRUE 2 FALSE 820078679 0to17w9rth3url8n7gvucdtobybdq5h 27783 per, for, Log, TFD 2010, 13 "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-12 23:28:39 FALSE "SporkBot" 12406635 Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier TRUE 2 FALSE 820078733 531dizmmloyxffbkdr5vph7owh921eg 27782 per, for, Log, TFD 2011, 17 "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-13 13:45:33 FALSE "Frietjes" 13791031 Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier FALSE 2 FALSE 820177382 nik9p2u2fuk4yazjxt8ymbicxv5qid9 27757 you, are, tor, you "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-24 01:35:22 FALSE "CommonsDelinker" 2304267 Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier FALSE 2 FALSE 822038928 gwk6pampl8si1v5pv3kwgteg710sfw3 27667 jpg, jpg, has, COM 16, 2018 "User:Editingaccount1994/sandbox"
FALSE 56237370 2018-01-07 10:42:20 FALSE "PamD" 1368779 FALSE 0 FALSE 819091874 n4ozbsgle13p9yywtfrz982ccj8woc9 25 alt "Anita del Rey"
FALSE 56237371 2018-01-07 10:42:27 FALSE "ClueBot NG" 13286072 FALSE 3 FALSE 819091883 ksohnvsbeuzwpl5vb8a3v8m18hva0a7 1274 119, 94, 96, 157, 119, 94, 96, 157, 1 "User talk:119.94.96.157"
FALSE 56237372 2018-01-07 10:42:50 FALSE "Underbar dk" 677153 FALSE 14 FALSE 819091914 je7aw21fedbwyqsyofpisdrynsu7olr 113 AES "Category:Ohmi Railway"
FALSE 56237375 2018-01-07 10:43:32 FALSE "TastyPoutine" 882433 FALSE 3 FALSE 819091968 cpm4tkzcx4hc6irr9ukbi06ogud8dtq 199 AES "User talk:92.226.219.222"
FALSE 56237375 2018-01-07 11:10:24 FALSE "AnomieBOT" 7611264 TRUE 3 FALSE 819094036 artmfz8b2gxhb3pp8a5p4ksplxqfkpg 1840 See, for "User talk:92.226.219.222"
FALSE 56237375 2018-01-07 14:33:36 FALSE "Only" 702940 FALSE 3 FALSE 819112363 dn9wj0n8d8pdd5lqe56uw5xamupowr1 2949 has, has "User talk:92.226.219.222"
FALSE 56237376 2018-01-07 10:44:01 FALSE "Dipayanacharya" 32794237 FALSE 2 FALSE 819092004 ofueugwatmmn7u73isw732neuza57gk 28 "User:Dipayanacharya"
FALSE 56237376 2018-01-07 10:49:08 FALSE "Dipayanacharya" 32794237 FALSE 2 FALSE 819092390 dsz55xv96ec2uv6w9c1z7c52ipfovbw 38 "User:Dipayanacharya"
FALSE 56237378 2018-01-07 10:44:56 FALSE "Vinegarymass911" 21516552 FALSE 0 FALSE 819092066 9ma38hak0ef1ew4fpiutxpnzd8oz1wd 65 AES, and "BSCIC"
FALSE 56237379 2018-01-07 10:45:21 FALSE "BrownHairedGirl" 754619 FALSE 14 FALSE 819092102 4dvakoat58bzyf5hmtthxukt29hip6n 285 AES, Non "Category:Women government ministers of Yemen"
FALSE 56237381 2018-01-07 10:45:54 FALSE "PRehse" 410898 FALSE 1 FALSE 819092135 2sjrxsc7os9k9pg4su2t4rk2j8nn0h7 103 AES, low, low "Talk:List of Morning Glories Characters"
FALSE 56237382 2018-01-07 10:45:56 FALSE "ClueBot NG" 13286072 FALSE 3 FALSE 819092138 3y9t5wpk6ur5jhone75rhm4wjf01fgi 1330 106, 207, 126, 114, 106, 207, 126, 114, 1 "User talk:106.207.126.114"
FALSE 56237382 2018-01-07 10:50:22 FALSE "HindWIKI" 31190506 FALSE 3 FALSE 819092495 8wvn6vh3isyt0dorpe89lztrburgupe 2355 "User talk:106.207.126.114"

1 anon articleid date_time deleted editor editor_id editorid li_cheval minor namespace revert reverteds revid sha1 text_chars three_cat three_letter three_number title
2 FALSE 56237363 2018-01-07 10:40:58 FALSE NinjaRobotPirate 3742946 None FALSE 3 FALSE 819091731 135nz8q6lfam6cojla7azb7k5alx3t3 1141 None has, has None User talk:86.139.142.254
3 FALSE 56237364 2018-01-07 10:41:10 FALSE Kavin kavitha 32792125 None FALSE 3 FALSE 819091755 0pwezjc6yopz0smc8al6ogc4fax5bwo 663 None AES, for 01, 12, 2001 User talk:Kavin kavitha
4 FALSE 56237365 2018-01-07 10:41:26 FALSE Amicable always 32621254 None FALSE 3 FALSE 819091788 sz3t2ap7z8bpkdvdvi195f3i35949bv 399 None new None User talk:Dr.vivek163
5 FALSE 56237366 2018-01-07 10:41:31 FALSE ClueBot NG 13286072 None FALSE 3 FALSE 819091796 r6s5j8j3iykenrhuhpnkpsmmd71vubf 1260 None None 1 User talk:Twistorl
6 FALSE 56237368 2018-01-07 10:41:51 FALSE Khruner 8409334 None FALSE 0 FALSE 819091825 tf5qz2yaswx61zrlm9ovxzuhl7r2dc4 2249 None AES, jpg, the, the, the, the, and, you, Tor 67, 119 Kom Firin
7 FALSE 56237368 2018-01-27 12:16:02 FALSE Khruner 8409334 None TRUE 0 FALSE 822610647 e6oa4g0qv64icdaq26uu1zzbyr5hcbh 2230 None None None Kom Firin
8 FALSE 56237369 2018-01-07 10:42:05 FALSE Editingaccount1994 32794215 Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier FALSE 2 FALSE 819091844 0fyvyh2a8xu41gt8obr34oba0bfixj6 27840 None AES, nom None User:Editingaccount1994/sandbox
9 FALSE 56237369 2018-01-07 11:09:52 FALSE AnomieBOT 7611264 Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier TRUE 2 FALSE 819093984 8gy52aolt5rg3eaketwj5v7eiw0apv2 27787 None web, See, for None User:Editingaccount1994/sandbox
10 FALSE 56237369 2018-01-12 21:45:50 FALSE SporkBot 12406635 Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier TRUE 2 FALSE 820064189 he8ydemaanxlrpftqxkez8jfpge1fsj 27784 None per, TFD, TFD None User:Editingaccount1994/sandbox
11 FALSE 56237369 2018-01-12 23:28:11 FALSE SporkBot 12406635 Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier TRUE 2 FALSE 820078679 0to17w9rth3url8n7gvucdtobybdq5h 27783 None per, for, Log, TFD 2010, 13 User:Editingaccount1994/sandbox
12 FALSE 56237369 2018-01-12 23:28:39 FALSE SporkBot 12406635 Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier TRUE 2 FALSE 820078733 531dizmmloyxffbkdr5vph7owh921eg 27782 None per, for, Log, TFD 2011, 17 User:Editingaccount1994/sandbox
13 FALSE 56237369 2018-01-13 13:45:33 FALSE Frietjes 13791031 Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier FALSE 2 FALSE 820177382 nik9p2u2fuk4yazjxt8ymbicxv5qid9 27757 None you, are, tor, you None User:Editingaccount1994/sandbox
14 FALSE 56237369 2018-01-24 01:35:22 FALSE CommonsDelinker 2304267 Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier, Li Chevalier FALSE 2 FALSE 822038928 gwk6pampl8si1v5pv3kwgteg710sfw3 27667 None jpg, jpg, has, COM 16, 2018 User:Editingaccount1994/sandbox
15 FALSE 56237370 2018-01-07 10:42:20 FALSE PamD 1368779 None FALSE 0 FALSE 819091874 n4ozbsgle13p9yywtfrz982ccj8woc9 25 None alt None Anita del Rey
16 FALSE 56237371 2018-01-07 10:42:27 FALSE ClueBot NG 13286072 None FALSE 3 FALSE 819091883 ksohnvsbeuzwpl5vb8a3v8m18hva0a7 1274 None None 119, 94, 96, 157, 119, 94, 96, 157, 1 User talk:119.94.96.157
17 FALSE 56237372 2018-01-07 10:42:50 FALSE Underbar dk 677153 None FALSE 14 FALSE 819091914 je7aw21fedbwyqsyofpisdrynsu7olr 113 None AES None Category:Ohmi Railway
18 FALSE 56237375 2018-01-07 10:43:32 FALSE TastyPoutine 882433 None FALSE 3 FALSE 819091968 cpm4tkzcx4hc6irr9ukbi06ogud8dtq 199 None AES None User talk:92.226.219.222
19 FALSE 56237375 2018-01-07 11:10:24 FALSE AnomieBOT 7611264 None TRUE 3 FALSE 819094036 artmfz8b2gxhb3pp8a5p4ksplxqfkpg 1840 None See, for None User talk:92.226.219.222
20 FALSE 56237375 2018-01-07 14:33:36 FALSE Only 702940 None FALSE 3 FALSE 819112363 dn9wj0n8d8pdd5lqe56uw5xamupowr1 2949 None has, has None User talk:92.226.219.222
21 FALSE 56237376 2018-01-07 10:44:01 FALSE Dipayanacharya 32794237 None FALSE 2 FALSE 819092004 ofueugwatmmn7u73isw732neuza57gk 28 None None None User:Dipayanacharya
22 FALSE 56237376 2018-01-07 10:49:08 FALSE Dipayanacharya 32794237 None FALSE 2 FALSE 819092390 dsz55xv96ec2uv6w9c1z7c52ipfovbw 38 None None None User:Dipayanacharya
23 FALSE 56237378 2018-01-07 10:44:56 FALSE Vinegarymass911 21516552 None FALSE 0 FALSE 819092066 9ma38hak0ef1ew4fpiutxpnzd8oz1wd 65 None AES, and None BSCIC
24 FALSE 56237379 2018-01-07 10:45:21 FALSE BrownHairedGirl 754619 None FALSE 14 FALSE 819092102 4dvakoat58bzyf5hmtthxukt29hip6n 285 None AES, Non None Category:Women government ministers of Yemen
25 FALSE 56237381 2018-01-07 10:45:54 FALSE PRehse 410898 None FALSE 1 FALSE 819092135 2sjrxsc7os9k9pg4su2t4rk2j8nn0h7 103 None AES, low, low None Talk:List of Morning Glories Characters
26 FALSE 56237382 2018-01-07 10:45:56 FALSE ClueBot NG 13286072 None FALSE 3 FALSE 819092138 3y9t5wpk6ur5jhone75rhm4wjf01fgi 1330 None None 106, 207, 126, 114, 106, 207, 126, 114, 1 User talk:106.207.126.114
27 FALSE 56237382 2018-01-07 10:50:22 FALSE HindWIKI 31190506 None FALSE 3 FALSE 819092495 8wvn6vh3isyt0dorpe89lztrburgupe 2355 None None None User talk:106.207.126.114

54
test/baseline_output/capturegroup_regextest_1.tsv
View File

@ -1,27 +1,27 @@
anon articleid date_time deleted editor editor_id minor namespace npov_neutral npov_npov revert reverteds revid sha1 testcase_a testcase_b testcase_c testcase_d text_chars title
FALSE 56237363 2018-01-07 10:40:58 FALSE "NinjaRobotPirate" 3742946 FALSE 3 None None FALSE 819091731 135nz8q6lfam6cojla7azb7k5alx3t3 None None None None 1141 "User talk:86.139.142.254"
FALSE 56237364 2018-01-07 10:41:10 FALSE "Kavin kavitha" 32792125 FALSE 3 None None FALSE 819091755 0pwezjc6yopz0smc8al6ogc4fax5bwo None None None None 663 "User talk:Kavin kavitha"
FALSE 56237365 2018-01-07 10:41:26 FALSE "Amicable always" 32621254 FALSE 3 None NPOV, NPOV FALSE 819091788 sz3t2ap7z8bpkdvdvi195f3i35949bv None None None None 399 "User talk:Dr.vivek163"
FALSE 56237366 2018-01-07 10:41:31 FALSE "ClueBot NG" 13286072 FALSE 3 None None FALSE 819091796 r6s5j8j3iykenrhuhpnkpsmmd71vubf None None None None 1260 "User talk:Twistorl"
FALSE 56237368 2018-01-07 10:41:51 FALSE "Khruner" 8409334 FALSE 0 None NPOV FALSE 819091825 tf5qz2yaswx61zrlm9ovxzuhl7r2dc4 None TestCaseB None None 2249 "Kom Firin"
FALSE 56237368 2018-01-27 12:16:02 FALSE "Khruner" 8409334 TRUE 0 None None FALSE 822610647 e6oa4g0qv64icdaq26uu1zzbyr5hcbh None None None None 2230 "Kom Firin"
FALSE 56237369 2018-01-07 10:42:05 FALSE "Editingaccount1994" 32794215 FALSE 2 None None FALSE 819091844 0fyvyh2a8xu41gt8obr34oba0bfixj6 None None None None 27840 "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-07 11:09:52 FALSE "AnomieBOT" 7611264 TRUE 2 None None FALSE 819093984 8gy52aolt5rg3eaketwj5v7eiw0apv2 None None None None 27787 "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-12 21:45:50 FALSE "SporkBot" 12406635 TRUE 2 None None FALSE 820064189 he8ydemaanxlrpftqxkez8jfpge1fsj None None None None 27784 "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-12 23:28:11 FALSE "SporkBot" 12406635 TRUE 2 None None FALSE 820078679 0to17w9rth3url8n7gvucdtobybdq5h None None None None 27783 "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-12 23:28:39 FALSE "SporkBot" 12406635 TRUE 2 None None FALSE 820078733 531dizmmloyxffbkdr5vph7owh921eg None None None None 27782 "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-13 13:45:33 FALSE "Frietjes" 13791031 FALSE 2 None None FALSE 820177382 nik9p2u2fuk4yazjxt8ymbicxv5qid9 None None None TestCaseD 27757 "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-24 01:35:22 FALSE "CommonsDelinker" 2304267 FALSE 2 None None FALSE 822038928 gwk6pampl8si1v5pv3kwgteg710sfw3 None None None None 27667 "User:Editingaccount1994/sandbox"
FALSE 56237370 2018-01-07 10:42:20 FALSE "PamD" 1368779 FALSE 0 None None FALSE 819091874 n4ozbsgle13p9yywtfrz982ccj8woc9 None None None None 25 "Anita del Rey"
FALSE 56237371 2018-01-07 10:42:27 FALSE "ClueBot NG" 13286072 FALSE 3 None None FALSE 819091883 ksohnvsbeuzwpl5vb8a3v8m18hva0a7 None None None None 1274 "User talk:119.94.96.157"
FALSE 56237372 2018-01-07 10:42:50 FALSE "Underbar dk" 677153 FALSE 14 None None FALSE 819091914 je7aw21fedbwyqsyofpisdrynsu7olr None None None None 113 "Category:Ohmi Railway"
FALSE 56237375 2018-01-07 10:43:32 FALSE "TastyPoutine" 882433 FALSE 3 None None FALSE 819091968 cpm4tkzcx4hc6irr9ukbi06ogud8dtq None None None None 199 "User talk:92.226.219.222"
FALSE 56237375 2018-01-07 11:10:24 FALSE "AnomieBOT" 7611264 TRUE 3 None None FALSE 819094036 artmfz8b2gxhb3pp8a5p4ksplxqfkpg None None None None 1840 "User talk:92.226.219.222"
FALSE 56237375 2018-01-07 14:33:36 FALSE "Only" 702940 FALSE 3 None None FALSE 819112363 dn9wj0n8d8pdd5lqe56uw5xamupowr1 None None None None 2949 "User talk:92.226.219.222"
FALSE 56237376 2018-01-07 10:44:01 FALSE "Dipayanacharya" 32794237 FALSE 2 None None FALSE 819092004 ofueugwatmmn7u73isw732neuza57gk None None None None 28 "User:Dipayanacharya"
FALSE 56237376 2018-01-07 10:49:08 FALSE "Dipayanacharya" 32794237 FALSE 2 None None FALSE 819092390 dsz55xv96ec2uv6w9c1z7c52ipfovbw None None None None 38 "User:Dipayanacharya"
FALSE 56237378 2018-01-07 10:44:56 FALSE "Vinegarymass911" 21516552 FALSE 0 None None FALSE 819092066 9ma38hak0ef1ew4fpiutxpnzd8oz1wd None None None None 65 "BSCIC"
FALSE 56237379 2018-01-07 10:45:21 FALSE "BrownHairedGirl" 754619 FALSE 14 None None FALSE 819092102 4dvakoat58bzyf5hmtthxukt29hip6n None None None None 285 "Category:Women government ministers of Yemen"
FALSE 56237381 2018-01-07 10:45:54 FALSE "PRehse" 410898 FALSE 1 None None FALSE 819092135 2sjrxsc7os9k9pg4su2t4rk2j8nn0h7 None None None None 103 "Talk:List of Morning Glories Characters"
FALSE 56237382 2018-01-07 10:45:56 FALSE "ClueBot NG" 13286072 FALSE 3 None None FALSE 819092138 3y9t5wpk6ur5jhone75rhm4wjf01fgi None None None None 1330 "User talk:106.207.126.114"
FALSE 56237382 2018-01-07 10:50:22 FALSE "HindWIKI" 31190506 FALSE 3 None None FALSE 819092495 8wvn6vh3isyt0dorpe89lztrburgupe None None None None 2355 "User talk:106.207.126.114"
anon articleid date_time deleted editor editorid minor namespace npov_neutral npov_npov revert reverteds revid sha1 testcase_a testcase_b testcase_c testcase_d text_chars title
FALSE 56237363 2018-01-07 10:40:58 FALSE "NinjaRobotPirate" 3742946 FALSE 3 FALSE 819091731 135nz8q6lfam6cojla7azb7k5alx3t3 1141 "User talk:86.139.142.254"
FALSE 56237364 2018-01-07 10:41:10 FALSE "Kavin kavitha" 32792125 FALSE 3 FALSE 819091755 0pwezjc6yopz0smc8al6ogc4fax5bwo 663 "User talk:Kavin kavitha"
FALSE 56237365 2018-01-07 10:41:26 FALSE "Amicable always" 32621254 FALSE 3 NPOV, NPOV FALSE 819091788 sz3t2ap7z8bpkdvdvi195f3i35949bv 399 "User talk:Dr.vivek163"
FALSE 56237366 2018-01-07 10:41:31 FALSE "ClueBot NG" 13286072 FALSE 3 FALSE 819091796 r6s5j8j3iykenrhuhpnkpsmmd71vubf 1260 "User talk:Twistorl"
FALSE 56237368 2018-01-07 10:41:51 FALSE "Khruner" 8409334 FALSE 0 NPOV FALSE 819091825 tf5qz2yaswx61zrlm9ovxzuhl7r2dc4 TestCaseB 2249 "Kom Firin"
FALSE 56237368 2018-01-27 12:16:02 FALSE "Khruner" 8409334 TRUE 0 FALSE 822610647 e6oa4g0qv64icdaq26uu1zzbyr5hcbh 2230 "Kom Firin"
FALSE 56237369 2018-01-07 10:42:05 FALSE "Editingaccount1994" 32794215 FALSE 2 FALSE 819091844 0fyvyh2a8xu41gt8obr34oba0bfixj6 27840 "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-07 11:09:52 FALSE "AnomieBOT" 7611264 TRUE 2 FALSE 819093984 8gy52aolt5rg3eaketwj5v7eiw0apv2 27787 "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-12 21:45:50 FALSE "SporkBot" 12406635 TRUE 2 FALSE 820064189 he8ydemaanxlrpftqxkez8jfpge1fsj 27784 "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-12 23:28:11 FALSE "SporkBot" 12406635 TRUE 2 FALSE 820078679 0to17w9rth3url8n7gvucdtobybdq5h 27783 "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-12 23:28:39 FALSE "SporkBot" 12406635 TRUE 2 FALSE 820078733 531dizmmloyxffbkdr5vph7owh921eg 27782 "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-13 13:45:33 FALSE "Frietjes" 13791031 FALSE 2 FALSE 820177382 nik9p2u2fuk4yazjxt8ymbicxv5qid9 TestCaseD 27757 "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-24 01:35:22 FALSE "CommonsDelinker" 2304267 FALSE 2 FALSE 822038928 gwk6pampl8si1v5pv3kwgteg710sfw3 27667 "User:Editingaccount1994/sandbox"
FALSE 56237370 2018-01-07 10:42:20 FALSE "PamD" 1368779 FALSE 0 FALSE 819091874 n4ozbsgle13p9yywtfrz982ccj8woc9 25 "Anita del Rey"
FALSE 56237371 2018-01-07 10:42:27 FALSE "ClueBot NG" 13286072 FALSE 3 FALSE 819091883 ksohnvsbeuzwpl5vb8a3v8m18hva0a7 1274 "User talk:119.94.96.157"
FALSE 56237372 2018-01-07 10:42:50 FALSE "Underbar dk" 677153 FALSE 14 FALSE 819091914 je7aw21fedbwyqsyofpisdrynsu7olr 113 "Category:Ohmi Railway"
FALSE 56237375 2018-01-07 10:43:32 FALSE "TastyPoutine" 882433 FALSE 3 FALSE 819091968 cpm4tkzcx4hc6irr9ukbi06ogud8dtq 199 "User talk:92.226.219.222"
FALSE 56237375 2018-01-07 11:10:24 FALSE "AnomieBOT" 7611264 TRUE 3 FALSE 819094036 artmfz8b2gxhb3pp8a5p4ksplxqfkpg 1840 "User talk:92.226.219.222"
FALSE 56237375 2018-01-07 14:33:36 FALSE "Only" 702940 FALSE 3 FALSE 819112363 dn9wj0n8d8pdd5lqe56uw5xamupowr1 2949 "User talk:92.226.219.222"
FALSE 56237376 2018-01-07 10:44:01 FALSE "Dipayanacharya" 32794237 FALSE 2 FALSE 819092004 ofueugwatmmn7u73isw732neuza57gk 28 "User:Dipayanacharya"
FALSE 56237376 2018-01-07 10:49:08 FALSE "Dipayanacharya" 32794237 FALSE 2 FALSE 819092390 dsz55xv96ec2uv6w9c1z7c52ipfovbw 38 "User:Dipayanacharya"
FALSE 56237378 2018-01-07 10:44:56 FALSE "Vinegarymass911" 21516552 FALSE 0 FALSE 819092066 9ma38hak0ef1ew4fpiutxpnzd8oz1wd 65 "BSCIC"
FALSE 56237379 2018-01-07 10:45:21 FALSE "BrownHairedGirl" 754619 FALSE 14 FALSE 819092102 4dvakoat58bzyf5hmtthxukt29hip6n 285 "Category:Women government ministers of Yemen"
FALSE 56237381 2018-01-07 10:45:54 FALSE "PRehse" 410898 FALSE 1 FALSE 819092135 2sjrxsc7os9k9pg4su2t4rk2j8nn0h7 103 "Talk:List of Morning Glories Characters"
FALSE 56237382 2018-01-07 10:45:56 FALSE "ClueBot NG" 13286072 FALSE 3 FALSE 819092138 3y9t5wpk6ur5jhone75rhm4wjf01fgi 1330 "User talk:106.207.126.114"
FALSE 56237382 2018-01-07 10:50:22 FALSE "HindWIKI" 31190506 FALSE 3 FALSE 819092495 8wvn6vh3isyt0dorpe89lztrburgupe 2355 "User talk:106.207.126.114"

1 anon articleid date_time deleted editor editor_id editorid minor namespace npov_neutral npov_npov revert reverteds revid sha1 testcase_a testcase_b testcase_c testcase_d text_chars title
2 FALSE 56237363 2018-01-07 10:40:58 FALSE NinjaRobotPirate 3742946 FALSE 3 None None FALSE 819091731 135nz8q6lfam6cojla7azb7k5alx3t3 None None None None 1141 User talk:86.139.142.254
3 FALSE 56237364 2018-01-07 10:41:10 FALSE Kavin kavitha 32792125 FALSE 3 None None FALSE 819091755 0pwezjc6yopz0smc8al6ogc4fax5bwo None None None None 663 User talk:Kavin kavitha
4 FALSE 56237365 2018-01-07 10:41:26 FALSE Amicable always 32621254 FALSE 3 None NPOV, NPOV FALSE 819091788 sz3t2ap7z8bpkdvdvi195f3i35949bv None None None None 399 User talk:Dr.vivek163
5 FALSE 56237366 2018-01-07 10:41:31 FALSE ClueBot NG 13286072 FALSE 3 None None FALSE 819091796 r6s5j8j3iykenrhuhpnkpsmmd71vubf None None None None 1260 User talk:Twistorl
6 FALSE 56237368 2018-01-07 10:41:51 FALSE Khruner 8409334 FALSE 0 None NPOV FALSE 819091825 tf5qz2yaswx61zrlm9ovxzuhl7r2dc4 None TestCaseB None None 2249 Kom Firin
7 FALSE 56237368 2018-01-27 12:16:02 FALSE Khruner 8409334 TRUE 0 None None FALSE 822610647 e6oa4g0qv64icdaq26uu1zzbyr5hcbh None None None None 2230 Kom Firin
8 FALSE 56237369 2018-01-07 10:42:05 FALSE Editingaccount1994 32794215 FALSE 2 None None FALSE 819091844 0fyvyh2a8xu41gt8obr34oba0bfixj6 None None None None 27840 User:Editingaccount1994/sandbox
9 FALSE 56237369 2018-01-07 11:09:52 FALSE AnomieBOT 7611264 TRUE 2 None None FALSE 819093984 8gy52aolt5rg3eaketwj5v7eiw0apv2 None None None None 27787 User:Editingaccount1994/sandbox
10 FALSE 56237369 2018-01-12 21:45:50 FALSE SporkBot 12406635 TRUE 2 None None FALSE 820064189 he8ydemaanxlrpftqxkez8jfpge1fsj None None None None 27784 User:Editingaccount1994/sandbox
11 FALSE 56237369 2018-01-12 23:28:11 FALSE SporkBot 12406635 TRUE 2 None None FALSE 820078679 0to17w9rth3url8n7gvucdtobybdq5h None None None None 27783 User:Editingaccount1994/sandbox
12 FALSE 56237369 2018-01-12 23:28:39 FALSE SporkBot 12406635 TRUE 2 None None FALSE 820078733 531dizmmloyxffbkdr5vph7owh921eg None None None None 27782 User:Editingaccount1994/sandbox
13 FALSE 56237369 2018-01-13 13:45:33 FALSE Frietjes 13791031 FALSE 2 None None FALSE 820177382 nik9p2u2fuk4yazjxt8ymbicxv5qid9 None None None TestCaseD 27757 User:Editingaccount1994/sandbox
14 FALSE 56237369 2018-01-24 01:35:22 FALSE CommonsDelinker 2304267 FALSE 2 None None FALSE 822038928 gwk6pampl8si1v5pv3kwgteg710sfw3 None None None None 27667 User:Editingaccount1994/sandbox
15 FALSE 56237370 2018-01-07 10:42:20 FALSE PamD 1368779 FALSE 0 None None FALSE 819091874 n4ozbsgle13p9yywtfrz982ccj8woc9 None None None None 25 Anita del Rey
16 FALSE 56237371 2018-01-07 10:42:27 FALSE ClueBot NG 13286072 FALSE 3 None None FALSE 819091883 ksohnvsbeuzwpl5vb8a3v8m18hva0a7 None None None None 1274 User talk:119.94.96.157
17 FALSE 56237372 2018-01-07 10:42:50 FALSE Underbar dk 677153 FALSE 14 None None FALSE 819091914 je7aw21fedbwyqsyofpisdrynsu7olr None None None None 113 Category:Ohmi Railway
18 FALSE 56237375 2018-01-07 10:43:32 FALSE TastyPoutine 882433 FALSE 3 None None FALSE 819091968 cpm4tkzcx4hc6irr9ukbi06ogud8dtq None None None None 199 User talk:92.226.219.222
19 FALSE 56237375 2018-01-07 11:10:24 FALSE AnomieBOT 7611264 TRUE 3 None None FALSE 819094036 artmfz8b2gxhb3pp8a5p4ksplxqfkpg None None None None 1840 User talk:92.226.219.222
20 FALSE 56237375 2018-01-07 14:33:36 FALSE Only 702940 FALSE 3 None None FALSE 819112363 dn9wj0n8d8pdd5lqe56uw5xamupowr1 None None None None 2949 User talk:92.226.219.222
21 FALSE 56237376 2018-01-07 10:44:01 FALSE Dipayanacharya 32794237 FALSE 2 None None FALSE 819092004 ofueugwatmmn7u73isw732neuza57gk None None None None 28 User:Dipayanacharya
22 FALSE 56237376 2018-01-07 10:49:08 FALSE Dipayanacharya 32794237 FALSE 2 None None FALSE 819092390 dsz55xv96ec2uv6w9c1z7c52ipfovbw None None None None 38 User:Dipayanacharya
23 FALSE 56237378 2018-01-07 10:44:56 FALSE Vinegarymass911 21516552 FALSE 0 None None FALSE 819092066 9ma38hak0ef1ew4fpiutxpnzd8oz1wd None None None None 65 BSCIC
24 FALSE 56237379 2018-01-07 10:45:21 FALSE BrownHairedGirl 754619 FALSE 14 None None FALSE 819092102 4dvakoat58bzyf5hmtthxukt29hip6n None None None None 285 Category:Women government ministers of Yemen
25 FALSE 56237381 2018-01-07 10:45:54 FALSE PRehse 410898 FALSE 1 None None FALSE 819092135 2sjrxsc7os9k9pg4su2t4rk2j8nn0h7 None None None None 103 Talk:List of Morning Glories Characters
26 FALSE 56237382 2018-01-07 10:45:56 FALSE ClueBot NG 13286072 FALSE 3 None None FALSE 819092138 3y9t5wpk6ur5jhone75rhm4wjf01fgi None None None None 1330 User talk:106.207.126.114
27 FALSE 56237382 2018-01-07 10:50:22 FALSE HindWIKI 31190506 FALSE 3 None None FALSE 819092495 8wvn6vh3isyt0dorpe89lztrburgupe None None None None 2355 User talk:106.207.126.114

2
test/baseline_output/collapse-user_sailormoon.tsv
View File

@ -1,4 +1,4 @@
anon articleid collapsed_revs date_time deleted editor editor_id minor namespace revert reverteds revid sha1 text_chars title
anon articleid collapsed_revs date_time deleted editor editorid minor namespace revert reverteds revid sha1 text_chars title
FALSE 1 1 2004-11-11 23:33:14 FALSE "Jasonr" 1 FALSE 6 FALSE 274 da39a3ee5e6b4b0d3255bfef95601890afd80709 0 "File:Wiki.png"
FALSE 3 1 2004-11-11 23:33:14 FALSE "Jasonr" 1 FALSE 6 FALSE 276 da39a3ee5e6b4b0d3255bfef95601890afd80709 0 "File:Search logo.png"
FALSE 6 3 2005-07-29 18:37:42 FALSE "Purplefeltangel" 5055 TRUE 4 FALSE 259 f9cc2c62a91bb1ee47fe6dee6ed6a38ba1e708ea 224 "Sailor Moon Wiki:About"

1 anon articleid collapsed_revs date_time deleted editor editor_id editorid minor namespace revert reverteds revid sha1 text_chars title
2 FALSE 1 1 2004-11-11 23:33:14 FALSE Jasonr 1 FALSE 6 FALSE 274 da39a3ee5e6b4b0d3255bfef95601890afd80709 0 File:Wiki.png
3 FALSE 3 1 2004-11-11 23:33:14 FALSE Jasonr 1 FALSE 6 FALSE 276 da39a3ee5e6b4b0d3255bfef95601890afd80709 0 File:Search logo.png
4 FALSE 6 3 2005-07-29 18:37:42 FALSE Purplefeltangel 5055 TRUE 4 FALSE 259 f9cc2c62a91bb1ee47fe6dee6ed6a38ba1e708ea 224 Sailor Moon Wiki:About

2
test/baseline_output/namespaces_ikwiki-20180301-pages-meta-history.tsv
View File

@ -1,4 +1,4 @@
anon articleid date_time deleted editor editor_id minor namespace revert reverteds revid sha1 text_chars title
anon articleid date_time deleted editor editorid minor namespace revert reverteds revid sha1 text_chars title
FALSE 2 2004-01-25 13:40:25 FALSE "Brion VIBBER" 1 FALSE 0 FALSE 3094 a6dzmzujf5owfsclacifhewzvp6q8ya 24 "HomePage"
TRUE 2 2007-01-01 14:51:20 FALSE "125.229.3.76" FALSE 0 FALSE 5938 46va13zuh9l56lj7ppey9a11l99zamg 337 "HomePage"
FALSE 2 2008-02-24 15:07:26 FALSE "Alexbot" 147 TRUE 0 FALSE 7322 5yib0zj4fa2lxe8xxw6l702097371u2 343 "HomePage"

Can't render this file because it is too large.

2
test/baseline_output/noargs_sailormoon.tsv
View File

@ -1,4 +1,4 @@
anon articleid date_time deleted editor editor_id minor namespace revert reverteds revid sha1 text_chars title
anon articleid date_time deleted editor editorid minor namespace revert reverteds revid sha1 text_chars title
FALSE 1 2004-11-11 23:33:14 FALSE "Jasonr" 1 FALSE 6 FALSE 274 da39a3ee5e6b4b0d3255bfef95601890afd80709 0 "File:Wiki.png"
FALSE 3 2004-11-11 23:33:14 FALSE "Jasonr" 1 FALSE 6 FALSE 276 da39a3ee5e6b4b0d3255bfef95601890afd80709 0 "File:Search logo.png"
FALSE 6 2005-07-16 21:46:43 FALSE "Purplefeltangel" 5055 FALSE 4 FALSE 3 83d73a7b77b4de495037c6236808e8518d545591 184 "Sailor Moon Wiki:About"

Can't render this file because it is too large.

2
test/baseline_output/persistence_legacy_sailormoon.tsv
View File

@ -1,4 +1,4 @@
anon articleid date_time deleted editor editor_id minor namespace revert reverteds revid sha1 text_chars title token_revs tokens_added tokens_removed tokens_window
anon articleid date_time deleted editor editorid minor namespace revert reverteds revid sha1 text_chars title token_revs tokens_added tokens_removed tokens_window
FALSE 1 2004-11-11 23:33:14 FALSE "Jasonr" 1 FALSE 6 FALSE 274 da39a3ee5e6b4b0d3255bfef95601890afd80709 0 "File:Wiki.png" 0 0 0 0
FALSE 3 2004-11-11 23:33:14 FALSE "Jasonr" 1 FALSE 6 FALSE 276 da39a3ee5e6b4b0d3255bfef95601890afd80709 0 "File:Search logo.png" 0 0 0 0
FALSE 6 2005-07-16 21:46:43 FALSE "Purplefeltangel" 5055 FALSE 4 FALSE 3 83d73a7b77b4de495037c6236808e8518d545591 184 "Sailor Moon Wiki:About" 390 65 0 6

Can't render this file because it is too large.

9304
test/baseline_output/persistence_sailormoon.tsv
View File

File diff suppressed because it is too large Load Diff

9304
test/baseline_output/persistence_segment_sailormoon.tsv
View File

File diff suppressed because it is too large Load Diff

54
test/baseline_output/regextest.tsv
View File

@ -1,27 +1,27 @@
anon articleid chev_com date_time deleted editor editor_id minor namespace revert reverteds revid sha1 text_chars title warning wiki_welcome
FALSE 56237363 None 2018-01-07 10:40:58 FALSE "NinjaRobotPirate" 3742946 FALSE 3 FALSE 819091731 135nz8q6lfam6cojla7azb7k5alx3t3 1141 "User talk:86.139.142.254" None None
FALSE 56237364 None 2018-01-07 10:41:10 FALSE "Kavin kavitha" 32792125 FALSE 3 FALSE 819091755 0pwezjc6yopz0smc8al6ogc4fax5bwo 663 "User talk:Kavin kavitha" None None
FALSE 56237365 None 2018-01-07 10:41:26 FALSE "Amicable always" 32621254 FALSE 3 FALSE 819091788 sz3t2ap7z8bpkdvdvi195f3i35949bv 399 "User talk:Dr.vivek163" None None
FALSE 56237366 None 2018-01-07 10:41:31 FALSE "ClueBot NG" 13286072 FALSE 3 FALSE 819091796 r6s5j8j3iykenrhuhpnkpsmmd71vubf 1260 "User talk:Twistorl" Warning welcome to Wikipedia
FALSE 56237368 None 2018-01-07 10:41:51 FALSE "Khruner" 8409334 FALSE 0 FALSE 819091825 tf5qz2yaswx61zrlm9ovxzuhl7r2dc4 2249 "Kom Firin" None None
FALSE 56237368 None 2018-01-27 12:16:02 FALSE "Khruner" 8409334 TRUE 0 FALSE 822610647 e6oa4g0qv64icdaq26uu1zzbyr5hcbh 2230 "Kom Firin" None None
FALSE 56237369 Chevalier, Chevalier 2018-01-07 10:42:05 FALSE "Editingaccount1994" 32794215 FALSE 2 FALSE 819091844 0fyvyh2a8xu41gt8obr34oba0bfixj6 27840 "User:Editingaccount1994/sandbox" None None
FALSE 56237369 None 2018-01-07 11:09:52 FALSE "AnomieBOT" 7611264 TRUE 2 FALSE 819093984 8gy52aolt5rg3eaketwj5v7eiw0apv2 27787 "User:Editingaccount1994/sandbox" None None
FALSE 56237369 None 2018-01-12 21:45:50 FALSE "SporkBot" 12406635 TRUE 2 FALSE 820064189 he8ydemaanxlrpftqxkez8jfpge1fsj 27784 "User:Editingaccount1994/sandbox" None None
FALSE 56237369 None 2018-01-12 23:28:11 FALSE "SporkBot" 12406635 TRUE 2 FALSE 820078679 0to17w9rth3url8n7gvucdtobybdq5h 27783 "User:Editingaccount1994/sandbox" None None
FALSE 56237369 None 2018-01-12 23:28:39 FALSE "SporkBot" 12406635 TRUE 2 FALSE 820078733 531dizmmloyxffbkdr5vph7owh921eg 27782 "User:Editingaccount1994/sandbox" None None
FALSE 56237369 None 2018-01-13 13:45:33 FALSE "Frietjes" 13791031 FALSE 2 FALSE 820177382 nik9p2u2fuk4yazjxt8ymbicxv5qid9 27757 "User:Editingaccount1994/sandbox" None None
FALSE 56237369 Chevalier, Chevalier 2018-01-24 01:35:22 FALSE "CommonsDelinker" 2304267 FALSE 2 FALSE 822038928 gwk6pampl8si1v5pv3kwgteg710sfw3 27667 "User:Editingaccount1994/sandbox" None None
FALSE 56237370 None 2018-01-07 10:42:20 FALSE "PamD" 1368779 FALSE 0 FALSE 819091874 n4ozbsgle13p9yywtfrz982ccj8woc9 25 "Anita del Rey" None None
FALSE 56237371 None 2018-01-07 10:42:27 FALSE "ClueBot NG" 13286072 FALSE 3 FALSE 819091883 ksohnvsbeuzwpl5vb8a3v8m18hva0a7 1274 "User talk:119.94.96.157" Warning welcome to Wikipedia
FALSE 56237372 None 2018-01-07 10:42:50 FALSE "Underbar dk" 677153 FALSE 14 FALSE 819091914 je7aw21fedbwyqsyofpisdrynsu7olr 113 "Category:Ohmi Railway" None None
FALSE 56237375 None 2018-01-07 10:43:32 FALSE "TastyPoutine" 882433 FALSE 3 FALSE 819091968 cpm4tkzcx4hc6irr9ukbi06ogud8dtq 199 "User talk:92.226.219.222" None None
FALSE 56237375 None 2018-01-07 11:10:24 FALSE "AnomieBOT" 7611264 TRUE 3 FALSE 819094036 artmfz8b2gxhb3pp8a5p4ksplxqfkpg 1840 "User talk:92.226.219.222" None None
FALSE 56237375 None 2018-01-07 14:33:36 FALSE "Only" 702940 FALSE 3 FALSE 819112363 dn9wj0n8d8pdd5lqe56uw5xamupowr1 2949 "User talk:92.226.219.222" None None
FALSE 56237376 None 2018-01-07 10:44:01 FALSE "Dipayanacharya" 32794237 FALSE 2 FALSE 819092004 ofueugwatmmn7u73isw732neuza57gk 28 "User:Dipayanacharya" None None
FALSE 56237376 None 2018-01-07 10:49:08 FALSE "Dipayanacharya" 32794237 FALSE 2 FALSE 819092390 dsz55xv96ec2uv6w9c1z7c52ipfovbw 38 "User:Dipayanacharya" None None
FALSE 56237378 None 2018-01-07 10:44:56 FALSE "Vinegarymass911" 21516552 FALSE 0 FALSE 819092066 9ma38hak0ef1ew4fpiutxpnzd8oz1wd 65 "BSCIC" None None
FALSE 56237379 None 2018-01-07 10:45:21 FALSE "BrownHairedGirl" 754619 FALSE 14 FALSE 819092102 4dvakoat58bzyf5hmtthxukt29hip6n 285 "Category:Women government ministers of Yemen" None None
FALSE 56237381 None 2018-01-07 10:45:54 FALSE "PRehse" 410898 FALSE 1 FALSE 819092135 2sjrxsc7os9k9pg4su2t4rk2j8nn0h7 103 "Talk:List of Morning Glories Characters" None None
FALSE 56237382 None 2018-01-07 10:45:56 FALSE "ClueBot NG" 13286072 FALSE 3 FALSE 819092138 3y9t5wpk6ur5jhone75rhm4wjf01fgi 1330 "User talk:106.207.126.114" Warning welcome to Wikipedia
FALSE 56237382 None 2018-01-07 10:50:22 FALSE "HindWIKI" 31190506 FALSE 3 FALSE 819092495 8wvn6vh3isyt0dorpe89lztrburgupe 2355 "User talk:106.207.126.114" None welcome to Wikipedia
anon articleid chev_com date_time deleted editor editorid minor namespace revert reverteds revid sha1 text_chars title warning wiki_welcome
FALSE 56237363 2018-01-07 10:40:58 FALSE "NinjaRobotPirate" 3742946 FALSE 3 FALSE 819091731 135nz8q6lfam6cojla7azb7k5alx3t3 1141 "User talk:86.139.142.254"
FALSE 56237364 2018-01-07 10:41:10 FALSE "Kavin kavitha" 32792125 FALSE 3 FALSE 819091755 0pwezjc6yopz0smc8al6ogc4fax5bwo 663 "User talk:Kavin kavitha"
FALSE 56237365 2018-01-07 10:41:26 FALSE "Amicable always" 32621254 FALSE 3 FALSE 819091788 sz3t2ap7z8bpkdvdvi195f3i35949bv 399 "User talk:Dr.vivek163"
FALSE 56237366 2018-01-07 10:41:31 FALSE "ClueBot NG" 13286072 FALSE 3 FALSE 819091796 r6s5j8j3iykenrhuhpnkpsmmd71vubf 1260 "User talk:Twistorl" Warning welcome to Wikipedia
FALSE 56237368 2018-01-07 10:41:51 FALSE "Khruner" 8409334 FALSE 0 FALSE 819091825 tf5qz2yaswx61zrlm9ovxzuhl7r2dc4 2249 "Kom Firin"
FALSE 56237368 2018-01-27 12:16:02 FALSE "Khruner" 8409334 TRUE 0 FALSE 822610647 e6oa4g0qv64icdaq26uu1zzbyr5hcbh 2230 "Kom Firin"
FALSE 56237369 Chevalier, Chevalier 2018-01-07 10:42:05 FALSE "Editingaccount1994" 32794215 FALSE 2 FALSE 819091844 0fyvyh2a8xu41gt8obr34oba0bfixj6 27840 "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-07 11:09:52 FALSE "AnomieBOT" 7611264 TRUE 2 FALSE 819093984 8gy52aolt5rg3eaketwj5v7eiw0apv2 27787 "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-12 21:45:50 FALSE "SporkBot" 12406635 TRUE 2 FALSE 820064189 he8ydemaanxlrpftqxkez8jfpge1fsj 27784 "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-12 23:28:11 FALSE "SporkBot" 12406635 TRUE 2 FALSE 820078679 0to17w9rth3url8n7gvucdtobybdq5h 27783 "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-12 23:28:39 FALSE "SporkBot" 12406635 TRUE 2 FALSE 820078733 531dizmmloyxffbkdr5vph7owh921eg 27782 "User:Editingaccount1994/sandbox"
FALSE 56237369 2018-01-13 13:45:33 FALSE "Frietjes" 13791031 FALSE 2 FALSE 820177382 nik9p2u2fuk4yazjxt8ymbicxv5qid9 27757 "User:Editingaccount1994/sandbox"
FALSE 56237369 Chevalier, Chevalier 2018-01-24 01:35:22 FALSE "CommonsDelinker" 2304267 FALSE 2 FALSE 822038928 gwk6pampl8si1v5pv3kwgteg710sfw3 27667 "User:Editingaccount1994/sandbox"
FALSE 56237370 2018-01-07 10:42:20 FALSE "PamD" 1368779 FALSE 0 FALSE 819091874 n4ozbsgle13p9yywtfrz982ccj8woc9 25 "Anita del Rey"
FALSE 56237371 2018-01-07 10:42:27 FALSE "ClueBot NG" 13286072 FALSE 3 FALSE 819091883 ksohnvsbeuzwpl5vb8a3v8m18hva0a7 1274 "User talk:119.94.96.157" Warning welcome to Wikipedia
FALSE 56237372 2018-01-07 10:42:50 FALSE "Underbar dk" 677153 FALSE 14 FALSE 819091914 je7aw21fedbwyqsyofpisdrynsu7olr 113 "Category:Ohmi Railway"
FALSE 56237375 2018-01-07 10:43:32 FALSE "TastyPoutine" 882433 FALSE 3 FALSE 819091968 cpm4tkzcx4hc6irr9ukbi06ogud8dtq 199 "User talk:92.226.219.222"
FALSE 56237375 2018-01-07 11:10:24 FALSE "AnomieBOT" 7611264 TRUE 3 FALSE 819094036 artmfz8b2gxhb3pp8a5p4ksplxqfkpg 1840 "User talk:92.226.219.222"
FALSE 56237375 2018-01-07 14:33:36 FALSE "Only" 702940 FALSE 3 FALSE 819112363 dn9wj0n8d8pdd5lqe56uw5xamupowr1 2949 "User talk:92.226.219.222"
FALSE 56237376 2018-01-07 10:44:01 FALSE "Dipayanacharya" 32794237 FALSE 2 FALSE 819092004 ofueugwatmmn7u73isw732neuza57gk 28 "User:Dipayanacharya"
FALSE 56237376 2018-01-07 10:49:08 FALSE "Dipayanacharya" 32794237 FALSE 2 FALSE 819092390 dsz55xv96ec2uv6w9c1z7c52ipfovbw 38 "User:Dipayanacharya"
FALSE 56237378 2018-01-07 10:44:56 FALSE "Vinegarymass911" 21516552 FALSE 0 FALSE 819092066 9ma38hak0ef1ew4fpiutxpnzd8oz1wd 65 "BSCIC"
FALSE 56237379 2018-01-07 10:45:21 FALSE "BrownHairedGirl" 754619 FALSE 14 FALSE 819092102 4dvakoat58bzyf5hmtthxukt29hip6n 285 "Category:Women government ministers of Yemen"
FALSE 56237381 2018-01-07 10:45:54 FALSE "PRehse" 410898 FALSE 1 FALSE 819092135 2sjrxsc7os9k9pg4su2t4rk2j8nn0h7 103 "Talk:List of Morning Glories Characters"
FALSE 56237382 2018-01-07 10:45:56 FALSE "ClueBot NG" 13286072 FALSE 3 FALSE 819092138 3y9t5wpk6ur5jhone75rhm4wjf01fgi 1330 "User talk:106.207.126.114" Warning welcome to Wikipedia
FALSE 56237382 2018-01-07 10:50:22 FALSE "HindWIKI" 31190506 FALSE 3 FALSE 819092495 8wvn6vh3isyt0dorpe89lztrburgupe 2355 "User talk:106.207.126.114" welcome to Wikipedia

1 anon articleid chev_com date_time deleted editor editor_id editorid minor namespace revert reverteds revid sha1 text_chars title warning wiki_welcome
2 FALSE 56237363 None 2018-01-07 10:40:58 FALSE NinjaRobotPirate 3742946 FALSE 3 FALSE 819091731 135nz8q6lfam6cojla7azb7k5alx3t3 1141 User talk:86.139.142.254 None None
3 FALSE 56237364 None 2018-01-07 10:41:10 FALSE Kavin kavitha 32792125 FALSE 3 FALSE 819091755 0pwezjc6yopz0smc8al6ogc4fax5bwo 663 User talk:Kavin kavitha None None
4 FALSE 56237365 None 2018-01-07 10:41:26 FALSE Amicable always 32621254 FALSE 3 FALSE 819091788 sz3t2ap7z8bpkdvdvi195f3i35949bv 399 User talk:Dr.vivek163 None None
5 FALSE 56237366 None 2018-01-07 10:41:31 FALSE ClueBot NG 13286072 FALSE 3 FALSE 819091796 r6s5j8j3iykenrhuhpnkpsmmd71vubf 1260 User talk:Twistorl Warning welcome to Wikipedia
6 FALSE 56237368 None 2018-01-07 10:41:51 FALSE Khruner 8409334 FALSE 0 FALSE 819091825 tf5qz2yaswx61zrlm9ovxzuhl7r2dc4 2249 Kom Firin None None
7 FALSE 56237368 None 2018-01-27 12:16:02 FALSE Khruner 8409334 TRUE 0 FALSE 822610647 e6oa4g0qv64icdaq26uu1zzbyr5hcbh 2230 Kom Firin None None
8 FALSE 56237369 Chevalier, Chevalier 2018-01-07 10:42:05 FALSE Editingaccount1994 32794215 FALSE 2 FALSE 819091844 0fyvyh2a8xu41gt8obr34oba0bfixj6 27840 User:Editingaccount1994/sandbox None None
9 FALSE 56237369 None 2018-01-07 11:09:52 FALSE AnomieBOT 7611264 TRUE 2 FALSE 819093984 8gy52aolt5rg3eaketwj5v7eiw0apv2 27787 User:Editingaccount1994/sandbox None None
10 FALSE 56237369 None 2018-01-12 21:45:50 FALSE SporkBot 12406635 TRUE 2 FALSE 820064189 he8ydemaanxlrpftqxkez8jfpge1fsj 27784 User:Editingaccount1994/sandbox None None
11 FALSE 56237369 None 2018-01-12 23:28:11 FALSE SporkBot 12406635 TRUE 2 FALSE 820078679 0to17w9rth3url8n7gvucdtobybdq5h 27783 User:Editingaccount1994/sandbox None None
12 FALSE 56237369 None 2018-01-12 23:28:39 FALSE SporkBot 12406635 TRUE 2 FALSE 820078733 531dizmmloyxffbkdr5vph7owh921eg 27782 User:Editingaccount1994/sandbox None None
13 FALSE 56237369 None 2018-01-13 13:45:33 FALSE Frietjes 13791031 FALSE 2 FALSE 820177382 nik9p2u2fuk4yazjxt8ymbicxv5qid9 27757 User:Editingaccount1994/sandbox None None
14 FALSE 56237369 Chevalier, Chevalier 2018-01-24 01:35:22 FALSE CommonsDelinker 2304267 FALSE 2 FALSE 822038928 gwk6pampl8si1v5pv3kwgteg710sfw3 27667 User:Editingaccount1994/sandbox None None
15 FALSE 56237370 None 2018-01-07 10:42:20 FALSE PamD 1368779 FALSE 0 FALSE 819091874 n4ozbsgle13p9yywtfrz982ccj8woc9 25 Anita del Rey None None
16 FALSE 56237371 None 2018-01-07 10:42:27 FALSE ClueBot NG 13286072 FALSE 3 FALSE 819091883 ksohnvsbeuzwpl5vb8a3v8m18hva0a7 1274 User talk:119.94.96.157 Warning welcome to Wikipedia
17 FALSE 56237372 None 2018-01-07 10:42:50 FALSE Underbar dk 677153 FALSE 14 FALSE 819091914 je7aw21fedbwyqsyofpisdrynsu7olr 113 Category:Ohmi Railway None None
18 FALSE 56237375 None 2018-01-07 10:43:32 FALSE TastyPoutine 882433 FALSE 3 FALSE 819091968 cpm4tkzcx4hc6irr9ukbi06ogud8dtq 199 User talk:92.226.219.222 None None
19 FALSE 56237375 None 2018-01-07 11:10:24 FALSE AnomieBOT 7611264 TRUE 3 FALSE 819094036 artmfz8b2gxhb3pp8a5p4ksplxqfkpg 1840 User talk:92.226.219.222 None None
20 FALSE 56237375 None 2018-01-07 14:33:36 FALSE Only 702940 FALSE 3 FALSE 819112363 dn9wj0n8d8pdd5lqe56uw5xamupowr1 2949 User talk:92.226.219.222 None None
21 FALSE 56237376 None 2018-01-07 10:44:01 FALSE Dipayanacharya 32794237 FALSE 2 FALSE 819092004 ofueugwatmmn7u73isw732neuza57gk 28 User:Dipayanacharya None None
22 FALSE 56237376 None 2018-01-07 10:49:08 FALSE Dipayanacharya 32794237 FALSE 2 FALSE 819092390 dsz55xv96ec2uv6w9c1z7c52ipfovbw 38 User:Dipayanacharya None None
23 FALSE 56237378 None 2018-01-07 10:44:56 FALSE Vinegarymass911 21516552 FALSE 0 FALSE 819092066 9ma38hak0ef1ew4fpiutxpnzd8oz1wd 65 BSCIC None None
24 FALSE 56237379 None 2018-01-07 10:45:21 FALSE BrownHairedGirl 754619 FALSE 14 FALSE 819092102 4dvakoat58bzyf5hmtthxukt29hip6n 285 Category:Women government ministers of Yemen None None
25 FALSE 56237381 None 2018-01-07 10:45:54 FALSE PRehse 410898 FALSE 1 FALSE 819092135 2sjrxsc7os9k9pg4su2t4rk2j8nn0h7 103 Talk:List of Morning Glories Characters None None
26 FALSE 56237382 None 2018-01-07 10:45:56 FALSE ClueBot NG 13286072 FALSE 3 FALSE 819092138 3y9t5wpk6ur5jhone75rhm4wjf01fgi 1330 User talk:106.207.126.114 Warning welcome to Wikipedia
27 FALSE 56237382 None 2018-01-07 10:50:22 FALSE HindWIKI 31190506 FALSE 3 FALSE 819092495 8wvn6vh3isyt0dorpe89lztrburgupe 2355 User talk:106.207.126.114 None welcome to Wikipedia

2
test/baseline_output/revert_radius_ikwiki-20180301-pages-meta-history.tsv
View File

@ -1,4 +1,4 @@
anon articleid date_time deleted editor editor_id minor namespace revert reverteds revid sha1 text_chars title
anon articleid date_time deleted editor editorid minor namespace revert reverteds revid sha1 text_chars title
FALSE 2 2004-01-25 13:40:25 FALSE "Brion VIBBER" 1 FALSE 0 FALSE 3094 a6dzmzujf5owfsclacifhewzvp6q8ya 24 "HomePage"
TRUE 2 2007-01-01 14:51:20 FALSE "125.229.3.76" FALSE 0 FALSE 5938 46va13zuh9l56lj7ppey9a11l99zamg 337 "HomePage"
FALSE 2 2008-02-24 15:07:26 FALSE "Alexbot" 147 TRUE 0 FALSE 7322 5yib0zj4fa2lxe8xxw6l702097371u2 343 "HomePage"

Can't render this file because it is too large.

2
test/baseline_output/url-encode_ikwiki-20180301-pages-meta-history.tsv
View File

@ -1,4 +1,4 @@
anon articleid date_time deleted editor editor_id minor namespace revert reverteds revid sha1 text_chars title
anon articleid date_time deleted editor editorid minor namespace revert reverteds revid sha1 text_chars title
FALSE 2 2004-01-25 13:40:25 FALSE %22Brion%20VIBBER%22 1 FALSE 0 FALSE 3094 a6dzmzujf5owfsclacifhewzvp6q8ya 24 %22HomePage%22
TRUE 2 2007-01-01 14:51:20 FALSE %22125.229.3.76%22 FALSE 0 FALSE 5938 46va13zuh9l56lj7ppey9a11l99zamg 337 %22HomePage%22
FALSE 2 2008-02-24 15:07:26 FALSE %22Alexbot%22 147 TRUE 0 FALSE 7322 5yib0zj4fa2lxe8xxw6l702097371u2 343 %22HomePage%22

Can't render this file because it is too large.

2
test/baseline_output/url-encode_sailormoon.tsv
View File

@ -1,4 +1,4 @@
anon articleid date_time deleted editor editor_id minor namespace revert reverteds revid sha1 text_chars title
anon articleid date_time deleted editor editorid minor namespace revert reverteds revid sha1 text_chars title
FALSE 1 2004-11-11 23:33:14 FALSE %22Jasonr%22 1 FALSE 6 FALSE 274 da39a3ee5e6b4b0d3255bfef95601890afd80709 0 %22File%3AWiki.png%22
FALSE 3 2004-11-11 23:33:14 FALSE %22Jasonr%22 1 FALSE 6 FALSE 276 da39a3ee5e6b4b0d3255bfef95601890afd80709 0 %22File%3ASearch%20logo.png%22
FALSE 6 2005-07-16 21:46:43 FALSE %22Purplefeltangel%22 5055 FALSE 4 FALSE 3 83d73a7b77b4de495037c6236808e8518d545591 184 %22Sailor%20Moon%20Wiki%3AAbout%22

Can't render this file because it is too large.

710
wikiq
View File

@ -6,8 +6,9 @@
import argparse
import sys
import os, os.path
import os.path
import re
from datetime import datetime, timezone
from subprocess import Popen, PIPE
from collections import deque
@ -19,48 +20,57 @@ from deltas.tokenizers import wikitext_split
import mwpersistence
import mwreverts
from urllib.parse import quote
TO_ENCODE = ('title', 'editor')
PERSISTENCE_RADIUS=7
PERSISTENCE_RADIUS = 7
from deltas import SequenceMatcher
from deltas import SegmentMatcher
import dataclasses as dc
from dataclasses import dataclass
import pyarrow as pa
import pyarrow.parquet as pq
class PersistMethod:
none = 0
sequence = 1
segment = 2
legacy = 3
def calculate_persistence(tokens_added):
return(sum([(len(x.revisions)-1) for x in tokens_added]),
len(tokens_added))
return (sum([(len(x.revisions) - 1) for x in tokens_added]),
len(tokens_added))
class WikiqIterator():
class WikiqIterator:
def __init__(self, fh, collapse_user=False):
self.fh = fh
self.collapse_user = collapse_user
self.mwiterator = Dump.from_file(self.fh)
self.namespace_map = { ns.id : ns.name for ns in
self.mwiterator.site_info.namespaces }
self.namespace_map = {ns.id: ns.name for ns in
self.mwiterator.site_info.namespaces}
self.__pages = self.load_pages()
def load_pages(self):
for page in self.mwiterator:
yield WikiqPage(page,
namespace_map = self.namespace_map,
namespace_map=self.namespace_map,
collapse_user=self.collapse_user)
def __iter__(self):
return self.__pages
def __next__(self):
return next(self._pages)
return next(self.__pages)
class WikiqPage():
class WikiqPage:
__slots__ = ('id', 'title', 'namespace', 'redirect',
'restrictions', 'mwpage', '__revisions',
'collapse_user')
def __init__(self, page, namespace_map, collapse_user=False):
self.id = page.id
self.namespace = page.namespace
@ -87,10 +97,11 @@ class WikiqPage():
# 3 A B True
# 4 A A False
# Post-loop A Always
collapsed_revs = 0
for i, rev in enumerate(self.mwpage):
# never yield the first time
if i == 0:
if self.collapse_user:
if self.collapse_user:
collapsed_revs = 1
rev.collapsed_revs = collapsed_revs
@ -128,6 +139,13 @@ class WikiqPage():
return next(self.__revisions)
"""
A RegexPair is defined by a regular expression (pattern) and a label.
The pattern can include capture groups. If it does then each capture group will have a resulting column in the output.
If the pattern does not include a capture group, then only one output column will result.
"""
class RegexPair(object):
def __init__(self, pattern, label):
self.pattern = re.compile(pattern)
@ -135,12 +153,21 @@ class RegexPair(object):
self.has_groups = bool(self.pattern.groupindex)
if self.has_groups:
self.capture_groups = list(self.pattern.groupindex.keys())
def get_pyarrow_fields(self):
if self.has_groups:
fields = [pa.field(self._make_key(cap_group), pa.list_(pa.string()))
for cap_group in self.capture_groups]
else:
fields = [pa.field(self.label, pa.list_(pa.string()))]
return fields
def _make_key(self, cap_group):
return ("{}_{}".format(self.label, cap_group))
return "{}_{}".format(self.label, cap_group)
def matchmake(self, content, rev_data):
temp_dict = {}
# if there are named capture groups in the regex
if self.has_groups:
@ -155,11 +182,11 @@ class RegexPair(object):
temp_list = []
for match in matchobjects:
# we only want to add the match for the capture group if the match is not None
if match.group(cap_group) != None:
if match.group(cap_group) is not None:
temp_list.append(match.group(cap_group))
# if temp_list of matches is empty just make that column None
if len(temp_list)==0:
if len(temp_list) == 0:
temp_dict[key] = None
# else we put in the list we made in the for-loop above
else:
@ -173,28 +200,186 @@ class RegexPair(object):
# there are no capture groups, we just search for all the matches of the regex
else:
#given that there are matches to be made
if self.pattern.search(content) is not None:
m = self.pattern.findall(content)
temp_dict[self.label] = ', '.join(m)
else:
temp_dict[self.label] = None
# given that there are matches to be made
if type(content) in (str, bytes):
if self.pattern.search(content) is not None:
m = self.pattern.findall(content)
temp_dict[self.label] = ', '.join(m)
else:
temp_dict[self.label] = None
# update rev_data with our new columns
rev_data.update(temp_dict)
for k, v in temp_dict.items():
setattr(rev_data, k, v)
return rev_data
class WikiqParser():
def __init__(self, input_file, output_file, regex_match_revision, regex_match_comment, regex_revision_label, regex_comment_label, collapse_user=False, persist=None, urlencode=False, namespaces = None, revert_radius=15):
"""
We used to use a dictionary to collect fields for the output.
Now we use dataclasses. Compared to a dictionary, this should help:
- prevent some bugs
- make it easier to output parquet data.
- use class attribute '.' syntax instead of dictionary syntax.
- improve support for tooling (autocomplete, type hints)
- use type information to define formatting rules
Depending on the parameters passed into Wikiq, the output schema can be different.
Therefore, we need to end up constructing a dataclass with the correct output schema.
It also needs to have the correct pyarrow schema so we can write parquet files.
The RevDataBase type has all the fields that will be output no matter how wikiq is invoked.
"""
@dataclass()
class RevDataBase:
revid: int
date_time: datetime
articleid: int
editorid: int
title: str
namespace: int
deleted: bool
text_chars: int = None
revert: bool = None
reverteds: list[int] = None
sha1: str = None
minor: bool = None
editor: str = None
anon: bool = None
# toggles url encoding. this isn't a dataclass field since it doesn't have a type annotation
urlencode = False
# defines pyarrow schema.
# each field in the data class needs an entry in this array.
# the names should match and be in the same order.
# this isn't a dataclass field since it doesn't have a type annotation
pa_schema_fields = [
pa.field("revid", pa.int64()),
pa.field("date_time", pa.timestamp('ms')),
pa.field("articleid", pa.int64()),
pa.field("editorid", pa.int64()),
pa.field("title", pa.string()),
pa.field("namespace", pa.int32()),
pa.field("deleted", pa.bool_()),
pa.field("test_chars", pa.int32()),
pa.field("revert", pa.bool_()),
pa.field("reverteds", pa.list_(pa.int64())),
pa.field("sha1", pa.string()),
pa.field("minor", pa.bool_()),
pa.field("editor", pa.string()),
pa.field("anon", pa.bool_())
]
# pyarrow is a columnar format, so most of the work happens in the flush_parquet_buffer function
def to_pyarrow(self):
return dc.astuple(self)
# logic to convert each field into the wikiq tsv format goes here.
def to_tsv_row(self):
row = []
for f in dc.fields(self):
val = getattr(self, f.name)
if getattr(self, f.name) is None:
row.append("")
elif f.type == bool:
row.append("TRUE" if val else "FALSE")
elif f.type == datetime:
row.append(val.strftime('%Y-%m-%d %H:%M:%S'))
elif f.name in {'editor', 'title'}:
s = '"' + val + '"'
if self.urlencode and f.name in TO_ENCODE:
row.append(quote(str(s)))
else:
row.append(s)
elif f.type == list[int]:
row.append('"' + ",".join([str(x) for x in val]) + '"')
elif f.type == str:
if self.urlencode and f.name in TO_ENCODE:
row.append(quote(str(val)))
else:
row.append(val)
else:
row.append(val)
return '\t'.join(map(str, row))
def header_row(self):
return '\t'.join(map(lambda f: f.name, dc.fields(self)))
"""
If collapse=True we'll use a RevDataCollapse dataclass.
This class inherits from RevDataBase. This means that it has all the same fields and functions.
It just adds a new field and updates the pyarrow schema.
"""
@dataclass()
class RevDataCollapse(RevDataBase):
collapsed_revs: int = None
pa_collapsed_revs_schema = pa.field('collapsed_revs', pa.int64())
pa_schema_fields = RevDataBase.pa_schema_fields + [pa_collapsed_revs_schema]
"""
If persistence data is to be computed we'll need the fields added by RevDataPersistence.
"""
@dataclass()
class RevDataPersistence(RevDataBase):
token_revs: int = None
tokens_added: int = None
tokens_removed: int = None
tokens_window: int = None
pa_persistence_schema_fields = [
pa.field("token_revs", pa.int64()),
pa.field("tokens_added", pa.int64()),
pa.field("tokens_removed", pa.int64()),
pa.field("tokens_window", pa.int64())]
pa_schema_fields = RevDataBase.pa_schema_fields + pa_persistence_schema_fields
"""
class RevDataCollapsePersistence uses multiple inheritance to make a class that has both persistence and collapse fields.
"""
@dataclass()
class RevDataCollapsePersistence(RevDataCollapse, RevDataPersistence):
pa_schema_fields = RevDataCollapse.pa_schema_fields + RevDataPersistence.pa_persistence_schema_fields
class WikiqParser:
def __init__(self, input_file, output_file, regex_match_revision, regex_match_comment, regex_revision_label,
regex_comment_label, collapse_user=False, persist=None, urlencode=False, namespaces=None,
revert_radius=15, output_parquet=True, parquet_buffer_size=2000):
"""
Parameters:
persist : what persistence method to use. Takes a PersistMethod value
"""
self.input_file = input_file
self.output_file = output_file
self.collapse_user = collapse_user
self.persist = persist
self.printed_header = False
self.namespaces = []
self.urlencode = urlencode
self.revert_radius = revert_radius
@ -204,25 +389,74 @@ class WikiqParser():
else:
self.namespace_filter = None
self.regex_schemas = []
self.regex_revision_pairs = self.make_matchmake_pairs(regex_match_revision, regex_revision_label)
self.regex_comment_pairs = self.make_matchmake_pairs(regex_match_comment, regex_comment_label)
# This is where we set the type for revdata.
if self.collapse_user is True:
if self.persist == PersistMethod.none:
revdata_type = RevDataCollapse
else:
revdata_type = RevDataCollapsePersistence
elif self.persist != PersistMethod.none:
revdata_type = RevDataPersistence
else:
revdata_type = RevDataBase
# if there are regex fields, we need to add them to the revdata type.
regex_fields = [(field.name, list[str], dc.field(default=None)) for field in self.regex_schemas]
# make_dataclass is a function that defines a new dataclass type.
# here we extend the type we have already chosen and add the regular expression types
self.revdata_type = dc.make_dataclass('RevData_Parser',
fields=regex_fields,
bases=(revdata_type,))
# we also need to make sure that we have the right pyarrow schema
self.revdata_type.pa_schema_fields = revdata_type.pa_schema_fields + self.regex_schemas
self.revdata_type.urlencode = self.urlencode
self.schema = pa.schema(self.revdata_type.pa_schema_fields)
# here we initialize the variables we need for output.
if output_parquet is True:
self.output_parquet = True
self.pq_writer = None
self.output_file = output_file
self.parquet_buffer = []
self.parquet_buffer_size = parquet_buffer_size
else:
self.print_header = True
if output_file == sys.stdout:
self.output_file = output_file
else:
self.output_file = open(output_file, 'w')
self.output_parquet = False
def make_matchmake_pairs(self, patterns, labels):
if (patterns is not None and labels is not None) and \
(len(patterns) == len(labels)):
return [RegexPair(pattern, label) for pattern, label in zip(patterns, labels)]
elif (patterns is None and labels is None):
(len(patterns) == len(labels)):
result = []
for pattern, label in zip(patterns, labels):
rp = RegexPair(pattern, label)
result.append(rp)
self.regex_schemas = self.regex_schemas + rp.get_pyarrow_fields()
return result
elif (patterns is None) and (labels is None):
return []
else:
sys.exit('Each regular expression *must* come with a corresponding label and vice versa.')
def matchmake(self, rev, rev_data):
rev_data = self.matchmake_revision(rev.text, rev_data)
def matchmake_revision(self, rev, rev_data):
rev_data = self.matchmake_text(rev.text, rev_data)
rev_data = self.matchmake_comment(rev.comment, rev_data)
return rev_data
def matchmake_revision(self, text, rev_data):
def matchmake_text(self, text, rev_data):
return self.matchmake_pairs(text, rev_data, self.regex_revision_pairs)
def matchmake_comment(self, comment, rev_data):
@ -238,7 +472,7 @@ class WikiqParser():
for ns in self.namespaces:
# skip if the namespace is not defined
if ns == None:
if ns is None:
default_ns = self.namespaces[ns]
continue
@ -248,7 +482,6 @@ class WikiqParser():
# if we've made it this far with no matches, we return the default namespace
return default_ns
def process(self):
# create a regex that creates the output filename
@ -259,13 +492,12 @@ class WikiqParser():
# Construct dump file iterator
dump = WikiqIterator(self.input_file, collapse_user=self.collapse_user)
# extract list of namspaces
self.namespaces = {ns.name : ns.id for ns in dump.mwiterator.site_info.namespaces}
# extract list of namespaces
self.namespaces = {ns.name: ns.id for ns in dump.mwiterator.site_info.namespaces}
page_count = 0
rev_count = 0
# Iterate through pages
for page in dump:
namespace = page.namespace if page.namespace is not None else self.__get_namespace_from_title(page.title)
@ -275,17 +507,17 @@ class WikiqParser():
if namespace not in self.namespace_filter:
continue
rev_detector = mwreverts.Detector(radius = self.revert_radius)
rev_detector = mwreverts.Detector(radius=self.revert_radius)
if self.persist != PersistMethod.none:
window = deque(maxlen=PERSISTENCE_RADIUS)
if self.persist == PersistMethod.sequence:
state = mwpersistence.DiffState(SequenceMatcher(tokenizer = wikitext_split),
state = mwpersistence.DiffState(SequenceMatcher(tokenizer=wikitext_split),
revert_radius=PERSISTENCE_RADIUS)
elif self.persist == PersistMethod.segment:
state = mwpersistence.DiffState(SegmentMatcher(tokenizer = wikitext_split),
state = mwpersistence.DiffState(SegmentMatcher(tokenizer=wikitext_split),
revert_radius=PERSISTENCE_RADIUS)
# self.persist == PersistMethod.legacy
@ -295,28 +527,20 @@ class WikiqParser():
# Iterate through a page's revisions
for rev in page:
# initialize rev_data
rev_data = {
'revid':rev.id,
'date_time' : rev.timestamp.strftime('%Y-%m-%d %H:%M:%S'),
'articleid' : page.id,
'editor_id' : "" if rev.deleted.user == True or rev.user.id is None else rev.user.id,
'title' : '"' + page.title + '"',
'namespace' : namespace,
'deleted' : "TRUE" if rev.deleted.text else "FALSE"
}
rev_data = self.matchmake(rev, rev_data)
# create a new data object instead of a dictionary.
rev_data = self.revdata_type(revid=rev.id,
date_time=datetime.fromtimestamp(rev.timestamp.unix(), tz=timezone.utc),
articleid=page.id,
editorid="" if rev.deleted.user == True or rev.user.id is None else rev.user.id,
title=page.title,
deleted=rev.deleted.text,
namespace=namespace
)
# if revisions are deleted, /many/ things will be missing
if rev.deleted.text:
rev_data['text_chars'] = ""
rev_data['sha1'] = ""
rev_data['revert'] = ""
rev_data['reverteds'] = ""
rev_data = self.matchmake_revision(rev, rev_data)
else:
if not rev.deleted.text:
# rev.text can be None if the page has no text
if not rev.text:
rev.text = ""
@ -325,70 +549,62 @@ class WikiqParser():
if rev.sha1:
text_sha1 = rev.sha1
else:
text_sha1 = sha1(bytes(rev.text, "utf8")).hexdigest()
rev_data['sha1'] = text_sha1
rev_data.sha1 = text_sha1
# TODO rev.bytes doesn't work.. looks like a bug
rev_data['text_chars'] = len(rev.text)
rev_data.text_chars = len(rev.text)
# generate revert data
revert = rev_detector.process(text_sha1, rev.id)
if revert:
rev_data['revert'] = "TRUE"
rev_data['reverteds'] = '"' + ",".join([str(x) for x in revert.reverteds]) + '"'
rev_data.revert = True
rev_data.reverteds = revert.reverteds
else:
rev_data['revert'] = "FALSE"
rev_data['reverteds'] = ""
rev_data.revert = False
# if the fact that the edit was minor can be hidden, this might be an issue
rev_data['minor'] = "TRUE" if rev.minor else "FALSE"
rev_data.minor = rev.minor
if not rev.deleted.user:
# wrap user-defined editors in quotes for fread
rev_data['editor'] = '"' + rev.user.text + '"'
rev_data['anon'] = "TRUE" if rev.user.id == None else "FALSE"
else:
rev_data['anon'] = ""
rev_data['editor'] = ""
rev_data.editor = rev.user.text
rev_data.anon = rev.user.id is None
#if re.match(r'^#redirect \[\[.*\]\]', rev.text, re.I):
# if re.match(r'^#redirect \[\[.*\]\]', rev.text, re.I):
# redirect = True
#else:
# else:
# redirect = False
#TODO missing: additions_size deletions_size
# if collapse user was on, lets run that
if self.collapse_user:
rev_data['collapsed_revs'] = rev.collapsed_revs
# TODO missing: additions_size deletions_size
# if collapse user was on, let's run that
if self.collapse_user:
rev_data.collapsed_revs = rev.collapsed_revs
# get the
if self.persist != PersistMethod.none:
if rev.deleted.text:
for k in ["token_revs", "tokens_added", "tokens_removed", "tokens_window"]:
old_rev_data[k] = None
else:
if not rev.deleted.text:
if self.persist != PersistMethod.legacy:
_, tokens_added, tokens_removed = state.update(rev.text, rev.id)
else:
_, tokens_added, tokens_removed = state.process(rev.text, rev.id, text_sha1)
window.append((rev.id, rev_data, tokens_added, tokens_removed))
if len(window) == PERSISTENCE_RADIUS:
old_rev_id, old_rev_data, old_tokens_added, old_tokens_removed = window[0]
num_token_revs, num_tokens = calculate_persistence(old_tokens_added)
old_rev_data["token_revs"] = num_token_revs
old_rev_data["tokens_added"] = num_tokens
old_rev_data["tokens_removed"] = len(old_tokens_removed)
old_rev_data["tokens_window"] = PERSISTENCE_RADIUS-1
old_rev_data.token_revs = num_token_revs
old_rev_data.tokens_added = num_tokens
old_rev_data.tokens_removed = len(old_tokens_removed)
old_rev_data.tokens_window = PERSISTENCE_RADIUS - 1
self.print_rev_data(old_rev_data)
@ -407,11 +623,10 @@ class WikiqParser():
rev_id, rev_data, tokens_added, tokens_removed = item
num_token_revs, num_tokens = calculate_persistence(tokens_added)
rev_data["token_revs"] = num_token_revs
rev_data["tokens_added"] = num_tokens
rev_data["tokens_removed"] = len(tokens_removed)
rev_data["tokens_window"] = len(window)-(i+1)
rev_data.token_revs = num_token_revs
rev_data.tokens_added = num_tokens
rev_data.tokens_removed = len(tokens_removed)
rev_data.tokens_window = len(window) - (i + 1)
self.print_rev_data(rev_data)
page_count += 1
@ -419,155 +634,236 @@ class WikiqParser():
print("Done: %s revisions and %s pages." % (rev_count, page_count),
file=sys.stderr)
def print_rev_data(self, rev_data):
# if it's the first time through, print the header
if self.urlencode:
for field in TO_ENCODE:
rev_data[field] = quote(str(rev_data[field]))
# remember to flush the parquet_buffer if we're done
if self.output_parquet is True:
self.flush_parquet_buffer()
self.pq_writer.close()
if not self.printed_header:
print("\t".join([str(k) for k in sorted(rev_data.keys())]), file=self.output_file)
self.printed_header = True
print("\t".join([str(v) for k, v in sorted(rev_data.items())]), file=self.output_file)
else:
self.output_file.close()
"""
For performance reasons it's better to write parquet in batches instead of one row at a time.
So this function just puts the data on a buffer. If the buffer is full, then it gets flushed (written).
"""
def write_parquet_row(self, rev_data):
padata = rev_data.to_pyarrow()
self.parquet_buffer.append(padata)
if len(self.parquet_buffer) >= self.parquet_buffer_size:
self.flush_parquet_buffer()
"""
Function that actually writes data to the parquet file.
It needs to transpose the data from row-by-row to column-by-column
"""
def flush_parquet_buffer(self):
"""
Returns the pyarrow table that we'll write
"""
def rows_to_table(rg, schema):
cols = []
first = rg[0]
for col in first:
cols.append([col])
for row in rg[1:]:
for j in range(len(cols)):
cols[j].append(row[j])
arrays = []
for col, typ in zip(cols, schema.types):
arrays.append(pa.array(col, typ))
return pa.Table.from_arrays(arrays, schema=schema)
outtable = rows_to_table(self.parquet_buffer, self.schema)
if self.pq_writer is None:
self.pq_writer = pq.ParquetWriter(self.output_file, self.schema, flavor='spark')
self.pq_writer.write_table(outtable)
self.parquet_buffer = []
# depending on if we are configured to write tsv or parquet, we'll call a different function.
def print_rev_data(self, rev_data):
if self.output_parquet is False:
printfunc = self.write_tsv_row
else:
printfunc = self.write_parquet_row
printfunc(rev_data)
def write_tsv_row(self, rev_data):
if self.print_header:
print(rev_data.header_row(), file=self.output_file)
self.print_header = False
line = rev_data.to_tsv_row()
print(line, file=self.output_file)
def open_input_file(input_filename):
if re.match(r'.*\.7z$', input_filename):
cmd = ["7za", "x", "-so", input_filename, '*']
cmd = ["7za", "x", "-so", input_filename, "*.xml"]
elif re.match(r'.*\.gz$', input_filename):
cmd = ["zcat", input_filename]
cmd = ["zcat", input_filename]
elif re.match(r'.*\.bz2$', input_filename):
cmd = ["bzcat", "-dk", input_filename]
cmd = ["bzcat", "-dk", input_filename]
else:
raise ValueError("Unrecognized file type: %s" % input_filename)
try:
input_file = Popen(cmd, stdout=PIPE).stdout
return Popen(cmd, stdout=PIPE).stdout
except NameError:
input_file = open(input_filename, 'r')
return open(input_filename, 'r')
def get_output_filename(input_filename, parquet=False):
output_filename = re.sub(r'\.(7z|gz|bz2)?$', '', input_filename)
output_filename = re.sub(r'\.xml', '', output_filename)
if parquet is False:
output_filename = output_filename + ".tsv"
else:
output_filename = output_filename + ".parquet"
return output_filename
return input_file
def open_output_file(input_filename):
# create a regex that creates the output filename
output_filename = re.sub(r'\.(7z|gz|bz2)?$', '', input_filename)
output_filename = re.sub(r'\.xml', '', output_filename)
output_filename = output_filename + ".tsv"
output_filename = get_output_filename(input_filename, parquet=False)
output_file = open(output_filename, "w")
return output_file
parser = argparse.ArgumentParser(description='Parse MediaWiki XML database dumps into tab delimitted data.')
# arguments for the input direction
parser.add_argument('dumpfiles', metavar="DUMPFILE", nargs="*", type=str,
help="Filename of the compressed or uncompressed XML database dump. If absent, we'll look for content on stdin and output on stdout.")
def main():
parser = argparse.ArgumentParser(description='Parse MediaWiki XML database dumps into tab delimited data.')
parser.add_argument('-o', '--output-dir', metavar='DIR', dest='output_dir', type=str, nargs=1,
help="Directory for output files.")
# arguments for the input direction
parser.add_argument('dumpfiles', metavar="DUMPFILE", nargs="*", type=str,
help="Filename of the compressed or uncompressed XML database dump. If absent, we'll look for content on stdin and output on stdout.")
parser.add_argument('-s', '--stdout', dest="stdout", action="store_true",
help="Write output to standard out (do not create dump file)")
parser.add_argument('-o', '--output-dir', metavar='DIR', dest='output_dir', type=str, nargs=1,
help="Directory for output files. If it ends with .parquet output will be in parquet format.")
parser.add_argument('--collapse-user', dest="collapse_user", action="store_true",
help="Operate only on the final revision made by user a user within all sequences of consecutive edits made by a user. This can be useful for addressing issues with text persistence measures.")
parser.add_argument('-s', '--stdout', dest="stdout", action="store_true",
help="Write output to standard out (do not create dump file)")
parser.add_argument('-p', '--persistence', dest="persist", default=None, const='', type=str, choices = ['','segment','sequence','legacy'], nargs='?',
help="Compute and report measures of content persistent: (1) persistent token revisions, (2) tokens added, and (3) number of revision used in computing the first measure. This may by slow. The defualt is -p=sequence, which uses the same algorithm as in the past, but with improvements to wikitext parsing. Use -p=legacy for old behavior used in older research projects. Use -p=segment for advanced persistence calculation method that is robust to content moves, but prone to bugs, and slower.")
parser.add_argument('--collapse-user', dest="collapse_user", action="store_true",
help="Operate only on the final revision made by user a user within all sequences of consecutive edits made by a user. This can be useful for addressing issues with text persistence measures.")
parser.add_argument('-u', '--url-encode', dest="urlencode", action="store_true",
help="Output url encoded text strings. This works around some data issues like newlines in editor names. In the future it may be used to output other text data.")
parser.add_argument('-p', '--persistence', dest="persist", default=None, const='', type=str,
choices=['', 'segment', 'sequence', 'legacy'], nargs='?',
help="Compute and report measures of content persistent: (1) persistent token revisions, (2) tokens added, and (3) number of revision used in computing the first measure. This may by slow. The default is -p=sequence, which uses the same algorithm as in the past, but with improvements to wikitext parsing. Use -p=legacy for old behavior used in older research projects. Use -p=segment for advanced persistence calculation method that is robust to content moves, but prone to bugs, and slower.")
parser.add_argument('-n', '--namespace-include', dest="namespace_filter", type=int, action='append',
help="Id number of namspace to include. Can be specified more than once.")
parser.add_argument('-u', '--url-encode', dest="urlencode", action="store_true",
help="Output url encoded text strings. This works around some data issues like newlines in editor names. In the future it may be used to output other text data.")
parser.add_argument('-rr',
'--revert-radius',
dest="revert_radius",
type=int,
action='store',
default=15,
help="Number of edits to check when looking for reverts (default: 15)")
parser.add_argument('-n', '--namespace-include', dest="namespace_filter", type=int, action='append',
help="Id number of namespace to include. Can be specified more than once.")
parser.add_argument('-RP', '--revision-pattern', dest="regex_match_revision", default=None, type=str, action='append',
help="The regular expression to search for in revision text. The regex must be surrounded by quotes.")
parser.add_argument('-rr',
'--revert-radius',
dest="revert_radius",
type=int,
action='store',
default=15,
help="Number of edits to check when looking for reverts (default: 15)")
parser.add_argument('-RPl', '--revision-pattern-label', dest="regex_revision_label", default=None, type=str, action='append',
help="The label for the outputted column based on matching the regex in revision text.")
parser.add_argument('-RP', '--revision-pattern', dest="regex_match_revision", default=None, type=str,
action='append',
help="The regular expression to search for in revision text. The regex must be surrounded by quotes.")
parser.add_argument('-CP', '--comment-pattern', dest="regex_match_comment", default=None, type=str, action='append',
help="The regular expression to search for in comments of revisions.")
parser.add_argument('-RPl', '--revision-pattern-label', dest="regex_revision_label", default=None, type=str,
action='append',
help="The label for the outputted column based on matching the regex in revision text.")
parser.add_argument('-CPl', '--comment-pattern-label', dest="regex_comment_label", default=None, type=str, action='append',
help="The label for the outputted column based on matching the regex in comments.")
parser.add_argument('-CP', '--comment-pattern', dest="regex_match_comment", default=None, type=str, action='append',
help="The regular expression to search for in comments of revisions.")
args = parser.parse_args()
parser.add_argument('-CPl', '--comment-pattern-label', dest="regex_comment_label", default=None, type=str,
action='append',
help="The label for the outputted column based on matching the regex in comments.")
# set persistence method
args = parser.parse_args()
if args.persist is None:
persist = PersistMethod.none
elif args.persist == "segment":
persist = PersistMethod.segment
elif args.persist == "legacy":
persist = PersistMethod.legacy
else:
persist = PersistMethod.sequence
# set persistence method
if args.namespace_filter is not None:
namespaces = args.namespace_filter
else:
namespaces = None
if args.persist is None:
persist = PersistMethod.none
elif args.persist == "segment":
persist = PersistMethod.segment
elif args.persist == "legacy":
persist = PersistMethod.legacy
else:
persist = PersistMethod.sequence
if len(args.dumpfiles) > 0:
for filename in args.dumpfiles:
input_file = open_input_file(filename)
if args.namespace_filter is not None:
namespaces = args.namespace_filter
else:
namespaces = None
# open directory for output
if args.output_dir:
output_dir = args.output_dir[0]
else:
output_dir = "."
if len(args.dumpfiles) > 0:
output_parquet = False
for filename in args.dumpfiles:
input_file = open_input_file(filename)
print("Processing file: %s" % filename, file=sys.stderr)
# open directory for output
if args.output_dir:
output_dir = args.output_dir[0]
else:
output_dir = "."
if args.stdout:
output_file = sys.stdout
else:
filename = os.path.join(output_dir, os.path.basename(filename))
output_file = open_output_file(filename)
if output_dir.endswith(".parquet"):
output_parquet = True
wikiq = WikiqParser(input_file,
output_file,
print("Processing file: %s" % filename, file=sys.stderr)
if args.stdout:
output_file = sys.stdout
else:
filename = os.path.join(output_dir, os.path.basename(filename))
output_file = get_output_filename(filename, parquet=output_parquet)
wikiq = WikiqParser(input_file,
output_file,
collapse_user=args.collapse_user,
persist=persist,
urlencode=args.urlencode,
namespaces=namespaces,
revert_radius=args.revert_radius,
regex_match_revision=args.regex_match_revision,
regex_revision_label=args.regex_revision_label,
regex_match_comment=args.regex_match_comment,
regex_comment_label=args.regex_comment_label,
output_parquet=output_parquet)
wikiq.process()
# close things
input_file.close()
else:
wikiq = WikiqParser(sys.stdin,
sys.stdout,
collapse_user=args.collapse_user,
persist=persist,
# persist_legacy=args.persist_legacy,
urlencode=args.urlencode,
namespaces=namespaces,
revert_radius=args.revert_radius,
regex_match_revision = args.regex_match_revision,
regex_revision_label = args.regex_revision_label,
regex_match_comment = args.regex_match_comment,
regex_comment_label = args.regex_comment_label)
regex_match_revision=args.regex_match_revision,
regex_revision_label=args.regex_revision_label,
regex_match_comment=args.regex_match_comment,
regex_comment_label=args.regex_comment_label)
wikiq.process()
# close things
input_file.close()
output_file.close()
else:
wikiq = WikiqParser(sys.stdin,
sys.stdout,
collapse_user=args.collapse_user,
persist=persist,
#persist_legacy=args.persist_legacy,
urlencode=args.urlencode,
namespaces=namespaces,
revert_radius=args.revert_radius,
regex_match_revision = args.regex_match_revision,
regex_revision_label = args.regex_revision_label,
regex_match_comment = args.regex_match_comment,
regex_comment_label = args.regex_comment_label)
# stop_words = "a,able,about,across,after,all,almost,also,am,among,an,and,any,are,as,at,be,because,been,but,by,can,cannot,could,dear,did,do,does,either,else,ever,every,for,from,get,got,had,has,have,he,her,hers,him,his,how,however,i,if,in,into,is,it,its,just,least,let,like,likely,may,me,might,most,must,my,neither,no,nor,not,of,off,often,on,only,or,other,our,own,rather,said,say,says,she,should,since,so,some,than,that,the,their,them,then,there,these,they,this,tis,to,too,twas,us,wants,was,we,were,what,when,where,which,while,who,whom,why,will,with,would,yet,you,your"
# stop_words = stop_words.split(",")
wikiq.process()
# stop_words = "a,able,about,across,after,all,almost,also,am,among,an,and,any,are,as,at,be,because,been,but,by,can,cannot,could,dear,did,do,does,either,else,ever,every,for,from,get,got,had,has,have,he,her,hers,him,his,how,however,i,if,in,into,is,it,its,just,least,let,like,likely,may,me,might,most,must,my,neither,no,nor,not,of,off,often,on,only,or,other,our,own,rather,said,say,says,she,should,since,so,some,than,that,the,their,them,then,there,these,they,this,tis,to,too,twas,us,wants,was,we,were,what,when,where,which,while,who,whom,why,will,with,would,yet,you,your"
# stop_words = stop_words.split(",")
if __name__ == "__main__":
main()