collective/mediawiki_dump_tools
18
0
Fork 0
Code Issues Pull Requests Activity

Refactor revision parsing logic to be columnar #1

Merged
beason merged 27 commits from test-parquet into parquet_support 2025-06-17 18:22:26 +00:00
Conversation 10 Commits 27 Files Changed 24 +3
1 changed files with 3 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Showing only changes of commit 17c7f208ab - Show all commits

3
wikiq
View File

@@ -488,6 +488,9 @@ class WikiqParser:
tables.RevisionIsAnon(),
])
if self.collapse_user:
table.columns.append(tables.RevisionCollapsed())
# extract list of namespaces
self.namespaces = {ns.name: ns.id for ns in dump.mwiterator.site_info.namespaces}