{ "cells": [ { "cell_type": "code", "execution_count": 1, "id": "fcc726a8-44a4-48cf-a1cd-937b05bd4d08", "metadata": {}, "outputs": [], "source": [ "import pandas as pd" ] }, { "cell_type": "code", "execution_count": 2, "id": "1fceca29-48c1-4ba3-93ba-88724dea22a7", "metadata": {}, "outputs": [], "source": [ "first_resolved_path = \"/mmfs1/gscratch/comdata/users/mjilg/mw-repo-lifecycles/case3/051725_coref_rel_phab_comments_to_2014.csv\"\n", "first_resolved_df = pd.read_csv(first_resolved_path)\n", "second_resolved_path = \"/mmfs1/gscratch/comdata/users/mjilg/mw-repo-lifecycles/case3/051725_coref_rel_phab_comments_2014_to_2015.csv\"\n", "second_resolved_df = pd.read_csv(second_resolved_path)" ] }, { "cell_type": "code", "execution_count": 3, "id": "f26c31e7-bee1-4100-821f-769e5b1791bd", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "8621" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "len(second_resolved_df)" ] }, { "cell_type": "code", "execution_count": 4, "id": "dfa81ca2-4d66-4679-bc3e-192d0cac67fa", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "5007" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "len(first_resolved_df)" ] }, { "cell_type": "code", "execution_count": 5, "id": "6dc11bda-f0f6-4eb6-96f5-02ed9a3492ba", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "13628" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "combined_df = pd.concat([first_resolved_df, second_resolved_df])\n", "unique_df = combined_df.drop_duplicates()\n", "len(unique_df)" ] }, { "cell_type": "code", "execution_count": 6, "id": "0c903199-8159-455c-aa7f-e57ef07ce03e", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
task_titlecomment_textdate_createdspeakermeta.affilconversation_idcomment_typestatusmeta.gerritidreply_totimestampis_relevantis_migratedtextresolved_text
0User with unattached accounts unable to login ...User:NickK reported in IRC that they're gettin...1411541280PHID-USER-v7vgzvvcw7v2umf737riFalsePHID-TASK-mio2uq45ny7mms72syuttask_descriptionresolvedFalse243215NaN2014-09-24 06:48:00+00:00TrueFalseUser:NickK reported in IRC that they're gettin...User:NickK reported in IRC that they're gettin...
1User with unattached accounts unable to login ...Revert has been deployed.1411573104PHID-USER-v7vgzvvcw7v2umf737riFalsePHID-TASK-mio2uq45ny7mms72syuttask_subcommentNaNFalse243216243215.02014-09-24 15:38:24+00:00TrueFalseRevert has been deployed.Revert has been deployed.
2User with unattached accounts unable to login ...**gerritadmin** wrote:\\n\\nChange 162550 merged...1411572378PHID-USER-ynivjflmc2dcl6w5ut5vFalsePHID-TASK-mio2uq45ny7mms72syuttask_subcommentNaNFalse243217243216.02014-09-24 15:26:18+00:00TrueFalse**gerritadmin** wrote:\\n\\nChange 162550 merged...**gerritadmin** wrote:\\n\\nChange 162550 merged...
3User with unattached accounts unable to login ...(In reply to Kunal Mehta (Legoktm) from commen...1411545535PHID-USER-v7bwpq3rs3zdxegibdbhFalsePHID-TASK-mio2uq45ny7mms72syuttask_subcommentNaNFalse243218243217.02014-09-24 07:58:55+00:00TrueFalse(In reply to Kunal Mehta (Legoktm) from commen...(In reply to Kunal Mehta (Legoktm) from commen...
4User with unattached accounts unable to login ...**gerritadmin** wrote:\\n\\nChange 162549 merged...1411542640PHID-USER-ynivjflmc2dcl6w5ut5vFalsePHID-TASK-mio2uq45ny7mms72syuttask_subcommentNaNFalse243219243218.02014-09-24 07:10:40+00:00TrueFalse**gerritadmin** wrote:\\n\\nChange 162549 merged...**gerritadmin** wrote:\\n\\nChange 162549 merged...
...................................................
8616OAuth login refers to mediawiki.org:/ instead ...> When I registered, phabricator linked mediaw...1413205650PHID-USER-hgn5uw2jafgjgfvxibhhFalsePHID-TASK-yeaxsfxhhtbn26koo5fitask_subcommentNaNFalse378799378798.02014-10-13 13:07:30+00:00TrueFalse> When I registered, phabricator linked mediaw...> When I registered, phabricator linked mediaw...
8617OAuth login refers to mediawiki.org:/ instead ...See {T574} for a related discussion.1412958953PHID-USER-lluzkul4z7us4sxkayssFalsePHID-TASK-yeaxsfxhhtbn26koo5fitask_subcommentNaNFalse378800378799.02014-10-10 16:35:53+00:00TrueFalseSee {T574} for a related discussion.See {T574} for a related discussion.
8618Improvements to Wikimedia SUL login dialog UI:...Some improvements to the Wikimedia SUL dialog:...1412362816PHID-USER-lluzkul4z7us4sxkayssFalsePHID-TASK-j6czqxlv5fzcx3tmq23ntask_descriptiondeclinedFalse378858NaN2014-10-03 19:00:16+00:00TrueFalseSome improvements to the Wikimedia SUL dialog:...Some improvements to the Wikimedia SUL dialog:...
8619Improvements to Wikimedia SUL login dialog UI:...I guess the same restrictions as in T543 apply...1412415111PHID-USER-lluzkul4z7us4sxkayssFalsePHID-TASK-j6czqxlv5fzcx3tmq23ntask_subcommentNaNFalse378860378859.02014-10-04 09:31:51+00:00TrueFalseI guess the same restrictions as in T543 apply...I guess the same restrictions as in T543 apply...
8620Improvements to Wikimedia SUL login dialog UI:...It's not entirely trivial to change1412366627PHID-USER-fn7qnpccfbitivgtw2rtFalsePHID-TASK-j6czqxlv5fzcx3tmq23ntask_subcommentNaNFalse378861378860.02014-10-03 20:03:47+00:00TrueFalseIt's not entirely trivial to changeIt's not entirely trivial to change
\n", "

13628 rows × 16 columns

\n", "
" ], "text/plain": [ " task_title \\\n", "0 User with unattached accounts unable to login ... \n", "1 User with unattached accounts unable to login ... \n", "2 User with unattached accounts unable to login ... \n", "3 User with unattached accounts unable to login ... \n", "4 User with unattached accounts unable to login ... \n", "... ... \n", "8616 OAuth login refers to mediawiki.org:/ instead ... \n", "8617 OAuth login refers to mediawiki.org:/ instead ... \n", "8618 Improvements to Wikimedia SUL login dialog UI:... \n", "8619 Improvements to Wikimedia SUL login dialog UI:... \n", "8620 Improvements to Wikimedia SUL login dialog UI:... \n", "\n", " comment_text date_created \\\n", "0 User:NickK reported in IRC that they're gettin... 1411541280 \n", "1 Revert has been deployed. 1411573104 \n", "2 **gerritadmin** wrote:\\n\\nChange 162550 merged... 1411572378 \n", "3 (In reply to Kunal Mehta (Legoktm) from commen... 1411545535 \n", "4 **gerritadmin** wrote:\\n\\nChange 162549 merged... 1411542640 \n", "... ... ... \n", "8616 > When I registered, phabricator linked mediaw... 1413205650 \n", "8617 See {T574} for a related discussion. 1412958953 \n", "8618 Some improvements to the Wikimedia SUL dialog:... 1412362816 \n", "8619 I guess the same restrictions as in T543 apply... 1412415111 \n", "8620 It's not entirely trivial to change 1412366627 \n", "\n", " speaker meta.affil \\\n", "0 PHID-USER-v7vgzvvcw7v2umf737ri False \n", "1 PHID-USER-v7vgzvvcw7v2umf737ri False \n", "2 PHID-USER-ynivjflmc2dcl6w5ut5v False \n", "3 PHID-USER-v7bwpq3rs3zdxegibdbh False \n", "4 PHID-USER-ynivjflmc2dcl6w5ut5v False \n", "... ... ... \n", "8616 PHID-USER-hgn5uw2jafgjgfvxibhh False \n", "8617 PHID-USER-lluzkul4z7us4sxkayss False \n", "8618 PHID-USER-lluzkul4z7us4sxkayss False \n", "8619 PHID-USER-lluzkul4z7us4sxkayss False \n", "8620 PHID-USER-fn7qnpccfbitivgtw2rt False \n", "\n", " conversation_id comment_type status meta.gerrit \\\n", "0 PHID-TASK-mio2uq45ny7mms72syut task_description resolved False \n", "1 PHID-TASK-mio2uq45ny7mms72syut task_subcomment NaN False \n", "2 PHID-TASK-mio2uq45ny7mms72syut task_subcomment NaN False \n", "3 PHID-TASK-mio2uq45ny7mms72syut task_subcomment NaN False \n", "4 PHID-TASK-mio2uq45ny7mms72syut task_subcomment NaN False \n", "... ... ... ... ... \n", "8616 PHID-TASK-yeaxsfxhhtbn26koo5fi task_subcomment NaN False \n", "8617 PHID-TASK-yeaxsfxhhtbn26koo5fi task_subcomment NaN False \n", "8618 PHID-TASK-j6czqxlv5fzcx3tmq23n task_description declined False \n", "8619 PHID-TASK-j6czqxlv5fzcx3tmq23n task_subcomment NaN False \n", "8620 PHID-TASK-j6czqxlv5fzcx3tmq23n task_subcomment NaN False \n", "\n", " id reply_to timestamp is_relevant is_migrated \\\n", "0 243215 NaN 2014-09-24 06:48:00+00:00 True False \n", "1 243216 243215.0 2014-09-24 15:38:24+00:00 True False \n", "2 243217 243216.0 2014-09-24 15:26:18+00:00 True False \n", "3 243218 243217.0 2014-09-24 07:58:55+00:00 True False \n", "4 243219 243218.0 2014-09-24 07:10:40+00:00 True False \n", "... ... ... ... ... ... \n", "8616 378799 378798.0 2014-10-13 13:07:30+00:00 True False \n", "8617 378800 378799.0 2014-10-10 16:35:53+00:00 True False \n", "8618 378858 NaN 2014-10-03 19:00:16+00:00 True False \n", "8619 378860 378859.0 2014-10-04 09:31:51+00:00 True False \n", "8620 378861 378860.0 2014-10-03 20:03:47+00:00 True False \n", "\n", " text \\\n", "0 User:NickK reported in IRC that they're gettin... \n", "1 Revert has been deployed. \n", "2 **gerritadmin** wrote:\\n\\nChange 162550 merged... \n", "3 (In reply to Kunal Mehta (Legoktm) from commen... \n", "4 **gerritadmin** wrote:\\n\\nChange 162549 merged... \n", "... ... \n", "8616 > When I registered, phabricator linked mediaw... \n", "8617 See {T574} for a related discussion. \n", "8618 Some improvements to the Wikimedia SUL dialog:... \n", "8619 I guess the same restrictions as in T543 apply... \n", "8620 It's not entirely trivial to change \n", "\n", " resolved_text \n", "0 User:NickK reported in IRC that they're gettin... \n", "1 Revert has been deployed. \n", "2 **gerritadmin** wrote:\\n\\nChange 162550 merged... \n", "3 (In reply to Kunal Mehta (Legoktm) from commen... \n", "4 **gerritadmin** wrote:\\n\\nChange 162549 merged... \n", "... ... \n", "8616 > When I registered, phabricator linked mediaw... \n", "8617 See {T574} for a related discussion. \n", "8618 Some improvements to the Wikimedia SUL dialog:... \n", "8619 I guess the same restrictions as in T543 apply... \n", "8620 It's not entirely trivial to change \n", "\n", "[13628 rows x 16 columns]" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "unique_df" ] }, { "cell_type": "code", "execution_count": 7, "id": "0c392d70-6236-4dfe-b6d4-bbe3f422b09e", "metadata": {}, "outputs": [], "source": [ "unique_df.to_csv(\"/mmfs1/gscratch/comdata/users/mjilg/mw-repo-lifecycles/case3/0050825_coref-rel-first.csv\", index=False)" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.11" } }, "nbformat": 4, "nbformat_minor": 5 }