mgaughan/adaptation-slr
1
0
Fork 0
Code

~final~ update to categorization script

Browse Source
This commit is contained in:
mgaughan 2025-05-30 16:39:24 -05:00
parent 86e2cd3ed8
commit c3bb0801a2
1 changed files with 8 additions and 7 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

15
models/p1-categorization.py
View File

@ -6,21 +6,23 @@ import pandas as pd
#load in the different models
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
print(device)
print(torch.cuda.get_device_name(0))
print(torch.cuda.get_device_properties(0))
olmo = AutoModelForCausalLM.from_pretrained("allenai/OLMo-2-1124-7B").to(device)
tokenizer = AutoTokenizer.from_pretrained("allenai/OLMo-2-1124-7B")
#priming prompt
prompt_1 = "For the GIVEN DATA, Please categorize it based on the following numbered characteristics: \n\n 1: YES/NO (Characteristic 1. This is an English language empirical study. English language empirical studies are academic papers written in English that study or analyze evidence. Literature reviews are not empirical studies.) \n 2: YES/NO (Characteristic 2. This focuses on free and open source software (FOSS). The focus of this paper is on FOSS projects and ecosystems.) \n 3: YES/NO (Characteristic 3. This focuses on FOSS project evolution. FOSS project evolution is the study of longitudinal changes to the characteristics of free and open source projects.) \n 4: YES/NO (Characteristic 4. This focuses on FOSS project adaptation. FOSS project adaptation describes the intentional changes made to the characteristics of FOSS projects to better align with the project's broader environment.) \n\n Only respond with the appropriate number followed by 'YES' if the characteristic is present in the provided data or 'NO' if it is not (e.g. '1. NO; 2. YES;'). Do not provide any additional information."
example_1 = "Example 1: TITLE -Understanding the OSS Communities of Deep Learning Frameworks: A Comparative Case Study of PYTORCH and TENSORFLOW \n ABSTRACT - Over the past two decades, deep learning has received tremendous success in developing software systems across various domains. Deep learning frameworks have been proposed to facilitate the development of such software systems, among which, PYTORCH and TENSORFLOW stand out as notable examples. Considerable attention focuses on exploring software engineering practices and addressing diverse technical aspects in developing and deploying deep learning frameworks and software systems. Despite these efforts, little is known about the open source software communities involved in the development of deep learning frameworks. In this article, we perform a comparative investigation into the open source software communities of the two representative deep learning frameworks, PYTORCH and TENSORFLOW. To facilitate the investigation, we compile a dataset of 2,792 and 3,288 code commit authors, along with 9,826 and 19,750 participants engaged in issue events on GITHUB, from the two communities, respectively. With the dataset, we first characterize the structures of the two communities by employing four operationalizations to classify contributors into various roles and inspect the contributions made by common contributors across the two communities. We then conduct a longitudinal analysis to characterize the evolution of the two communities across various releases, in terms of the numbers of contributors with various roles and role transitions among contributors. Finally, we explore the causal effects between community characteristics and the popularity of the two frameworks. We find that the TENSORFLOW community harbors a larger base of contributors, encompassing a higher proportion of core developers and a more extensive cohort of active users compared to the PYTORCH community. In terms of the technical background of the developers, 64.4% and 56.1% developers in the PYTORCH and TENSORFLOW communities are employed by the leading companies of the corresponding open source software projects, Meta and Google, respectively; 25.9% and 21.9% core developers in the PYTORCH and TENSORFLOW communities possess Ph.D. degrees, while 77.2% and 77.7% contribute to other machine learning or deep learning open source projects, respectively. Developers contributing to both communities demonstrate spatial and temporal similarities to some extent in their pull requests across the respective projects. The evolution of contributors with various roles exhibits a consistent upward trend over time in the PYTORCH community. Conversely, a noticeable turning point in the growth of contributors characterizes the evolution of the TENSORFLOW community. Both communities show a statistically significant decreasing trend in the inflow rates of core developers. Furthermore, we observe statistically significant causal effects between the expansion of communities and retention of core developers and the popularity of deep learning frameworks. Based on our findings, we discuss implications, provide recommendations for sustaining open source software communities of deep learning frameworks, and outline directions for future research.\n CATEGORIES: 1. YES; 2. YES; 3.YES; 4. NO"
example_1 = "Example 1: TITLE - Analysis of Open Source Software Evolution Using Evolution Curve Method \n ABSTRACT - Design and evolution of modem information systems is influenced by many factors: technical, organizational, social, and psychological. This is especially true for open source software systems (OSSS), when many developers from different backgrounds interact, share their ideas and contribute towards the development and improvement of a software product. The evolution of all OSSS is a continuous process of source code development, adaptation, improvement and maintenance. Studying changes to the various characteristics of source code can help us understand the evolution of a software system. In this paper, the software evolution process is analyzed using a proposed Evolution curve (E-curve) method, which is based on information theoretic metrics of source code. The method allows identifying major evolution stages and transition points of an analyzed software system. The application of the E-curves is demonstrated for the eMule system. .\n CATEGORIES: 1. YES; 2. YES; 3.YES; 4. NO"
example_4 = "Example 4: TITLE - “Needle” hidden in silk floss: Inactivation effect and mechanism of melamine sponge loaded bismuth oxide composite copper-metal organic framework (MS/Bi2O3@Cu-MOF) as floating photocatalyst on Microcystis aeruginosa \n ABSTRACT - Photocatalytic technology showed significant potential for addressing the issue of cyanobacterial blooms resulting from eutrophication in bodies of water. However, the traditional powder materials were easy to agglomerate and settle, which led to the decrease of photocatalytic activity. The emergence of floating photocatalyst was important for the practical application of controlling harmful algal blooms. This study was based on the efficient powder photocatalyst bismuth oxide composite copper-metal organic framework (Bi2O3 @Cu-MOF), which was successfully loaded onto melamine sponge (MS) by sodium alginate immobilization to prepare a floating photocatalyst MS/Bi2O3 @Cu-MOF for the inactivation of Microcystis aeruginosa (M. aeruginosa) under visible light. When the capacity was 0.4 g (CA0.4), MS/Bi2O3 @Cu-MOF showed good photocatalytic activity, and the inactivation rate of M. aeruginosa reached 74.462% after 120 h. MS/Bi2O3 @Cu-MOF-CA0.4 showed a large specific surface area of 30.490 m2/g and an average pore size of 22.862 nm, belonging to mesoporous materials. After 120 h of treatment, the content of soluble protein in the MS/Bi2O3 @Cu-MOF-CA0.4 treatment group decreased to 0.365 mg/L, the content of chlorophyll a (chla) was 0.023 mg/L, the content of malondialdehyde (MDA) increased to 3.168 nmol/mgprot, and the contents of various antioxidant enzymes experienced drastic changes, first increasing and then decreasing. The photocatalytic process generated center dot OH and center dot O2-, which played key role in inactivating the algae cells. Additionally, the release of Cu2+ and adsorption of the material also contributed to the process. \n CATEGORIES: 1. YES; 2. NO; 3. NO; 4. NO;"
example_4 = "Example 4: TITLE - Thermal Insulation Properties of Milkweed Floss Nonwovens: Influence of Temperature, Relative Humidity, and Fiber Content \n ABSTRACT - This study investigated the influence of fiber content, temperature, and relative humidity on the thermal insulation properties of nonwoven mats made of seed fibers from Asclepias Syriaca, commonly known as milkweed floss. Nonwoven mats with a 1-inch thickness were produced by uniformly arranging milkweed fibers within a mold. Various quantities of fiber were employed to obtain nonwoven mats with a fiber content ranging from 5 to 35 kg/m3. Thermal conductivity and thermal diffusivity were measured across diverse relative humidity levels and temperatures. Simultaneously, milkweed floss samples were exposed to identical environmental conditions to assess the moisture regain and specific heat capacities of the fiber. The specific heat capacity of milkweed and thermal conductivity of the nonwovens exhibited a linear increase with temperature. The thermal diffusivity and thermal conductivity of the nonwovens decreased with rising fiber content. The thermal insulation properties of the nonwovens remained partially stable below 30\\% relative humidity but substantially deteriorated at higher levels. The nonwovens exhibited optimal thermal insulation properties at a fiber content between 20 and 25 kg/m3. The results of this study highlighted several technical advantages of employing milkweed floss as a sustainable and lightweight solution for thermal insulation. \n CATEGORIES: 1. YES; 2. NO; 3. NO; 4. NO;"
example_3 = "Example 3: TITLE - Open Source Software Community Inclusion Initiatives to Support Women Participation \n ABSTRACT - This paper focuses on the inclusion initiatives of Open Source Software (OSS) Communities to support women who participate in their online communities. In recent years, media and research has highlighted the negative experiences of women in OSS and we believe that could be detrimental to the women of OSS. Therefore, in this research, we built upon the research that demonstrates the value of Codes of Conduct for minorities in an online community. Additionally, we focus on women only spaces in OSS, because past research on women and IT shows that women perform better when they can build connections and mentoring networks with other women. We investigated 355 OSS websites for presence of women only spaces and searched for, collected and analyzed the Codes of Conduct on the websites of these OSS. Qualitative content analysis of the websites show that only 12 out of 355 websites have women only sections. Less than ten percent (28) of the analyzed websites had a code of conduct.\n CATEGORIES: 1. YES; 2. YES; 3. NO; 4. NO"
example_3 = "Example 3: TITLE - Social network structures in open source software development teams \n ABSTRACT - Drawing on social network theories and previous studies, this research examines the dynamics of social network structures in open source software (OSS) teams. Three projects were selected from SourceForge.net in terms of their similarities as well as their differences. Monthly data were extracted from the bug tracking systems in order to achieve a longitudinal view of the interaction pattern of each project. Social network analysis was used to generate the indices of social structure. The finding suggests that the interaction pattern of OSS projects evolves from a single hub at the beginning to a corel periphery model as the projects move forward.\n CATEGORIES: 1. YES; 2. YES; 3. NO; 4. NO"
example_2 = "Example 2: TITLE - An Exploratory Mixed-methods Study on General Data Protection Regulation (GDPR) Compliance in Open-Source Software \n ABSTRACT- Background: Governments worldwide are considering data privacy regulations. These laws, such as the European Union’s General Data Protection Regulation (GDPR), require software developers to meet privacy-related requirements when interacting with users’ data. Prior research describes the impact of such laws on software development, but only for commercial software. Although open-source software is commonly integrated into regulated software, and thus must be engineered or adapted for compliance, we do not know how such laws impact open-source software development. Aims: To understand how data privacy laws affect open-source software (OSS) development, we focus on the European Union’s GDPR, as it is the most prominent such law. We investigated how GDPR compliance activities influence OSS developer activity (RQ1), how OSS developers perceive fulfilling GDPR requirements (RQ2), the most challenging GDPR requirements to implement (RQ3), and how OSS developers assess GDPR compliance (RQ4). Method: We distributed an online survey to explore perceptions of GDPR implementations from open-source developers (N=56). To augment this analysis, we further conducted a repository mining study to analyze development metrics on pull requests (N=31,462) submitted to open-source GitHub repositories. Results: Our results suggest GDPR policies complicate OSS development and introduce challenges, primarily regarding the management of users’ data, implementation costs and time, and assessments of compliance. Moreover, we observed negative perceptions of the GDPR from OSS developers and significant increases in development activity, in particular metrics related to coding and reviewing, on GitHub pull requests related to GDPR compliance. Conclusions: Our findings provide future research directions and implications for improving data privacy policies, motivating the need for relevant resources and automated tools to support data privacy regulation implementation and compliance efforts in OSS. \n CATEGORIES: 1. YES; 2. YES; 3. YES; 4. YES;"
with open("cites/auto-dedup-cites.csv", mode='r', newline='') as file:
with open("cites/053025_man_filtered_dedup.csv", mode='r', newline='') as file:
reader = csv.reader(file)
array_of_categorizations = []
index = -1
@ -31,8 +33,8 @@ with open("cites/auto-dedup-cites.csv", mode='r', newline='') as file:
cite_dict = {}
#organizing the data from each citation
cite_dict['key'] = row[0]
cite_dict['title'] = row[4]
cite_dict['abstract'] = row[10]
cite_dict['title'] = row[1]
cite_dict['abstract'] = row[2]
#prompt construction
given_data = f"GIVEN DATA: Title - {cite_dict['title']} \n Abstract - {cite_dict['abstract']}"
prompt = f"{prompt_1}\n\n{example_1}\n\n{example_2}\n\n{example_3}\n\n{example_4}\n\n{given_data}\n"
@ -58,7 +60,6 @@ with open("cites/auto-dedup-cites.csv", mode='r', newline='') as file:
value = key_value[1]
cite_dict[key] = value
array_of_categorizations.append(cite_dict)
#CSV everything
df = pd.DataFrame(array_of_categorizations)
df.to_csv('053025_olmo_categorized_citations.csv', index=False)