Digital Commons Network^™

Improving Document Classification By Integrating Human-Crafted Semantic Knowledge, Lubomir Stanchev, Zachary Weinfeld

Computer Science and Software Engineering

Document classification is a pivotal task in various domains, warranting the development of robust algorithms. Among these, the Bidirectional Encoder Representations from Transformers (BERT) algorithm, introduced by Google, has proven to perform well when fine-tuned for the task at hand. Leveraging transformer architecture, BERT demonstrates stellar language understanding capabilities. However, the integration of BERT with a range of approaches has shown potential for further enhancing classification accuracy. This paper investigates three techniques that leverage semantic understanding to improve the performance of document classification models trained with BERT. First, we balance corpora afflicted by imbalanced training data distributions. Next, we substitute …

Bert-Based Detection Of Ai-Generated Text For Content Verification, Soham Biren Katlariwala

2024 REYES Proceedings

With advancements in AI-driven natural language generation, distinguishing between AI-generated and human-written text has become imperative for ensuring content authenticity across industries. This study explores the effectiveness of Bidirectional Encoder Representations from Transformers (BERT) in addressing this classification challenge. Utilizing a diverse dataset and robust preprocessing techniques, BERT achieved a peak F1-score of 0.94364, outperforming traditional models such as Logistic Regression and Support Vector Machines. The results underscore the potential of transformer-based models in addressing real-world con- tent verification problems. Future enhancements include fine-tuning and expanding datasets for greater generalizability.

Evaluating The Deductive Competence Of Large Language Models, Spencer M. Seals, Valerie L. Shalin

Psychology Faculty Publications

The development of highly fluent large language models (LLMs) has prompted increased interest in assessing their reasoning and problem-solving capabilities. We investigate whether several LLMs can solve a classic type of deductive reasoning problem from the cognitive science literature. The tested LLMs have limited abilities to solve these problems in their conventional form. We performed follow up experiments to investigate if changes to the presentation format and content improve model performance. We do find performance differences between conditions; however, they do not improve overall performance. Moreover, we find that performance interacts with presentation format and content in unexpected ways that …

การจัดลำดับความสำคัญของบทวิจารณ์ของผู้ใช้ซอฟต์แวร์โดยคำนึงถึงประสบการณ์ผู้ใช้โดยใช้การเรียนรู้ของเครื่อง, ลักษณ์สิปาง สาครวิจิตรี

Chulalongkorn University Theses and Dissertations (Chula ETD)

ปัจจุบันตลาดโมไบล์แอปพลิเคชันมีการแข่งขันสูง นักพัฒนาจึงจำเป็นต้องให้ความสำคัญกับบทวิจารณ์ของผู้ใช้ซึ่งบ่งบอกถึงความคิดเห็นและประสบการณ์ของผู้ใช้งานจริง เพื่อนำไปสู่การปรับปรุงและพัฒนาฟังก์ชันการทำงานให้ตรงตามความต้องการของผู้ใช้มากยิ่งขึ้น อย่างไรก็ตาม แอปพลิเคชันยอดนิยมมักมีบทวิจารณ์จำนวนมาก ทำให้เกิดข้อจำกัดในการที่นักพัฒนาจะสามารถอ่านและวิเคราะห์บทวิจารณ์ทั้งหมดได้อย่างมีประสิทธิภาพ เพื่อจัดการกับปัญหาดังกล่าว งานวิจัยฉบับนี้จึงนำเทคโนโลยีการเรียนรู้ของเครื่องเข้ามาช่วยในการจำแนกประเภทของบทวิจารณ์ โดยพิจารณาจากเนื้อหาในบทวิจารณ์ของผู้ใช้ซึ่งอาจสะท้อนถึงปัญหา ข้อบกพร่อง หรือข้อเสนอในการพัฒนาฟังก์ชันใหม่ ข้อมูลที่ได้จะถูกนำไปใช้จัดลำดับความสำคัญของการแก้ไขปรับปรุงตามผลกระทบที่มีต่อประสบการณ์ของผู้ใช้ งานวิจัยนี้ได้กำหนดหมวดหมู่ของบทวิจารณ์ไว้ทั้งหมด 5 ประเภท ได้แก่ ปัญหาด้านความจำเป็นพื้นฐาน, ปัญหาด้านการปฏิบัติ, ปัญหาด้านความเพลิดเพลิน, ปัญหาด้านความแปลกใหม่ และปัญหาอื่น ๆ ซึ่งสะท้อนลำดับความสำคัญของปัญหาจากมุมมองของผู้ใช้ ในการพัฒนาโมเดลการเรียนรู้ของเครื่องพบว่าโมเดลเบิร์ตมีประสิทธิภาพสูงกว่าโมเดลเอสวีเอ็ม แรนดอมฟอเรสต์ และโลจิสติกรีเกรสชัน โดยมีค่าความเที่ยงเป็น 0.72 ค่าเรียกกลับเป็น 0.719 ค่าเอฟวันเป็น 0.719 และค่าความแม่นเป็น 0.722 นอกจากนี้ งานวิจัยยังได้พัฒนาเว็บแอปพลิเคชันต้นแบบที่ใช้โมเดลเบิร์ตที่สร้างขึ้น เพื่อช่วยให้นักพัฒนาสามารถนำไปใช้วิเคราะห์และจัดลำดับความสำคัญของงานการบำรุงรักษาโมไบล์แอปพลิเคชัน โดยเว็บแอปพลิเคชันจะช่วยลดภาระในการอ่านบทวิจารณ์จำนวนมาก พร้อมทั้งสกัดข้อมูลที่สำคัญออกมาให้เห็นภาพรวมของความต้องการของผู้ใช้

Using Ai For Qualitative Labeling: Consistency And Comparisons, James Mcintyre

Honors Program Theses

This paper details a research study evaluating AI's ability to perform qualitative deductive coding. Multiple AI models were utilized and compared against three human coders and one expert coder. A series of 107 statements were sourced from a group discussion for a qualitative impact assessment of an organization. The AI models were provided these statements and directed to code them using the Community Capitals Framework. Two generations of AI models were evaluated. Overall, the AI achieved a fair level of agreement with the human annotators, but the alignment was far from perfect. Newer AI models did not increase agreement with …

Using Ai For Qualitative Labeling: Consistency And Comparisons, James Temple

Honors Program Theses

This paper continues research that evaluates the capacity of artificial intelligence (AI) to perform qualitative coding tasks. The previous study found that AI models lacked consistency with themselves and did not agree with human coded data. Since that study, AI’s general level of intelligence has increased. Hence, this study re-evaluates how well the newest set of AI models (Claude 3 and Gemini) can perform qualitative coding tasks. When tested, the new AI models perform about the same or better than previous models depending on the metric tested. While Gemini and Claude 3 do not agree with human output any more …

Enhancing Cross-Modal Contextual Congruence For Crowdfunding Success Using Knowledge-Infused Learning, Trilok Padhi, Ugur Kursuncu, Yaman Kumar, Valerie L. Shalin, Lane Peterson Fronczek

Psychology Faculty Publications

The digital landscape continually evolves with multimodality, enriching the online experience for users. Creators and marketers aim to weave subtle contextual cues from various modalities into congruent content to engage users with a harmonious message. This interplay of multimodal cues is often a crucial factor in attracting users' attention. However, this richness of multimodality presents a challenge to computational modeling, as the semantic contextual cues spanning across modalities need to be unified to capture the true holistic meaning of the multimodal content. This contextual meaning is critical in attracting user engagement as it conveys the intended message of the brand …

The Specter Of Representation: Computational Images And Algorithmic Capitalism, Samine Joudat

CGU Theses & Dissertations

The processes of computation and automation that produce digitized objects have displaced the concept of an image once conceived through optical devices such as a photographic plate or a camera mirror that were invented to accommodate the human eye. Computational images exist as information within networks mediated by machines. They are increasingly less about what art history understands as representation or photography considers indexing and more an operational product of data processing.

Through genealogical, theoretical, and practice-based investigation, this dissertation project traces a lineage of computation through images from early cybernetics to contemporary machine learning under algorithmic capitalist conditions of …

How Economically Marginalized Adolescents Of Color Negotiate Critical Pedagogy In A Computing Classroom, Jean Salac, Lena Armstrong, Megumi Kivuva, Jayne Everson, Amy J. Ko

Computer Science Faculty Work

Background and Context: With the growing movement to adopt critical framings of computing, scholars have worked to reframe computing education from the narrow development of programming skills to skills in identifying and resisting oppressive structures in computing. However, we have little guidance on how these framings may manifest in classroom practice. Objectives: To better understand the processes and practice of critical pedagogy in a computing classrooms, we taught a critically conscious computing elective within a summer academic program at a northwest United States university targeted at secondary students (ages 14–18) from low-income backgrounds and would be the first …

Conceptthread: Visualizing Threaded Concepts In Mooc Videos, Zhiguang Zhou, Li Ye, Lihong Cai, Lei Wang, Yigang Wang, Yongheng Wang, Wei Chen, Yong Wang

Research Collection School Of Computing and Information Systems

Massive Open Online Courses (MOOCs) platforms are becoming increasingly popular in recent years. Online learners need to watch the whole course video on MOOC platforms to learn the underlying new knowledge, which is often tedious and time-consuming due to the lack of a quick overview of the covered knowledge and their structures. In this paper, we propose ConceptThread , a visual analytics approach to effectively show the concepts and the relations among them to facilitate effective online learning. Specifically, given that the majority of MOOC videos contain slides, we first leverage video processing and speech analysis techniques, including shot recognition, …

Understanding Social Dynamics In Toxic Conversations And Public Health Intervention Acceptance On Social Media, Ana Aleksandric

Computer Science and Engineering Dissertations - Archive

Social media is now central to daily life, offering users a space to share content and opinions. However, these platforms also facilitate the spread of hate speech and misinformation, which can negatively impact public health. This dissertation develops methodologies to analyze social media data for insights that could inform health interventions. The research first examines user responses to toxic content, focusing on behavioral and emotional reactions, as well as group dynamics and bystander effects in toxic interactions. Another key focus is public opinion toward health interventions, particularly COVID-19 vaccination, using geolocated posts and analyzing factors such as race, ethnicity, and …

Identification And Quantification Of Authorial Style Similarity, Mary E. Koone Phd

Computer Science and Engineering Dissertations - Archive

This thesis studies the topic of identifying author similarity, grouping authors together based on that similarity. To solve that problem, the thesis proposes concrete solutions to a series of subproblems. The initial sub-problems are: how to identify a pool of possible features for representing documents, and how to select and combine some of those features to map a document into a feature vector. Another sub-problem is how to evaluate the usefulness of such feature vectors in identifying language style similarity. This thesis proposes, as part of addressing that sub-problem, a novel method for evaluating the quality of document representations obtained, …

A Prototype Of A Conversational Virtual University Support Agent Powered By A Large Language Model That Addresses Inquiries About Policies In The Student Handbook, Joseph Benjamin R. Ilagan, Jose Ramon Ilagan

Quantitative Methods and Information Technology Faculty Publications

Universities gain a competitive advantage by deliberately improving overall service, student, faculty, and staff experience, leading to attractiveness, retention, and improved outcomes. Quality services are achieved partly by addressing employee satisfaction, specifically in the work environment. This paper presents a prototype study of a virtual university support agent, a system grounded in a Large Language Model (LLM) engineered to address inquiries from university students, faculty and staff related to the student handbook. The study investigates the integration of generative artificial intelligence and natural conversation properties inherent in LLMs to overcome customer service shortcomings identified in previous chatbot applications. The LLMs' …

Graph-Partitioning Entity Resolution For Resolving Noisy Product Names In Ocr Scans Of Retail Receipts, Jose Ramon Ilagan, Joseph Benjamin R. Ilagan

Quantitative Methods and Information Technology Faculty Publications

In business intelligence for retail, it is critical to ensure consistent and unambiguous product dimension information. This is challenging, especially if an organization does not have full control over the source of either transaction or master data. Such lack of control is the case when brands rely on data provided directly by consumers through images of receipts. Product name strings obtained from the digitization of receipts often contain substitution, insertion, and deletion errors. These errors prevent product names from serving as a useful dimension for further analysis. This paper proposes a clustering-based approach to link error-laden product names to underlying …

Towards Dynamic Context Detection From Voice Commands And Conversations With Smart Assistants In Smart Homes, Jeniya Sultana

Graduate Theses/Dissertations

Voice-enabled interactions have become increasingly popular with the rise of voice assistants. Identifying contexts or meanings from voice commands and conversations with smart assistants can contribute to the autonomous control of smart home devices and appliances. To improve automation, there is a growing need for efficient context detection that eliminates the need to memorize voice commands. To address this need, I followed a two-step approach in my research. In the first step, I developed a unique context recognition model using a transformer, an attention mechanism, and a fully connected neural network. I trained this model on a conversational dataset of …

Contributors, Beth Buyserie, Travis N. Thurston

Teaching and Generative AI: Pedagogical Possibilities and Productive Tensions

No abstract provided.

34. "Language Weaves Its Tapestry": Crafting Found Poetry Using Ai Tools, Ruth Li

Teaching and Generative AI: Pedagogical Possibilities and Productive Tensions

In this chapter, I offer a pedagogical approach to teaching creative writing in the era of Al technologies. I share an activity that engages students in crafting found poetry using Al tools such as ChatGPT. Found poems are collage-like assemblages drawn from other sources such as newspaper or magazine articles. By immersing students in crafting found poems using Al, I inspire creative experimentation with Al-generated texts. Even as writing processes become automated and writing styles standardized, I posit that this found poem activity nurtures students' careful attunement to the materiality of language, to its shape and texture as words unfold …

30. Revising Llm Text To (Re)Discover Rhetoric In A Graduate Seminar, Clancy Ratliff

Teaching and Generative AI: Pedagogical Possibilities and Productive Tensions

In this essay, I share an assignment that asks students to engage in rhetorical revision of LLM text using a changelog table as a tool to document the revision. To demonstrate to the students what I was asking them to do, I did the assignment myself, and my example is included in this chapter as well. I argue that the style of generated text from LLMs calls for increased resistance to the norms and conventions of school writing and an embrace of writing that's more rhetorical: a bit more personal, risky, rule-breaking, creative, and experimental, intentionally centering purpose and audience. …

24. Revisioning A Bibliography Assignment To Center Discovery And Critical Source Engagement, Lillian Campbell, Jenna Green, Nicole Bungert

Teaching and Generative AI: Pedagogical Possibilities and Productive Tensions

This chapter shares context, assignment details, and impact of revisions of a research journal assignment in our first-year writing program at a mid-sized private university in the Midwest. With the rise of access to large language processing models like ChatGPT in Spring 2023, we reviewed our curriculum to consider what assignments might be most vulnerable to Al-generated writing. Our bibliography assignment was an obvious contender, since it focused on summary of individual sources. Thus, we worked with our research librarian liaison to reimagine this assignment, guided by a conversation about learning goals. After reviewing relevant scholarship on source use and …

21. My Summer With Chatgpt, Mary Lourdes Silva

Teaching and Generative AI: Pedagogical Possibilities and Productive Tensions

Out of necessity, I first used ChatGPT to process mentally and emotionally two major injuries. Trapped at home for the entire summer, I learned to write with ChatGPT to complete a large-scale research project. The anthropomorphizing experience left me feeling less alone. For nearly a year, news about Al-generated writing software sparked nationwide concern about the future of traditional essays. As an early adopter of most digital technologies and the type of person who welcomes chaos, I chose to learn everything | could about ChatGPT, which meant I needed to learn how to "cheat" with ChatGPT. Inspired by similar assignments …

20. Fit To Resist In Post-Product Space: Underserved Student Populations And Generative Ai's Writing Norms, John Paul Tassoni

Teaching and Generative AI: Pedagogical Possibilities and Productive Tensions

This essay describes ways the specter of Al altered interactions with students in a first-year composition course and how the response speaks to the broader impact of generative Al on underserved populations-especially in the sense that large language models and writing instruction can both affirm center/dominant discourses, values, and practices. The essay argues that the emergence of Al should shift writing instruction for "at-risk" and other marginalized student populations into post-product space. In such space, course work emphasizes students' negotiations with the expectations of higher education's hidden curriculum alongside the finished products that Al can now always already provide.

8. Ai And Writing Classrooms: A Study Of Purposeful Use And Student Responses To The Technology, Laura Dumin

Teaching and Generative AI: Pedagogical Possibilities and Productive Tensions

Given that Al is here for the foreseeable future, I revamped my spring 2023 English courses to include using Al in purposeful ways. I submitted an IRB to test different approaches to integrating Al into my classrooms. The study asked students in three different English courses to complete pre- and post-reflections about their understanding of Al and its uses.

This chapter will discuss how my students responded to the use of Al in the writing classroom, how student attitudes toward Al shifted over the course of the semester, ideas for how to use Al in writing classrooms, and the main …

2. Some Ethical Considerations For Teaching And Generative Ai In Higher Education, Lydia Wilkes

Teaching and Generative AI: Pedagogical Possibilities and Productive Tensions

The chapter discusses some ethical considerations for teaching and generative Al in higher education with a focus on large language models like ChatGPT. These considerations include generative Al's effects on the environment and labor, its reliance on data and tendency to violate privacy, the biases encoded in it, and the digital divide. As more sophisticated generative Al is integrated into word processors, these ethical concerns will become more difficult to perceive and the ethics of using Al more difficult to parse. This chapter concludes with a note on power and justice.

Introduction, Beth Buyserie, Travis N. Thurston

Teaching and Generative AI: Pedagogical Possibilities and Productive Tensions

With the rapid development of generative Al, teachers are experiencing a new pedagogical challenge, one that promises to forever change the way we approach teaching and learning. As a response to this unprecedented teaching context, this collection-Teaching and Generative Al: Pedagogical Possibilities and Productive Tensions-provides interdisciplinary teachers, librarians, and instructional designers with practical and thoughtful pedagogical resources for navigating the possibilities and challenges of teaching in an Al era. Because our goal with this edited collection is to present nuanced discussions of Al technologies across disciplines, the chapters collectively acknowledge or explore both possibilities and tensions-including the strengths, …

Unveiling User Perspectives: Exploring Themes In Femtech Mobile App Reviews For Enhanced Usability And Privacy, Nidhi Nellore, Tania Mishra, Michael Zimmer

Computer Science Faculty Research and Publications

Femtech, a growing sector in mobile healthcare technology, caters to women's needs across various life stages with digital solutions like period tracking and pregnancy management apps. Maintaining robust data privacy is crucial due to the sensitive nature of the information involved, such as menstrual cycles and pregnancy status. Our research analyzes user feedback from platforms like the Apple App Store and Google Play Store to understand perceptions of femtech apps, covering accessibility, interface, features, and privacy concerns. Understanding user perspectives helps developers enhance usability and trust, driving further adoption. Prioritizing privacy also fosters industry advancement. This paper stresses the importance …

Review Of Who Wrote This?: How Ai And The Lure Of Efficiency Threaten Human Writing, By Naomi S. Baron, Taylor J. Greene

Library Articles and Research

A review of Who Wrote This?: How AI and the Lure of Efficiency Threaten Human Writing, by Naomi S. Baron.

Content Moderation On Social Media: Social And Computational Standards And Implications, Mohit Singhal

Computer Science and Engineering Dissertations - Archive

Social media has become a powerful tool that reflects human communication's best and worst aspects. They allow individuals to freely express opinions, communicate with others, and learn about new stories. On the other hand, they have become fertile grounds for several forms of abuse, harassment, and the dissemination of misinformation. Social media platforms have established and employed content moderation to counteract the spread of abuse and misinformation.

Some critical challenges hinder the understanding of the social media content moderation ecosystem. This dissertation investigates various aspects of content moderation, including their coverage, fairness, and effectiveness. Firstly, it investigates how, in practice, …

Natural Language Generation From Large-Scale Open-Domain Knowledge Graphs, Xiao Shi

Computer Science and Engineering Dissertations - Archive

This dissertation delves into the realm of natural language generation (NLG) from expansive open-domain knowledge graphs, aiming to bridge the gap between existing methods primarily tested on limited datasets and the demands of real-world large-scale, diverse graph structures. Prior works in NLG often relied on small-scale or restricted datasets, neglecting the complexities of broader knowledge graphs. To address this, we introduce a new dataset called GraphNarrative, designed to encompass a wide range of graph structures and enhance the realism of NLG tasks.

The core contribution of this research lies in devising a novel approach to mitigating information hallucination, a common …

Claim Sensing: A Study Linking Factual Claims To Human Behaviors On Social Media, Zeyu Zhang

Computer Science and Engineering Dissertations - Archive

The ubiquity of social media has transformed it into a rich source for reflecting people's opinions, behaviors, and interactions. Users frequently encounter factual claims in news, stories, and political statements, which can be either true or false. These claims significantly shape people's minds and behaviors, influencing not only individual perspectives but also broader public discourse. This study explores individuals' behaviors and perceptions toward factual claims by leveraging the concept of "check-worthiness" to analyze the relationship between such claims and user behaviors across datasets containing tens of millions of social media posts, particularly tweets from the platform X (formerly Twitter). It …

Using Generative Large Language Models For Hierarchical Relationship Prediction In Medical Ontologies, Hao Liu, Shuxin Zhou, Zhehuan Chen, Yehoshua Perl, Jiayin Wang

School of Computing Faculty Scholarship and Creative Works

This study extends the exploration of ontology enrichment by evaluating the performance of various open-sourced Large Language Models (LLMs) on the task of predicting hierarchical relationships (IS-A) in medical ontologies including SNOMED CT Clinical Finding and Procedure hierarchies and the human Disease Ontology. With the previous finetuned BERT models for hierarchical relationship prediction as the baseline, we assessed eight open-source generative LLMs for the same task. We observed only three models, without finetuning, demonstrated comparable or superior performance compared to the baseline BERT -based models. The best performance model OpenChat achieved a macro average F1 score of 0.96 (0.95) on …