Human-Computer Interaction Commons^™

Designing Classifact: Towards A Transparent And Secure Platform For Stakeholder-In-The-Loop Data Annotation, Philippine Waisvisz

Communications of the IIMA

AI systems depend on human judgment, yet many annotation workflows are either designed for data-science specialists or managed through external commercial platforms. The first approach may demand more technical skills than relevant stakeholders possess. The second can require organizations to transfer data, expertise, and governance to an outside provider. Both can limit the involvement of people who understand what data means in its real-world context. Human-in-the-loop approaches introduce human judgment. Stakeholder-in-the-loop annotation focuses on selecting and organizing people whose contextual knowledge fits the AI application.

This paper presents Classifact, a transparent and secure platform for stakeholder-in-the-loop data annotation and validation. …

Ai-Augmented Financial Advisors: Comparing Ai And Human Analyst Investment Recommendations In Agreement, Performance, And Firm Size Effects, Crystal Chen

Honors College Theses

This study examines the level of agreement and performance between artificial intelligence (AI) generated investment recommendations and human analyst recommendations across U.S. publicly traded firms. Using a sample of twelve companies categorized by firm size (large, mid, and small), the study collects buy, hold, or sell recommendations from generative AI systems and human analysts. Agreement between AI-to-AI and AI-to-human recommendations is measured using Cohen’s Kappa agreement. Portfolio performance is evaluated by constructing equal-weighted portfolios for each recommendation source and size category. Risk-adjusted returns are measured using the Sharpe ratio over 1-, 2-, and 3-month periods. Furthermore, the study tests whether …