Human-Computer Interaction Commons^™

Designing Classifact: Towards A Transparent And Secure Platform For Stakeholder-In-The-Loop Data Annotation, Philippine Waisvisz

Communications of the IIMA

AI systems depend on human judgment, yet many annotation workflows are either designed for data-science specialists or managed through external commercial platforms. The first approach may demand more technical skills than relevant stakeholders possess. The second can require organizations to transfer data, expertise, and governance to an outside provider. Both can limit the involvement of people who understand what data means in its real-world context. Human-in-the-loop approaches introduce human judgment. Stakeholder-in-the-loop annotation focuses on selecting and organizing people whose contextual knowledge fits the AI application.

This paper presents Classifact, a transparent and secure platform for stakeholder-in-the-loop data annotation and validation. …