Psychology Commons^™

The Relationship Between Rater Agreement, Behavioral Observability And Overall Impressions, Jennifer N. Scott

Masters Theses & Specialist Projects

This study examined two item characteristics believed to influence rater agreement: observability and difficulty. The first goal of this study was to replicate the findings of Roch, Paquin and Littlejohn (2009), which found that rater agreement was negatively related to item observability (Hypothesis 1) and rating difficulty (Hypothesis 2). The study also explored whether participants had closer item performance ratings to their overall impression when items were less observable (Hypothesis 3) and more difficult to rate (Hypothesis 4). A sample of 254 Undergraduate psychology students viewed a video of a leaderless group discussion and then filled out a rating form …