Digital Commons Network^™

The Behaviors Of Bert Attention Heads In Stereotype Detection, Joseph H. Hajjar

Dartmouth College Master’s Theses

We are living in the age of information, where it has become increasingly easy to share ideas, news, and content which are seen by an increasingly large number of people. This increasing scope of the increasing amount of data that is being shared lends itself to the question: how can we determine whether what we are reading promotes a stereotype? Previous work has applied transformer based models in this domain yielding impressive performance, but few studies exist interpreting the nature of attention heads in this task. Our work explores the feature encoding and extraction behaviors of attention heads in transformer …