Physical Sciences and Mathematics Commons^™

Mapping Relationships And Positions Of Objects In Images Using Mask And Bounding Box Data, Jaime M. Villanueva Jr, Anantharam Subramanian, Vishal Ahir, Andrew Pollock

SMU Data Science Review

In this paper we present novel methods for automatically annotating images with relationship and position tags that are derived using mask and bounding box data. A Mask Region-based Convolutional Neural Network (Mask R-CNN) is used as the foundation for the ob- ject detection process. The relationships are found by manipulating the bounding box and mask segmentation outputs of a Mask R-CNN. The absolute positions, the positions of the objects relative to the image, and the relative positions, the positions of objects relative to the other objects, are then associated with the images as annotations that are out- put in order …