Digital Commons Network^™

Automatic Speechreading With Applications To Human-Computer Interfaces, Xiaozheng Zhang, Charles C. Broun, Russell M. Mersereau, Mark A. Clements

Electrical Engineering

There has been growing interest in introducing speech as a new modality into the human-computer interface (HCI). Motivated by the multimodal nature of speech, the visual component is considered to yield information that is not always present in the acoustic signal and enables improved system performance over acoustic-only methods, especially in noisy environments. In this paper, we investigate the usefulness of visual speech information in HCI related applications. We first introduce a new algorithm for automatically locating the mouth region by using color and motion information and segmenting the lip region by making use of both color and edge information …