Engineering Commons^™

Long Future Frame Prediction Using Optical Flow Informed Deep Neural Networks For Enhancement Of Robotic Teleoperation In High Latency Environments, Md Moniruzzaman, Alexander Rassau, Douglas Chai, Syed M. S. Islam

Research outputs 2022 to 2026

High latency in teleoperation has a significant negative impact on operator performance. While deep learning has revolutionized many domains recently, it has not previously been applied to teleoperation enhancement. We propose a novel approach to predict video frames deep into the future using neural networks informed by synthetically generated optical flow information. This can be employed in teleoperated robotic systems that rely on video feeds for operator situational awareness. We have used the image-to-image translation technique as a basis for the prediction of future frames. The Pix2Pix conditional generative adversarial network (cGAN) has been selected as a base network. Optical …