Engineering Commons^™

Improving Feature Generalizability With Multitask Learning In Class Incremental Learning, Dong Ma, Chi Ian Tang, Cecilia Mascolo

Research Collection School Of Computing and Information Systems

Many deep learning applications, like keyword spotting [1], [2], require the incorporation of new concepts (classes) over time, referred to as Class Incremental Learning (CIL). The major challenge in CIL is catastrophic forgetting, i.e., preserving as much of the old knowledge as possible while learning new tasks. Various techniques, such as regularization, knowledge distillation, and the use of exemplars, have been proposed to resolve this issue. However, prior works primarily focus on the incremental learning step, while ignoring the optimization during the base model training. We hypothesise that a more transferable and generalizable feature representation from the base model would …