Computer Engineering Commons^™

Exploring Alternative Approaches To Language Modeling For Learning From Data And Knowledge, Yuxin Zi, Kaushik Roy, Vignesh Narayanan, Amit Sheth

Publications

Despite their wide applications to language understanding tasks, large language models (LLMs) still face challenges such as hallucinations - the occasional fabrication of information, and alignment issues - the lack of associations with human-curated world models (e.g., intuitive physics or common-sense knowledge). Additionally, the black-box nature of LLMs makes it highly challenging to train them meaningfully in order to achieve a desired behavior. Specifically, the attempt to adjust LLMs’ concept embedding spaces can be highly intractable, which involves analyzing the implicit impact on LLMs’ numerous parameters and the resulting inductive biases. This paper proposes a novel architecture that wraps powerful …