Haphazardness in LLMs and Its Implications for Agentic AI Systems (Nov 7)

Speaker: Xiuwen Liu

Date: Nov 7, 2:15 – 3:05 pm

Abstract:

As large language models (LLMs) are increasingly deployed in agentic AI systems to iteratively solve complex tasks, empirical studies have revealed a persistent lack of reproducibility and stability in their behaviors.
However, the underlying mechanisms responsible for these phenomena remain poorly understood.

In this talk, I will present our recent progress in understanding, quantifying, and improving the stability of LLMs. Using the last pseudo token (LPT) representation—a deterministic function of the input sequence and the input to the language modeling head—we show that for each input sequence, there exists a region where “unpredictable” noise dominates. This leads to an empirically estimated total rate of change several orders of magnitude larger than the theoretical prediction. Moreover, we find that regions near theoretical decision boundaries exhibit chaotic behavior. To mitigate this effect, we develop a multi-sample algorithm that significantly reduces noise on the same GPU. Our experiments further reveal consistent behaviors across different floating-point precisions (FP16, FP32, FP64) and across models (Llama 3.1 and an OpenAI OSS model).

In addition, we observe that LPTs form large angles with unembedding tokens, resulting in token swapping effects when comparing the results of the same

This is joint work with Chashi Islam and other collaborators in my lab.

Location: LOV 307 and ZOOM