← Research

Where Do LLMs Retrieve Network Information? Identifying the Responsible Heads and Neurons

Md Abdul Kadir, Md Mohasin Hossain, Omair Shahzad Bhatti, Abdulrahman Mohamed Selim, Daniel Sonntag

Large language models (LLMs) used for Web information processing rely on retrieving relationships between network entities, such as domain names and IP addresses, but the internal components that causally enable this retrieval remain unclear, hindering targeted safeguards against unsafe actions. Therefore, we investigate whether their retrieval capabilities can be localized to specific attention heads or individual neurons within these LLMs. Across five LLMs from three architecture families, we identify heads through causal ablation screening, test their selectivity against matched negative and context-free controls, rank neurons within these heads using causal ablation and correlational measures, and leverage features from both causally validated heads and individual neurons to build network information retrieval detectors, which we evaluate without retraining on an independent reverse-DNS dataset. In every model, a small set of causally validated heads (1--9 out of 128--1152 candidates) builds a detector with 99.5--100\% held-out accuracy. At the neuron level, the results are model-specific, e.g., in one model, causal and correlational rankings identify the same neuron, matching full-head performance (AUC = 1.000), while in another, the best causally ranked neuron achieves only AUC = 0.665 despite the full head reaching 1.000. On independent CAIDA data, head detectors maintain AUC $\geq 0.971$, while single-neuron detectors fall below chance in three of five models. Overall, these findings highlight the importance of choosing an appropriate level of granularity when causally localizing how LLMs retrieve structured network information from context.

PDF ↗

Models

Loading
EVIDENCE / COMPARISON

Benchmarks

Explore every benchmark

Loading

These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

Install TensorTrace