NeFut Logo NeFut
中 Admin Login

[CS.AI] From Encoding to Generation: Towards Empathetic Large Language Models

Published at: 2026-10-22 22:00 Last updated: 2026-10-25 13:04
#AI #Machine Learning #Artificial Intelligence

This work investigates the two capacities of empathetic interaction—cognitive empathy (representing another’s emotional state) and affective empathy (producing an appropriate response)—and applies this framework to examine how Llama‑3.2‑3B‑Instruct internally organizes emotion‑related information across ten emotion categories from the EmpatheticDialogues dataset.\

Layer‑wise linear probing reveals a broad middle‑layer region (approximately layers 10‑12) where emotion‑category decodability peaks. The layer‑11 L2 probe achieves 63.6% held‑out accuracy, far surpassing the 10% chance baseline and the 54.6% TF‑IDF lexical baseline.\

During response generation, user tokens that convey emotion attract 1.89× more attention per token than other utterance tokens, with the effect concentrated in layers 10‑15. Target‑emotion alignment then drops from a mean cosine similarity of 0.239 for input representations to 0.125 for generated responses; a positive drift occurs in 77% of examples, with the largest shifts observed for angry and furious emotions.\

These findings indicate that emotion‑category representation, attention allocation, and response‑level alignment emerge as distinct components of model processing that can be analyzed through a functional cognitive‑affective empathy framework. The study provides a foundation for monitoring emotion representations during generation and leveraging the model’s inherent affective structure to produce more naturally empathetic replies without external prompting or explicit emotion conditioning.\

Review

Original Source: https://arxiv.org/abs/2610.27603

[h] Back to Home