Research Mentorship
Image credit:Works mentored:
Beyond Discourse Units: Exploring the Role of Context in Relation Recognition.
Sujeeth Vankudari, Vasudha Varadarajan and H. Andrew Schwartz. In Under Review at Association for Computational Linguistics Rolling Review, 2024.
Discourse frameworks have traditionally centered on _minimal spans of “discourse units” or arguments, as defined by annotation schemas in frameworks like PDTB or RST. While discourse relations have been understood to not be viewed in full isolation, this approach may still be limiting, as annotators typically have access to the entire context when labeling spans and relations. In this study, we empirically evaluate the inclusion of contextual information in discourse modeling. Further, we also evaluate the effect of including explicit modeling of interactions between the spans. Our findings reveal that context-inclusive models outperform non-contextual baselines in case of explicit relations, with the inclusion of context proving more beneficial than explicit inter-argument modeling, but not beneficial in the case of implicit relations. We observe average improvements of 10.04% for PDTB3-L1, and 16.25% for L2. This work suggests that discourse units are not as minimal as previously assumed and contributes to a more nuanced understanding of discourse structure, opening new avenues for improving NLP for discourse comprehension.
A Systematic Evaluation of Transformer-LM Representations for Capturing Author States and Traits
Khushboo Singh, Vasudha Varadarajan, Adithya V. Ganesan, August Håkan Nilsson, Syeda Mahwish,Pranav Chitale, Ryan L. Boyd, Lyle Ungar, Richard N. Rosenthal, H. Andrew Schwartz In Under Review at Association for Computational Linguistics Rolling Review, 2024.
Large Language Models (LLMs) are increasingly used for human-centered tasks, yet their ability to model diverse psychological constructs is not well understood. This study systematically evaluates the capabilities of diverse Transformer-based LLMs in modeling human psychological constructs across varying levels of temporal stability. Using a unique dataset of Ecological Momentary Assessments (EMAs) at varying levels of aggregation from none (EMA-level) to waves (quarterly), and users (averaged over ∼ 2 years), we explore how autoencoder, encoder-decoder, and autoregressive models capture traits and states. The findings reveal that the performance of LLMs is influenced by the level of analysis, with models excelling at specific combinations of outcome stability and construct characteristics. Aggregation strategies play a critical role in enhancing the reliability of predictions for rapidly changing states, moderately stable dispositions, and enduring traits. These results suggest actionable insights into the design of LLM-based approaches for psychological assessments, emphasizing the importance of selecting appropriate model architectures and temporal aggregation techniques.
