2026 · The American Journal of Bioethics
Federation Opacity Makes Medical Machine Learning’s Failures Harder to Detect and Correct ↗Open accessFree PDF ↗Artificial Intelligence in Healthcare and Education Abstract
Our target article argued that federated learning (FL) generates a distinctive variety of opacity, federation opacity, in which no party can access, analyze, or curate the model’s full training data...
2026 · arXiv (Cornell University) · preprint
Structured Sentiment Analysis Using Sequence Labeling as Dependency Graph Parsing ↗Free copyFree PDF ↗Sentiment Analysis and Opinion Mining Abstract
This study addresses the problem of structured sentiment analysis, whose goal is to obtain a fine-grained sentiment graph where the nodes represent spans of sentiment holders, targets, and expressions, while the arcs define the relationships among them. Our proposed approach casts the task as dependency graph parsing, but departs from traditional parsing methods by solving it through sequence labeling. To do so, we leverage recent advances in linearized graph encodings that allow each word in the input to be assigned a label, effectively capturing the structure of the dependency graph. We conducted experiments on seven datasets spanning five languages (English, Spanish, Norwegian, Basque, and Catalan), showing performance competitive with leading, more complex single-model approaches.
2026 · arXiv (Cornell University) · preprint
Attention-Based Adaptive Policies for Simultaneous Speech-to-Text Translation ↗Free copyFree PDF ↗Speech Recognition and Synthesis Abstract
Simultaneous speech-to-text translation (Simul-S2TT) consists of generating partial translations while the incoming audio frames are processed by the system. However, the streaming nature of this setup creates the challenge of deciding the best moment to perform an accurate translation while minimizing the delay. To address this challenge, we utilize the cross-attention mechanism of the encoder-decoder architecture to find the right alignment between the input speech frames and the target text tokens. In this paper, we propose the Recent Frame Attention Policy (RFAP) and the Dual-Condition Attention Policy (DCAP) that allow offline trained speech-to-text translation models to be used in streaming scenarios without requiring additional training. Results on three different language translation pairs over the CVSS-C corpus show that the RFAP is able to surpass other policies with gains of up to 4.0 BLEU while reducing the translation delay by almost 1 second. Moreover, the DCAP is able to preserve a high translation quality when the latency is very low.
2026 · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · conference-paper
Brainrot: Deskilling and Addiction are Overlooked AI Risks ↗AI ActOpen accessEthics and Social Impacts of AI
Abstract
The scope of AI safety and alignment work in generative artificial intelligence (GenAI) has so far mostly been limited to harms related to: (a) discrimination and hate speech, (b) harmful/inappropriate (violent, sexual, illegal) content, (c) information hazards, and (d) use cases related to malicious actors, such as cybersecurity, child abuse, and chemical, biological, radiological, and nuclear threats. The public conversation around AI, on the other hand, has also been focusing on threats to our cognition, mental health, and welfare at large, related to over-relying on new technologies, most recently, those related to GenAI. Examples include deskilling associated with cognitive offloading and the atrophy of critical thinking as a result of over-reliance on GenAI systems, and addiction associated with attachment and dependence on GenAI systems. Such risks are rarely addressed, if at all, in the AI safety and alignment literature. In this paper, we highlight and quantify this discrepancy and discuss some initial thoughts on how safety and alignment work could address cognitive and mental health concerns. Finally, we discuss how information campaigns and regulation can be used to mitigate such prominent risks.
2026 · Underline Science Inc. · other
Evaluating Adjective-Noun Compositionality in LLMs: Functional vs Representational Perspectives ↗Free copyTopic Modeling
Abstract
Compositionality is considered central to language abilities. As performant language systems, how do large language models (LLMs) do on compositional tasks? We evaluate adjective–noun compositionality in LLMs using two complementary setups: prompt-based functional assessment and a representational analysis of internal model states. Our results reveal a striking divergence between task performance and internal states. While LLMs reliably develop compositional representations, they fail to translate consistently into functional task success across model variants. Consequently, we highlight the importance of contrastive evaluation for obtaining a more complete understanding of model capabilities.
2026 · Underline Science Inc. · other
LLM Beliefs Are in Their Heads ↗Free copyPhilosophy and Theoretical Science
Abstract
We investigate belief-like representations in decoder-only autoregressive LLMs using linear controlled probes on residual stream activations and single attention heads. Following Herrmann and Levinstein’s (2025) criteria (Accuracy, Use, Coherence, and Uniformity) we find that large models exhibit strong truth sensitivity (Accuracy), and steering activations along probe directions reliably changes downstream behavior (Use). Coherence, measured via calibrated probes and cross-dataset probing, is moderate across models, while training on diverse data yields domain-consistent truth directions (Uniformity). The results are particularly encouraging at the head level and align with some standard philosophical accounts of belief, e.g., minimal functionalism, supporting the view that LLMs can maintain propositional attitudes under such theoretical frameworks.
2026 · Underline Science Inc. · other
FrameNet-Cultures: A Benchmark for Evaluating LLMs via Cross-Cultural Frame Semantics ↗Free copyTopic Modeling
Abstract
Large language models (LLMs) exhibit cultural biases, yet existing benchmarks rely on closed-form, domain-specific questionnaires. We introduce FRAMENET-CULTURES, a benchmark for evaluating cultural alignment in LLMs based on Fillmore-style frame semantics. Using the EveryCulture encyclopedia, we construct a lexicon of 18 cultural frames (e.g., greeting,child-rearing) across 20 countries, treating it as a structured reference for comparison rather than a definitive representation of contemporary societies. For each frame, we prompt five major LLMs, ChatGPT-5, Gemini-2.5-Flash, Mistral-Large, Qwen-3-Max, DeepSeek-V3.2, three times to generate open-ended instantiations, which are manually annotated and binarized. We measure alignment with country- and continent-level profiles using normalized Hamming distance, and validate cultural recognizability through human evaluation of generated dialogues. Under culture-neutral prompting, outputs align most closely with European profiles, followed by Asian and American ones, indicating a consistent cross-model pattern. With culture-specific prompting, models shift toward the target regions, aligning most strongly with Africa for Ethiopia and with Asia for India. FRAMENET-CULTURES is the first open-ended benchmark for cultural alignment relying on frame semantics. Data, prompts, and annotations are publicly available at https://github.com/neda-jamshidi/FrameNet-Cultures.
2026 · IEEE International Conference on Acoustics Speech and Signal Processing · conference-paper
Towards Fair ASR for Second Language Speakers using Fairness Prompted Finetuning ↗Free copyFree PDF ↗Speech Recognition and Synthesis Abstract
In this work, we address the challenge of building fair English ASR systems for second-language speakers. Our analysis of widely used ASR models, Whisper and Seamless-M4T, reveals large fluctuations in word error rate (WER) across 26 accent groups, indicating significant fairness gaps. To mitigate this, we propose fairness-prompted finetuning with lightweight adapters, incorporating Spectral Decoupling (SD), Group Distributionally Robust Optimization (Group-DRO), and Invariant Risk Minimization (IRM). Our proposed fusion of traditional empirical risk minimization (ERM) with cross-entropy and fairness-driven objectives (SD, Group DRO, and IRM) enhances fairness across accent groups while maintaining overall recognition accuracy. In terms of macro-averaged word error rate, our approach achieves a relative improvement of 58.7% and 58.5% over the large pretrained Whisper and Seamless-M4T, and 9.7% and 7.8% over them, finetuning with standard empirical risk minimization with cross-entropy loss.
2026 · arXiv (Cornell University) · preprint
Postmortem avatars in grief therapy: Prospects, ethics, and governance ↗AI ActFree copyFree PDF ↗Grief, Bereavement, and Mental Health Abstract
Postmortem avatars (PMAs) -- AI systems that simulate a deceased person by being fine-tuned on data they generated or that was generated about them -- have attracted growing scholarly attention, yet their potential role in clinical settings remains largely unexplored. This paper examines the ethics of deploying PMAs as therapeutic tools in grief therapy. Drawing on the dual-process model of grief, the theory of continuing bonds, and the philosophical framework of fictionalism, we propose two potential therapeutic applications: incorporating PMAs into established imaginal exercises such as the empty chair exercise, and treating the process of PMA creation as an art-therapeutic exercise in its own right. We consider five ethical objections to these applications and argue that none constitute knock-down arguments against therapeutic use, particularly given the risk-mitigating role of the clinical context. We conclude by identifying outstanding governance challenges and calling for empirical research, without which neither the promise nor the dangers of therapeutic PMAs can be adequately assessed.
2026 · arXiv (Cornell University) · preprint
On Challenges in Evaluation Methodology: A Review ↗Free copyFree PDF ↗Natural Language Processing Techniques Abstract
Recent advances in language models have prompted a growing body of work that questions prevailing evaluation practices. However, many such critiques have already been discussed in natural language processing (NLP), a field with a long history of methodological reflection on evaluation. We conduct a scoping review spanning four decades and develop a taxonomy of 22 recurring concerns. For each concern, we examine how recently it was discussed in the reviewed literature and whether it receives explicit attention in institutional checklists, finding that some concerns have become less visible in literature and many lack coverage in these checklists. Our work provides an overview of the major concerns in evaluation methodology that have shaped the field and identifies potential targets for renewed investigation and field-level intervention by the research community.
2026 · Proceedings of the AAAI Conference on Artificial Intelligence · conference-paper
Realist and Pluralist Conceptions of Intelligence and Their Implications on AI Research ↗Open accessFree PDF ↗Psychiatry, Mental Health, Neuroscience Abstract
In this paper, we argue that current AI research operates on a spectrum between two different underlying conceptions of intelligence: Intelligence Realism, which holds that intelligence represents a single, universal capacity measurable across all systems, and Intelligence Pluralism, which views intelligence as diverse, context-dependent capacities that cannot be reduced to a single universal measure. Through an analysis of current debates in AI research, we demonstrate how the conceptions remain largely implicit yet fundamentally shape how empirical evidence gets interpreted across a wide range of areas. These underlying views generate fundamentally different research approaches across three areas. Methodologically, they produce different approaches to model selection, benchmark design, and experimental validation. Interpretively, they lead to contradictory readings of the same empirical phenomena, from capability emergence to system limitations. Regarding AI risk, they generate categorically different assessments: realists view superintelligence as the primary risk and search for unified alignment solutions, while pluralists see diverse threats across different domains requiring context-specific solutions. We argue that making explicit these underlying assumptions can contribute to a clearer understanding of disagreements in AI research.
2026 · Proceedings of the AAAI Conference on Artificial Intelligence · conference-paper
LiRA: A Multi-Agent Framework for Reliable and Readable Literature Review Generation ↗Open accessFree PDF ↗Academic Writing and Publishing Abstract
The rapid growth of scientific publications has made it increasingly difficult to keep literature reviews comprehensive and up-to-date. Though prior work has focused on automating retrieval and screening, the writing phase of systematic reviews remains largely under-explored, especially with regard to readability and factual accuracy. To address this, we present LiRA (Literature Review Agents), a multi-agent collaborative workflow which emulates the human literature review process. LiRA utilizes specialized agents for content outlining, subsection writing, editing, and reviewing, producing cohesive and comprehensive review articles. Evaluated on SciReviewGen and a proprietary ScienceDirect dataset, LiRA outperforms current baselines such as AutoSurvey and MASS-Survey in writing and citation quality, while maintaining competitive similarity to human-written reviews. We further evaluate LiRA in real-world scenarios using document retrieval and assess its robustness to reviewer model variation. Our findings highlight the potential of agentic LLM workflows, even without domain-specific tuning, to improve the reliability and usability of automated scientific writing.
2026 · Minds and Machines
Epistemic Drift in Mind-Model Systems ↗Open accessFree PDF ↗Cybernetics and Technology in Society Abstract
Abstract LLMs have been shown to induce representations akin to real-world information structures. According to the standard account, this is because LLMs model the human mind, and minds model the world. Thus, LLMs and minds share properties that ultimately converge upon similar structures. We contribute to this discourse in two ways: a) We argue for an alternative account: LLMs model minds, and minds model worlds, but minds also model LLMs, which in turn extend the capacities of minds. In sum, the world-mind-LLM system constitutes a three-player game, which b) we show does not necessarily converge. We explore the epistemological consequences of non-convergence for world-mind-model systems, introducing the concept of epistemic drift. We offer a new explanatory account of epistemic drift as a social phenomenon, and explore the potential safety concerns of epistemic drift with the introduction of LLMs. Finally, we contribute to existing discourse surrounding LLM agency, arguing that agency is not found in the content of LLM output, but in the way they functionally interact with minds.
2026 · The American Journal of Bioethics
Federation Opacity and the Promise of Federated Learning in Healthcare ↗Open accessFree PDF ↗Privacy-Preserving Technologies in Data Abstract
Federated learning (FL) is a machine learning approach that allows multiple devices or institutions to collaboratively train a model without sharing their local data with a third-party. FL is considered a promising way to address patient privacy concerns in medical artificial intelligence. The ethical risks of medical FL systems themselves, however, have thus far been underexamined. This paper aims to address this gap. We argue that medical FL presents a new variety of opacity-federation opacity-that, in turn, generates a distinctive double black box problem in healthcare AI. We highlight several instances in which the anticipated benefits of medical FL may be exaggerated, and conclude by highlighting key challenges that must be overcome to make FL ethically feasible in medicine.
2026 · arXiv (Cornell University) · preprint
Can Small Agents Collaborate to Beat a Single Large Language Model? ↗Free copyFree PDF ↗Natural Language Processing Techniques Abstract
Recent progress in language modeling has largely relied on scaling model size, yet larger models do not reliably improve performance on tasks requiring multi-step reasoning and tool use. Multi-agent collaboration offers a potential alternative, raising a key question: can well-organized systems built from smaller models outperform much larger language models? We address this question using a minimally designed multi-agent system with a single orchestrator and a small set of specialized sub-agents with restricted communication. On tool-intensive benchmarks spanning factual retrieval, multi-hop reasoning, scientific question answering, and mathematical problem solving, we conduct controlled comparisons between small multi-agent systems and large single-agent models. We find that small multi-agent systems can outperform substantially larger single-agent models, even when the latter have direct access to tools. Reasoning at the orchestrator yields the largest gains, while enabling reasoning in sub-agents provides limited or negative benefits. Overall system performance is driven primarily by orchestrator capacity rather than sub-agent capacity. These results suggest that improved agentic performance depends more on architectural orchestration than on raw model scaling.
2026 · Underline Science Inc. · other
1460 - LiRA: A Multi-Agent Framework for Reliable and Readable Literature Review Generation ↗Free copyAcademic Writing and Publishing
Abstract
The rapid growth of scientific publications has made it increasingly difficult to keep literature reviews comprehensive and up-to-date. Though prior work has focused on automating retrieval and screening, the writing phase of systematic reviews remains largely under-explored, especially with regard to readability and factual accuracy. To address this, we present LiRA (Literature Review Agents), a multi-agent collaborative workflow which emulates the human literature review process. LiRA utilizes specialized agents for content outlining, subsection writing, editing, and reviewing, producing cohesive and comprehensive review articles. Evaluated on SciReviewGen and a proprietary ScienceDirect dataset, LiRA outperforms current baselines such as AutoSurvey and MASS-Survey in writing and citation quality, while maintaining competitive similarity to human-written reviews. We further evaluate LiRA in real-world scenarios using document retrieval and assess its robustness to reviewer model variation. Our findings highlight the potential of agentic LLM workflows, even without domain-specific tuning, to improve the reliability and usability of automated scientific writing.
2026 · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · conference-paper
Latent-Space Intervention for Cross-Lingual Factual Consistency: Consistency Improvements without Accuracy Drops ↗Abstract
Large Language Models (LLMs) often answer the same factual question differently across languages.We study whether cross-lingual latent-space intervention can reduce this inconsistency.We train layer-specific autoencoders on parallel multilingual representations and apply inference-time corrections to factual QA prompts.We find that latent intervention improves geometric alignment between languages, and that this improvement translates into consistent gains in cross-lingual consistency with English across both open-ended and multiplechoice QA formats, without degrading factual accuracy.In open-ended QA, Spearman's rank correlation between English and non-English languages improves substantially, with gains of 0.16 for English-Arabic and 0.20 for English-Russian pairs.In multiple-choice QA, answer agreement with English improves consistently across both KLAR and mParaRel.Ablations show that AE reconstruction drives these gains, whereas PCA-based and mean-shift subspace projections contribute only marginally to the observed improvements in consistency. 1
2026 · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · conference-paper
Does Item Response Theory Generalize Across Languages? Evidence from Cross-Lingual LLM Benchmarks ↗Abstract
The application of Item Response Theory (IRT) for data selection in benchmarks has mostly been limited to monolingual settings.In this work, we ask: how does IRT generalize when the same benchmark is translated across languages?We conduct experiments on Multilingual GSM and Global MMLU, covering over 100 language models and 11 languages.Fitting 1PL, 2PL, and 3PL IRT models per language, we assess cross-lingual consistency of parameters and whether IRT-filtered subsets preserve model rankings across languages: item difficulty emerges as the most stable parameter followed by guessing and then discrimination.A differential item analysis also shows that lack of stability can be tracked to specific items shifting under translation across languages.Our findings suggest IRT-based selection in multilingual settings must be both parameter-aware and item-aware, validating model choice and screening individual items before assuming cross-lingual generalization.
2026 · Conference on Empirical Methods in Natural Language Processing (EMNLP) · conference-paper
Does Alignment Debias or Just Suppress? Evaluating Stereotypical Bias Across Base-Instruct Pairs ↗Open accessFree PDF ↗Ethics and Social Impacts of AI Abstract
Warning: This paper contains offensive stereotypes about gender, race, and religion.Alignment has been widely reported to reduce stereotypical bias in large language models, but it is unclear whether these reductions reflect debiasing or suppression at the output level.Across 27 base-instruct pairs from four open-weight families and four bias benchmarks, three patterns provide substantial evidence that the measured reduction is largely suppression.(i) The chat template drives about half the alignment effect on CrowS-Pairs and essentially all on StereoSet, where 18 of 27 improve without it (p = 0.061) versus 26 of 27 with it (p < 0.001).Per-pair effects remain below d = 0.2 in both conditions.IAT lies at the weight-level end of the spectrum, with template contribution indistinguishable from zero.(ii) Decomposing BBQ's ambiguous-context score shows instruct models defer rather than debias (28% → 53%) and are more biased when they answer (conditional bias 0.53 → 0.59).Suppressed bias is recoverable: the most aggressive adversarial framing restores CrowS-Pairs bias to near base levels, while on BBQ it partly reverses deferral without reactivating conditional bias.(iii) Across eight probed pairs, base and instruct gender directions remain geometrically aligned (median cosine 0.86 vs. 0.006 randompair baseline).Together, these findings suggest benchmark improvements after alignment may overstate genuine bias reduction.
2025 · OSF Preprints (OSF Preprints) · preprint
Establishing the Danish Multi Generation Register ↗Insurance, Mortality, Demography, Risk Management
Abstract
The aim of this project is to establish the Danish Multi Generation Register (MGR) including family relations for current and past Danish residents since ~1920. The MGR is established for research and statistical purposes, including among others research and statistics related to demography, health, socioeconomic factors, economics, family demographics, genetics, history, geography, onomastics, history, sociology, environment, behavior, psychology, business and social sciences. In the generation of the MGR we utilize among others information from official and archival versions of the Civil Registration System, Parish Registers (“ministerialbøger”), population registries, censuses and the cause of death register. More info here: https://doi.org/10.1177/14034948221147096 Data access statement Access to individual-level data in Denmark is governed by Danish authorities. These include the Danish Data Protection Agency, the Danish Health Data Authority, the Ethical Committee, Statistics Denmark, and the Danish National Archives. Each scientific project must be approved before initiation, and approval is granted to a specific Danish research institution. Researchers at Danish research institutions may obtain the relevant approval and data. International researchers may gain data access if governed by a Danish research institution having needed approval and data access.
2025 · Science and Engineering Ethics
Mele’s Digital Zygote: Developer Responsibility for Neural Networks ↗Open accessFree PDF ↗Ethics and Social Impacts of AI Abstract
Should developers be held responsible for the predictions of their neural networks-and if not, does that introduce a responsibility gap? The claim that neural networks introduce a responsibility gap has seen significant pushback, with philosophers arguing that the gap can be bridged, or did not exist in the first place. We show how the responsibility gap turns on whether we can distinguish between foreseeable and unforeseeable neural network predictions. Empirical facts about neural networks tell us we cannot, which seems to force developers to either assume full responsibility or no responsibility at all, introducing a responsibility gap-unless, of course, the same empirical facts hold true of humans, in which case there is no gap, but the trouble is simply with the classical notion of responsibility. We revisit and revise Mele's Zygote, as well as the famous Palsgraf case, and argue that in fact, what complicates responsibility assignment for neural networks also complicates responsibility assignment for humans, and humans seem to confront us with the same all-or-nothing dilemma. Thus, we agree there is no technology-induced responsibility gap (there was no gap in the first place), but for slightly different reasons than our predecessors.
2025 · arXiv (Cornell University) · preprint
On the Measure of a Model: From Intelligence to Generality ↗Abstract
Benchmarks such as ARC, Raven-inspired tests, and the Blackbird Task are widely used to evaluate the intelligence of large language models (LLMs). Yet, the concept of intelligence remains elusive- lacking a stable definition and failing to predict performance on practical tasks such as question answering, summarization, or coding. Optimizing for such benchmarks risks misaligning evaluation with real-world utility. Our perspective is that evaluation should be grounded in generality rather than abstract notions of intelligence. We identify three assumptions that often underpin intelligence-focused evaluation: generality, stability, and realism. Through conceptual and formal analysis, we show that only generality withstands conceptual and empirical scrutiny. Intelligence is not what enables generality; generality is best understood as a multitask learning problem that directly links evaluation to measurable performance breadth and reliability. This perspective reframes how progress in AI should be assessed and proposes generality as a more stable foundation for evaluating capability across diverse and evolving tasks.
2025 · VBN Forskningsportal (Aalborg Universitet) · other
Artificial General Intelligence and its Societal Implications ↗AI ActFree copyFree PDF ↗Ethics and Social Impacts of AI Abstract
The prospect of artificial general intelligence (AGI) has led some researchers and entrepreneurs to foresee an imminent "intelligence explosion," leading to either humanity's near-extinction, or a future of abundance.This brief examines the philosophical and technical premises underpinning such predictions, and their implications for governance.There are two such premises: first, that AI systems can be intelligent and become more intelligent with time; and second, that advanced AI can direct its behavior towards autonomously set goals.However, these premises rest on relatively weak foundations.CAISA recommends a clear-headed approach to AGI, as long as the concept and governance suggestions remain vague.The rush to prioritize AGI development-and/or its governance-risks diverting attention from the tangible risks posed by existing AI technologies, such as algorithmic bias, privacy erosion, and unchecked power concentrations.At the same time, to have a more meaningful discussion of AGI further research is needed, spanning philosophy, cognitive science, cybersecurity as well as the societal and political impact of AGI narratives.
2025 · arXiv (Cornell University) · preprint
EvalCards: A Framework for Standardized Evaluation Reporting ↗AI ActFree copyFree PDF ↗Scientific Computing and Data Management Abstract
Evaluation reports guide the use of general-purpose AI (GPAI) models, yet inconsistent reporting often leaves essential questions unanswered. Through a review of prior work, we identify three central issues in evaluation reporting practices today, reproducibility, accessibility, and accountability, and examine how existing reporting artifacts fall short in addressing them. We introduce the EvalCards framework, outlining its key dimensions and presenting a preliminary format for concise evaluation summaries. We conclude by discussing how the framework can guide the future development of evaluation reporting standards.
2025 · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · conference-paper
Beyond Technocratic XAI: The Who, What & How in Explanation Design ↗AI ActFree to readFree PDF ↗Explainable Artificial Intelligence (XAI) Abstract
The field of Explainable AI (XAI) offers a wide range of techniques for making complex models interpretable. Yet, in practice, generating meaningful explanations is a context-dependent task that requires intentional design choices to ensure accessibility and transparency. This paper reframes explanation as a situated design process, an approach particularly relevant for practitioners involved in building and deploying explainable systems. Drawing on prior research and principles from design thinking, we propose a three-part framework for explanation design in XAI: asking Who needs the explanation, What they need explained, and How that explanation should be delivered. We also emphasize the need for ethical considerations, including risks of epistemic inequality, reinforcing social inequities, and obscuring accountability and governance. By treating explanation as a sociotechnical design process, this framework encourages a context-aware approach to XAI that supports effective communication and the development of ethically responsible explanations.
2025 · Routledge eBooks · book-chapter
Large Language Models and Linguistic Understanding1 ↗Philosophy and Theoretical Science
Abstract
Can large language models (LLMs) understand linguistic utterances? It remains hard to assess such claims as long as we are not clear on what is meant by “linguistic understanding” and “linguistic meaning”. One question at the forefront of recent debates has been whether an LLM can meaningfully use words to talk about the world or is, in a meaningless way, simply predicting the most likely next token. We ask a different but related question. Our question is what form of understanding we should attribute to an LLM as a receiver of our utterances. We introduce a number of notions of linguistic understanding from philosophy of mind and philosophy of language. On this basis, we ask how we should assess the claim that LLMs can understand the linguistic utterances they receive, for instance when prompted by users. Given plausible constraints on semantic and communicative understanding, we discuss how we should evaluate the claim that LLMs have the required competencies to understand linguistic utterances. The aim of the chapter is not to provide any definitive answer to the question of linguistic understanding in LLMs but rather to provide a roadmap of relevant distinctions and considerations.
2025 · arXiv (Cornell University) · preprint
Goal-Directedness is in the Eye of the Beholder ↗Free copyFree PDF ↗Multi-Agent Systems and Negotiation Abstract
Our ability to predict the behavior of complex agents turns on the attribution of goals. Probing for goal-directed behavior comes in two flavors: Behavioral and mechanistic. The former proposes that goal-directedness can be estimated through behavioral observation, whereas the latter attempts to probe for goals in internal model states. We work through the assumptions behind both approaches, identifying technical and conceptual problems that arise from formalizing goals in agent systems. We arrive at the perhaps surprising position that goal-directedness cannot be measured objectively. We outline new directions for modeling goal-directedness as an emergent property of dynamic, multi-agent systems.
2025 · Interspeech (USB) · conference-paper
A Study on The Impact of Foundation Models on Automatic Depression Detection from Speech Signals ↗Free copyEmotion and Mood Recognition
Abstract
An automatic depression detection (ADD) system using spoken language offers the opportunity to develop practical, low-cost tools to detect symptoms early. However, limited data availability, privacy concerns, and transcription efforts pose significant challenges. Recent advancements in foundational models, capable of understanding and processing multimodal inputs, present opportunities for enhancing ADD systems. This study explores various speech foundation models to investigate their impact on ADD. We leverage Whisper and MMS for automatic transcription and integrate speech and text embeddings into a language model optimized with low-rank adaptation (LoRA). In addition, we examine the effects of fine-tuning strategies and prompt formats on model performance. We used English and Bengali datasets to demonstrate the potential of our method in ADD, even with moderate-quality transcriptions. The best speech and language foundation models outperform baseline models on both datasets.
2025 · AI and Ethics · erratum
Correction: Navigating the informativeness-compression trade-off in XAI ↗Open accessFree PDF ↗Explainable Artificial Intelligence (XAI) 2025 · arXiv (Cornell University) · preprint
What if Othello-Playing Language Models Could See? ↗Free copyFree PDF ↗Multimodal Machine Learning Applications Abstract
Language models are often said to face a symbol grounding problem. While some have argued the problem can be solved without resort to other modalities, many have speculated that grounded learning is more efficient. We explore this question in Othello, a simplified, rule-based world that offers a controlled and interpretable testbed for studying world understanding. Building on prior work, we introduce VISOTHELLO, a multi-modal model trained jointly on move sequences and board images. Using the Othello rule understanding task, we examine whether multi-modal learning provides advantages over text-only approaches. We further evaluate robustness under semantically irrelevant perturbations and analyze the consistency of cross-modal alignment. Our results suggest that multi-modal training not only improves performance and robustness but also promotes convergence toward shared internal representations across different model architectures.
2025 · The Lancet Psychiatry
Absolute and relative risks of mental disorders in families: a Danish register-based study ↗Mental Health Treatment and Access
2025 · arXiv (Cornell University) · preprint
Real-Time Progress Prediction in Reasoning Language Models ↗Abstract
Recent reasoning language models, particularly those that employ long latent chains of thought, achieve strong performance on complex agentic tasks. However, as these models operate over increasingly long time horizons, their internal progress becomes opaque to users, making expectation management and real-time oversight difficult. In this work, we investigate whether real-time progress prediction is feasible for such models. We first test whether hidden states encode progress information by discretizing reasoning trajectories and training a linear probe to classify reasoning states. We then fine-tune models to generate progress estimates from 0--100\% during chain-of-thought reasoning. Our strongest progress-reporting checkpoint reaches 0.161 MAE on mathematical reasoning traces and outperforms position baselines in this setting. Finally, we quantify the intrinsic ambiguity of progress labels by measuring how much the implied progress value varies from the same partial rollout. This ambiguity is lowest for Qwen3-4B, whose continuations produce the smallest rollout dispersion, suggesting that larger models can make progress labels more stable by reducing variation in remaining solution length.
2025 · arXiv (Cornell University) · preprint
Lost at the Beginning of Reasoning ↗Abstract
Recent advancements in large language models (LLMs) have significantly advanced complex reasoning capabilities, particularly through extended chain-of-thought (CoT) reasoning that incorporates mechanisms such as backtracking, self-reflection, and self-correction. Despite these developments, the self-correction abilities of LLMs during long CoT reasoning remain underexplored. And recent findings on overthinking suggest that such models often engage in unnecessarily redundant reasoning. In this work, we empirically show that the first reasoning step exerts a disproportionately large influence on the final prediction. I.e., errors introduced at this stage can substantially degrade subsequent reasoning quality. This phenomenon is consistently observed across various state-of-the-art open- and closed-source reasoning models. Leveraging this insight, we propose an efficient sampling strategy that leverages a reward model to identify and retain high-quality first reasoning steps while discarding suboptimal ones, achieving up to a 70% reduction in inference cost without sacrificing any accuracy. Our work highlights the central role of the first reasoning step in generating a high-quality reasoning trajectory, and thus enabling significantly efficient sampling.
2025 · arXiv (Cornell University) · preprint
Mechanistic Interpretability Needs Philosophy ↗Free copyFree PDF ↗Explainable Artificial Intelligence (XAI) Abstract
Mechanistic interpretability (MI) aims to explain how neural networks work by uncovering their underlying mechanisms. As the field grows in influence, it is increasingly important to examine not just models themselves, but the assumptions, concepts and explanatory strategies implicit in MI research. We argue that mechanistic interpretability needs philosophy as an ongoing partner in clarifying its concepts, refining its methods, and navigating the epistemic and ethical complexities of interpreting AI systems. There is significant unrealised potential for progress in MI to be gained through deeper engagement with philosophers and philosophical frameworks. Taking three open problems from the MI literature as examples, this paper illustrates the value philosophy can add to MI research, and outlines a path toward deeper interdisciplinary dialogue.
2025 · Repository for Publications and Research Data (ETH Zurich) · conference-paper
NLP for Social Good: A Survey and Outlook of Challenges, Opportunities, and Responsible Deployment ↗Free copyEthics and Social Impacts of AI
Abstract
Natural language processing (NLP) now shapes many aspects of our world, yet its potential for positive social impact is underexplored. This paper surveys work in ``NLP for Social Good" (NLP4SG) across nine domains relevant to global development and risk agendas, summarizing principal tasks and challenges. We analyze ACL Anthology trends, finding that inclusion and AI harms attract the most research, while domains such as poverty, peacebuilding, and environmental protection remain underexplored. Guided by our review, we outline opportunities for responsible and equitable NLP and conclude with a call for cross-disciplinary partnerships and human-centered approaches to ensure that future NLP technologies advance the public good.
2025 · AI and Ethics
Navigating the informativeness-compression trade-off in XAI ↗Open accessFree PDF ↗Explainable Artificial Intelligence (XAI) Abstract
Abstract Every explanation faces a trade-off between informativeness and compression (Kinney and Lombrozo, 2022). On the one hand, we want to aim for as much detailed and correct information as possible, informativeness, on the other hand, we want to ensure that a human can process and comprehend the explanation, compression. Current methods in eXplainable AI (XAI) try to satisfy this trade-off statically, outputting one fixed, non-adjustable explanation that sits somewhere on the spectrum between informativeness and compression. However, some current XAI methods fail to meet the expectations of users and developers such that several failures have been reported in the literature which often come with user-specific knowledge gaps and good-enough understanding. In this work, we propose Dynamic XAI to navigate the trade-off interactively. We argue how this simple idea can help overcome the trade-off by eliminating gaps in user-specific understanding and preventing misunderstandings. We conclude by situating our approach within the broader ethical considerations around XAI.
2025 · arXiv (Cornell University) · preprint
RAVENEA: A Benchmark for Multimodal Retrieval-Augmented Visual Culture Understanding ↗Free copyFree PDF ↗Multimodal Machine Learning Applications Abstract
As vision-language models (VLMs) become increasingly integrated into daily life, the need for accurate visual culture understanding is becoming critical. Yet, these models frequently fall short in interpreting cultural nuances effectively. Prior work has demonstrated the effectiveness of retrieval-augmented generation (RAG) in enhancing cultural understanding in text-only settings, while its application in multimodal scenarios remains underexplored. To bridge this gap, we introduce RAVENEA (Retrieval-Augmented Visual culturE uNdErstAnding), a new benchmark designed to advance visual culture understanding through retrieval, focusing on two tasks: culture-focused visual question answering (cVQA) and culture-informed image captioning (cIC). RAVENEA extends existing datasets by integrating over 11,396 unique Wikipedia documents curated and ranked by human annotators. Through the extensive evaluation on seven multimodal retrievers and fifteen VLMs, RAVENEA reveals some undiscovered findings: (i) In general, cultural grounding annotations can enhance multimodal retrieval and corresponding downstream tasks. (ii) VLMs, when augmented with culture-aware retrieval, generally outperform their non-augmented counterparts (by averaging +6% on cVQA and +11% on cIC). (iii) Performance of culture-aware retrieval augmented varies widely across countries. These findings highlight the limitations of current multimodal retrievers and VLMs, underscoring the need to enhance visual culture understanding within RAG systems. We believe RAVENEA offers a valuable resource for advancing research on retrieval-augmented visual culture understanding.
2025 · arXiv (Cornell University) · preprint
SemEval-2025 Task 7: Multilingual and Crosslingual Fact-Checked Claim Retrieval ↗Abstract
The rapid spread of online disinformation presents a global challenge, and machine learning has been widely explored as a potential solution. However, multilingual settings and low-resource languages are often neglected in this field. To address this gap, we conducted a shared task on multilingual claim retrieval at SemEval 2025, aimed at identifying fact-checked claims that match newly encountered claims expressed in social media posts across different languages. The task includes two subtracks: (1) a monolingual track, where social posts and claims are in the same language, and (2) a crosslingual track, where social posts and claims might be in different languages. A total of 179 participants registered for the task contributing to 52 test submissions. 23 out of 31 teams have submitted their system papers. In this paper, we report the best-performing systems as well as the most common and the most effective approaches across both subtracks. This shared task, along with its dataset and participating systems, provides valuable insights into multilingual claim retrieval and automated fact-checking, supporting future research in this field.
2025 · Intercultural Pragmatics
Language acquisition in vector space ↗Topic Modeling
Abstract
Abstract Language models are mathematical functions and, as such, induce vector spaces in which input is embedded. Comparing the point clouds of concept vectors across such language models and similar computer vision models, we see surprising similarities. This sheds new light on the Innateness Debate. Much linguistic structure can be induced from extra-linguistic data. Language models are generally thought to be too sample-inefficient to be good models of language acquisition, but what about language models initialized by computer vision models?
2025 · arXiv (Cornell University) · preprint
Evaluating Multimodal Language Models as Visual Assistants for Visually Impaired Users ↗Free copyFree PDF ↗Assistive Technology in Communication and Mobility Abstract
This paper explores the effectiveness of Multimodal Large Language models (MLLMs) as assistive technologies for visually impaired individuals. We conduct a user survey to identify adoption patterns and key challenges users face with such technologies. Despite a high adoption rate of these models, our findings highlight concerns related to contextual understanding, cultural sensitivity, and complex scene understanding, particularly for individuals who may rely solely on them for visual interpretation. Informed by these results, we collate five user-centred tasks with image and video inputs, including a novel task on Optical Braille Recognition. Our systematic evaluation of twelve MLLMs reveals that further advancements are necessary to overcome limitations related to cultural context, multilingual support, Braille reading comprehension, assistive object recognition, and hallucinations. This work provides critical insights into the future direction of multimodal AI for accessibility, underscoring the need for more inclusive, robust, and trustworthy visual assistance technologies.
2025 · arXiv (Cornell University) · preprint
Revisiting the Othello World Model Hypothesis ↗Abstract
Li et al. (2023) used the Othello board game as a test case for the ability of GPT-2 to induce world models, and were followed up by Nanda et al. (2023b). We briefly discuss the original experiments, expanding them to include more language models with more comprehensive probing. Specifically, we analyze sequences of Othello board states and train the model to predict the next move based on previous moves. We evaluate seven language models (GPT-2, T5, Bart, Flan-T5, Mistral, LLaMA-2, and Qwen2.5) on the Othello task and conclude that these models not only learn to play Othello, but also induce the Othello board layout. We find that all models achieve up to 99% accuracy in unsupervised grounding and exhibit high similarity in the board features they learned. This provides considerably stronger evidence for the Othello World Model Hypothesis than previous works.
2025 · Frontiers in artificial intelligence and applications · book-chapter
Captivation Lures and Social Robots ↗Ethics and Social Impacts of AI
Abstract
Social robots, including chatbots and virtual agents, can be fine-tuned on engagement feedback from end users, and their behavior optimized to keep users on the platforms. We refer to strategies induced by social robots in this way as captivation lures, to reflect their manipulative and trap-like nature. We argue that captivation lures differ from phenomena previously studied, including nudging, hypernudging and non-argumentative influence. Researchers have studied nudging - say, in commercials - and hypernudging - say, in recommender systems - for decades, but with recent developments in generative artificial intelligence and social robotics, new ways of manipulating end users have become possible. Captivation lures can be induced from human feedback, and unlike traditional nudging and hypernudging they are not limited to a repository of human-produced content and known nudging strategies. The fact that social robots can induce new, unseen, and hard-to-detect strategies at scale, in the absence of such intentions, has important philosophical and moral implications.
2025 · Frontiers in artificial intelligence and applications · book-chapter
How Good Are We at Assessing the Trustworthiness of LLMs? ↗Explainable Artificial Intelligence (XAI)
Abstract
Chain-of-thought prompting has been proposed as a technique that would increase people’s trust in instruction-tuned large language models such as ChatGPT-3.5 or LLaMA-2. We find, somewhat surprisingly, that while people prefer chain-of-thought explanations, such explanations only increase trust when they are not read, but decrease trust when they are read. Moreover, the question type is predictive of trust. Across these two psychological biases, much of people’s trust in instruction-tuned large language models seems independent of the content of their responses.
2025 · Open Philosophy
Digital Friends and Empathy Blindness ↗Open accessFree PDF ↗Social Robot Interaction and HRI Abstract
Abstract Can chatbot-based virtual relationships replace physical ones? One possible bottleneck is lack of empathy in chatbots, as well as the attraction of physical relationships. Through a mixture of survey and respondent interviews, we investigate perceived chatbot empathy with devoted users, as well as how virtual relationships have affected their physical relationships. We found that Replika users experience high levels of empathy in their interactions with the chatbot, and that extensive use leads to (reported) reduced interest in physical relationships. We speculate whether extensive use of social chatbots can lead to empathy blindness and apathy over time.
2025 · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · conference-paper
Trick or Neat: Adversarial Ambiguity and Language Model Evaluation ↗Open accessFree PDF ↗Natural Language Processing Techniques Abstract
Detecting ambiguity is important for language understanding, including uncertainty estimation, humour detection, and processing garden path sentences.We assess language models' sensitivity to ambiguity by introducing an adversarial ambiguity dataset that includes syntactic, lexical, and phonological ambiguities along with adversarial variations (e.g., word-order changes, synonym replacements, and randombased alterations).Our findings show that direct prompting fails to robustly identify ambiguity, while linear probes trained on model representations can decode ambiguity with high accuracy, sometimes exceeding 90%.Our results offer insights into the prompting paradigm and how language models encode ambiguity at different layers.We release both our code and data: coastalcph/lm_ambiguity. spaces/lmsys/chatbot-arena-leaderboard a asa aulllllll_s lb___srssj_ayyylysnnnnydoooononnnnomyyyynmmm y|m T|||| hTTTT|ehhhhT eeeebh oebbbbyoooob yyyyso aysssswaaaa swww at hwttttehhhh t eeeegth eieggddrl eiieelwrrswttl eeocwwiccomthiittpttiia hhvavetn eeh ttw ehhwwwi eetdiiiht ttrgtthhh eetl shtl ltetehshssehes.eeeccke toomseseppp.yloeysn.g..soclasoscpsoe.p.e.p p t pa*aoooo bllbpp|ll jhayyynndomdonnamynm tne .peoepoblbppl_jjjjj_sjT_jj__j_____rrssrrsrsrrsssaayayayynennnddoodoooononnonnmymyyy__yym mm_r |__|||rrrs||||| rhhT|hT| T||ahT| yyh|hT|||| hyh hTwy TTyhhTTTmm hTeeehmhmeehhth e eeem m T ehe hhnh ne am s a eem me|| m a|mma|aa || a TT nTnanwaaTnnaaaa h n nhos snnnseess e aea sssa sssm wann t swm awawmmawaaaw aawwnat ttatattth hh n h htttnttttnwetAJMJehehss h oeehhh s i ls seeh ertth te eantaa ewweammdpbhoocrraaeh twn rnx awo uuowiocwaaiqdacsstrylmgbnkeft yaeotiai mmmdctwho yoppilinoxuueoinatrmlnuowsurtaaittahhowmmiieemrppcytryslhitcialhhagawndwtntcs eennnetish olylwo htiehne aoa attcitihreinwtpahwtlge hh t riniintn iitihrtrtwgksllwhde tieistntliiih t wie wi iti hhrtnviitm tnsvootiihhwrwtiiiethvihihwtwtti esmmti i cettwntwehhiiietiitech nttthti etii ihehwetnt naahto ththitieie wtetth t ehwheetnnihhpt htelehlewnt tteeltei c tt eihltett tti elhhtthteww hstthlithl wtethdewt .tseehhltlcpdbhtbs*tleltieih leects shitltseoloc elciuolorrl msmpgbcltcptetltoshehcsese ptosheltlhierxpaoycclsheolitseoptltoethhcocnatusssilpelttekes.eymolcpsohsnktllpch..
2025 · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · conference-paper
How Do Multilingual Language Models Remember Facts? ↗Abstract
Large Language Models (LLMs) store and retrieve vast amounts of factual knowledge acquired during pre-training.Prior research has localized and identified mechanisms behind knowledge recall; however, it has only focused on English monolingual models.The question of how these mechanisms generalize to non-English languages and multilingual LLMs remains unexplored.In this paper, we address this gap by conducting a comprehensive analysis of three multilingual LLMs.First, we show that previously identified recall mechanisms in English largely apply to multilingual contexts, with nuances based on language and architecture.Next, through patching intermediate representations, we localize the role of language during recall, finding that subject enrichment is language-independent, while object extraction is language-dependent.Additionally, we discover that the last token representation acts as a Function Vector (FV), encoding both the language of the query and the content to be extracted from the subject.Furthermore, in decoder-only LLMs, FVs compose these two pieces of information in two separate stages.These insights reveal unique mechanisms in multilingual LLMs for recalling information, highlighting the need for new methodologies-such as knowledge evaluation, fact editing, and knowledge acquisition-that are specifically tailored for multilingual LLMs. 1
2025 · Proceedings of the conference - Association for Computational Linguistics. Meeting · conference-paper
Do Language Models Have Semantics? On the Five Standard Positions ↗Open accessPhilosophy and Theoretical Science
Abstract
We identify five positions on whether large language models (LLMs) and chatbots can be said to exhibit semantic understanding. These positions differ in whether they attribute semantics to LLMs and/or chatbots trained on feedback, what kind of semantics they attribute (inferential or referential), and in virtue of what they attribute referential semantics (internal or external causes). This allows for 24 = 16 logically possible positions, only five of which have been argued for. Based on a pairwise comparison of these five positions, we conclude that the better theory of semantics in large language models is, in fact, a sixth combination: Both large language models and chatbots have inferential and referential semantics, grounded in both internal and external causes.
2025 · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · conference-paper
Lost in Embeddings: Information Loss in Vision–Language Models ↗Open accessMultimodal Machine Learning Applications
Abstract
Vision--language models (VLMs) often process visual inputs through a pretrained vision encoder, followed by a projection into the language model's embedding space via a connector component. While crucial for modality fusion, the potential information loss induced by this projection step and its direct impact on model capabilities remain understudied. We introduce two complementary approaches to examine and quantify this loss by analyzing the latent representation space. First, we evaluate semantic information preservation by analyzing changes in k-nearest neighbor relationships between image representations, before and after projection. Second, we directly measure information loss by reconstructing visual embeddings from the projected representation, localizing loss at an image patch level. Experiments reveal that connectors substantially distort the local geometry of visual representations, with k-nearest neighbors diverging by 40--60\% post-projection, correlating with degradation in retrieval performance. The patch-level embedding reconstruction provides interpretable insights for model behavior on visually grounded question-answering tasks, finding that areas of high information loss reliably predict instances where models struggle.
2025 · Conference on Empirical Methods in Natural Language Processing (EMNLP) · conference-paper
Understanding Subword Compositionality of Large Language Models ↗Abstract
Large language models (LLMs) take sequences of subwords as input, requiring them to effective compose subword representations into meaningful word-level representations.In this paper, we present a comprehensive set of experiments to probe how LLMs compose subword information, focusing on three key aspects: structural similarity, semantic decomposability, and form retention.Our analysis of the experiments suggests that five LLM families can be classified into three distinct groups, likely reflecting difference in their underlying composition strategies.Specifically, we observe (i) three distinct patterns in the evolution of structural similarity between subword compositions and whole-word representations across layers; (ii) great performance when probing layer by layer their sensitivity to semantic decompositionality; and (iii) three distinct patterns when probing sensitivity to formal features, e.g., character sequence length.These findings provide valuable insights into the compositional dynamics of LLMs and highlight different compositional pattens in how LLMs encode and integrate subword information.
2025 · Conference on Empirical Methods in Natural Language Processing (EMNLP) · conference-paper
Debiasing Multilingual LLMs in Cross-lingual Latent Space ↗Abstract
Debiasing techniques such as SentDebias aim to reduce bias in large language models (LLMs).Previous studies have evaluated their cross-lingual transferability by directly applying these methods to LLM representations, revealing their limited effectiveness across languages.In this work, we therefore propose to perform debiasing in a joint latent space rather than directly on LLM representations.We construct a well-aligned cross-lingual latent space using an autoencoder trained on parallel TED talk scripts.Our experiments with Aya-expanse and two debiasing techniques across four languages (English, French, German, Dutch) demonstrate that a) autoencoders effectively construct a well-aligned cross-lingual latent space, and b) applying debiasing techniques in the learned cross-lingual latent space significantly improves both the overall debiasing performance and cross-lingual transferability.
2024 · Theoria
Externalist XAI ? ↗Free to readFree PDF ↗Explainable Artificial Intelligence (XAI) Abstract
Abstract Developers of artificial intelligence (AI) often cannot explain the inferences their neural networks make, at least not in ways that satisfy user needs. XAI, explainable artificial intelligence, aims to develop techniques for providing such explanations. XAI researchers have adopted techniques that, to philosophers, seem representationalist–internalist, leading some philosophers to call for more externalist alternatives. But is explaining AI models through causally related external factors feasible? I suggest we compare the idea of externalist XAI to so‐called functionalist XAI. Two common arguments against functionalist XAI are the Epistemic Argument and the Trust Argument, respectively. I discuss whether these arguments can be met by externalists, as well as a new, previously unnoticed challenge from random initialization.
2024 · Research at the University of Copenhagen (University of Copenhagen) · conference-paper
Ethical Concern Identification in NLP: A Corpus of ACL Anthology Ethics Statements ↗Free copyEthics and Social Impacts of AI
Abstract
What ethical concerns, if any, do LLM researchers have? We introduce EthiCon, a corpus of 1,580 ethical concern statements extracted from scientific papers published in the ACL Anthology. We extract ethical concern keywords from the statements and show promising results in automating the concern identification process. Through a survey, we compare the ethical concerns of the corpus to the concerns listed by the general public and professionals in the field. Finally, we compare our retrieved ethical concerns with existing taxonomies pointing to gaps and future research directions.
2024 · arXiv (Cornell University) · preprint
MEG: Medical Knowledge-Augmented Large Language Models for Question Answering ↗Abstract
Question answering is a natural language understanding task that involves reasoning over both explicit context, and unstated relevant domain knowledge. Despite the high cost of training, large language models (LLMs) -- the backbone of most modern question-answering systems -- still struggle to reliably capture the nuanced relationships between concepts that are crucial for reasoning in specialized fields like medicine. In this work, we present MEG, a parameter-efficient approach for medical knowledge-augmented LLMs. MEG uses a lightweight mapping network to incorporate knowledge graph embeddings into the LLM, enabling it to leverage external knowledge in a cost-effective way. We evaluate our method on four popular medical multiple-choice datasets and show that LLMs i) can effectively interpret knowledge graph embeddings and ii) gain significant advantages from the factual grounding these embeddings provide. MEG attains an average of +6.7% and +9.9% accuracy over specialized models like BioMistral-7B and MediTron-7B, respectively. Finally, we show that MEG's performance remains robust to the choice of graph encoder.
2024 · CBS Research Portal (Copenhagen Business School) · report
Insights and Recommendations for News Publishers ↗Journalism and Media Studies
Abstract
This report presents insights and recommendations to news publishers on how to integrate artificial intelligence (AI) in their work processes and news experience from the Platform Intelligence in News project (PIN). PIN was a comprehensive applied research project that aimed to develop effective and responsible AI systems for news. The project was a collaboration between JP/ Politikens Media Group and three universities that began in October 2020 and concluded in June 2024 and received funding from Innovation Fund Denmark.
2024 · Minds and Machines
Is Unsupervised Clustering Somehow Truer? ↗Open accessFree PDF ↗Philosophy of Biology and Scientific Explanation Abstract
Abstract Scientists increasingly approach the world through machine learning techniques, but philosophers of science often question their epistemic status. Some philosophers have argued that the use of unsupervised clustering algorithms is more justified than the use of supervised classification, because supervised classification is more biased, and because (parametric) simplicity plays a different and more interesting role in unsupervised clustering. I call these arguments the No-Bias Argument and the Simplicity-Truth Argument. I show how both arguments are fallacious and how, on the contrary, the use of supervised classification is at least as justified as the use of unsupervised clustering.
2024 · arXiv (Cornell University) · preprint
From Words to Worlds: Compositionality for Cognitive Architectures ↗Abstract
Large language models (LLMs) are very performant connectionist systems, but do they exhibit more compositionality? More importantly, is that part of why they perform so well? We present empirical analyses across four LLM families (12 models) and three task categories, including a novel task introduced below. Our findings reveal a nuanced relationship in learning of compositional strategies by LLMs -- while scaling enhances compositional abilities, instruction tuning often has a reverse effect. Such disparity brings forth some open issues regarding the development and improvement of large language models in alignment with human cognitive capacities.
2024 · Language Resources and Evaluation Conference (LREC) · conference-paper
Evaluating Webcam-based Gaze Data as an Alternative for Human Rationale Annotations ↗Free to readFree PDF ↗Gaze Tracking and Assistive Technology Abstract
Rationales in the form of manually annotated input spans usually serve as ground truth when evaluating explainability methods in NLP.They are, however, time-consuming and often biased by the annotation process.In this paper, we debate whether human gaze, in the form of webcam-based eye-tracking recordings, poses a valid alternative when evaluating importance scores.We evaluate the additional information provided by gaze data, such as total reading times, gaze entropy, and decoding accuracy with respect to human rationale annotations.We compare WebQAmGaze, a multilingual dataset for information-seeking QA, with attention and explainability-based importance scores for 4 different multilingual Transformer-based language models (mBERT, distil-mBERT, XLMR, and XLMR-L) and 3 languages (English, Spanish, and German).Our pipeline can easily be applied to other tasks and languages.Our findings suggest that gaze data offers valuable linguistic insights that could be leveraged to infer task difficulty and further show a comparable ranking of explainability methods to that of human rationales.
2024 · arXiv (Cornell University) · preprint
Does Instruction Tuning Make LLMs More Consistent? ↗Abstract
The purpose of instruction tuning is enabling zero-shot performance, but instruction tuning has also been shown to improve chain-of-thought reasoning and value alignment (Si et al., 2023). Here we consider the impact on $\textit{consistency}$, i.e., the sensitivity of language models to small perturbations in the input. We compare 10 instruction-tuned LLaMA models to the original LLaMA-7b model and show that almost across-the-board they become more consistent, both in terms of their representations and their predictions in zero-shot and downstream tasks. We explain these improvements through mechanistic analyses of factual recall.
2024 · Intercultural Pragmatics
What is lost when a language dies? ↗Linguistic and Sociocultural Studies
Abstract
Abstract Nowak argues that the problem with language loss is not linguists’ loss of data or that the loss of a language is often a result of discrimination against its speakers. Instead, the real problem is its speakers’ loss of illocutionary force. I argue that Nowak’s argument rests on two premises that are both empirically unjustified: that cultural knowledge is a prerequisite for illocutionary force, and that language is a prerequisite for illocutionary force. Languages are among the most fascinating accomplishments of mankind, surpassing Machu Picchu and the Great Wall of China in the eyes of many. I think language loss is comparable to loss of species. The intuition that the death of a language is a significant event, reflects that: Something that evolved gradually over hundreds of years, passed on through hundreds of generations and thousands of individual speakers, is irreversibly gone, once and for all. The illocutionary force of its individual speakers is not.
2024 · arXiv (Cornell University) · preprint
Word Order and World Knowledge ↗Abstract
Word order is an important concept in natural language, and in this work, we study how word order affects the induction of world knowledge from raw text using language models. We use word analogies to probe for such knowledge. Specifically, in addition to the natural word order, we first respectively extract texts of six fixed word orders from five languages and then pretrain the language models on these texts. Finally, we analyze the experimental results of the fixed word orders on word analogies and show that i) certain fixed word orders consistently outperform or underperform others, though the specifics vary across languages, and ii) the Wov2Lex hypothesis is not hold in pre-trained language models, and the natural word order typically yields mediocre results. The source code will be made publicly available at https://github.com/lshowway/probing_by_analogy.
2024 · Acta Analytica
Identity Theory and Falsifiability ↗Open accessFree PDF ↗Philosophy and Theoretical Science Abstract
Abstract I identify a class of arguments against multiple realization (MR): BookofSand arguments. The arguments are in their general form successful under reasonably uncontroversial assumptions, but this, on the other hand, turns the table on identity theory: If arguments from MR can always be refuted by BookofSand arguments, is identity theory falsifiable? In the absence of operational demarcation criteria, it is not. I suggest a parameterized formal demarcation principle for brain state/process types and show how it can be used to identify previously unconsidered contenders for evidence for MR, e.g., binary classification, division, and sorting. For these to be actual instances of MR, the corresponding psychological kinds must be verifiably, relevantly similar. I also briefly discuss possible linguistic, behavioral, and experimental demarcation criteria for psychological kinds.
2024 · Inquiry
On Hedden's proof that machine learning fairness metrics are flawed ↗AI ActOpen accessEthics and Social Impacts of AI
Abstract
Brian Hedden, in a recent article in Philosophy and Public Affairs [Hedden Citation2021. “On Statistical Criteria of Algorithmic Fairness.” Philosophy and Public Affairs 49 (2): 209–231. https://doi.org/10.1111/papa.v49.2.], presented a thought experiment designed to probe the validity of the fairness metrics used in machine learning (ML). The thought experiment has caused a great stir, also within machine learning [Viganó et al. “People are Not Coins: Morally Distinct Types of Predictions Necessitate Different Fairness Constraints.” In 2022 ACM Conference on Fairness, Accountability, and Transparency, FAccT '22, 2293–2301, New York, NY: Association for Computing Machinery.]. Brian Hedden describes a particular prediction problem p – involving 40 people divided into two rooms flipping biased coins – and a binary classification model m for predicting the outcome of these 40 coin flips. Brian Hedden argues that in the thought experiment, m is ‘perfectly fair’, but at the same time, he shows that almost all existing fairness metrics would score m as unfair. He concludes that almost all existing fairness metrics are flawed. If he is right, this seriously undermines most recent work on fair ML. We present three counter-arguments to Brian Hedden's thought experiment, of which the first is the most important: (a) the prediction problem p is irrelevant for ML because p is not (evaluated as) a learning problem, (b) the model m is not actually fair and (c) the prediction problem p is irrelevant for fairness metrics, because group assignment in p is random.
2024 · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · conference-paper
MuLan: A Study of Fact Mutability in Language Models ↗Open accessFree PDF ↗Natural Language Processing Techniques Abstract
Constanza Fierro, Nicolas Garneau, Emanuele Bugliarello, Yova Kementchedjhieva, Anders Søgaard. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers). 2024.
2024 · Findings of the Association for Computational Linguistics: NAACL · conference-paper
Group Fairness in Multilingual Speech Recognition Models ↗Open accessFree PDF ↗Speech Recognition and Synthesis Abstract
We evaluate the performance disparity of the Whisper and MMS families of ASR models across the VoxPopuli and Common Voice multilingual datasets, with an eye toward intersectionality.Our two most important findings are that model size, surprisingly, correlates logarithmically with worst-case performance disparities, meaning that larger (and better) models are less fair.We also observe the importance of intersectionality.In particular, models often exhibit significant performance disparity across binary gender for adolescents.
2024 · Transactions of the Association for Computational Linguistics
CreoleVal: Multilingual Multitask Benchmarks for Creoles ↗Open accessFree PDF ↗Natural Language Processing Techniques Abstract
Abstract Creoles represent an under-explored and marginalized group of languages, with few available resources for NLP research. While the genealogical ties between Creoles and a number of highly resourced languages imply a significant potential for transfer learning, this potential is hampered due to this lack of annotated data. In this work we present CreoleVal, a collection of benchmark datasets spanning 8 different NLP tasks, covering up to 28 Creole languages; it is an aggregate of novel development datasets for reading comprehension relation classification, and machine translation for Creoles, in addition to a practical gateway to a handful of preexisting benchmarks. For each benchmark, we conduct baseline experiments in a zero-shot setting in order to further ascertain the capabilities and limitations of transfer learning for Creoles. Ultimately, we see CreoleVal as an opportunity to empower research on Creoles in NLP and computational linguistics, and in general, a step towards more equitable language technology around the globe.
2024 · conference-paper
Vision-Language Models under Cultural and Inclusive Considerations ↗Open accessFree PDF ↗Multimodal Machine Learning Applications Abstract
Large vision-language models (VLMs) can assist visually impaired people by describing images from their daily lives.Current evaluation datasets may not reflect diverse cultural user backgrounds or the situational context of this use case.To address this problem, we create a survey to determine caption preferences and propose a culture-centric evaluation benchmark by filtering VizWiz, an existing dataset with images taken by people who are blind.We then evaluate several VLMs, investigating their reliability as visual assistants in a culturally diverse setting.While our results for state-of-the-art models are promising, we identify challenges such as hallucination and misalignment of automatic evaluation metrics with human judgment.We make our survey, data, code, and model outputs publicly available.coastalcph/vizwiz-culture
2024 · Transactions of the Association for Computational Linguistics
Do Vision and Language Models Share Concepts? A Vector Space Alignment Study ↗Open accessMultimodal Machine Learning Applications
Abstract
Abstract Large-scale pretrained language models (LMs) are said to “lack the ability to connect utterances to the world” (Bender and Koller, 2020), because they do not have “mental models of the world” (Mitchell and Krakauer, 2023). If so, one would expect LM representations to be unrelated to representations induced by vision models. We present an empirical evaluation across four families of LMs (BERT, GPT-2, OPT, and LLaMA-2) and three vision model architectures (ResNet, SegFormer, and MAE). Our experiments show that LMs partially converge towards representations isomorphic to those of vision models, subject to dispersion, polysemy, and frequency. This has important implications for both multi-modal processing and the LM understanding debate (Mitchell and Krakauer, 2023).1
2024 · Conference on Empirical Methods in Natural Language Processing (EMNLP) · conference-paper
FoodieQA: A Multimodal Dataset for Fine-Grained Understanding of Chinese Food Culture ↗Open accessMultimodal Machine Learning Applications
Abstract
Wenyan Li, Crystina Zhang, Jiaang Li, Qiwei Peng, Raphael Tang, Li Zhou, Weijia Zhang, Guimin Hu, Yifei Yuan, Anders Søgaard, Daniel Hershcovich, Desmond Elliott. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024.
2024 · Conference on Empirical Methods in Natural Language Processing (EMNLP) · conference-paper
Defining Knowledge: Bridging Epistemology and Large Language Models ↗Open accessFree PDF ↗Epistemology, Ethics, and Metaphysics Abstract
Knowledge claims are abundant in the literature on large language models (LLMs); but can we say that GPT-4 truly "knows" the Earth is round?To address this question, we review standard definitions of knowledge in epistemology and we formalize interpretations applicable to LLMs.In doing so, we identify inconsistencies and gaps in how current NLP research conceptualizes knowledge with respect to epistemological frameworks.Additionally, we conduct a survey of 100 professional philosophers and computer scientists to compare their preferences in knowledge definitions and their views on whether LLMs can really be said to know.Finally, we suggest evaluation protocols for testing knowledge in accordance to the most relevant definitions.
2024 · Conference on Empirical Methods in Natural Language Processing (EMNLP) · conference-paper
Concept Space Alignment in Multilingual LLMs ↗Abstract
Multilingual large language models (LLMs) seem to generalize somewhat across languages.We hypothesize this is a result of implicit vector space alignment.Evaluating such alignment, we see that larger models exhibit very highquality linear alignments between corresponding concepts in different languages.Our experiments show that multilingual LLMs suffer from two familiar weaknesses: generalization works best for languages with similar typology, and for abstract concepts.For some models, e.g., the Llama-2 family of models, prompt-based embeddings align better than word embeddings, but the projections are less linear -an observation that holds across almost all model families, indicating that some of the implicitly learned alignments are broken somewhat by promptbased methods.
2024 · Conference on Empirical Methods in Natural Language Processing (EMNLP) · conference-paper
On Mitigating Performance Disparities in Multilingual Speech Recognition ↗Open accessFree PDF ↗Speech Recognition and Synthesis Abstract
How far have we come in mitigating performance disparities across genders in multilingual speech recognition?We compare the impact on gender disparity of different finetuning algorithms for automated speech recognition across model sizes, languages and gender.We look at both performance-focused and fairness-promoting algorithms.Across languages, we see slightly better performance for female speakers for larger models regardless of the fine-tuning algorithm.The best tradeoff between performance and parity is found using adapter fusion.Fairness-promoting finetuning algorithms (Group-DRO and Spectral Decoupling) hurt performance compared to adapter fusion with only slightly better performance parity.LoRA increases disparities slightly.Fairness-mitigating fine-tuning techniques led to slightly higher variance in performance across languages, with the exception of adapter fusion.
2024 · Conference on Empirical Methods in Natural Language Processing (EMNLP) · conference-paper
Unlocking Markets: A Multilingual Benchmark to Cross-Market Question Answering ↗Open accessFree PDF ↗Information Retrieval and Search Behavior Abstract
Users post numerous product-related questions on e-commerce platforms, affecting their purchase decisions.Product-related question answering (PQA) entails utilizing product-related resources to provide precise responses to users.We propose a novel task of Multilingual Crossmarket Product-based Question Answering (MCPQA) and define the task as providing answers to product-related questions in a main marketplace by utilizing information from another resource-rich auxiliary marketplace in a multilingual context.We introduce a largescale dataset comprising over 7 million questions from 17 marketplaces across 11 languages.We then perform automatic translation on the Electronics category of our dataset, naming it as McMarket.We focus on two subtasks: review-based answer generation and productrelated question ranking.For each subtask, we label a subset of McMarket using an LLM and further evaluate the quality of the annotations via human assessment.We then conduct experiments to benchmark our dataset, using models ranging from traditional lexical models to LLMs in both single-market and cross-market scenarios across McMarket and the corresponding LLM subset.Results show that incorporating cross-market information significantly enhances performance in both tasks.
2023 · Frontiers in artificial intelligence and applications · book-chapter
Re-Framing Case Law Citation Prediction from a Paragraph Perspective ↗Open accessFree PDF ↗Artificial Intelligence in Law Abstract
Case law citation prediction, i.e., predicting what historical cases are relevant for your current case, can assist legal discovery and decision-making, but legal documents are long, and often only parts of them are relevant for a particular use case. We therefore reframe case law citation prediction as a paragraph-to-paragraph citation task, introduce a new dataset, and train and evaluate new models. We also evaluate our models qualitatively. Our resources provide a first step toward discovering citation patterns and modeling legal rules in EU law from precedent documents.
2023 · AI and Ethics
Can machines be trustworthy? ↗AI ActOpen accessFree PDF ↗Ethics and Social Impacts of AI Abstract
Abstract AI regulators promote ‘trustworthy AI’, but what exactly does trustworthy AI mean, and what does it have to do with trust? Many philosophers argue that the phrase is a contradiction of terms. Trust, unlike reliance, is said to be a uniquely human relationship involving direct responsiveness or intent. I argue that the objective of trustworthy AI can be real trust in the general sense of Karen Jones and others, and very similar to the kind of trust we place in institutions. The idea that trustworthiness does not apply to machines, stems from a petitio principii fallacy. We show how to escape this fallacy, providing a better and less anthropomorphic definition of trustworthiness. We briefly discuss how transparency modulates trustworthiness on our revised definition, as well as a possible challenge from intentionality.
2023 · arXiv (Cornell University) · preprint
Large language models converge toward human-like concept organization ↗Abstract
Large language models show human-like performance in knowledge extraction, reasoning and dialogue, but it remains controversial whether this performance is best explained by memorization and pattern matching, or whether it reflects human-like inferential semantics and world knowledge. Knowledge bases such as WikiData provide large-scale, high-quality representations of inferential semantics and world knowledge. We show that large language models learn to organize concepts in ways that are strikingly similar to how concepts are organized in such knowledge bases. Knowledge bases model collective, institutional knowledge, and large language models seem to induce such knowledge from raw text. We show that bigger and better models exhibit more human-like concept organization, across four families of language models and three knowledge graph embeddings.
2023 · Research at the University of Copenhagen (University of Copenhagen) · conference-paper
Differential Privacy, Linguistic Fairness, and Training Data Influence: Impossibility and Possibility Theorems for Multilingual Language Models ↗Free copyPrivacy-Preserving Technologies in Data
Abstract
Language models such as mBERT, XLM-R, and BLOOM aim to achieve multilingual generalization or compression to facilitate transfer to a large number of (potentially unseen) languages. However, these models should ideally also be private, linguistically fair, and transparent, by relating their predictions to training data. Can these requirements be simultaneously satisfied? We show that multilingual compression and linguistic fairness are compatible with differential privacy, but that differential privacy is at odds with training data influence sparsity, an objective for transparency. We further present a series of experiments on two common NLP tasks and evaluate multilingual compression and training data influence sparsity under different privacy guarantees, exploring these trade-offs in more detail. Our results suggest that we need to develop ways to jointly optimize for these objectives in order to find practical trade-offs.
2023 · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · conference-paper
Private Meeting Summarization Without Performance Loss ↗Free to readFree PDF ↗Privacy-Preserving Technologies in Data Abstract
Meeting summarization has an enormous business potential, but in addition to being a hard problem, roll-out is challenged by privacy concerns. We explore the problem of meeting summarization under differential privacy constraints and find, to our surprise, that while differential privacy leads to slightly lower performance on in-sample data, differential privacy improves performance when evaluated on unseen meeting types. Since meeting summarization systems will encounter a great variety of meeting types in practical employment scenarios, this observation makes safe meeting summarization seem much more feasible. We perform extensive error analysis and identify potential risks in meeting summarization under differential privacy, including a faithfulness analysis.
2023 · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · conference-paper
On the Independence of Association Bias and Empirical Fairness in Language Models ↗Free to readFree PDF ↗Ethics and Social Impacts of AI Abstract
The societal impact of pre-trained language models has prompted researchers to probe them for strong associations between protected attributes and value-loaded terms, from slur to prestigious job titles. Such work is said to probe models for bias or fairness, or such probes ‘into representational biases’ are said to be ‘motivated by fairness’, suggesting an intimate connection between bias and fairness. We provide conceptual clarity by distinguishing between association biases [11] and empirical fairness [56] and show the two can be independent. Our main contribution, however, is showing why this should not come as a surprise. To this end, we first provide a thought experiment, showing how association bias and empirical fairness can be completely orthogonal. Next, we provide empirical evidence that there is no correlation between bias metrics and fairness metrics across the most widely used language models. Finally, we survey the sociological and psychological literature and show how this literature provides ample support for expecting these metrics to be uncorrelated.
2023 · Scandinavian Journal of Public Health
Towards more comprehensive nationwide familial aggregation studies in Denmark: The Danish Civil Registration System versus the lite Danish Multi-Generation Register ↗Open accessFree PDF ↗Genetic Associations and Epidemiology Abstract
Aim: Linking information on family members in the Danish Civil Registration System (CRS) with information in Danish national registers provides unique possibilities for research on familial aggregation of diseases, health patterns, social factors and demography. However, the CRS is limited in the number of generations that it can identify. To allow more complete familial linkages, we introduce the lite Danish Multi-Generation Register (lite MGR) and the future full Danish MGR that is currently being developed. Methods: We generated the lite MGR by linking the current version of the CRS with historical versions stored by the Danish National Archives in the early 1970s, which contain familial links not saved in the current CRS. We describe and compare the completeness of familial links in the lite MGR and the current version of the CRS. We also describe planned procedures for generating the full MGR by linking the current CRS with scanned archived records from Parish Registers. Results: Among people born in Denmark in 1960 or later, the current CRS contains information on both parents. However, it has limited parental information for people born earlier. Among the 732,232 people born in Denmark during 1950–1959, 444,084 (60.65%) had information on both parents in the CRS. In the lite MGR, it was 560,594 (76.56%). Conclusions: The lite MGR offers more complete information on familial relationships than the current CRS. The lite and full MGR will offer an infrastructure tying together existing research infrastructures, registers and biobanks, raising their joint research value to an unparalleled level.
2023 · Canadian Journal of Philosophy
On the Opacity of Deep Neural Networks ↗Open accessFree PDF ↗Explainable Artificial Intelligence (XAI) Abstract
Abstract Deep neural networks are said to be opaque, impeding the development of safe and trustworthy artificial intelligence, but where this opacity stems from is less clear. What are the sufficient properties for neural network opacity? Here, I discuss five common properties of deep neural networks and two different kinds of opacity. Which of these properties are sufficient for what type of opacity? I show how each kind of opacity stems from only one of these five properties, and then discuss to what extent the two kinds of opacity can be mitigated by explainability methods.
2023 · arXiv (Cornell University) · preprint
WebQAmGaze: A Multilingual Webcam Eye-Tracking-While-Reading Dataset ↗Free copyFree PDF ↗Gaze Tracking and Assistive Technology Abstract
We present WebQAmGaze, a multilingual low-cost eye-tracking-while-reading dataset, designed as the first webcam-based eye-tracking corpus of reading to support the development of explainable computational language processing models. WebQAmGaze includes webcam eye-tracking data from 600 participants of a wide age range naturally reading English, German, Spanish, and Turkish texts. Each participant performs two reading tasks composed of five texts each, a normal reading and an information-seeking task, followed by a comprehension question. We compare the collected webcam data to high-quality eye-tracking recordings. The results show a moderate to strong correlation between the eye movement measures obtained with the webcam compared to those obtained with a commercial eye-tracking device. When validating the data, we find that higher fixation duration on relevant text spans accurately indicates correctness when answering the corresponding questions. This dataset advances webcam-based reading studies and opens avenues to low-cost and diverse data collection. WebQAmGaze is beneficial to learn about the cognitive processes behind question-answering and to apply these insights to computational models of language understanding.
2023 · Minds and Machines
Grounding the Vector Space of an Octopus: Word Meaning from Raw Text ↗Open accessFree PDF ↗Natural Language Processing Techniques Abstract
Abstract Most, if not all, philosophers agree that computers cannot learn what words refers to from raw text alone. While many attacked Searle’s Chinese Room thought experiment, no one seemed to question this most basic assumption. For how can computers learn something that is not in the data? Emily Bender and Alexander Koller (2020) recently presented a related thought experiment, the so-called Octopus thought experiment, which replaces the rule-based interlocutor of Searle’s thought experiment with a neural language model. The Octopus thought experiment was awarded a best paper prize and was widely debated in the AI community. Again, however, even its fiercest opponents accepted the premise that what a word refers to cannot be induced in the absence of direct supervision. I will argue that what a word refers to is probably learnable from raw text alone. Here’s why: higher-order concept co-occurrence statistics are stable across languages and across modalities, because language use (universally) reflects the world we live in (which is relatively stable). Such statistics are sufficient to establish what words refer to. My conjecture is supported by a literature survey, a thought experiment, and an actual experiment.
2023 · Proceedings of the conference - Association for Computational Linguistics. Meeting · conference-paper
LeXFiles and LegalLAMA: Facilitating English Multinational Legal Language Model Development ↗AI ActOpen accessFree PDF ↗Artificial Intelligence in Law Abstract
In this work, we conduct a detailed analysis on the performance of legal-oriented pretrained language models (PLMs).We examine the interplay between their original objective, acquired knowledge, and legal language understanding capacities which we define as the upstream, probing, and downstream performance, respectively.We consider not only the models' size but also the pre-training corpora used as important dimensions in our study.To this end, we release a multinational English legal corpus (LeXFiles) and a legal knowledge probing benchmark (LegalLAMA) to facilitate training and detailed analysis of legal-oriented PLMs.We release two new legal PLMs trained on LeXFiles and evaluate them alongside others on LegalLAMA and LexGLUE.We find that probing performance strongly correlates with upstream performance in related legal topics.On the other hand, downstream performance is mainly driven by the model's size and prior legal knowledge which can be estimated by upstream and probing performance.Based on these findings, we can conclude that both dimensions are important for those seeking the development of domain-specific PLMs.
2023 · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · conference-paper
Mapping Brains with Language Models: A Survey ↗Open accessFree PDF ↗Neural dynamics and brain function Abstract
Over the years, many researchers have seemingly made the same observation: Brain and language model activations exhibit some structural similarities, enabling linear partial mappings between features extracted from neural recordings and computational language models.In an attempt to evaluate how much evidence has been accumulated for this observation, we survey over 30 studies spanning 10 datasets and 8 metrics.How much evidence has been accumulated, and what, if anything, is missing before we can draw conclusions?Our analysis of the evaluation methods used in the literature reveals that some of the metrics are less conservative.We also find that the accumulated evidence, for now, remains ambiguous, but correlations with model size and quality provide grounds for cautious optimism.
2023 · Proceedings of the conference - Association for Computational Linguistics. Meeting · conference-paper
What does the Failure to Reason with “Respectively” in Zero/Few-Shot Settings Tell Us about Language Models? ↗Abstract
Humans can effortlessly understand the coordinate structure of sentences such as "Niels Bohr and Kurt Cobain were born in Copenhagen and Seattle, respectively".In the context of natural language inference (NLI), we examine how language models (LMs) reason with respective readings (Gawron and Kehler, 2004) from two perspectives: syntactic-semantic and commonsense-world knowledge.We propose a controlled synthetic dataset WikiResNLI and a naturally occurring dataset NatResNLI to encompass various explicit and implicit realizations of "respectively".We show that finetuned NLI models struggle with understanding such readings without explicit supervision.While few-shot learning is easy in the presence of explicit cues, longer training is required when the reading is evoked implicitly, leaving models to rely on common sense inferences.Furthermore, our fine-grained analysis indicates models fail to generalize across different constructions.To conclude, we demonstrate that LMs still lag behind humans in generalizing to the long tail of linguistic constructions.
2023 · Proceedings of the conference - Association for Computational Linguistics. Meeting · conference-paper
Being Right for Whose Right Reasons? ↗Open accessFree PDF ↗Explainable Artificial Intelligence (XAI) Abstract
Explainability methods are used to benchmark the extent to which model predictions align with human rationales i.e., are 'right for the right reasons'.Previous work has failed to acknowledge, however, that what counts as a rationale is sometimes subjective.This paper presents what we think is a first of its kind, a collection of human rationale annotations augmented with the annotators demographic information.We cover three datasets spanning sentiment analysis and common-sense reasoning, and six demographic groups (balanced across age and ethnicity).Such data enables us to ask both what demographics our predictions align with and whose reasoning patterns our models' rationales align with.We find systematic inter-group annotator disagreement and show how 16 Transformer-based models align better with rationales provided by certain demographic groups: We find that models are biased towards aligning best with older and/or white annotators.We zoom in on the effects of model size and model distillation, finding -contrary to our expectations -negative correlations between model size and rationale agreement as well as no evidence that either model size or model distillation improves fairness.
2023 · Findings of the Association for Computational Linguistics: EACL · conference-paper
Grammatical Error Correction through Round-Trip Machine Translation ↗Open accessFree PDF ↗Natural Language Processing Techniques Abstract
Machine Translation operates on the premise of an interlingua which abstracts away from the surface form while preserving the meaning.A decade ago, the idea of using round-trip MT to guide Grammatical Error Correction was proposed as a way to abstract away from potential errors in surface forms (Madnani et al., 2012).At the time, it did not pan out due to the low quality of MT systems of the day.Today much stronger MT systems are available so we re-evaluate this idea across five languages and models of various sizes.We find that for extra large models input augmentation through round-trip MT has little to no effect.For more 'workable' model sizes, however, it yields consistent improvements, sometimes bringing the performance of a base or large model up to that of a large or xl model, respectively.The round-trip translation comes at a computational cost though, so one would have to determine whether to opt for a larger model or for input augmentation on a case-by-case basis.
2023 · Conference of the European Chapter of the Association for Computational Linguistics · conference-paper
A Two-Sided Discussion of Preregistration of NLP Research ↗Open accessFree PDF ↗Meta-analysis and systematic reviews Abstract
Van Miltenburg et al. (2021) suggest NLP research should adopt preregistration to prevent fishing expeditions and to promote publication of negative results.At face value, this is a very reasonable suggestion, seemingly solving many methodological problems with NLP research.We discuss pros and cons-some old, some new: a) Preregistration is challenged by the practice of retrieving hypotheses after the results are known; b) preregistration may bias NLP toward confirmatory research; c) preregistration must allow for reclassification of research as exploratory; d) preregistration may increase publication bias; e) preregistration may increase flag-planting; f) preregistration may increase p-hacking; and finally, g) preregistration may make us less risk tolerant.We cast our discussion as a dialogue, presenting both sides of the debate.
2023 · Conference on Empirical Methods in Natural Language Processing (EMNLP) · paratext
Copyright Violations and Large Language Models ↗AI ActOpen accessFree PDF ↗Law, AI, and Intellectual Property Abstract
Language models may memorize more than just facts, including entire chunks of texts seen during training.Fair use exemptions to copyright laws typically allow for limited use of copyrighted material without permission from the copyright holder, but typically for extraction of information from copyrighted materials, rather than verbatim reproduction.This work explores the issue of copyright violations and large language models through the lens of verbatim memorization, focusing on possible redistribution of copyrighted text.We present experiments with a range of language models over a collection of popular books and coding problems, providing a conservative characterization of the extent to which language models can redistribute these materials.Overall, this research highlights the need for further examination and the potential impact on future developments in natural language processing to ensure adherence to copyright regulations.