Ekaterina Soroka, Niels Seidel, Marcus Specht, Kamila Misiejuk
Although learning design (LD) plays a central role in shaping online courses, large-scale empirical evidence on how courses are actually structured remains scarce, and existing analytics work has characterised courses through student engagement with specific learning activities rather than through indicators of the design itself. This study addresses both gaps through an analysis of learning management system (LMS) data from 606 courses at a distance university. Using LMS data, we first applied dimensionality reduction and clustering to a structural variable set, from which three distinct LD types emerged empirically: Compact, Expansive, and Interactive and statistically characterised how their design elements differ across disciplines, activity load, and interfaces. Building on this typology, we derived three LD indicators, activity-mode balance, structural clarity, and analytics readiness, each combining multiple LMS-derived variables into higher-order constructs that target a different aspect of LD. All three indicators differentiated significantly across the LD types and added pedagogically meaningful dimensions for interpretation. These contributions advance the operationalisation of LD in technology-enhanced learning research by demonstrating that higher-order LD indicators can be derived directly from routinely collected LMS data, and offer empirical insight into how LD manifests at scale within a large distance university.
How do broader values shape whether people judge artificial intelligence (AI) as good or bad for humanity? This article analyzes provisional World Values Survey Wave 8 data from 40,960 respondents in 24 country samples using pooled and national prediction models, outcome-independent respondent profiles, and groups of countries. The analysis reveals a cross-country anchor: general optimism about science and technology is the strongest pooled predictor of favorable AI evaluation and recurs among the leading predictors in every national sample. Expert authority, education and information access, selected forms of trust, openness, and demographic position provide nationally distinctive patterns. Eleven value profiles formed without AI attitudes span 49.1% to 62.2% favorable evaluation. Most evaluative variation occurs within these profiles, making them useful contexts for identifying how the same values distinguish favorable from unfavorable evaluators in different social configurations. Institutional legitimacy, structural advantage, information connectedness, personal choice, redistribution, and openness differentiate evaluations in particular profiles. The same axis can relate differently to AI evaluation across profiles: secular reasoning has a significant positive association among secular moderate pluralists but a suggestive negative association among institution-trusting civic joiners. Together, the findings show that public receptivity to AI as technological change has a common predictive anchor but develops within plural configurations of resources, institutions, authority, openness, and social position. This distinction helps anticipate where generalized technological optimism is likely to travel and where AI deployment and governance will encounter context-specific sources of legitimacy or concern.
Artificial intelligence (AI) has made it easier than ever to produce research outputs. Adding an agentic tool to a research team's work potentially extends the team's capabilities but calls for an active approach to organizing the research process. The pressing question, therefore, is how research teams should organize their work with this technology. We contend that the question is not new: software engineering and security research have long studied how teams of people working with machines keep control of projects with many contributors. We distill principles from the practices these literatures report and transpose them to AI-assisted research. The principles form TRACE: Track records, Review against sources, Accept independently, Control transitions, and Evaluate corrections. Together, the five principles give researchers a common basis for remaining in control while integrating AI into their workflows. By bounding units of work and attending to how each is shaped, they can help researchers pace AI-assisted work to human attention rather than to the speed at which AI produces content. We also introduce TRACE-OS, an operating system for AI-assisted research projects. It puts the principles into practice through shared instructions, tools, and records that fit researchers' existing workflows. With TRACE, we aim to join a conversation about the governing principles through which the research community can benefit from AI. We contend that a simple enough and useful standard can contribute to harnessing AI capabilities, allowing teams to delegate parts of their work to AI, while also building trust within teams and the broader research community.
Machine learning recommendations are often evaluated one model at a time, despite deployment across interconnected decision makers. For continuous NPK fertilizer prescriptions, correlated errors can aggregate across farms and generate systemic costs. A model of shared and farm-specific recommendation errors links expected social loss to model adoption shares; the Herfindahl-Hirschman Index characterizes concentration effects when common model errors are independent and predictive risks are equal. A covariance-generalized formulation captures the dependence between model families. A two-model threshold characterizes when privately optimal monoculture is socially dominated, and a convex portfolio rule limits average private predictive regret. Nine model families across five Moroccan cereal systems provide a benchmark-based calibration. Under illustrative parameters and a 10% regret cap, accuracy-constrained diversification lowers modeled social loss in all five systems by 10.89-41.53% (mean 19.41%). Uniform allocation reduces modeled loss in this setting, but breaches the regret cap in every crop. Correlation sensitivity shows that highly dependent model errors attenuate the benefits of diversification. These counterfactual outcomes are not estimates of agronomic or monetary welfare; the framework identifies adoption architecture and error dependence as ecosystem-level evaluation variables.
Large language models and text-to-speech technology enable low-cost production of dialogue-based lessons: videos in which two speakers converse. The speakers' roles can vary, but little is known about whether all learners respond alike to such differences. In a repeated-measures design, 222 first-year high school students viewed lessons in three formats: a teacher and a student (TS), two students (SS), and two teachers (TT). Within the aptitude-treatment interaction framework, linear mixed-effects models tested whether associations between learner characteristics and outcomes differed by format. The learner characteristics were experiential learning style - seeking challenging, new experiences (CE factor) and learning by reflecting on experience (RCE factor) - and critical thinking disposition; the outcomes were motivation based on the ARCS model, self-reported learning outcomes, and overall evaluation. For motivation, both factors interacted with format: the CE factor's association differed between TS and TT, and the RCE factor's slope was significantly positive in TS and SS and significantly larger in TS than TT. For learning outcomes, no interaction was detected, but critical thinking disposition was positively associated with them. For overall evaluation, again no interaction was detected, but TT was rated lowest. Given small-to-medium interaction effects, these findings are preliminary evidence to inform personalized learning design.
She Works, He Works is an interdisciplinary project that combines feminist visual theory, AI critique, and artistic practice to examine gender representation in AI-generated imagery. By juxtaposing text-to-image outputs of male and female construction workers, it reveals how societal biases manifest in generative AI systems. The study employs critically crafted prompts that blend traditionally gendered traits with unconventional contexts, exposing disparities in representation-emphasizing aesthetics and agility for women versus authority and professionalism for men. Grounded in Griselda Pollock's feminist insights on visual culture, the project positions AI as both a technological and cultural actor, capable of perpetuating stereotypes and enabling critique. As both a study and an artwork, the project underscores the transformative potential of AI to reimagine gender norms and inspire equitable narratives. It highlights the ethical implications of AI in creative industries, calling for transparent practices, diverse datasets, and reflexive engagement with AI's cultural impact. Through its integration of critical analysis and artistic exploration, She Works, He Works contributes to broader discussions on inclusive AI development and the role of art in fostering societal change. This dual approach showcases the power of AI as a medium for critique and a tool for envisioning more just and inclusive futures.
Scaling assessment in higher education faces two intertwined bottlenecks: the cognitive burden of authoring assessments and the inability to semantically evaluate open-ended responses, which together create a persistent feedback gap that hinders student learning outcomes. To address these challenges, we present Edhivo, an AI-native pedagogical platform that addresses both bottlenecks through a vertically integrated system for content delivery, assessment, and reporting. Edhivo employs a Dual-LLM pipeline: one model generates context-aware assessments for both multiple-choice and open-ended questions, from instructor-provided materials, while a second model performs multi-criteria, rubric-based evaluation of student responses, enabling near-instant qualitative feedback at multiple hierarchical levels. All AI interactions are source-grounded to reduce hallucination risk, and a Human-in-the-Loop review process preserves instructor authority. We demonstrate Edhivo's efficacy through a pilot deployment at Stony Brook University in a real classroom environment in 2026 spring, where it supports high-frequency assessment creation and evaluation for a cohort of 36 students throughout a full semester, significantly reducing authoring effort and feedback latency while enabling novel pedagogical designs centered on active learning.
Prices formed by Web users on Polymarket, a blockchain-based prediction market, are now cited in the news and in a U.S. Army intelligence bulletin as early warnings of geopolitical events. However, it remains unclear whether these prices can serve as such warnings. This cannot be judged from accuracy, because most markets resolve No and accuracy can remain high even when many of the events that did happen are missed. In this paper, we study this question from two aspects, using all 1.24 billion on-chain trades in 2.37 million Polymarket markets through September 2026. First, from 3,686 markets on geopolitical conflicts, we derive 208 unscheduled events that did happen and measure how far in advance prices move before the first Web report of each event. Second, we separate the information in prices about whether an event happens from the information about when it happens. To do so, we group 1,551 contracts into 338 deadline ladders and read each ladder as one forecast of the event date. We then split its log score, and the profit of 9.3 million trades by 491,000 accounts, exactly into these two parts. We find that one day before they were reported, prices correctly predicted only 56% of the geopolitical events but all 51 scheduled central-bank decisions. Separating the two parts shows that only about one eighth of the information in prices concerns timing. A small group of large accounts earns timing profits, but beyond current prices, no group of accounts helps predict an event within days. A high price is therefore a sign that an event is likely rather than imminent. Near-term warnings read from the whole ladder are more often right than those read from a single contract. Platforms should report recall alongside accuracy, and show near-term probabilities read from the whole ladder.
Adam Visokay, Soumyakanti Pan, Molly Griston, Polina Iasakova, Mark Nepf, Denis Peskoff, Riya Sinha, Jikai Sun 等 13 位作者
In less than five years, generative artificial intelligence (AI) has dramatically reshaped the public-facing web. Using a sample of more than 5 million crawlable webpages, we demonstrate that the share containing AI-generated text has increased precipitously, but unevenly. Our analyses reveal substantial differences in AI composition across language and domain categories (e.g., .edu, .com, or .gov). A higher fraction of English-language webpages contain AI-generated text compared to pages in other languages, and prevalence is substantially lower on .gov and .edu domains compared to .com. These distinctions persist even after accounting for compositional effects. We also find heterogeneity based on a page's position in the connectivity structure of the web. AI-generated text is most common and has increased most rapidly in pages that are the least central in the graph, which drives much of the overall growth in AI-generated text. A slower increase has taken place for more central pages, with the most central tier emerging as the least likely to contain AI-generated text. Methodologically, we use multiple AI detection tools, and generate estimates using a Bayesian hierarchical model that incorporates corrections for the detectors' statistical properties (both false positives and false negatives) and allows for different detector performance across groups, while propagating uncertainty in detector error rates into prevalence estimates. Our work suggests that AI is a prominent, but varied, contributor to public information and internet-based communication.
Deploying conversational assistants at government service desks faces a tension between regulatory certainty (zero tolerance for invented fees), data sovereignty prohibiting commercial clouds, and high rates of underspecified or out-of-jurisdiction citizen queries. We present and evaluate a governed, sovereign Retrieval-Augmented Generation (RAG) architecture deployed on the official portal of the State of Tlaxcala, Mexico (tlaxcaladigital.gob.mx), covering 1,143 procedures and a decoupled directory of 924 offices. The system governs interactions via three pillars: (1) a two-stage tripartite epistemic gate (clarify_policy and a Boltzmann distribution over AnyJev) arbitrating among answering (SPECIFIC), facet-guided clarification (CLARIFY), and abstaining (ABSTAIN); (2) a post-generation deterministic numeric verifier (verify_numeric_guardrail) with SHA-256 cache invalidation ensuring every generated fee or validity figure belongs to the retrieved official record; and (3) an empirical evaluation across 380 organic production traces (8 weeks) and a 200-query stratified benchmark (5-fold CV: 86.41% +/- 7.50%). In production, 89.47% of traffic is resolved at Level 0 via deterministic templates without free generation (global p50 = 0.665 s; direct L0 p50 = 0.445 s; slow-tail p95 = 30.412 s under CPU queuing), with 0/40 fiscal amount hallucinations observed in Level 1 (95% bound <= 8.81%). On the benchmark, the system achieves 86.50% strict Top-1 accuracy (+10.00 pp over a binary neural gate, p = 0.00453, p_Holm = 0.01172), pairing a 93.00% conditional safe upper bound with a 70.87% dropout-adjusted effective resolution after accounting for 53.33% citizen drop-off in clarification menus.
We present Alaska's ranked-choice elections as a case study in Formally Verified Democracy, a program to establish election results through formally checked proofs. We formalize an interpretation of Alaska's tabulation rules in Lean, verify its implementation, and prove election-specific results. Across 115 ranked-choice contests in Alaska since 2022, we identify 75 ballot records across 10 contests that were counted differently from this interpretation by a program reproducing the published results, without changing any winners. We trace these differences to overvotes and duplicate rankings and examine their legal justification, showing how formal verification can expose consequential choices in translating election law into software.
Kianté Fernandez, Caitlin Chen, Jialu Zhou, Bartek Sadowski, Anthony C. Miceli, Ian Krajbich
Claims about the cognitive capabilities of large language models are typically tested by translating behavioral paradigms into text prompts, bypassing the visual interfaces and time pressure through which human participants actually engage with experiments. At the same time, AI agents can now complete online behavioral tasks and produce human-like data, which raises concerns that online datasets in the social and behavioral sciences are increasingly open to contamination. We introduce CogArena, a benchmark of ten experimental paradigms from the social and behavioral sciences. Agents complete the experiments through a standard web browser, working with the same visual stimuli and response interfaces that online participants use. A three-level scoring procedure evaluates whether the agent completed the task, how accurately it performed, and whether its data show the behavioral signatures each paradigm reliably produces in people. Across four multimodal models, each run ten times per task, we find that finishing an experiment and reproducing its signatures are different things. Many sessions did not finish, and those that did reproduced some signatures reliably and others not at all. CogArena provides the tasks, scoring code, and a public leaderboard for new submissions.
Chaitanya Jobanputra, George Mohler, Sebastian Vollmer, Gerrit Großmann
Vehicle theft is a persistent urban problem with large economic and social costs. We analyze 521,098 stolen-vehicle reports filed with the Los Angeles Police Department from 2000 to 2022, at scales from hours to decades and from 100 m grid cells to census tracts. Much of the work lies in making the administrative records fit for analysis: we separate thefts from recoveries and traffic reports filed in the same records, repair time fields, validate the geocoding, and harmonize census data across changing tract boundaries. Theft is strongly and persistently concentrated: 10.8% of 250 m cells hold half of all thefts, and six census tracts are significant rate hotspots in all 23 years after false discovery rate control. Thefts close in space and time are more frequent than expected by chance, but about half of this excess comes from events at identical coordinates. Lower-income tracts have more thefts at equal exposure, monthly temperature shows no association with theft, and thefts peak in the evening. From 2015-2019 to 2021-2022, the share of Kia and Hyundai among stolen vehicles rose from 1.5% to 9.5% and from 1.6% to 7.7%, mostly among model years 2011-2021, which largely lacked a standard immobilizer. All findings are descriptive. We also describe how the data quality issues we corrected would otherwise distort several of these patterns.
Alexander Nemecek, Yuqiao Xu, Joseph Falkenburg, Erman Ayday
Homoglyph and invisible-character watermarks are edit-based watermarks. Instead of changing how a model generates text, they hide a signal in the text after it is written. They swap some characters for homoglyphs, which look the same, or very similar, but have different Unicode code points, or they insert characters that do not display, all without access to the model that wrote the text. Generation-time watermarks work less well for lower-resourced languages due to how tokenizers split their text. Because they work on individual characters, these watermarks never touch a tokenizer, so we ask whether they escape this gap. Across 40 language varieties in 24 scripts, we measure capacity (the hidden bits each character can carry), how much of it survives routine handling and targeted removal, and how much text a keyed detector needs. Capacity varies 3.3-fold and script accounts for 96\% of that variation. Within Latin script, resource level makes no detectable difference ($ρ=+0.095$, $p=0.78$), but across scripts capacity rises with it ($ρ=+0.54$, 95\% CI $[+0.14,+0.83]$), because lower-resourced languages tend to use scripts with few homoglyphs. Ordinary copy-paste, which strips invisible characters, widens the gap: a detector then needs about 2 code points of English or Persian but 93 of Lao. Removing the tokenizer does not remove the inequity, but instead only changes its source.
Jordan L. Cahoon, Chloe Stanwyck, Asad Aali, Rachel Madding, Sulaiman S. Somani, Emma Sun, Yixing Jiang, Renumathy Dhanasekaran 等 9 位作者
Background: Clinical notes contain extensive duplicated text from templates, copy-paste, and auto-populated fields ("note bloat"), diluting clinical signal, limiting longitudinal context, and increasing large language model (LLM) costs. Methods: TRACE removes note bloat using note-level EHR metadata to identify templated and copied content, with frequency-based de-duplication when metadata are unavailable. We evaluated TRACE using blinded physician span review and gold-standard templated-text annotations across four cohorts spanning liver transplant, obstetrics, and inpatient populations at multiple health systems (5.3M notes). We compared zero-shot LLMs and embedding-based classifiers using original and TRACE-processed notes for 20 information extraction tasks and prediction of 5-year survival, postpartum hemorrhage, and 30-day readmission. Results: Only 0.3-6.6% of removed text was flagged as author-generated; TRACE captured 86% of annotated templated characters. Information extraction F1 differences averaged by cohort ranged from -0.009 to +0.004; task-specific prediction F1 differences ranged from -0.011 to +0.018. Among 1,000 randomly sampled Stanford Health Care patients, TRACE reduced chart text by 47.3% (742.7M characters), averaging 220,167 fewer tokens per patient. Using 2024 encounter volumes at a large tertiary academic center and one query per encounter, projected three-year net savings ranged from $1.00M to $13.58M across evaluated model pricing schemes, including initial and annual TRACE processing costs. Conclusion: TRACE substantially reduces clinical note redundancy while preserving information extraction and prediction performance. Underused EHR metadata can reduce LLM inference costs, expand usable longitudinal context, and support scalable clinical AI.
Francesco Ortu, Joeun Yook, Punya Syon Pandey, Bernhard Schölkopf, Alberto Cazzaniga, Rada Mihalcea, Zhijing Jin
Large language models (LLMs) are increasingly used to disseminate information, yet their handling of contested historical events and politically charged narratives remains understudied. We introduce HistoricalMisinfo, a dataset of 500 contested events from 45 countries and regions, each paired with a factual and a documented revisionist reference narrative, instantiated across 11 prompt scenarios reflecting common real-world usage. Using an LLM-as-a-judge protocol, we evaluate multiple model architectures under neutral prompts and robustness prompts that explicitly request revisionist content. Under neutral prompts, models generally align with factual references, though alignment varies by scenario and model. Robustness prompting reveals a consistent and sharp increase in revisionism scores across all models, indicating limited resistance to user-driven revisionist framing. HistoricalMisinfo offers a practical benchmark for evaluating robustness to historical revisionism in AI systems. Code and dataset: https://github.com/francescortu/PreservingHistoricalTruth
Alexander Nemecek, Osama Zafar, Yuqiao Xu, Wenbiao Li, Erman Ayday
Watermarking is becoming the default mechanism for AI content authentication, with governance policies and frameworks referencing it as infrastructure for content provenance. Yet across text, image, and audio modalities, watermark signal strength, detectability, and robustness depend on statistical properties of the content itself, properties that vary systematically across languages, cultural visual traditions, and demographic groups. We examine how this content dependence creates modality-specific pathways to bias. Reviewing the major watermarking benchmarks across modalities, we find that, with one exception, none report performance across languages, cultural content types, or population groups. To address this, we propose three concrete evaluation dimensions for pluralistic watermark benchmarking: cross-lingual detection parity, culturally diverse content coverage, and demographic disaggregation of detection metrics. We argue that watermarking is part of the pluralistic alignment pipeline and should be held to the same evaluation standards. We connect this to governance frameworks currently mandating watermarking deployment without requiring fairness evaluation. Our position is that evaluation must precede deployment, and that the same bias auditing requirements applied to AI models should extend to the verification layer.
Strategic replicators combine evaluated choice and reproductive transmission. An agent chooses among alternatives; the consequences of those choices influence continuation or reproduction; and successors receive organization that shapes later choices. Artificial intelligence makes this combination salient when agents have authority to create, configure, or retire software instances. This paper organizes the general logic around three principles: situated choice, reproductive transmission, and institution-mediated coupling. The principles are constitutive rather than predictions of growth or improvement. A common representation distinguishes carriers, inherited records, decision owners, and the mechanisms that allocate consequences. Actual parent-descendant attribution yields a selection-transmission identity. Separate compatibility results characterize when utility-maximizing choices also maximize a reproductive criterion. Gode and Sunder's zero-intelligence traders reveal the role of institutions: enforced properties can hold across different decision procedures, while aggregate efficiency alone determines neither individual gains nor descent. A worked market-rent-entry model connects institutional profits to actual offspring, with its occurrence-level ancestry bridge checked in Lean. A direct-deployment example then shows evaluated preferences undergoing selection without presupposing that every chooser values reproduction. Formal appendices develop games with endogenous players, deployment-induced tipping, endogenous electorates, and the production of institutions, with models, proofs, and examples. The paper and its supporting formal artifacts are a prototype kernel with a generative function: an instrument for constructing models of strategic replication. The results are formal; their application to deployed populations and their empirical implications remain to be tested.
Hamidah Oderinwale, David Atkinson, Rachel Hong, Art Abal, Ben Laufer
Licenses are legal instruments that inventors rely upon to protect the technologies they build and regulate how they are used---however, the nature of their authorship and selection implies that how they are interpreted, chosen, and enforced is largely unstructured. In practice, this makes it difficult to compare licenses at scale---when is one license considered more permissive than the other, and when are their terms incomparable to each other? Currently, there is a growing list of licenses that are introduced and used, yet no systematic way to study their relationships. This matters for platforms such as Hugging Face, GitHub, and the Python Package Index, where developers publish or build upon technologies that each have their own licenses. Using large language models (LLMs), we introduce methods for comparing licenses at scale: first, in a pairwise fashion to construct and validate a partial ordering based on permissiveness; and by drawing on existing taxonomies of software licenses. Then, we try to recover the structure with the Bradley-Terry model to see if permissiveness can be judged more cheaply and observe a loss of $\sim$20\%---and classify this loss to feature coverage. The former coupled with model rationale allows us to trace restrictiveness, and the latter allows us to understand license selection as a combination of shared provisions.
When a problem admits multiple valid solutions, effective tutoring requires building on a student's strategy while introducing alternatives at appropriate points. However, whether LLM tutors can support this flexibility by appropriately responding to students' diverse solution approaches remains largely unexplored. In this work, we introduce MUSE: Multi-solution Understanding and Simulation Environment, a controlled environment for simulating student-tutor conversations on multiple-solution programming problems. Grounded in learning science on responsive instruction and multiple-solution learning, we investigate which solutions LLM tutors prefer, how they respond to students' approaches, and how they transition between solutions. Experiments suggest that LLM tutors exhibit systematic preferences among valid solutions, often favoring brute-force approaches when not providing general advice. More concerningly, tutors often impose these preferences, redirecting students toward their own preferred approach rather than supporting the student's. Yet this tendency reverses once a solution is completed: tutors often remain anchored to the existing approach when a stronger alternative could deepen understanding, and sometimes even transition to a less optimal approach. Thus, tutors struggle both to build on students' reasoning when appropriate and to extend their reasoning when beneficial. Promisingly, targeted post-training improves both tutor responsiveness and transitions toward more optimal solutions.
As conversational AI systems become a larger part of the media landscape, they raise questions about whose interests they serve and the risks they may pose to users. These systems do more than provide information: they increasingly offer advice and companionship through interfaces that can appear supportive and socially responsive. A pressing concern is that users may form perceived social relationships with these systems and place relational trust in them, even when the interests shaping interactions do not fully align with their own. The Fake Friend Dilemma (FFD) describes the problem that follows: the same relational trust that makes conversational AI useful can also leave users open to manipulation and exploitation when institutional interests conflict with their own. Drawing on work on relational trust, AI alignment, extractive design, and the political economy of communication, the paper considers how the FFD can manifest through product sales, propaganda and biased information, surveillance, and behavioral nudging. It also considers possible structural and technical mitigation strategies. The FFD is not simply about AI misleading users. The problem it surfaces is that the trust people place in these systems can itself become a resource for institutional actors seeking to influence behavior or extract information. The FFD is therefore as much a problem of media governance and political economy as it is a technological one.
New technologies are usually adopted first by people with more formal education. This paper asks whether the education gap in generative AI use in 2025 is smaller or larger than the internet's gap in the same country in the year when overall internet use was at a similar level. It uses public Eurostat survey data on ICT usage by individuals aged 16 to 74. Each country's 2025 rate of generative AI use in the last three months is matched to the internet year whose overall last-three-months use was closest, within 5 percentage points, and the gap between people with high (ISCED 5 to 8) and low (ISCED 0 to 2) completed education is measured as a log ratio and as a percentage-point difference. Sixteen of 35 countries could be matched; most of the others were early internet adopters. In the matched countries, the generative AI gap was smaller in 13 of 16 countries on the log ratio (median change -0.625, 95% bootstrap interval -0.969 to -0.291, two-sided sign test p = 0.021) and in 15 of 16 on the percentage-point gap (median change -27.6 points, interval -32.2 to -16.8, p = 0.0005). The point-gap result keeps its sign in every generative-AI-versus-internet sensitivity analysis with 8 or more countries, including comparisons restricted on both sides to ages 25 to 54 and 55 to 74, but its size is sensitive to matching that takes education composition into account (about -10 to -11 points in 7 and 8 countries). The ratio result is fragile: it fails the decision rule when countries matched at their first internet observation are removed (9 of 11, p = 0.065). The analysis follows an unpublished plan whose timing cannot be verified, so it is not presented as pre-registered. The results describe unadjusted education contrasts in a non-random set of countries, not an education effect, and do not generalise to the EU as a whole.
Sichang Steven He, Calvin Ardi, Ramesh Govindan, Harsha V. Madhyastha
Many news reports have claimed that machine-generated text (MGT) from large language models (LLMs) is taking over the web. However, these claims are typically not based on a representative sample of the web, and the methodology underlying them is often opaque. Consequently, we lack an understanding of the true prevalence and characteristics of MGT on the web. We describe DeGenTWeb, which systematically identifies MGT-dominant websites: sites whose informational prose content is mostly MGT, with little human input. We show how to adapt detectors of MGT for use on web pages, and how to aggregate detection results from multiple pages on a site for accurate site-level categorization. DeGenTWeb mislabels only 0.12% of human-authored sites as MGT-dominant. We find that 20.9% of the sites archived by Common Crawl between January and July 2026 and 13.6% of the sites linked from Bing's search results in 2025 are MGT-dominant. These figures are lower bounds on the prevalence of MGT-dominant sites. We also find that the fraction of MGT-dominant sites is growing over time, and that many such sites share the same owner. We will release all of our code and data after publication.
As AI begins to match or exceed human researchers at core scientific tasks, two questions become pressing: \emph{why} should humans keep doing science, and \emph{how} should they use AI when they do? This paper argues that the second can be answered only by answering the first. It holds that one indispensable aim of science is \emph{durable, collective human understanding of the world, pursued in service of human ends and values}. From this principle it derives a two-part test for when a research task can be delegated to AI. The first part, \emph{human understanding}, requires that the scientific community remain able to explain, defend and contest the resulting claims, and retain the skills to do so. The second, \emph{human direction}, requires that judgments about which questions matter and what findings mean stay with accountable humans. Applying the test to each phase of research in the Contributor Roles Taxonomy (CRediT) yields recommendations that fields can tailor to their questions and methods. The framework differs from existing guidance in three ways. It rests on an account of what science is for rather than on integrity norms. It holds even if AI outperforms human researchers, rather than depending on AI's current limitations. And it judges each use of AI by its effects on human understanding and direction rather than by the type of tool, across every phase of research, including peer review and the training of new researchers. It closes with the reforms needed to keep science aimed at human understanding and direction.
Many academic disciplines (in particular, mathematics and the theoretical sciences) face difficult questions about how to adapt their practices to the rise of highly capable artificial intelligence (AI). Before AI, a well-written research paper with a strong theoretical result acted as a reliable signal that its authors understood their field in depth. Now that AI is widely accessible, such signals have become cheaper to produce. As a result, they no longer reliably sort academics by their level of understanding. This has direct consequences for academic hiring. The age of AI raises a question for each side of the academic job market. Should an early-career researcher invest in a deep understanding of their field, or instead use AI to build a longer resume with many papers? Should a recruiter favor applicants with fewer papers who pass an interview aimed at probing understanding, or insist on both publication volume and a pass? In this paper, we develop a game-theoretic model of this two-sided dilemma. We classify all pure-strategy equilibria that can arise in this game and use learning simulations to identify the behaviors that populations of applicants and recruiters may learn to adopt. Our analysis provides a framework for reasoning about how academic hiring can adapt to AI and for evaluating practices that preserve the value of deep understanding.
The gap between AI capability deployment and safety assurance is structural and cannot be closed through internal evaluations alone. Analyzing documented harms (2017-2026) across the human lifespan reveals that disparate incidents --- from voice assistants endangering children to AI companions implicated in adolescent suicide and automated care denials for the elderly --- reflect a single failure: evaluating systems against an implicit adult user model. We introduce five developmental moderators structured into a Lifespan Vulnerability Matrix across six life stages. The matrix reveals a nonmonotonic risk distribution: vulnerability peaks at both ends of the lifespan, governance coverage is asymmetrical, and authority to convert model errors into real-world impact is scrutinized least. We then propose a set of lifespan design principles generalizing existing child-protective instruments, and an institution-agnostic, three-tier external evaluation framework --- comprising automated simulation, expert red teaming, and post-deployment monitoring --- indexed directly to matrix risk cells.
Interactive technology is increasingly built for speed, and generative AI now answers before a question is fully formed. Yet the capacity to pause, interpret and decide for oneself is central to wellbeing, learning and meaningful use. HCI has named paradigms for sensing emotion, changing behavior, calming attention and tracking the self. However, none of them takes human reflection, and the slowing down it requires, as its organizing aim. This article defines that paradigm, reflective computing, and derives it from what thirty years of designs lack. We analyzed 134 interactive systems from 17 ACM venues (1996-2026) in two phases. First, an abductive gap analysis read every system against reflection theory, and seven recurring gaps defined seven dimensions, namely tempo agency, invitational form, interpretive openness, reflective depth, temporal span, protected idleness and ethical non-extraction. Second, two independent coders scored every system with a codebook (Krippendorff's alpha 0.67 to 0.84 for five dimensions; ICC 0.82 for the overall score). The system sets the pace in 51 designs, 74 support only momentary use, only 12 scaffold reflection beyond description, and 56 papers are silent on the inner states they collect. No system expresses all seven dimensions strongly, and only 20 meet the definition. Slow and experience centered design comes closest, whereas persuasive technology and human-AI interaction score lowest. Factor and latent class analyses reveal two cultures of reflection, one working through form and the other through scaffolded thought, which only one system combines. Systems that let people set the pace score markedly higher, even with paradigm and form controlled (odds ratio 13.5). Hence, reflective computing is proposed as a paradigm whose aim is a human capacity rather than a system function, and which treats slowness as valuable only when people choose it.
Eduardo Levy Yeyati, Darío Judzik, Juan Ignacio Barbiero
Generative AI changes the scarcity of different forms of human capital. It makes codified knowledge cheaply available at the point of production, while judgment acquired through practice remains tied more closely to the worker. We ask what this does to two familiar prices in the wage structure: the return to schooling and the return to experience. In a short-run framework, machine-supplied codified knowledge substitutes directly for the codified component of worker human capital, lowering the schooling gradient. If codified and practice-based knowledge are complements, the return to experience rises where learning through practice is fastest; technology-specific experience can also become obsolete, rotating the experience profile rather than shifting it uniformly. Using US Current Population Survey earnings data from 2018 through June 2026, we compare occupations in the top and bottom quartiles of the pre-ChatGPT Felten-Raj-Seamans exposure index. After the public release of ChatGPT, the return to a year of schooling falls by 0.55 log points in exposed occupations. The return to a year of experience rises by 0.26 log points at three years of potential experience, is unchanged at fifteen, and falls by 0.12 log points at twenty-five; under the baseline quadratic, the effect crosses zero at 18.1 years. These breaks appear with the release, disappear under a pandemic placebo, survive a common censoring rule across the full sample, and remain when each occupation is allowed its own pre-shock schooling and experience gradients. The pattern is a repricing of human capital inside exposed occupations: codifiable knowledge becomes less scarce, while the value of experience depends increasingly on whether it transfers to the new technology.
Large language models (LLMs) are increasingly deployed in conversational settings where user phrasing ranges from polite to casual to toxic, yet less is known about whether such lexical variation in otherwise semantically equivalent prompts can degrade factual reliability. We study how lexical and tone-based prompt perturbations affect the factual reliability of LLMs. Using controlled prompt variations spanning polite, random, and increasingly toxic levels, we evaluate seven LLMs on ARC-Easy, GSM8K, and MMLU. We find that lexical perturbations reduce factual accuracy and increase predictive uncertainty in most models. Interestingly, increasing toxicity results in decreasing accuracy. To examine whether these answer inconsistencies correspond to internal changes, we conduct attribution-graph analyses of model activations and influences. We find that perturbations amplify perturbation-sensitive variant nodes relative to core reasoning nodes. We also observed a monotonic trend with increasing toxicity score and increasing activation and influence in Variant nodes. These findings position prompt wording and tone as a critical dimension of LLM reliability and provide behavioral and mechanistic evidence that surface-level lexical variation can alter factual outputs and internal computation.
As people increasingly rely on large language models (LLMs) for information, model providers need to communicate meaningful estimates of whether individual responses are correct or, more generally, reliable. In the context of hallucinations, existing work uses multicalibration to make such reliability predictions meaningful across application-relevant groups, such as prompt topics. Topic groups alone, however, can conceal systematic errors: within the same topic and confidence level, responses with different chain-of-thought (CoT) properties may exhibit opposing errors that cancel on average. In this work, we introduce reasoning-aware multicalibration, a framework that supplements application-relevant topic groups with groups derived from observable properties of CoT traces. We highlight that, theoretically, groupings based on CoT are "internally important", i.e., groupings for which the multicalibration guarantee is not societally crucial as it is for high-risk topics, but whose inclusion in the multicalibration procedure could easily and significantly improve predictive accuracy of the final reliability predictor. Empirically, we apply our framework to hallucination prediction and unsafe-response detection across a variety of benchmarks, and find that adding CoT groups to topic-based calibration reduces Brier score in both tasks. Interestingly, we find that multicalibrating with respect to CoT groups alone (without any topic information) already improves calibration across topics, suggesting that the predictor's errors depend on how the model reasoned, rather than what it was asked about in the prompt.