Yuchi Wang, Rongyao Fang, Weihao Yu, Xilin Wei, Tiancheng Gu, Dingkang Yang, Hang Zhang, Dayiheng Liu 等 10 位作者
Beyond basic tool use and coding skills, effectively incorporating visual references into agentic generation is equally essential for applying AI agents to real-world productivity tasks. We study this crucial capability through webpage development, a practical and readily evaluable setting in which existing benchmarks remain largely idealized: they typically focus on reconstructing visual targets or generating webpages from textual instructions alone, overlooking the more common workflow of developing from visual examples. Specifically, we introduce RefCraft, a benchmark comprising 300 realistic multimodal webpage-development tasks that combine natural-language instructions with screenshots from one to five reference webpages. Agents must selectively interpret and utilize these references while rendering, visually inspecting, and iteratively refining their outputs. We evaluate the resulting open-ended artifacts using independently verifiable keypoints and complementary programmatic and VLM judges. Across nine frontier (M)LLMs, the strongest system achieves an overall score of only 52.12 out of 100, revealing that visual-reference-conditioned agentic generation remains a substantial challenge for current models in practical webpage-development settings.
Medical Image Quality Assessment (IQA) serves as a critical safety gate for clinical AI, yet existing approaches rely on scalar metrics that fail to capture the descriptive, expert-like evaluation process central to clinical practice. We introduce MedQ-Bench, a comprehensive benchmark for language-based medical IQA evaluation with MLLMs, establishing a perception--description paradigm across 5 imaging modalities and over 40 quality attributes. The benchmark comprises two complementary tasks: MedQ-Perception probes low-level visual attribute discrimination via 2,600 expert-annotated perceptual queries; MedQ-Description covers both no-reference and comparison description sub-tasks with 708 expert-annotated samples, evaluated through a four-dimensional judging protocol. Comprehensive evaluation of 23 state-of-the-art MLLMs reveals that current models exhibit only preliminary and unstable capabilities, and highlights the urgent need for targeted optimization of MLLMs in medical IQA. Project & Dataset: https://github.com/liujiyaoFDU/MedQ-Bench