226 papers
  1. #0002 OpenReview
  2. #0004 OpenReview
  3. #0007 OpenReview
  4. #0011 OpenReview
  5. #0014 OpenReview
  6. #0018 OpenReview
  7. #0019 OpenReview
  8. #0020 OpenReview
  9. #0021 OpenReview
  10. #0024 OpenReview
  11. #0027 OpenReview
  12. #0030 OpenReview
  13. #0032 OpenReview
  14. #0038 OpenReview
  15. #0041
    Metrics That Write Themselves: Evolving an Evaluator from Its Own Blind Spots Xing Zhang, Yanwei CUI, Guanghui Wang, Zhihao Lin, Peiyang He
    OpenReview
  16. #0045 OpenReview
  17. #0048 OpenReview
  18. #0051 OpenReview
  19. #0054 OpenReview
  20. #0056 OpenReview
  21. #0057
    Q/R/M/C: An Intervention-Validated Evaluation Protocol for Memory-Based Agents Zamotaev Rodion Sergeevich, Natalia Gusarova, Ivan Tomilov, Aleksandra Vatian
    OpenReview
  22. #0060 OpenReview
  23. #0062 OpenReview
  24. #0064 OpenReview
  25. #0069 OpenReview
  26. #0070
    Detecting Contaminated Code-Generation Prompt Batches via Influence Functions Francesco Quinzan, Noor Munir, Yishun Lu, S Roberts
    OpenReview
  27. #0077 OpenReview
  28. #0079
    Is Generative Content Ready to Ship? A Production-Scale Playbook Malay Haldar, Yu Jin, Chin Hui Chew, Hongwei Zhang, Nalini Kartha, Winnie Xiong, Soumyadip Banerjee, Totte Harinen, Aditya Mukherji, Kedar Bellare, Phanindra Ganti, Stephanie Moyerman, Sanjeev Katariya
    OpenReview
  29. #0081 OpenReview
  30. #0085
    It's Not What the Image Shows: Irrelevant Context Destabilises VLM Judges Without Informing Them Nagham Omar, Mahmoud Jabarin, Kinan Ibraheem, Lotem Peled-Cohen
    OpenReview
  31. #0086 OpenReview
  32. #0090 OpenReview
  33. #0091 OpenReview
  34. #0093 OpenReview
  35. #0095 OpenReview
  36. #0096
    LLM judges are easier to talk down than up: a Bayesian item response analysis Guoyang Zheng, Susan Wei, Sevvandi Kandanaarachchi
    OpenReview
  37. #0097 OpenReview
  38. #0099
    Frozen LLMs as Recommenders for AutoML Pipeline Selection: Can We Trust the Evaluation? Roman Neruda, Juan Carlos Figueroa Garcia, Carlos Alberto Franco Franco
    OpenReview
  39. #0100 OpenReview
  40. #0101
    Conditional Evaluation of Language Models with Cheap Auxiliary Signals Zhi Zhang, Lingfeng Lyu, Yue Kang, Doudou Zhou
    OpenReview
  41. #0102 OpenReview
  42. #0103
    Certified Rank-Stability: Risk-Controlling Leaderboard Claims Under Distribution Shift ADITYA UPADHYAY, Arpan Verma, Chandan Sah, Dev Utkarsh Pal, Harshit Gupta
    OpenReview
  43. #0108 OpenReview
  44. #0109
    Evaluating the Evaluators: Investigating LLM Judges for Personalized Writing Style Assessment Natalie Mackraz, Andrew Silva, Aini Putkonen, Brihi Joshi, Joelle Alcaidinho, Arnav Arora, Barry-John Theobald, Katherine Metcalf
    OpenReview
  45. #0110
    Detecting and Mitigating Non-Deterministic Answer Flips for Reliable LLM Inference Qitao Tan, Joymallya Chakraborty, Navaneet K L, Kushal Kumar, Nicolas Dormont, Vidit Bansal, Abhishek Dan
    OpenReview
  46. #0111 OpenReview
  47. #0112
    Label-free detection and mitigation of judge bias Paul Albert, Quoc Phong Nguyen, Julien Monteil
    OpenReview
  48. #0115
    Defining Operational Conditions for Safety-Critical AI-Based Systems from Data Johann Maximilian Christensen, Elena Hoemann, Frank Köster, Sven Hallerbach
    OpenReview
  49. #0116 OpenReview
  50. #0119
    Overwhelmed by Choice: Studying LLM Decision Making at Scale Yu-Chi Lin, Aryan Seth, Anshul Aravind, Eugene Lee, Tanmay Parekh, Nanyun Peng, Kai-Wei Chang
    OpenReview
  51. #0120 OpenReview
  52. #0121 OpenReview
  53. #0125 OpenReview
  54. #0126
    Voice or Stereotype? Disentangling Acoustic and Content-Based Gender in Speech-to-Speech Models Xiaoqun Liu, Tanu Mitra, Harshit Rajgarhia, Abhishek Mukherji
    OpenReview
  55. #0129 OpenReview
  56. #0132
    BODYBENCH: Evaluating Adversarial Image Defenses Against AI Nudification Inpainting Li Qiwei, Salma Abdel Magid, Olga Russakovsky, Eric Gilbert, Sarita Schoenebeck
    OpenReview
  57. #0133
    Metric Match: A Subset Selection Approach to Evaluating LLM Judge Reliability Alyssa Unell, Natalie Dullerud, Naomi Boneh, Meena Jagadeesan, Tatsunori Hashimoto, Nigam Shah, Sanmi Koyejo
    OpenReview
  58. #0134
    Test Accuracy Conflates Learned Weights with Normalization State Sarthak Tayal, Suayb S. Arslan, Emirhan Inan
    OpenReview
  59. #0135 OpenReview
  60. #0137
    Efficient Language Model Evaluation Using Latent-Space Stratification Samuel Myren, Kurtis Shuler, Grant David Meadors, Emily Taylor
    OpenReview
  61. #0140 OpenReview
  62. #0149
    Evaluating Deception, Not Just Deepfakes Jessee Ho, Shweta Khushu, Shaina Raza
    OpenReview
  63. #0150 OpenReview
  64. #0151 OpenReview
  65. #0152 OpenReview
  66. #0155
    Faithful Dual-constrained Erasure for Robust LLM Safety Alignment Jiaqing Li, Shide Zhou, Zhibo Zhang, Yuxi Li, Tianlong Yu, Kailong Wang
    OpenReview
  67. #0156
    SIR: Self-improving Red-teaming for Compute Use Agents Chen Xiong, Zhiyuan He, Pin-Yu Chen, Stjepan Picek, Tsung-Yi Ho
    OpenReview
  68. #0158 OpenReview
  69. #0161
    Generalization Is Stability, Not Accuracy: Multi-Axis Evaluation of LLMs Nagham Omar, Mahmoud Jabarin, Maya Rozenshtein, Rom Himelstein, Avi Mendelson, Amit LeVi
    OpenReview
  70. #0162 OpenReview
  71. #0164
    Towards a Practical Understanding of On-Policy Lagrangian Safe Reinforcement Learning Lindsay Spoor, Alvaro Serra-Gomez, Aske Plaat, Thomas M. Moerland
    OpenReview
  72. #0165 OpenReview
  73. #0166
    Beyond Accuracy: On Measuring Logical Compliance of Predictive Models Guillaume Delplanque, Pierre Geneves, Nabil Layaïda
    OpenReview
  74. #0170
    Is Inter-Seed Cross-Play Enough? Assessing the Standard Evaluation Protocol for Zero-Shot Coordination Maksymilian Wolski, Nicholas Stephen Hoernle, Johannes Forkel, Jakob Nicolaus Foerster
    OpenReview
  75. #0171
    Mind the Tokenizer: Why a Single Space Decides What Multiple-Choice Evaluations Measure Prabhu Teja S, Frank Schneider, Martin Reinhardt, Fabien C. Y. Benureau, Thomas F Burns
    OpenReview
  76. #0173 OpenReview
  77. #0177
    BenchBench-Protocol: Evaluating Real-World Wet-Lab Protocol Reasoning and Modification Aditya Sivakumar, Nithin Parsan, Ashu Singhal, Nicholas Larus-Stone
    OpenReview
  78. #0178
    PoisonForge: Task-Level Targeted Poisoning Benchmark for Instruction-Tuned LLMs Luze Sun, Anshuman Suri, Harsh Chaudhari, Cristina Nita-Rotaru, Alina Oprea
    OpenReview
  79. #0180
    Do FID Evaluations Bias Generative Models Away from Statistical Relevance? Saumya Goyal, Dhruv Garg, Divyan Goyal, Barnabas Poczos
    OpenReview
  80. #0182
    Evolving LLM-Generated Features for Interpretable Classification Jack Butler, Zainab Afolabi, Nikita Kozodoi
    OpenReview
  81. #0183 OpenReview
  82. #0186 OpenReview
  83. #0187 OpenReview
  84. #0188 OpenReview
  85. #0192
    Benchmark Contamination: A Taxonomy Organized by Defeated Mitigation Johanna Angulo, Víctor Yeste, Hector Espinos-Morato
    OpenReview
  86. #0194
    Training LLMs to Verbalize Evaluation Awareness Usman Anwar, Sahar Abdelnabi, David Krueger
    OpenReview
  87. #0198
    Support-Set Target Leakage in Relational Foundation Models during In-Context Learning: Model Dependence and Evaluation Reliability Roshan Reddy Upendra, Alexandre Dorais, Joe Meyer, Andrew Pouret, Anastasios Lambrianos Stappas, Dinesh Katupputhur Ramprasath, Viswanath Ganapathy, Tom Palczewski, Minghua Li
    OpenReview
  88. #0202 OpenReview
  89. #0206
    COMPREHEND: Ultra-Fine-Grained Benchmark Understanding by Entailment Hierarchy Sivaraaman Balakrishnan, Prakriti Shankar Shetty, Jeremiah Hong, Hamed Zamani, Haw-Shiuan Chang
    OpenReview
  90. #0208
    Architecture-Induced Rank Reversal in Out-of-Distribution Detection Mustakim Ahmed Hasan, Ishtiak Mahmud Saad, Mominul Islam
    OpenReview
  91. #0209 OpenReview
  92. #0215 OpenReview
  93. #0217
    Speculative Evaluation of Stochastic LLMs Qianli Shen, Xiang Li, Ruomeng Ding, Yanxi Chen, Daoyuan Chen, Yaliang Li
    OpenReview
  94. #0218
    Measuring Stability via Minimal Context Interventions Erel Hadad, Rom Himelstein, Amit LeVi, Brit Youngmann, Avi Mendelson
    OpenReview
  95. #0220 OpenReview
  96. #0221 OpenReview
  97. #0224
    Visual Fairness Misalignment: AI Models Prefer People Based on How They Look Mostufa Jbareen, Amit LeVi, Rom Himelstein, Avi Mendelson, Brit Youngmann
    OpenReview
  98. #0225
    EvidenceFlip: Counterfactual Evidence Audits for LLM Judges in Digital Forensics Arash Mahboubi, Yue Wang, Duoyi Zhang, Seyit Camtepe
    OpenReview
  99. #0233 OpenReview
  100. #0234 OpenReview
  101. #0235
    LLM Judges Verify Presence, Not Absence Dr Sebastian Fox, Luke Markham, Ryan Lail, Michael Karotsieris
    OpenReview
  102. #0237 OpenReview
  103. #0239 OpenReview
  104. #0240
    Evaluation Metrics for Safe Reinforcement Learning Lindsay Spoor, Aske Plaat, Thomas M. Moerland
    OpenReview
  105. #0247 OpenReview
  106. #0250 OpenReview
  107. #0251
    Strength Is Not Soundness: Measuring the Exploitability of LLM Policies in Imperfect-Information Games Benjamin Regazzoni, Corentin Caillaud, Rui Miguel Azevedo Leitão
    OpenReview
  108. #0267 OpenReview
  109. #0268
    Ask the Model: Counterfactual Cue Reliance Predicts Worst-Group Failure and Targets Repair Prakhar Gupta, Terry Jingchen Zhang, Florent Draye, Bernhard Schölkopf, Zhijing Jin
    OpenReview
  110. #0269 OpenReview
  111. #0271 OpenReview
  112. #0272 OpenReview
  113. #0278
    DatedGPT: Preventing Lookahead Bias in Large Language Models with Time-Aware Pretraining Yutong Yan, Raphael Tang, Zhenyu Gao, Wenxi Jiang, Yao Lu
    OpenReview
  114. #0280 OpenReview
  115. #0281 OpenReview
  116. #0283 OpenReview
  117. #0284 OpenReview
  118. #0286
    What Do Compliance Detectors Read? An Audit of Activation Probes and Guard Models Saisab Sadhu, Aadit Sengupta, Vinay kumar Sankarapu, Pratinav Seth
    OpenReview
  119. #0288 OpenReview
  120. #0289 OpenReview
  121. #0295
    Guardrail erosion in domain fine-tuned large language models: A paired audit of knowledge-centric abstention in biomedical LLMs Stephen Peng, Vineet Jagadeesan Nair, Noah Broestl, Steven Mills, Anne Friederike Kleppe, Adi Zolotov
    OpenReview
  122. #0298
    PyMETA: Evaluating Student Code Diagnosis on and Beyond the First Execution Error Ziqi Tang, Chuyue Li, Jingyi Wang, Yu Wu, Kazuma Hashimoto, Lingyu Gao
    OpenReview
  123. #0300
    What Do Evaluation Awareness Metrics Measure Hugh Van Deventer, Laura Ruis, Joshua Joseph
    OpenReview
  124. #0301 OpenReview
  125. #0302 OpenReview
  126. #0305 OpenReview
  127. #0306 OpenReview
  128. #0308 OpenReview
  129. #0309
    Benchmarking Exploration Hacking Propensity in Frontier LLMs Eduard Hübner, Nina Żukowska, Alexander Panfilov, Joschka Braun
    OpenReview
  130. #0310
    PriorBPB: Reducing the Answer Length Confound of Bits-per-Byte While Improving Accuracy Prediction Thomas F Burns, Prabhu Teja S, Frank Schneider, Martin Reinhardt
    OpenReview
  131. #0311
    VerifierContractBench: Auditing the Safety Validity of Optimized Agent Evaluation Lakshya Narula, Saurabh Tripathi, Lily Bharati Sharma, Nikhil Rao
    OpenReview
  132. #0312 OpenReview
  133. #0315 OpenReview
  134. #0316
    Evaluating Agentic AI in Transportation: A Multidimensional Framework and Deterministic Stress Test for Traffic-Simulation Workflows Xiangyong Luo, Wan Li, Guanhao Xu, haowen xu, Abhilasha Jairam Saroj, Jinghui Yuan, Ross Wang
    OpenReview
  135. #0317 OpenReview
  136. #0318
    Stubborn or Sycophantic? Auditing What Sycophancy Evaluations Measure HanRui Zhang, Ashton Chew, Ryan Soe, Ahmed Taha, Ruizhe Li, Aditya Shah, Maheep Chaudhary
    OpenReview
  137. #0321
    Dutch Books for Language Models Isaiah Andrews, Suproteem K Sarkar
    OpenReview
  138. #0322 OpenReview
  139. #0323 OpenReview
  140. #0324
    Your Benchmark Is an Empirical Measure Over Difficulty Yifan Sun, Naicheng Yu, Jiawen Gong, Jingyan Shen, Huan Zhang
    OpenReview
  141. #0326
    How Much Can Repeated Running Buy? A Measured Variance Ceiling for LLM Code Evaluation Yongxi Zhou, Lai Yun Choi, Jiaxi Wen, Wenbo Ye, Junwei Yao
    OpenReview
  142. #0327 OpenReview
  143. #0328 OpenReview
  144. #0329
    CoBe: Counterfactual Benchmark for Text Editing Arvind Raghavan, Inwoo Hwang, Shreyas Havaldar, Kai-Zhan Lee, Aurghya Maiti, Yushu Pan, Jeffrey Wu, Mingxuan Li, Elias Bareinboim
    OpenReview
  145. #0332
    A Study of LLM Judge Capability-Dependent Biases and Multi-Judge Ensemble for Bias Calibration Gemma Zhang, Prachi Badarayani, Asmi Kumar, Sadid A. Hasan, Sulaiman Vesal
    OpenReview
  146. #0334 OpenReview
  147. #0337
    ECHO: Diagnosing Spatial Cue Access in Audio-Language Models Daniel Huang, Alvin Shen, Aarush Rachakonda, Aryan Shrivastava, Kyle Xu
    OpenReview
  148. #0341
    When the Metric Lies: Silent Evaluation Failures in RL Exploration Seyed Ali Alavi, Mohaddeseh Jahanabadi, Dustin Dannenhauer
    OpenReview
  149. #0342
    LLM-Generated Investment Research Memos Need Validation Chenri Luo, Xukang Gong, BINYAN YU, Xiao-Yang Liu
    OpenReview
  150. #0343 OpenReview
  151. #0344 OpenReview
  152. #0346 OpenReview
  153. #0350 OpenReview
  154. #0352 OpenReview
  155. #0353 OpenReview
  156. #0356
    PTXBench: Evaluating LLMs for GPU Kernel Optimization with Architecture-Specific PTX Genghan Zhang, Yixin Dong, Chengze Fan, Zhichen Zeng, Yueming Yuan, Shaowei Zhu, Kunle Olukotun
    OpenReview
  157. #0358
    Reading the Room, or Reading the Prompt? Measurement Fragility in Verbalized Evaluation-Awareness Neta Ravid, Daniel Dahl Juhl, Matthew Gregory, Guy Nachshon, Jacob Haimes
    OpenReview
  158. #0361
    Auditing Language-to-Action Control in Driving VLAs Aradhya Goel, Bhoomika Gupta, Muhammed Ustaomeroglu, Guannan Qu
    OpenReview
  159. #0362
    Asymptotically Log-Optimal Bayes-Assisted Confidence Sequences for Bounded Means Valentin Kilian, Stefano Cortinovis, Francois Caron
    OpenReview
  160. #0366 OpenReview
  161. #0367 OpenReview
  162. #0368 OpenReview
  163. #0369 OpenReview
  164. #0372 OpenReview
  165. #0375 OpenReview
  166. #0376
    Assessing the Gameability of Vision Language Model Judges for World Model Video Evaluation James Kim, Rahul Bellam, Joseph Le Nevez, Neil Porwal, Vasu Sharma, Nicholas Babey
    OpenReview
  167. #0377 OpenReview
  168. #0382 OpenReview
  169. #0383 OpenReview
  170. #0384
    LLM-as-Environment: Benchmarking Open-Ended Decision Making with Language-Model Simulators Caroline Choi, Neil Band, Nick Rui, Sandra Yang, Zeyneb N. Kaya, Ludwig Schmidt, Tengyu Ma
    OpenReview
  171. #0385
    The Camera, the Split, and the Label: A Three-Check Audit of Glaucoma-Progression Prediction Hao Li, Jared Joselowitz, Zachary Ellis, Yuxiang Zhou, Samuel Thio, Aisling Higham, Yajie Vera He
    OpenReview
  172. #0386
    Partition, Prompt, Aggregate: Statistical Self-Consistency in Language Models Patrik Wolf, Thomas Kleine Buening, Andreas Krause, Celestine Mendler-Dünner
    OpenReview
  173. #0387
    Policy Is Not Capability: Some Safety Benchmarks Measure Which Definition of "Unsafe" You Trained On Vahid Jebraeeli, Jonathon Romero, Yijin Wang, Adrian Chan, Jianbang Zhang
    OpenReview
  174. #0389
    Mean Accuracy Is the Wrong Estimand for a Release Decision Jaewon Choi, Jihye Kang, Namhyuk Ahn
    OpenReview
  175. #0390 OpenReview
  176. #0391 OpenReview
  177. #0395 OpenReview
  178. #0399
    Auditing and Enforcing Calendar-Shift Invariance in Clinical Prediction Models Wenhao Maxwell Ren, Everest Yang, Valarie Tran, Igor Shuryak
    OpenReview
  179. #0401 OpenReview
  180. #0402 OpenReview
  181. #0403 OpenReview
  182. #0404 OpenReview
  183. #0409 OpenReview
  184. #0410 OpenReview
  185. #0411 OpenReview
  186. #0412 OpenReview
  187. #0413 OpenReview
  188. #0418
    Stress-Testing RL-based Capability Elicitation with Randomized Underperformance Nina Żukowska, Eduard Hübner, Alexander Panfilov, Joschka Braun
    OpenReview
  189. #0423 OpenReview
  190. #0424 OpenReview
  191. #0428
    SeqPAC: PAC-Certified Labeling with Sequential Ensembles Qianyu Julie Zhu, Ashwin Chandramouli, Youssef Marzouk, Suleiman A. Khan, Fatemeh Mansoori
    OpenReview
  192. #0430 OpenReview
  193. #0444 OpenReview
  194. #0448 OpenReview
  195. #0450 OpenReview
  196. #0452 OpenReview
  197. #0453 OpenReview
  198. #0454 OpenReview
  199. #0455 OpenReview
  200. #0456
    Auditing VLM Benchmarks from Existing Evaluation Runs Denis Degtiarev, Anna Gromova, Sergey Katkov, Artem Mikhailin, Safina Mukhamedshina, Albert Nasybullin, Karina Shaikhutdinova, Artem Tuzov
    OpenReview
  201. #0458 OpenReview
  202. #0459 OpenReview
  203. #0461 OpenReview
  204. #0463 OpenReview
  205. #0465
    Epistemic Exams Loren K. Fryxell, Kola Ayonrinde
    OpenReview
  206. #0469 OpenReview
  207. #0470 OpenReview
  208. #0474 OpenReview
  209. #0475 OpenReview
  210. #0477
    Mind the Mic Tsz Chai Alan Liu
    OpenReview
  211. #0481
    Measuring Definition Sensitivity in LLM Annotation of Social Science Concepts Bogdan Mamaev, Qizhou Wang, Matteo Vergani, Christopher Leckie
    OpenReview
  212. #0482 OpenReview
  213. #0483 OpenReview
  214. #0485 OpenReview
  215. #0486 OpenReview
  216. #0487
    The Probability Term in Motion Forecasting Benchmarks Is Not a Calibration Score Jaewoong Yoo, Ganzorig Gankhuyag, Jin-Man Park, Haengseon Son, Kyoungwon Min
    OpenReview
  217. #0488 OpenReview
  218. #0498 OpenReview
  219. #0499 OpenReview
  220. #0500
    Can We Trust Small Language Models as Judges? Suchir Salhan, Mingchuan Zou, Paula Buttery
    OpenReview
  221. #0501
    Shortcut Sensitivity in Agent-Safety Evaluations Hasan Demirkiran, Jens Ernstberger
    OpenReview
  222. #0502 OpenReview
  223. #0504
    MemoReason: Evaluating the Effect of Parametric Memory on Contextual Reasoning in LLMs Zineddine Tighidet, Andrea Mogini, Jiali Mei, Patrick Gallinari, Benjamin Piwowarski
    OpenReview
  224. #0506 OpenReview
  225. #0513 OpenReview
  226. #0514 OpenReview