Gemini 2.5 Pro

Gheorghe Comanici, E. Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit S. Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, Luke Marris, Sam Petulla, Colin Gaffney, A. Aharoni, Nathan Lintz, T. C. Pais, Henrik Jacobsson, I. Szpektor, Nan-Jiang Jiang, Krishna Haridasan, Ahmed Omran, Nikunj Saunshi, Dara Bahri, Gaurav Mishra, Eric Chu, Toby Boyd, Brad Hekman, Aaron Parisi, Chaoyi Zhang, Kornraphop Kawintiranon, Tania Bedrax-Weiss, Oliver Wang, Ya Xu, Ollie Purkiss, Uri Mendlovic, Ilai Deutel, Nam Nguyen, A. Langley, Flip Korn, Lucia Rossazza, Alexandre Ramé, Sagar Waghmare, Helen Miller, Vaishakh Keshava, Ying Jian, Xiaofan Zhang, R. A. Popa, Kedar Dhamdhere, Blavz Bratanivc, Kyuyeun Kim, Terry Koo, Ferran Alet, Yi-Ting Chen, Arsha Nagrani, Hannah Muckenhirn, Zhiyuan Zhang, Corbin Quick, Filip Paveti'c, D. Nguyen, João Carreira, Michael Elabd, Ha-roon Qureshi, Fabian Mentzer, Yao Yang, Danielle Eisenbud, Anmol Gulati, Ellie Talius, Eric Ni, Sahra Ghalebikesabi, Edouard Yvinec, Alaa Saade, Thatcher Ulrich, Lorenzo Blanco, Dan A. Calian, Muhua Huang, Aäron van den Oord, Naman Goyal, Terry Chen, Praynaa Rawlani, Christian Schallhart, Swachhand Lokhande, Xianghong Luo, Jyn Shan, Ceslee Montgomery, Victoria Krakovna, Federico Piccinini, Omer Barak, Jingyu Cui, Yi-Ling Jia, Mikhail Dektiarev, A. Kolganov, Shi-Yu Huang, Zhe Chen, Xingyu Wang, Jessica Austin, Peter de Boursac, Evgeny Sluzhaev, F. Ding, Hui-Jian Li, Surya Bhupatiraju, M. Agarwal, Slawek Kwasiborski, Paramjit Sandhu, Patrick Siegler, Ahmet Iscen, Eyal Ben-David, Shiraz Butt, Miltos Allamanis, Seth Benjamin, R. Busa-Fekete, Félix Hernández-Campos, S. Goldshtein, Matt Dibb, Weiyan Zhang, Annie Marsden, Carey Radebaugh, Stephen Roller, Abhishek Nayyar, Jacob Austin, Tayfun Terzi, Bhargav Kanagal Shamanna, Peter Shaw, Aayush Singh, Florian Luisier, Artur Mendoncca, V. Aggarwal, L. Markeeva, Claudio Fantacci, S. Brin, H. Choe, Guan-Yu Wang, Hartwig Adam, Avigail Dabush, T. Kiyono, E. Marcus, Jeremy R. Cole, T. Weber, Hongrae Lee, Ronny Huang, Alex Muzio, Leandro Kieliger, Maigo Le, Courtney Biles, Long Le, Archit Sharma, Chengrun Yang, Avery Lamp, Dave Dopson, N. Hurley, Katrina Xu, Zhihao Shan, Shuang Song, Jiewen Tan, Alexandre Senges, G. Zhang, Chong You, Yennie Jun, David Raposo, Susanna Ricco, Xuan Yang, Weijie Chen, Prakhar Gupta, A. Szlam, Kevin Villela, C. Ferng, Daniel Kasenberg, Chen Liang, Rui Zhu, Arunachalam Narayanaswamy, Florence Perot, Paul Pucciarelli, A. Shekhawat, A. Stern, Rishikesh Ingale, Stefani Karp, Sanaz Bahargam, Adrian Goedeckemeyer, Jie Han, Sicheng Li, A. Tacchetti, Dian Yu, A. Chakladar, Zhiying Zhang, Mona Mahdy, Xu Gao, Dale S. Johnson, Samrat Phatale, A. Piergiovanni, Hyeontaek Lim, Clément Farabet, C. Lebsack, Theo Guidroz, John Blitzer, Nico Duduta, David Madras, Steven Li, D. V. Dincklage, X. Li, Mahdis Mahdieh, George Tucker, Ganesh Jawahar, Yunxuan Owen Xiao, Daniel Tarlow, Robert Geirhos, Noam Velan, Daniel Vlasic, Kalesha Bullard, Sk Park, Nishesh Gupta, Kellie Webster, Ayal Hitron, Jieming Mao, J. Eisenschlos, Laurel Prince, Nina D'Souza, K. Zheng, Sara Nasso, Gabriela Botea, Carl Doersch, Çağlar Ünlü, Chris Alberti, A. Svyatkovskiy, Ankit Goel, Krzysztof Choromanski, Pan-Pan Jiang, R. Nguyen, Four Flynn, Daria Ćurko, Peter Chen, Nicholas Roth, Kieran Milan, Caleb Habtegebriel, Shashi Narayan, M. Moffitt, Jake Marcus, T. Anthony, Brendan McMahan, Gowoon Cheon, Ruibo Liu, Megan Barnes, Lukasz Lew, Re-beca Santamaria-Fernandez, M. Upadhyay, A. Akula, A. M. Hrafnkelsson, A. Caceres, Andrew Bunner, Michal Sokolik, Subha Puttagunta, L. Moore, Berivan Isik, J. Hartford, Lawrence Chan, P. Shenoy, D. Holtmann-Rice, Jane Park, Fabio Viola, Alexandru Salcianu, Sujeevan Rajayogam, Ian Stewart-Binks, Zelin Wu, Richard Everett, Xi Xiong, Pierre-Antoine Manzagol, Gary Leung, Carl Saroufim, Bo Pang, Dawid Wegner, G. Papamakarios, J. Palomaki, Helena Pankov, Guangda Lai, G. Tubone, Shubin Zhao, T. Strinopoulos, Seth Neel, Ming-Qiu Wang, Joe Kelley, Li Li, Ping-mei Xu, Anitha Vijayakumar, Andrea D'olimpio, Omer Levy, Massimo Nicosia, Grigory Rozhdestvenskiy, Ni Lao, Si-Rui Xie, Yash Katariya, Jon Simon, Sanjiv Kumar, Florian Hartmann, M. Kilgore, Jinhyuk Lee, Aroma Mahendru, Roman Ring, T. Hennigan, Fiona Lang, Colin Cherry, David Steiner, Dawsen Hwang, Ray Smith, Pi-Dong Wang, Jeremy Chen, Ming-Hsuan Yang, S. Kwei, Philippe Schlattner, Donnie Kim, Ganesh Poomal Girirajan, Nikola Momchev, Ayushi Agarwal, Xingyi Zhou, Ilkin Safarli, Zachary Garrett, AJ Pierigiovanni, Sarthak Jauhari, Alif Raditya Rochman, Shikhar Vashishth, Quan Yuan, Christof Angermueller, Jon Blanton, Xiny-ing Song, N. B. Gundavarapu, Thi Avrahami, Maxine Deines, Subhrajit Roy, Manish Gupta, Christopher Semturs, Shobha Vasudevan, Aditya Srikanth Veerubhotla, Shriya Sharma, Joshy Jacob, Zhen Yang, Andreas Terzis, Dan Karliner, Auriel Wright, Tania Rojas-Esponda, Ashley Brown, A. Roy, Pawan Dogra, A. Kapishnikov, Peter Young, W. Kan, Vinodh K. Rajendran, M. Ivanova, S. Deshmukh, Chia-Hua Ho, Michael Kwong, S. Ginzburg, Annie Louis, KP Sawhney, Slav Petrov, Jing Xie, Yun-Fei Bai, G. Stoyanov, Alex Fabrikant, Rajesh Jayaram, Yu-Qi Li, Joseph Heyward, Justin Gilmer, Yaqing Wang, Radu Soricut, Lu Liu, Qing-ping Duan, Jamie Hayes, Maura O'Brien, Gau-rav Singh Tomar, Sivan Eiger, Bahare Fatemi, Jeffrey Hui, Catarina Barros, A. Chukwuka, Alena Butryna, Saksham Thakur, Austin Huang, Zhufeng Pan, Haotian Tang, Serkan Cabi, Tulsee Doshi, Michiel A. Bakker, S. Bagri, Ruy Ley-Wild, Á. Lelkes, J. Lees, P. Kane, David Greene, Shimu Wu, J. Bornschein, G. Surita, Sarah Hodkinson, Fangtao Li, Chris Hidey, Sébastien Pereira, Sean Ammirati, Phillip Lippe, Adam Kraft, Pu Han, S. Gerlach, Zifeng Wang, Liviu Panait, Feng Han, B. Farris, Y. Bi, Hannah DeBalsi, Miaosen Wang, Gladys Tyen, James Cohan, Susan Zhang, Jarred Barber, D. Chung, Jaeyoung Kim, M. Kunesch, S. Pecht, Nami Akazawa, Abe Friesen, James Lyon, Ali Eslami, Jun-Ru Wu, Jiewen Tan, Yue Song, Ravin Kumar, Christoper A. Welty, I. Akol'zin, Gena Gibson, Sean Augenstein, Arjun Pillai, N. Yuen, Du Phan, Xin Wang, Iain Barr, H. Zen, Nan Hua, Casper Liu, Jilei Wang, T. Bhatia, Hao Xu, Oded Elyada, Pushmeet Kohli, Miroslav Olvs'ak, Kelly Chen, Azalia Mirhoseini, Noam Shazeer, Shoshana Jakobovits, Maggie Tran, Nolan Ramsden, T. Bharti, Fred Alcober, Yunjie Li, S. Shetty, Jing Chen, Dmitry Kalashnikov, Megha Nawhal, Sercan Ö. Arik, Hanwen Chen, M. Blokzijl, Shubham Gupta, J. Rubin, Rigel Swavely, Sophie Bridgers, I. Gemp, Chenlin Su, A. Suggala, Juliette Pluto, Mary Cassin, A. Vaucher, Kaiyang Ji, Jiahao Cai, Andrew Audibert, Animesh Sinha, David Tian, E. Farkash, Amy Hua, Jilin Chen, Duc Tran, E. Loper, Nicole Brichtova, Lara McConnaughey, Ballie Sandhu, Robert Leland, Doug DeCarlo, A. Over, J. Huang, Xing Wu, C. Fan, Eric Li, Yun-Peng Lei, Deepak Sharma, Cosmin Paduraru, Luo Yu, Matko Bovsnjak, Phuong Q. Dao, Min Choi, Sneha Kudugunta, Jakub Adamek, C. Guia, Ali Khodaei, Jie Feng, Wenjun Zeng, David Welling, Sandeep Tata, Christina Butterfield

In this report, we introduce the Gemini 2.X model family: Gemini 2.5 Pro and Gemini 2.5 Flash, as well as our earlier Gemini 2.0 Flash and Flash-Lite models. Gemini 2.5 Pro is our most capable model yet, achieving SoTA performance on frontier coding and reasoning benchmarks. In addition to its incredible coding and reasoning skills, Gemini 2.5 Pro is a thinking model that excels at multimodal understanding and it is now able to process up to 3 hours of video content. Its unique combination of long context, multimodal and reasoning capabilities can be combined to unlock new agentic workflows. Gemini 2.5 Flash provides excellent reasoning abilities at a fraction of the compute and latency requirements and Gemini 2.0 Flash and Flash-Lite provide high performance at low latency and cost. Taken together, the Gemini 2.X model generation spans the full Pareto frontier of model capability vs cost, allowing users to explore the boundaries of what is possible with complex agentic problem solving.

Ali Şalbaş, R. Buyuktoka

Background/Objectives: Multimodal large language models (LLMs) are increasingly used in radiology. However, their ability to recognize fundamental imaging features, including modality, anatomical region, imaging plane, contrast-enhancement status, and particularly specific magnetic resonance imaging (MRI) sequences, remains underexplored. This study aims to evaluate and compare the performance of three advanced multimodal LLMs (ChatGPT-4o, Claude 4 Opus, and Gemini 2.5 Pro) in classifying brain MRI sequences. Methods: A total of 130 brain MRI images from adult patients without pathological findings were used, representing 13 standard MRI series. Models were tested using zero-shot prompts for identifying modality, anatomical region, imaging plane, contrast-enhancement status, and MRI sequence. Accuracy was calculated, and differences among models were analyzed using Cochran’s Q test and McNemar test with Bonferroni correction. Results: ChatGPT-4o and Gemini 2.5 Pro achieved 100% accuracy in identifying the imaging plane and 98.46% in identifying contrast-enhancement status. MRI sequence classification accuracy was 97.7% for ChatGPT-4o, 93.1% for Gemini 2.5 Pro, and 73.1% for Claude 4 Opus (p < 0.001). The most frequent misclassifications involved fluid-attenuated inversion recovery (FLAIR) sequences, often misclassified as T1-weighted or diffusion-weighted sequences. Claude 4 Opus showed lower accuracy in susceptibility-weighted imaging (SWI) and apparent diffusion coefficient (ADC) sequences. Gemini 2.5 Pro exhibited occasional hallucinations, including irrelevant clinical details such as “hypoglycemia” and “Susac syndrome.” Conclusions: Multimodal LLMs demonstrate high accuracy in basic MRI recognition tasks but vary significantly in specific sequence classification tasks. Hallucinations emphasize caution in clinical use, underlining the need for validation, transparency, and expert oversight.

Fatma E. A. Hassanein, Radwa R Hussein, H. Almalahy, S. Sarhan, Yousra Ahmed, Asmaa Abou-Bakr

The diagnostic potential of multimodal large language models (LLMs) in oral medicine remains underexplored, particularly in real-world clinical contexts. This study introduces Vision-Based Diagnostic Gain (VWDG) as a novel metric to quantify the incremental diagnostic value of incorporating images into AI-assisted diagnosis of oral lesions. We conducted a prospective, biopsy-validated, case-matched study including 200 oral lesion cases with clinical photographs and radiographs of variable quality. ChatGPT-5 and Gemini 2.5 Pro were evaluated against board-certified oral medicine experts. Each case was presented under two conditions: text-only and multimodal (text plus images). Diagnostic accuracy was measured across Top-1, Top-3, and Top-5 differentials. VWDG was defined as the absolute and relative improvement in diagnostic accuracy between multimodal and text-only conditions. Cochran’s Q and paired McNemar tests with effect sizes quantified differences across models and conditions, with analyses stratified by lesion type and diagnostic difficulty Both models demonstrated strong baseline diagnostic accuracy, but their performance diverged with image integration. ChatGPT-5 achieved significant VWDG across thresholds—Top-1 gain + 19% points, Top-3 gain + 18 pp, and Top-5 gain + 14 pp (all p < 0.001). In contrast, Gemini 2.5 Pro showed negligible or even negative gain (0 pp at Top-1/Top-3; − 2 pp at Top-5). Stratified analyses confirmed that ChatGPT-5 benefited most from visual input in malignant and diagnostically difficult cases, whereas Gemini’s strength remained in text-dominant contexts. Human experts consistently outperformed both models in simple and benign presentations. By introducing and applying VWDG, this study provides the first expert-anchored, head-to-head evaluation of next-generation multimodal LLMs in oral medicine. ChatGPT-5 functions as a visual synergist, Gemini as a textual expert, and their complementary strengths suggest a cooperative human–AI diagnostic paradigm. VWDG offers a clinically meaningful framework for benchmarking AI models and guiding safe, context-aware integration into practice.

J. Spörer

Comprehending long, dense annual reports is a critical task for financial analysts that is ripe for AI automation, yet model reliability remains a key concern. To address this, we introduce Financial Touchstone—a new, large-scale benchmark with 2,878 question-context-answer triplets across 480 international annual reports, guaranteed to be unseen by the models we evaluate. We test eleven frontier language models from leading labs, including reasoning-capable models like Google’s Gemini 2.5 Pro, Anthropic’s Claude Opus, OpenAI’s o3, and xAI’s Grok 4. Our analysis reveals that while reasoning models achieve high accuracy—with Gemini 2.5 Pro reaching 91.6% and hallucination rates as low as 3.2%—the primary bottleneck is not the models’ comprehension but the initial information retrieval step. Model accuracy plummets to 0.2% when the provided context is insufficient. This work demonstrates that future progress in automated financial analysis hinges more on solving the challenge of targeted information retrieval in complex documents than on incremental improvements in model reasoning alone.

Aleksandra Wielochowska, Aleksandra Stachowicz, Melania Olender, Ada Latkowska, Julia Glińska, Dawid Boczkowski, Dominika Radej, Anna Kowalczyk, Weronika Majchrowicz, Tomasz Dolata, Michalina Loson-Kawalec, Piotr Sawina, Wojciech Jaworski, Patrycja Dadyńska

Background: Artificial intelligence (AI) models are developing rapidly, with growing ability to process extensive and up-to-date medical knowledge. This makes them increasingly important as didactic tools, widely used by residents preparing for the Państwowy Egzamin Specjalizacyjny (PES) or the State Specialization Examination. However, concerns remain about the accuracy and reliability of AI-generated answers. Systematic validation is therefore essential, particularly in the context of exam questions, to ensure safe and effective use in medical education. The aim of this study was to assess the educational potential of Gemini 2.5 Pro in solving the PES in gynecology and obstetrics by comparing its responses with the official key and analyzing the declared confidence level for each answer. Materials and methods: This study was designed to empirically verify the ability of the multimodal language model Gemini 2.5 PRO to solve examination tasks at a specialist level. The research material was the PES paper in obstetrics and gynecology (spring session 2025), provided by the Center for Medical Examinations in Łódź. For the final analysis, 119 questions were qualified after one of the tasks was invalidated by the examination committee. During the exam simulation, two parameters were recorded for each question: the consistency of the model's answer with the official key and a subjective assessment of confidence expressed by the model on a 5-point scale. The collected data were used to check whether the model's effectiveness depended on the nature of the question (clinical vs. theoretical) using the chi-squared test, and to assess the correlation between confidence and the correctness of the answer using the Mann-Whitney U test. Results: According to the observations, Gemini 2.5 PRO passed the exam with a score of 96.63%, achieving 115 points. The model's effectiveness was similar, regardless of whether the question concerned a clinical case or theoretical knowledge (p=0.313). We cannot demonstrate that the level of confidence is correlated with the effectiveness of the answer (p=0.064). When comparing the model's confidence level depending on whether the question concerned a clinical case (12 questions) or theoretical knowledge (107 questions), the difference turned out to be statistically insignificant, which means that the level of confidence was similar regardless of the question category (clinical vs. theoretical). Conclusions: The results of the PES exam in gynecology and obstetrics clearly show that the Gemini Pro model achieved high effectiveness. A key observation was that no correlation can be demonstrated between the model's confidence level and the correctness of the answers given. Additionally, no statistically significant difference was shown in the model's confidence level between theoretical and clinical questions. The analysis proves that AI has enormous potential to support specialized education. Despite such spectacular effects, further in-depth research and constant substantive supervision by specialists are necessary to safely integrate AI with teaching programs.