A 10-year-old CNN outperforms frontier LLMs for skin cancer diagnosis

A ten-year-old convolutional neural network published by Google in 2015 outperformed two of the most capable multimodal artificial intelligence systems available in October 2026 when diagnosing biopsy-proven skin lesions, according to lynchlab.net's report. In an evaluation conducted by Dr. Magnus Lynch, a Consultant Dermatologist and Mohs Surgeon at Guy's Hospital and Honorary Reader, a fine-tuned Inception v3 model matched against OpenAI's GPT-6 Astra and Anthropic's Claude Fable 5.1 achieved a 62 percent diagnostic accuracy across 200 difficult test cases. The two frontier large language models correctly diagnosed only 42 percent of the same lesions, illustrating persistent clinical shortcomings in generalist multimodal models applied to dermatology.

This report examines the comparative benchmark methodology, individual diagnostic breakdowns across benign and malignant categories, and the clinical implications for medical triage. It details how specialized legacy computer vision pipelines trained on curated medical archives compare directly with trillion-parameter generalist architectures when interpreting clinical and dermoscopic photography. The analysis covers the specific failure modes observed across all three models and assesses what these findings mean for clinicians, software engineers, and health technology developers evaluating multimodal models for clinical decision support.

A 10-year-old CNN outperforms frontier LLMs: what actually changed

The benchmark tested all three models on 200 biopsy-proven lesions drawn from MILK10k, a public dataset released in 2025 through the International Skin Imaging Collaboration. Each lesion in the sample had two matched photographs: a clinical close-up and an illuminated magnifying lens dermoscopic image. The test set contained 100 lesions that should not be discharged, comprising 30 melanomas, 25 basal cell carcinomas, 25 squamous cell carcinomas including Bowen's disease, and 20 actinic keratoses. It also included 100 lesions safe to discharge, split evenly between 50 benign naevi and 50 seborrhoeic keratoses. Because every lesion in the dataset had been biopsied, the benign cases represented borderline abnormalities that had troubled human clinicians enough to warrant surgical removal.

The fine-tuned Inception v3 network diagnosed 125 of the 200 lesions correctly on its first choice, representing a 62 percent accuracy rate. GPT-6 Astra achieved 84 correct diagnoses, or 42 percent, while Claude Fable 5.1 correctly identified 85 lesions, or 42 percent. When allowing the models a second choice, Inception v3 correctly identified 173 of the 200 lesions, compared to 134 for GPT-6 Astra and 138 for Claude Fable 5.1. On the critical task of identifying invasive melanomas, the CNN found 13 of 30 as its first choice, while Claude Fable 5.1 caught 11 and GPT-6 Astra identified 8. In one test case featuring an invasive melanoma under 0.8 millimeters thick on the ear, both frontier language models diagnosed a harmless seborrhoeic keratosis and recommended discharging the patient, whereas the CNN identified the melanoma.

A 10-year-old CNN outperforms frontier LLMs: how it works

The researcher deliberately avoided using a state-of-the-art vision architecture for the dedicated benchmark. The tested CNN utilized Google's Inception v3 architecture, initially pretrained on standard everyday photographic datasets and then fine-tuned across all layers for approximately twenty minutes on a single Nvidia H100 GPU. Fine-tuning used 68,011 dermoscopic and clinical images taken from the public ISIC Archive. The 200 test lesions and all other photographs associated with the same lesions or patients were scrubbed from the training and validation sets before training began. The resulting model is lightweight enough to run locally inside a mobile web browser via WebGPU without uploading patient imagery to an external server.

Inference differed substantially between the specialized pipeline and the frontier language models. The CNN evaluated both the clinical photograph and the dermoscopic view, combined the two scores while applying the majority of the weight to the dermoscopic capture, and classified the lesion into one of six diagnoses. GPT-6 Astra and Claude Fable 5.1 received the identical image pair alongside default clinical prompts without prior fine-tuning, returning a first diagnosis, a numerical confidence level, a second choice, and a discharge decision. Neither language model refused a case. Both language models demonstrated pronounced systematic classification biases: GPT-6 Astra labeled 75 lesions as basal cell carcinoma when only 25 actually were, and Claude Fable 5.1 labeled 53 lesions as actinic keratosis when only 20 were present.

A 10-year-old CNN outperforms frontier LLMs: why it matters now

The results demonstrate the stark performance gap between generalist vision-language models and specialized image-classification models fine-tuned on targeted domain data. General-purpose multimodal models are widely promoted for automated medical intake and visual diagnostics, but their uncalibrated priors can introduce severe safety risks. The largest diagnostic accuracy gap occurred on benign lesions: the CNN correctly classified 37 of 50 moles and 26 of 50 seborrhoeic keratoses, whereas GPT-6 Astra identified 16 moles and 13 seborrhoeic keratoses, and Claude Fable 5.1 identified 33 moles and 6 seborrhoeic keratoses. By misclassifying safe lesions as malignant conditions, frontier language models introduce high false-positive rates that could overwhelm clinical dermatology services.

The benchmark also highlighted critical trade-offs between diagnostic classification accuracy and conservative triage thresholds. In terms of clinical triage, GPT-6 Astra retained 97 of the 100 unsafe lesions and Claude Fable 5.1 retained 96, discharging 3 and 4 melanomas respectively. Inception v3 discharged 18 melanomas under its default preset threshold while keeping 77 unsafe lesions, though when its classification threshold was recalibrated to match the frontier models, it kept 98 or 99 unsafe lesions while discharging only 1 or 2 melanomas. Conversely, when asked from a patient perspective using solely a clinical photograph with the question of whether to consult a doctor, both language models advised seeking clinical care in nearly every instance, flagging 99 and 98 of a separate 100 biopsy-proven melanomas.

A 10-year-old CNN outperforms frontier LLMs: who is affected

Dermatologists, general practitioners, and clinical workflow architects face direct consequences from these diagnostic discrepancies. Unspecialized frontier language models deployed directly in clinical workflows risk generating high volumes of incorrect secondary diagnoses or recommending inappropriate discharges on difficult, pigment-deficient melanomas. In one test case involving an amelanotic melanoma under 0.8 millimeters thick, GPT-6 Astra classified the lesion as a basal cell carcinoma with 95 percent confidence. While all three models successfully diagnosed a clear basal cell carcinoma on the trunk of a 75-year-old man, the failure of frontier models to recognize early-stage invasive melanomas on dermoscopy underscores the hazards of relying on unverified multimodal chat endpoints.

Health system administrators and software developers building diagnostic tools must consider compute and regulatory boundaries when selecting model architectures. Running frontier commercial models through proprietary remote application programming interfaces requires continuous data transmission and high operational costs. By contrast, specialized neural networks like Inception v3 can run client-side on consumer smartphones using WebGPU, preserving patient privacy and delivering superior categorical diagnostic accuracy without cloud dependencies. Regulated medical dermatology products typically rely on ensembles of modern vision transformers trained on extensive private datasets, which significantly exceed Inception v3 baseline performance while avoiding the hallucinated certainty observed in generalist language models.

A 10-year-old CNN outperforms frontier LLMs: what to watch

Researchers and clinicians are monitoring whether future multimodal foundational models can close the performance gap on domain-specific dermoscopic evaluations. The study's author noted several operational caveats regarding the trial: the sample size of 200 cases and 30 melanomas creates statistical uncertainty, the test images represented exclusively hard, biopsy-selected lesions where the CNN had an area under the receiver operating characteristic curve of 0.86 compared to 0.96 on ordinary archive images, and language model outputs can vary between individual runs. Because MILK10k was released publicly in 2025, the evaluated language models had potential prior training exposure to the images, whereas the CNN was strictly isolated from them.

The open release of evaluation materials provides an independent verification path for artificial intelligence developers. Dr. Lynch published the test cases, prompts, full model response logs, and Python scoring scripts alongside a browser-executable WebGPU build of the fine-tuned Inception v3 model. Independent research teams are expected to test newer vision transformers, specialized medical fine-tunes, and chain-of-thought clinical prompting strategies against the same benchmark. The central metric to observe going forward is whether general multimodal models can match dedicated domain-specific vision ensembles on fine-grained dermoscopic patterns without exhibiting systematic diagnostic skew.