google/medgemma-4b-it

Explore google/medgemma-4b-it: model specifications, capabilities, benchmark results, platform pricing and original sources.

huggingface/google/medgemma-4b-it

Specifications & capabilities

Published model ID
medgemma@medgemma-4b-it
Capabilities
multimodal, visionInput

Evaluations

1–14 / 14

LLM Stats · GitHub archive

0.889

source id
llmStatsArchive
leaderboard name
MIMIC CXR · Normalized (reported)
dimension
general
metric
MIMIC CXR · Normalized (reported)
rank
1
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
MIMIC CXR
language
en
modality
multimodal
verified
false
max_score
1
categories
  • healthcare
  • vision
  • multimodal
created_at
2025-07-19T19:56:14.017221+00:00
paper_link
https://arxiv.org/abs/1901.07042
updated_at
2025-07-19T19:56:14.017221+00:00
description
MIMIC-CXR is a large publicly available dataset of chest radiographs with free-text radiology reports. Contains 377,110 images corresponding to 227,835 radiographic studies from 65,379 patients at Beth Israel Deaconess Medical Center. The dataset is de-identified and widely used for medical imaging research, automated report generation, and medical AI development.
benchmark_id
mimic-cxr
multilingual
false
analysis_method
Average F1 for top 5 conditions
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
MIMIC CXR · Normalized (reported)
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/google/models/medgemma-4b-it/benchmarks.json
archive_kind
llmStats
evidence_url
https://huggingface.co/google/medgemma-4b-it
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
google/medgemma-4b-it
metric_population
1
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
medgemma-4b-it
Original result ↗

LLM Stats · GitHub archive

0.889

source id
llmStatsArchive
leaderboard name
MIMIC CXR · Raw
dimension
general
metric
MIMIC CXR · Raw
rank
1
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
MIMIC CXR
language
en
modality
multimodal
verified
false
max_score
1
categories
  • healthcare
  • vision
  • multimodal
created_at
2025-07-19T19:56:14.017221+00:00
paper_link
https://arxiv.org/abs/1901.07042
updated_at
2025-07-19T19:56:14.017221+00:00
description
MIMIC-CXR is a large publicly available dataset of chest radiographs with free-text radiology reports. Contains 377,110 images corresponding to 227,835 radiographic studies from 65,379 patients at Beth Israel Deaconess Medical Center. The dataset is de-identified and widely used for medical imaging research, automated report generation, and medical AI development.
benchmark_id
mimic-cxr
multilingual
false
analysis_method
Average F1 for top 5 conditions
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
MIMIC CXR · Raw
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/google/models/medgemma-4b-it/benchmarks.json
archive_kind
llmStats
evidence_url
https://huggingface.co/google/medgemma-4b-it
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
google/medgemma-4b-it
metric_population
1
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
medgemma-4b-it
Original result ↗

LLM Stats · GitHub archive

0.718

source id
llmStatsArchive
leaderboard name
DermMCQA · Raw
dimension
general
metric
DermMCQA · Raw
rank
1
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
DermMCQA
language
en
modality
text
verified
false
max_score
1
categories
  • healthcare
created_at
2025-07-19T19:56:14.024498+00:00
paper_link
https://arxiv.org/abs/2309.06961
updated_at
2025-07-19T19:56:14.024498+00:00
description
Dermatology multiple choice question assessment benchmark for evaluating medical knowledge and diagnostic reasoning in dermatological conditions and treatments.
benchmark_id
dermmcqa
multilingual
false
analysis_method
Accuracy
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
DermMCQA · Raw
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/google/models/medgemma-4b-it/benchmarks.json
archive_kind
llmStats
evidence_url
https://huggingface.co/google/medgemma-4b-it
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
google/medgemma-4b-it
metric_population
1
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
medgemma-4b-it
Original result ↗

LLM Stats · GitHub archive

0.718

source id
llmStatsArchive
leaderboard name
DermMCQA · Normalized (reported)
dimension
general
metric
DermMCQA · Normalized (reported)
rank
1
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
DermMCQA
language
en
modality
text
verified
false
max_score
1
categories
  • healthcare
created_at
2025-07-19T19:56:14.024498+00:00
paper_link
https://arxiv.org/abs/2309.06961
updated_at
2025-07-19T19:56:14.024498+00:00
description
Dermatology multiple choice question assessment benchmark for evaluating medical knowledge and diagnostic reasoning in dermatological conditions and treatments.
benchmark_id
dermmcqa
multilingual
false
analysis_method
Accuracy
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
DermMCQA · Normalized (reported)
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/google/models/medgemma-4b-it/benchmarks.json
archive_kind
llmStats
evidence_url
https://huggingface.co/google/medgemma-4b-it
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
google/medgemma-4b-it
metric_population
1
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
medgemma-4b-it
Original result ↗

LLM Stats · GitHub archive

0.698

source id
llmStatsArchive
leaderboard name
PathMCQA · Normalized (reported)
dimension
general
metric
PathMCQA · Normalized (reported)
rank
1
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
PathMCQA
language
en
modality
multimodal
verified
false
max_score
1
categories
  • healthcare
  • vision
  • multimodal
  • reasoning
created_at
2025-07-19T19:56:14.036453+00:00
paper_link
https://arxiv.org/abs/2401.16355
updated_at
2025-07-19T19:56:14.036453+00:00
description
PathMMU is a massive multimodal expert-level benchmark for understanding and reasoning in pathology, containing 33,428 multimodal multi-choice questions and 24,067 images validated by seven pathologists. It evaluates Large Multimodal Models (LMMs) performance on pathology tasks, with the top-performing model GPT-4V achieving only 49.8% zero-shot performance compared to 71.8% for human pathologists.
benchmark_id
pathmcqa
multilingual
false
analysis_method
Accuracy
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
PathMCQA · Normalized (reported)
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/google/models/medgemma-4b-it/benchmarks.json
archive_kind
llmStats
evidence_url
https://huggingface.co/google/medgemma-4b-it
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
google/medgemma-4b-it
metric_population
1
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
medgemma-4b-it
Original result ↗

LLM Stats · GitHub archive

0.698

source id
llmStatsArchive
leaderboard name
PathMCQA · Raw
dimension
general
metric
PathMCQA · Raw
rank
1
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
PathMCQA
language
en
modality
multimodal
verified
false
max_score
1
categories
  • healthcare
  • vision
  • multimodal
  • reasoning
created_at
2025-07-19T19:56:14.036453+00:00
paper_link
https://arxiv.org/abs/2401.16355
updated_at
2025-07-19T19:56:14.036453+00:00
description
PathMMU is a massive multimodal expert-level benchmark for understanding and reasoning in pathology, containing 33,428 multimodal multi-choice questions and 24,067 images validated by seven pathologists. It evaluates Large Multimodal Models (LMMs) performance on pathology tasks, with the top-performing model GPT-4V achieving only 49.8% zero-shot performance compared to 71.8% for human pathologists.
benchmark_id
pathmcqa
multilingual
false
analysis_method
Accuracy
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
PathMCQA · Raw
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/google/models/medgemma-4b-it/benchmarks.json
archive_kind
llmStats
evidence_url
https://huggingface.co/google/medgemma-4b-it
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
google/medgemma-4b-it
metric_population
1
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
medgemma-4b-it
Original result ↗

LLM Stats · GitHub archive

0.623

source id
llmStatsArchive
leaderboard name
SlakeVQA · Raw
dimension
general
metric
SlakeVQA · Raw
rank
1
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
SlakeVQA
language
en
modality
multimodal
verified
false
max_score
1
categories
  • vision
  • healthcare
  • multimodal
  • reasoning
created_at
2025-07-19T19:56:14.027646+00:00
paper_link
https://arxiv.org/abs/2102.09542
updated_at
2025-07-19T19:56:14.027646+00:00
description
A semantically-labeled knowledge-enhanced dataset for medical visual question answering. Contains 642 radiology images (CT scans, MRI scans, X-rays) covering five body parts and 14,028 bilingual English-Chinese question-answer pairs annotated by experienced physicians. Features comprehensive semantic labels and a structural medical knowledge base with both vision-only and knowledge-based questions requiring external medical knowledge reasoning.
benchmark_id
slakevqa
multilingual
true
analysis_method
Tokenized F1
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
SlakeVQA · Raw
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/google/models/medgemma-4b-it/benchmarks.json
archive_kind
llmStats
evidence_url
https://huggingface.co/google/medgemma-4b-it
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
google/medgemma-4b-it
metric_population
1
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
medgemma-4b-it
Original result ↗

LLM Stats · GitHub archive

0.623

source id
llmStatsArchive
leaderboard name
SlakeVQA · Normalized (reported)
dimension
general
metric
SlakeVQA · Normalized (reported)
rank
1
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
SlakeVQA
language
en
modality
multimodal
verified
false
max_score
1
categories
  • vision
  • healthcare
  • multimodal
  • reasoning
created_at
2025-07-19T19:56:14.027646+00:00
paper_link
https://arxiv.org/abs/2102.09542
updated_at
2025-07-19T19:56:14.027646+00:00
description
A semantically-labeled knowledge-enhanced dataset for medical visual question answering. Contains 642 radiology images (CT scans, MRI scans, X-rays) covering five body parts and 14,028 bilingual English-Chinese question-answer pairs annotated by experienced physicians. Features comprehensive semantic labels and a structural medical knowledge base with both vision-only and knowledge-based questions requiring external medical knowledge reasoning.
benchmark_id
slakevqa
multilingual
true
analysis_method
Tokenized F1
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
SlakeVQA · Normalized (reported)
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/google/models/medgemma-4b-it/benchmarks.json
archive_kind
llmStats
evidence_url
https://huggingface.co/google/medgemma-4b-it
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
google/medgemma-4b-it
metric_population
1
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
medgemma-4b-it
Original result ↗

LLM Stats · GitHub archive

0.499

source id
llmStatsArchive
leaderboard name
VQA-Rad · Normalized (reported)
dimension
general
metric
VQA-Rad · Normalized (reported)
rank
1
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
VQA-Rad
language
en
modality
multimodal
verified
false
max_score
1
categories
  • vision
  • healthcare
  • multimodal
created_at
2025-07-19T19:56:14.031802+00:00
paper_link
https://doi.org/10.1038/sdata.2018.251
updated_at
2025-07-19T19:56:14.031802+00:00
description
VQA-RAD (Visual Question Answering in Radiology) is the first manually constructed dataset of medical visual question answering containing 3,515 clinically generated visual questions and answers about radiology images. The dataset includes questions created by clinical trainees on 315 radiology images from MedPix covering head, chest, and abdominal scans, designed to support AI development for medical image analysis and improve patient care.
benchmark_id
vqa-rad
multilingual
false
analysis_method
Tokenized F1
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
VQA-Rad · Normalized (reported)
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/google/models/medgemma-4b-it/benchmarks.json
archive_kind
llmStats
evidence_url
https://huggingface.co/google/medgemma-4b-it
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
google/medgemma-4b-it
metric_population
1
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
medgemma-4b-it
Original result ↗

LLM Stats · GitHub archive

0.499

source id
llmStatsArchive
leaderboard name
VQA-Rad · Raw
dimension
general
metric
VQA-Rad · Raw
rank
1
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
VQA-Rad
language
en
modality
multimodal
verified
false
max_score
1
categories
  • vision
  • healthcare
  • multimodal
created_at
2025-07-19T19:56:14.031802+00:00
paper_link
https://doi.org/10.1038/sdata.2018.251
updated_at
2025-07-19T19:56:14.031802+00:00
description
VQA-RAD (Visual Question Answering in Radiology) is the first manually constructed dataset of medical visual question answering containing 3,515 clinically generated visual questions and answers about radiology images. The dataset includes questions created by clinical trainees on 315 radiology images from MedPix covering head, chest, and abdominal scans, designed to support AI development for medical image analysis and improve patient care.
benchmark_id
vqa-rad
multilingual
false
analysis_method
Tokenized F1
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
VQA-Rad · Raw
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/google/models/medgemma-4b-it/benchmarks.json
archive_kind
llmStats
evidence_url
https://huggingface.co/google/medgemma-4b-it
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
google/medgemma-4b-it
metric_population
1
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
medgemma-4b-it
Original result ↗

LLM Stats · GitHub archive

0.481

source id
llmStatsArchive
leaderboard name
CheXpert CXR · Normalized (reported)
dimension
general
metric
CheXpert CXR · Normalized (reported)
rank
1
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
CheXpert CXR
language
en
modality
image
verified
false
max_score
1
categories
  • healthcare
  • vision
created_at
2025-07-19T19:56:14.021015+00:00
paper_link
https://arxiv.org/abs/1901.07031
updated_at
2025-07-19T19:56:14.021015+00:00
description
CheXpert is a large dataset of 224,316 chest radiographs from 65,240 patients for automated chest X-ray interpretation. The dataset includes uncertainty labels for 14 medical observations extracted from radiology reports. It serves as a benchmark for developing and evaluating automated chest radiograph interpretation models.
benchmark_id
chexpert-cxr
multilingual
false
analysis_method
Average F1 for top 5 conditions
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
CheXpert CXR · Normalized (reported)
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/google/models/medgemma-4b-it/benchmarks.json
archive_kind
llmStats
evidence_url
https://huggingface.co/google/medgemma-4b-it
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
google/medgemma-4b-it
metric_population
1
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
medgemma-4b-it
Original result ↗

LLM Stats · GitHub archive

0.481

source id
llmStatsArchive
leaderboard name
CheXpert CXR · Raw
dimension
general
metric
CheXpert CXR · Raw
rank
1
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
CheXpert CXR
language
en
modality
image
verified
false
max_score
1
categories
  • healthcare
  • vision
created_at
2025-07-19T19:56:14.021015+00:00
paper_link
https://arxiv.org/abs/1901.07031
updated_at
2025-07-19T19:56:14.021015+00:00
description
CheXpert is a large dataset of 224,316 chest radiographs from 65,240 patients for automated chest X-ray interpretation. The dataset includes uncertainty labels for 14 medical observations extracted from radiology reports. It serves as a benchmark for developing and evaluating automated chest radiograph interpretation models.
benchmark_id
chexpert-cxr
multilingual
false
analysis_method
Average F1 for top 5 conditions
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
CheXpert CXR · Raw
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/google/models/medgemma-4b-it/benchmarks.json
archive_kind
llmStats
evidence_url
https://huggingface.co/google/medgemma-4b-it
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
google/medgemma-4b-it
metric_population
1
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
medgemma-4b-it
Original result ↗

LLM Stats · GitHub archive

0.188

source id
llmStatsArchive
leaderboard name
MedXpertQA · Normalized (reported)
dimension
general
metric
MedXpertQA · Normalized (reported)
rank
1
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
MedXpertQA
language
en
modality
multimodal
verified
false
max_score
1
categories
  • healthcare
  • reasoning
  • multimodal
created_at
2025-07-19T19:56:14.040381+00:00
paper_link
https://arxiv.org/abs/2501.18362
updated_at
2025-07-19T19:56:14.040381+00:00
description
A comprehensive benchmark to evaluate expert-level medical knowledge and advanced reasoning, featuring 4,460 questions spanning 17 specialties and 11 body systems. Includes both text-only and multimodal subsets with expert-level exam questions incorporating diverse medical images and rich clinical information.
benchmark_id
medxpertqa
multilingual
false
analysis_method
Accuracy
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
MedXpertQA · Normalized (reported)
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/google/models/medgemma-4b-it/benchmarks.json
archive_kind
llmStats
evidence_url
https://huggingface.co/google/medgemma-4b-it
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
google/medgemma-4b-it
metric_population
1
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
medgemma-4b-it
Original result ↗

LLM Stats · GitHub archive

0.188

source id
llmStatsArchive
leaderboard name
MedXpertQA · Raw
dimension
general
metric
MedXpertQA · Raw
rank
1
aggregation scope
status
archived
license
CC-BY-4.0
direction
higherIsBetter
conditions
language
en
benchmark
name
MedXpertQA
language
en
modality
multimodal
verified
false
max_score
1
categories
  • healthcare
  • reasoning
  • multimodal
created_at
2025-07-19T19:56:14.040381+00:00
paper_link
https://arxiv.org/abs/2501.18362
updated_at
2025-07-19T19:56:14.040381+00:00
description
A comprehensive benchmark to evaluate expert-level medical knowledge and advanced reasoning, featuring 4,460 questions spanning 17 specialties and 11 body systems. Includes both text-only and multimodal subsets with expert-level exam questions incorporating diverse medical images and rich clinical information.
benchmark_id
medxpertqa
multilingual
false
analysis_method
Accuracy
is_self_reported
true
verified_by_llmstats
false
attribution
Jonathan Chavez Tamales and LLM Stats contributors (copyright 2024 jc); TensorTrace structures original results and derives scoped ranks.
license_url
https://creativecommons.org/licenses/by/4.0/
metric_kind
MedXpertQA · Raw
rank_method
competition rank among reported evaluations in identical conditions, including unlinked observations
source_file
data/organizations/google/models/medgemma-4b-it/benchmarks.json
archive_kind
llmStats
evidence_url
https://huggingface.co/google/medgemma-4b-it
git_revision
872b75f63b8d853c46ed17856424029a0c47b1d2
published_at
2025-10-24T17:47:59Z
snapshot_date
2025-10-24T17:47:59Z
source_revision
6e27aa41f51fa8cb327602349240077d1c7acae9fead8ba2f07db0d549927ddc
publisher_anchor
google/medgemma-4b-it
metric_population
1
current_catalog_url
https://llm-stats.com/
benchmark_evaluation
true
source model id
medgemma-4b-it
Original result ↗

Provider specifications & access

1–1 / 1

MedGemma-4b-it ↗

source model id
publishers/google/models/medgemma@medgemma-4b-it
hosting
vertex-self-deployed
platforms
  • Google Cloud Vertex AI
architecture
input_modalities
  • image
  • text
output_modalities
  • text
capabilities
  • multimodal
  • visionInput

Safety assessments

1–1 / 1

Foundation Model Transparency Index 2025 ↗

41 / 100

source id
stanfordFMTI
canonical model id
huggingface/google/medgemma-4b-it
taxonomy
transparency
hazard category
transparency
hazard name
Organization transparency
policy area
Google
assessment type
organization-level disclosure
metric
disclosure_score
result label
41 / 100
sample size
100
updated at
2026-11-09T18:08:05.227Z

Original sources

1–17 / 17

Hugging Face · google/medgemma-4b-it ↗

Google Cloud · Model page ↗

arXiv · Paper · 2009.13081 ↗

arXiv · Paper · 2102.09542 ↗

arXiv · Paper · 2106.14463 ↗

arXiv · Paper · 2303.15343 ↗

arXiv · Paper · 2404.05590 ↗

arXiv · Paper · 2405.03162 ↗

arXiv · Paper · 2411.15640 ↗

arXiv · Paper · 2412.03555 ↗

arXiv · Paper · 2501.18362 ↗

arXiv · Paper · 2501.19393 ↗

arXiv · Paper · 2507.05201 ↗

GitHub · Catalog ↗

GitHub · Catalog ↗

Google Cloud · Catalog ↗

Hugging Face · Catalog ↗

Source links

1–4 / 4

Hugging Face model card

1–1 / 1

Explore more models

Models

Loading
EVIDENCE / COMPARISON

Benchmarks

Explore every benchmark

Loading

These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

Install TensorTrace