NeuML/pubmedbert-base-embeddings-2M

`NeuML/pubmedbert-base-embeddings-2M` powered by Hugging Face Inference Toolkit - [Original Model Card](https://huggingface.co/NeuML/pubmedbert-base-embeddings-2M) - [`feature-extraction` Task on Hug…

azureFoundry/HuggingFace/neuml-pubmedbert-base-embeddings-2m@1

NeuML/pubmedbert-base-embeddings-2M powered by Hugging Face Inference Toolkit

Send Request

You can use cURL or any REST Client to send a request to the AzureML endpoint with your AzureML token.

curl <AZUREML_ENDPOINT_URL> \
    -X POST \
    -H "Authorization: Bearer <AZUREML_TOKEN>" \
    -H "Content-Type: application/json" \
    -d '{"inputs":"Today is a sunny day and I will get some ice cream."}'

Supported Parameters

  • inputs (string or string[]): Either the text to create the embeddings for, or a list of texts.
  • normalize (boolean): Whether to normalize the embedding to be generated or not.
  • prompt_name (string): The name of the prompt that should be used by for encoding. If not set, no prompt will be applied. Must be a key in the sentence-transformers configuration prompts dictionary. For example if prompt_name is "query" and the prompts is {"query": "query: ", …}, then the sentence "What is the capital of France?" will be encoded as "query: What is the capital of France?" because the prompt text will be prepended before any text to encode.
  • truncate (boolean): Whether to truncate the input to match the max allowed input tokens or not.
  • truncation_direction (enum): Possible values: Left, Right.

Check the full API Specification at the Hugging Face Inference documentation.

Specifications & capabilities

Input
text
Output
embeddings
Published model ID
pubmedbert-base-embeddings-2M
Capabilities
embeddings

Evaluations

1–9 / 9

MTEB

33.005%

source id
mteb
leaderboard name
MTEB Clustering
dimension
embedding
metric
Average task score
rank
507
sample size
2
aggregation scope
splits
  • test
subsets
  • default
task_count
2
task_names
  • MedrxivClusteringP2P.v2
  • MedrxivClusteringS2S.v2
sample_size
2
subset_count
1
subset_sample
  • default
aggregation_method
mean-of-task-means
model_revision_count
1
model_revision_sample
  • 1d7bbe04d6713e425161146bfdc71473cbed498a
source model id
NeuML/pubmedbert-base-embeddings-2M
Original result ↗

MTEB

5.705%

source id
mteb
leaderboard name
MTEB Reranking
dimension
embedding
metric
Average task score
rank
560
sample size
1
aggregation scope
splits
  • test
subsets
  • default
task_count
1
task_names
  • CMedQAv2-reranking
sample_size
1
subset_count
1
subset_sample
  • default
aggregation_method
mean-of-task-means
model_revision_count
1
model_revision_sample
  • 1d7bbe04d6713e425161146bfdc71473cbed498a
source model id
NeuML/pubmedbert-base-embeddings-2M
Original result ↗

MTEB

3.691%

source id
mteb
leaderboard name
MTEB Semantic Similarity
dimension
embedding
metric
Average task score
rank
571
sample size
2
aggregation scope
splits
  • test
  • validation
subsets
  • default
task_count
1
task_names
  • STSB
sample_size
2
subset_count
1
subset_sample
  • default
aggregation_method
mean-of-task-means
model_revision_count
1
model_revision_sample
  • 175c20e200ec8780103bb998e1018df0b704a64b
source model id
NeuML/pubmedbert-base-embeddings-2M
Original result ↗

MTEB

5.381%

source id
mteb
leaderboard name
MTEB Clustering
dimension
embedding
metric
Average task score
rank
699
sample size
4
aggregation scope
splits
  • test
subsets
  • default
task_count
4
task_names
  • CLSClusteringP2P
  • CLSClusteringS2S
  • ThuNewsClusteringP2P
  • ThuNewsClusteringS2S
sample_size
4
subset_count
1
subset_sample
  • default
aggregation_method
mean-of-task-means
model_revision_count
1
model_revision_sample
  • 175c20e200ec8780103bb998e1018df0b704a64b
source model id
NeuML/pubmedbert-base-embeddings-2M
Original result ↗

MTEB

25.267%

source id
mteb
leaderboard name
MTEB Other
dimension
embedding
metric
Average task score
rank
764
sample size
46
aggregation scope
splits
  • all
  • dentistry_and_oral_health
  • dermatology
  • gastroenterology
  • genetics
  • neuroscience_and_neurology
  • orthopedic_surgery
  • otorhinolaryngology
  • plastic_surgery
  • psychiatry_and_psychology
  • pulmonology
  • test
subsets
  • arabic
  • chinese
  • default
  • en
  • english
  • es
  • fr
  • french
  • korean
  • russian
  • spanish
  • vietnamese
task_count
7
task_names
  • CUREv1
  • NFCorpus
  • PublicHealthQA
  • SciFact
  • SciFact-PL
  • TRECCOVID
  • TRECCOVID-PL
sample_size
46
subset_count
12
subset_sample
  • arabic
  • chinese
  • default
  • en
  • english
  • es
  • fr
  • french
  • korean
  • russian
  • spanish
  • vietnamese
aggregation_method
mean-of-task-means
model_revision_count
1
model_revision_sample
  • 1d7bbe04d6713e425161146bfdc71473cbed498a
source model id
NeuML/pubmedbert-base-embeddings-2M
Original result ↗

MTEB

23.65%

source id
mteb
leaderboard name
MTEB Other
dimension
embedding
metric
Average task score
rank
778
sample size
17
aggregation scope
splits
  • test
  • validation
subsets
  • default
task_count
14
task_names
  • AFQMC
  • ATEC
  • BQ
  • Cmnli
  • IFlyTek
  • JDReview
  • LCQMC
  • MultilingualSentiment
  • Ocnli
  • OnlineShopping
  • PAWSX
  • QBQTC
  • TNews
  • Waimai
sample_size
17
subset_count
1
subset_sample
  • default
aggregation_method
mean-of-task-means
model_revision_count
1
model_revision_sample
  • 175c20e200ec8780103bb998e1018df0b704a64b
source model id
NeuML/pubmedbert-base-embeddings-2M
Original result ↗

MTEB

17.333%

source id
mteb
leaderboard name
MTEB Retrieval
dimension
embedding
metric
Average task score
rank
823
sample size
2
aggregation scope
splits
  • dev
  • test
subsets
  • default
task_count
2
task_names
  • CmedqaRetrieval
  • MedicalQARetrieval
sample_size
2
subset_count
1
subset_sample
  • default
aggregation_method
mean-of-task-means
model_revision_count
1
model_revision_sample
  • 1d7bbe04d6713e425161146bfdc71473cbed498a
source model id
NeuML/pubmedbert-base-embeddings-2M
Original result ↗

MTEB

23.604%

source id
mteb
leaderboard name
MTEB Overall
dimension
embedding
metric
Average task score
rank
933
sample size
51
aggregation scope
splits
  • all
  • dentistry_and_oral_health
  • dermatology
  • dev
  • gastroenterology
  • genetics
  • neuroscience_and_neurology
  • orthopedic_surgery
  • otorhinolaryngology
  • plastic_surgery
  • psychiatry_and_psychology
  • pulmonology
  • test
subsets
  • arabic
  • chinese
  • default
  • en
  • english
  • es
  • fr
  • french
  • korean
  • russian
  • spanish
  • vietnamese
task_count
12
task_names
  • CMedQAv2-reranking
  • CUREv1
  • CmedqaRetrieval
  • MedicalQARetrieval
  • MedrxivClusteringP2P.v2
  • MedrxivClusteringS2S.v2
  • NFCorpus
  • PublicHealthQA
  • SciFact
  • SciFact-PL
  • TRECCOVID
  • TRECCOVID-PL
sample_size
51
subset_count
12
subset_sample
  • arabic
  • chinese
  • default
  • en
  • english
  • es
  • fr
  • french
  • korean
  • russian
  • spanish
  • vietnamese
aggregation_method
mean-of-task-means
model_revision_count
1
model_revision_sample
  • 1d7bbe04d6713e425161146bfdc71473cbed498a
source model id
NeuML/pubmedbert-base-embeddings-2M
Original result ↗

MTEB

18.754%

source id
mteb
leaderboard name
MTEB Overall
dimension
embedding
metric
Average task score
rank
956
sample size
23
aggregation scope
splits
  • test
  • validation
subsets
  • default
task_count
19
task_names
  • AFQMC
  • ATEC
  • BQ
  • CLSClusteringP2P
  • CLSClusteringS2S
  • Cmnli
  • IFlyTek
  • JDReview
  • LCQMC
  • MultilingualSentiment
  • Ocnli
  • OnlineShopping
  • PAWSX
  • QBQTC
  • STSB
  • TNews
  • ThuNewsClusteringP2P
  • ThuNewsClusteringS2S
  • Waimai
sample_size
23
subset_count
1
subset_sample
  • default
aggregation_method
mean-of-task-means
model_revision_count
1
model_revision_sample
  • 175c20e200ec8780103bb998e1018df0b704a64b
source model id
NeuML/pubmedbert-base-embeddings-2M
Original result ↗

Provider specifications & access

1–1 / 1

NeuML/pubmedbert-base-embeddings-2M ↗

source model id
HuggingFace/neuml-pubmedbert-base-embeddings-2m@1
hosting
foundry-managed-compute
platforms
  • Microsoft Foundry
license
apache-2.0
architecture
input_modalities
  • text
output_modalities
  • embeddings
capabilities

    Original sources

    1–5 / 5

    Hugging Face · NeuML/pubmedbert-base-embeddings-2M ↗

    Microsoft Foundry · Model page ↗

    Microsoft Foundry · Model documentation ↗

    Hugging Face · Catalog ↗

    Microsoft Foundry · Catalog ↗

    Source links

    1–2 / 2

    Explore more models

    Models

    Loading
    EVIDENCE / COMPARISON

    Benchmarks

    Explore every benchmark

    Loading

    These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

    Install TensorTrace