NeuML/pubmedbert-base-embeddings-8M

`NeuML/pubmedbert-base-embeddings-8M` powered by Hugging Face Inference Toolkit - [Original Model Card](https://huggingface.co/NeuML/pubmedbert-base-embeddings-8M) - [`feature-extraction` Task on Hug…

azureFoundry/HuggingFace/neuml-pubmedbert-base-embeddings-8m@1

NeuML/pubmedbert-base-embeddings-8M powered by Hugging Face Inference Toolkit

Send Request

You can use cURL or any REST Client to send a request to the AzureML endpoint with your AzureML token.

curl <AZUREML_ENDPOINT_URL> \
    -X POST \
    -H "Authorization: Bearer <AZUREML_TOKEN>" \
    -H "Content-Type: application/json" \
    -d '{"inputs":"Today is a sunny day and I will get some ice cream."}'

Supported Parameters

  • inputs (string or string[]): Either the text to create the embeddings for, or a list of texts.
  • normalize (boolean): Whether to normalize the embedding to be generated or not.
  • prompt_name (string): The name of the prompt that should be used by for encoding. If not set, no prompt will be applied. Must be a key in the sentence-transformers configuration prompts dictionary. For example if prompt_name is "query" and the prompts is {"query": "query: ", …}, then the sentence "What is the capital of France?" will be encoded as "query: What is the capital of France?" because the prompt text will be prepended before any text to encode.
  • truncate (boolean): Whether to truncate the input to match the max allowed input tokens or not.
  • truncation_direction (enum): Possible values: Left, Right.

Check the full API Specification at the Hugging Face Inference documentation.

Specifications & capabilities

Input
text
Output
embeddings
Published model ID
pubmedbert-base-embeddings-8M
Capabilities
embeddings

Evaluations

1–9 / 9

MTEB

32.047%

source id
mteb
leaderboard name
MTEB Clustering
dimension
embedding
metric
Average task score
rank
528
sample size
2
aggregation scope
splits
  • test
subsets
  • default
task_count
2
task_names
  • MedrxivClusteringP2P.v2
  • MedrxivClusteringS2S.v2
sample_size
2
subset_count
1
subset_sample
  • default
aggregation_method
mean-of-task-means
model_revision_count
1
model_revision_sample
  • 387d350015e963744f4fafe56a574b7cd48646c9
source model id
NeuML/pubmedbert-base-embeddings-8M
Original result ↗

MTEB

6.185%

source id
mteb
leaderboard name
MTEB Reranking
dimension
embedding
metric
Average task score
rank
559
sample size
1
aggregation scope
splits
  • test
subsets
  • default
task_count
1
task_names
  • CMedQAv2-reranking
sample_size
1
subset_count
1
subset_sample
  • default
aggregation_method
mean-of-task-means
model_revision_count
1
model_revision_sample
  • 387d350015e963744f4fafe56a574b7cd48646c9
source model id
NeuML/pubmedbert-base-embeddings-8M
Original result ↗

MTEB

3.656%

source id
mteb
leaderboard name
MTEB Semantic Similarity
dimension
embedding
metric
Average task score
rank
574
sample size
2
aggregation scope
splits
  • test
  • validation
subsets
  • default
task_count
1
task_names
  • STSB
sample_size
2
subset_count
1
subset_sample
  • default
aggregation_method
mean-of-task-means
model_revision_count
1
model_revision_sample
  • b6a0de4964b05d69fea61532cd6ae1241a3ee9fd
source model id
NeuML/pubmedbert-base-embeddings-8M
Original result ↗

MTEB

5.86%

source id
mteb
leaderboard name
MTEB Clustering
dimension
embedding
metric
Average task score
rank
699
sample size
4
aggregation scope
splits
  • test
subsets
  • default
task_count
4
task_names
  • CLSClusteringP2P
  • CLSClusteringS2S
  • ThuNewsClusteringP2P
  • ThuNewsClusteringS2S
sample_size
4
subset_count
1
subset_sample
  • default
aggregation_method
mean-of-task-means
model_revision_count
1
model_revision_sample
  • b6a0de4964b05d69fea61532cd6ae1241a3ee9fd
source model id
NeuML/pubmedbert-base-embeddings-8M
Original result ↗

MTEB

30.604%

source id
mteb
leaderboard name
MTEB Other
dimension
embedding
metric
Average task score
rank
744
sample size
46
aggregation scope
splits
  • all
  • dentistry_and_oral_health
  • dermatology
  • gastroenterology
  • genetics
  • neuroscience_and_neurology
  • orthopedic_surgery
  • otorhinolaryngology
  • plastic_surgery
  • psychiatry_and_psychology
  • pulmonology
  • test
subsets
  • arabic
  • chinese
  • default
  • en
  • english
  • es
  • fr
  • french
  • korean
  • russian
  • spanish
  • vietnamese
task_count
7
task_names
  • CUREv1
  • NFCorpus
  • PublicHealthQA
  • SciFact
  • SciFact-PL
  • TRECCOVID
  • TRECCOVID-PL
sample_size
46
subset_count
12
subset_sample
  • arabic
  • chinese
  • default
  • en
  • english
  • es
  • fr
  • french
  • korean
  • russian
  • spanish
  • vietnamese
aggregation_method
mean-of-task-means
model_revision_count
1
model_revision_sample
  • 387d350015e963744f4fafe56a574b7cd48646c9
source model id
NeuML/pubmedbert-base-embeddings-8M
Original result ↗

MTEB

23.654%

source id
mteb
leaderboard name
MTEB Other
dimension
embedding
metric
Average task score
rank
779
sample size
17
aggregation scope
splits
  • test
  • validation
subsets
  • default
task_count
14
task_names
  • AFQMC
  • ATEC
  • BQ
  • Cmnli
  • IFlyTek
  • JDReview
  • LCQMC
  • MultilingualSentiment
  • Ocnli
  • OnlineShopping
  • PAWSX
  • QBQTC
  • TNews
  • Waimai
sample_size
17
subset_count
1
subset_sample
  • default
aggregation_method
mean-of-task-means
model_revision_count
1
model_revision_sample
  • b6a0de4964b05d69fea61532cd6ae1241a3ee9fd
source model id
NeuML/pubmedbert-base-embeddings-8M
Original result ↗

MTEB

23.049%

source id
mteb
leaderboard name
MTEB Retrieval
dimension
embedding
metric
Average task score
rank
795
sample size
2
aggregation scope
splits
  • dev
  • test
subsets
  • default
task_count
2
task_names
  • CmedqaRetrieval
  • MedicalQARetrieval
sample_size
2
subset_count
1
subset_sample
  • default
aggregation_method
mean-of-task-means
model_revision_count
1
model_revision_sample
  • 387d350015e963744f4fafe56a574b7cd48646c9
source model id
NeuML/pubmedbert-base-embeddings-8M
Original result ↗

MTEB

27.551%

source id
mteb
leaderboard name
MTEB Overall
dimension
embedding
metric
Average task score
rank
909
sample size
51
aggregation scope
splits
  • all
  • dentistry_and_oral_health
  • dermatology
  • dev
  • gastroenterology
  • genetics
  • neuroscience_and_neurology
  • orthopedic_surgery
  • otorhinolaryngology
  • plastic_surgery
  • psychiatry_and_psychology
  • pulmonology
  • test
subsets
  • arabic
  • chinese
  • default
  • en
  • english
  • es
  • fr
  • french
  • korean
  • russian
  • spanish
  • vietnamese
task_count
12
task_names
  • CMedQAv2-reranking
  • CUREv1
  • CmedqaRetrieval
  • MedicalQARetrieval
  • MedrxivClusteringP2P.v2
  • MedrxivClusteringS2S.v2
  • NFCorpus
  • PublicHealthQA
  • SciFact
  • SciFact-PL
  • TRECCOVID
  • TRECCOVID-PL
sample_size
51
subset_count
12
subset_sample
  • arabic
  • chinese
  • default
  • en
  • english
  • es
  • fr
  • french
  • korean
  • russian
  • spanish
  • vietnamese
aggregation_method
mean-of-task-means
model_revision_count
1
model_revision_sample
  • 387d350015e963744f4fafe56a574b7cd48646c9
source model id
NeuML/pubmedbert-base-embeddings-8M
Original result ↗

MTEB

18.856%

source id
mteb
leaderboard name
MTEB Overall
dimension
embedding
metric
Average task score
rank
957
sample size
23
aggregation scope
splits
  • test
  • validation
subsets
  • default
task_count
19
task_names
  • AFQMC
  • ATEC
  • BQ
  • CLSClusteringP2P
  • CLSClusteringS2S
  • Cmnli
  • IFlyTek
  • JDReview
  • LCQMC
  • MultilingualSentiment
  • Ocnli
  • OnlineShopping
  • PAWSX
  • QBQTC
  • STSB
  • TNews
  • ThuNewsClusteringP2P
  • ThuNewsClusteringS2S
  • Waimai
sample_size
23
subset_count
1
subset_sample
  • default
aggregation_method
mean-of-task-means
model_revision_count
1
model_revision_sample
  • b6a0de4964b05d69fea61532cd6ae1241a3ee9fd
source model id
NeuML/pubmedbert-base-embeddings-8M
Original result ↗

Provider specifications & access

1–1 / 1

NeuML/pubmedbert-base-embeddings-8M ↗

source model id
HuggingFace/neuml-pubmedbert-base-embeddings-8m@1
hosting
foundry-managed-compute
platforms
  • Microsoft Foundry
license
apache-2.0
architecture
input_modalities
  • text
output_modalities
  • embeddings
capabilities

    Original sources

    1–5 / 5

    Hugging Face · NeuML/pubmedbert-base-embeddings-8M ↗

    Microsoft Foundry · Model page ↗

    Microsoft Foundry · Model documentation ↗

    Hugging Face · Catalog ↗

    Microsoft Foundry · Catalog ↗

    Source links

    1–2 / 2

    Explore more models

    Models

    Loading
    EVIDENCE / COMPARISON

    Benchmarks

    Explore every benchmark

    Loading

    These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

    Install TensorTrace