NeuML/pubmedbert-base-embeddings-100K

`NeuML/pubmedbert-base-embeddings-100K` powered by Hugging Face Inference Toolkit - [Original Model Card](https://huggingface.co/NeuML/pubmedbert-base-embeddings-100K) - [`feature-extraction` Task on…

azureFoundry/HuggingFace/neuml-pubmedbert-base-embeddings-100k@1

NeuML/pubmedbert-base-embeddings-100K powered by Hugging Face Inference Toolkit

Send Request

You can use cURL or any REST Client to send a request to the AzureML endpoint with your AzureML token.

curl <AZUREML_ENDPOINT_URL> \
    -X POST \
    -H "Authorization: Bearer <AZUREML_TOKEN>" \
    -H "Content-Type: application/json" \
    -d '{"inputs":"Today is a sunny day and I will get some ice cream."}'

Supported Parameters

  • inputs (string or string[]): Either the text to create the embeddings for, or a list of texts.
  • normalize (boolean): Whether to normalize the embedding to be generated or not.
  • prompt_name (string): The name of the prompt that should be used by for encoding. If not set, no prompt will be applied. Must be a key in the sentence-transformers configuration prompts dictionary. For example if prompt_name is "query" and the prompts is {"query": "query: ", …}, then the sentence "What is the capital of France?" will be encoded as "query: What is the capital of France?" because the prompt text will be prepended before any text to encode.
  • truncate (boolean): Whether to truncate the input to match the max allowed input tokens or not.
  • truncation_direction (enum): Possible values: Left, Right.

Check the full API Specification at the Hugging Face Inference documentation.

Specifications & capabilities

Input
text
Output
embeddings
Published model ID
pubmedbert-base-embeddings-100K
Capabilities
embeddings

Evaluations

1–5 / 5

MTEB

30.38%

source id
mteb
leaderboard name
MTEB Clustering
dimension
embedding
metric
Average task score
rank
550
sample size
2
aggregation scope
splits
  • test
subsets
  • default
task_count
2
task_names
  • MedrxivClusteringP2P.v2
  • MedrxivClusteringS2S.v2
sample_size
2
subset_count
1
subset_sample
  • default
aggregation_method
mean-of-task-means
model_revision_count
1
model_revision_sample
  • bac5e3b12fb8c650e92a19c41b436732c4f16e9e
source model id
NeuML/pubmedbert-base-embeddings-100K
Original result ↗

MTEB

5.581%

source id
mteb
leaderboard name
MTEB Reranking
dimension
embedding
metric
Average task score
rank
563
sample size
1
aggregation scope
splits
  • test
subsets
  • default
task_count
1
task_names
  • CMedQAv2-reranking
sample_size
1
subset_count
1
subset_sample
  • default
aggregation_method
mean-of-task-means
model_revision_count
1
model_revision_sample
  • bac5e3b12fb8c650e92a19c41b436732c4f16e9e
source model id
NeuML/pubmedbert-base-embeddings-100K
Original result ↗

MTEB

11.054%

source id
mteb
leaderboard name
MTEB Other
dimension
embedding
metric
Average task score
rank
809
sample size
46
aggregation scope
splits
  • all
  • dentistry_and_oral_health
  • dermatology
  • gastroenterology
  • genetics
  • neuroscience_and_neurology
  • orthopedic_surgery
  • otorhinolaryngology
  • plastic_surgery
  • psychiatry_and_psychology
  • pulmonology
  • test
subsets
  • arabic
  • chinese
  • default
  • en
  • english
  • es
  • fr
  • french
  • korean
  • russian
  • spanish
  • vietnamese
task_count
7
task_names
  • CUREv1
  • NFCorpus
  • PublicHealthQA
  • SciFact
  • SciFact-PL
  • TRECCOVID
  • TRECCOVID-PL
sample_size
46
subset_count
12
subset_sample
  • arabic
  • chinese
  • default
  • en
  • english
  • es
  • fr
  • french
  • korean
  • russian
  • spanish
  • vietnamese
aggregation_method
mean-of-task-means
model_revision_count
1
model_revision_sample
  • bac5e3b12fb8c650e92a19c41b436732c4f16e9e
source model id
NeuML/pubmedbert-base-embeddings-100K
Original result ↗

MTEB

3.174%

source id
mteb
leaderboard name
MTEB Retrieval
dimension
embedding
metric
Average task score
rank
882
sample size
2
aggregation scope
splits
  • dev
  • test
subsets
  • default
task_count
2
task_names
  • CmedqaRetrieval
  • MedicalQARetrieval
sample_size
2
subset_count
1
subset_sample
  • default
aggregation_method
mean-of-task-means
model_revision_count
1
model_revision_sample
  • bac5e3b12fb8c650e92a19c41b436732c4f16e9e
source model id
NeuML/pubmedbert-base-embeddings-100K
Original result ↗

MTEB

12.506%

source id
mteb
leaderboard name
MTEB Overall
dimension
embedding
metric
Average task score
rank
975
sample size
51
aggregation scope
splits
  • all
  • dentistry_and_oral_health
  • dermatology
  • dev
  • gastroenterology
  • genetics
  • neuroscience_and_neurology
  • orthopedic_surgery
  • otorhinolaryngology
  • plastic_surgery
  • psychiatry_and_psychology
  • pulmonology
  • test
subsets
  • arabic
  • chinese
  • default
  • en
  • english
  • es
  • fr
  • french
  • korean
  • russian
  • spanish
  • vietnamese
task_count
12
task_names
  • CMedQAv2-reranking
  • CUREv1
  • CmedqaRetrieval
  • MedicalQARetrieval
  • MedrxivClusteringP2P.v2
  • MedrxivClusteringS2S.v2
  • NFCorpus
  • PublicHealthQA
  • SciFact
  • SciFact-PL
  • TRECCOVID
  • TRECCOVID-PL
sample_size
51
subset_count
12
subset_sample
  • arabic
  • chinese
  • default
  • en
  • english
  • es
  • fr
  • french
  • korean
  • russian
  • spanish
  • vietnamese
aggregation_method
mean-of-task-means
model_revision_count
1
model_revision_sample
  • bac5e3b12fb8c650e92a19c41b436732c4f16e9e
source model id
NeuML/pubmedbert-base-embeddings-100K
Original result ↗

Provider specifications & access

1–1 / 1

NeuML/pubmedbert-base-embeddings-100K ↗

source model id
HuggingFace/neuml-pubmedbert-base-embeddings-100k@1
hosting
foundry-managed-compute
platforms
  • Microsoft Foundry
license
apache-2.0
architecture
input_modalities
  • text
output_modalities
  • embeddings
capabilities

    Original sources

    1–5 / 5

    Hugging Face · NeuML/pubmedbert-base-embeddings-100K ↗

    Microsoft Foundry · Model page ↗

    Microsoft Foundry · Model documentation ↗

    Hugging Face · Catalog ↗

    Microsoft Foundry · Catalog ↗

    Source links

    1–2 / 2

    Explore more models

    Models

    Loading
    EVIDENCE / COMPARISON

    Benchmarks

    Explore every benchmark

    Loading

    These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

    Install TensorTrace