sbunlp/fabert

`sbunlp/fabert` powered by Hugging Face Inference Toolkit - [Original Model Card](https://huggingface.co/sbunlp/fabert) - [`fill-mask` Task on Hugging Face](https://huggingface.co/tasks/fill-mask) ##…

azureFoundry/HuggingFace/sbunlp-fabert@1

sbunlp/fabert powered by Hugging Face Inference Toolkit

Send Request

You can use cURL or any REST Client to send a request to the AzureML endpoint with your AzureML token.

curl <AZUREML_ENDPOINT_URL> \
    -X POST \
    -H "Authorization: Bearer <AZUREML_TOKEN>" \
    -H "Content-Type: application/json" \
    -d '{"inputs":"The answer to the universe is undefined."}'

Supported Parameters

  • inputs (string): The text with masked tokens
  • parameters (object):
    • top_k (integer): When passed, overrides the number of predictions to return.
    • targets (string[]): When passed, the model will limit the scores to the passed targets instead of looking up in the whole vocabulary. If the provided targets are not in the model vocab, they will be tokenized and the first resulting token will be used (with a warning, and that might be slower).

Check the full API Specification at the Hugging Face Inference documentation.

Specifications & capabilities

Published model ID
fabert

Evaluations

1–7 / 7

MTEB

55.689%

source id
mteb
leaderboard name
MTEB Clustering
dimension
embedding
metric
Average task score
rank
121
sample size
3
aggregation scope
splits
  • test
subsets
  • default
task_count
3
task_names
  • BeytooteClustering
  • DigikalamagClustering
  • NLPTwitterAnalysisClustering
sample_size
3
subset_count
1
subset_sample
  • default
aggregation_method
mean-of-task-means
model_revision_count
1
model_revision_sample
  • a0e3973064c97768e121b9b95f21adc94e0ca3fb
source model id
sbunlp/fabert
Original result ↗

MTEB

77.158%

source id
mteb
leaderboard name
MTEB Semantic Similarity
dimension
embedding
metric
Average task score
rank
223
sample size
1
aggregation scope
splits
  • test
subsets
  • default
task_count
1
task_names
  • SynPerSTS
sample_size
1
subset_count
1
subset_sample
  • default
aggregation_method
mean-of-task-means
model_revision_count
1
model_revision_sample
  • a0e3973064c97768e121b9b95f21adc94e0ca3fb
source model id
sbunlp/fabert
Original result ↗

MTEB

65.659%

source id
mteb
leaderboard name
MTEB Classification
dimension
embedding
metric
Average task score
rank
237
sample size
28
aggregation scope
splits
  • test
  • validation
subsets
  • default
  • fa
  • fas
task_count
25
task_names
  • DigikalamagClassification
  • MassiveIntentClassification
  • MassiveScenarioClassification
  • MultilingualSentimentClassification
  • NLPTwitterAnalysisClassification
  • NLPTwitterAnalysisClassification.v2
  • PerShopDomainClassification
  • PerShopIntentClassification
  • PersianFoodSentimentClassification
  • SIDClassification
  • SIDClassification.v2
  • StyleClassification
  • SynPerChatbotConvSAClassification
  • SynPerChatbotConvSAToneChatbotClassification
  • SynPerChatbotConvSAToneClassification
  • SynPerChatbotConvSAToneUserClassification
  • SynPerChatbotRAGToneChatbotClassification
  • SynPerChatbotRAGToneClassification
  • SynPerChatbotRAGToneUserClassification
  • SynPerChatbotSatisfactionLevelClassification
  • SynPerChatbotToneChatbotClassification
  • SynPerChatbotToneClassification
  • SynPerChatbotToneUserClassification
  • SynPerTextToneClassification
  • SynPerTextToneClassification.v3
sample_size
28
subset_count
3
subset_sample
  • default
  • fa
  • fas
aggregation_method
mean-of-task-means
model_revision_count
1
model_revision_sample
  • a0e3973064c97768e121b9b95f21adc94e0ca3fb
source model id
sbunlp/fabert
Original result ↗

MTEB

50.575%

source id
mteb
leaderboard name
MTEB Reranking
dimension
embedding
metric
Average task score
rank
291
sample size
2
aggregation scope
splits
  • dev
  • test
subsets
  • fa
task_count
2
task_names
  • MIRACLReranking
  • WikipediaRerankingMultilingual
sample_size
2
subset_count
1
subset_sample
  • fa
aggregation_method
mean-of-task-means
model_revision_count
1
model_revision_sample
  • a0e3973064c97768e121b9b95f21adc94e0ca3fb
source model id
sbunlp/fabert
Original result ↗

MTEB

37.64%

source id
mteb
leaderboard name
MTEB Other
dimension
embedding
metric
Average task score
rank
689
sample size
49
aggregation scope
splits
  • dev
  • test
subsets
  • default
task_count
49
task_names
  • ArguAna-Fa
  • ArguAna-Fa.v2
  • CExaPPC
  • ClimateFEVER-Fa
  • DBPedia-Fa
  • DeepSentiPers
  • DeepSentiPers.v2
  • FEVER-FaHardNegatives
  • FarsTail
  • FarsiParaphraseDetection
  • Farsick
  • FiQA2018-Fa
  • FiQA2018-Fa.v2
  • HamshahriClustring
  • HotpotQA-Fa
  • HotpotQA-FaHardNegatives
  • MSMARCO-Fa
  • MSMARCO-FaHardNegatives
  • NFCorpus-Fa
  • NQ-Fa
  • NQ-FaHardNegatives
  • ParsinluEntail
  • ParsinluQueryParaphPC
  • PersianTextEmotion
  • PersianTextEmotion.v2
  • Query2Query
  • SAMSumFa
  • SCIDOCS-Fa
  • SCIDOCS-Fa.v2
  • SIDClustring
  • SciFact-Fa
  • SciFact-Fa.v2
  • SentimentDKSF
  • SynPerChatbotConvSAAnger
  • SynPerChatbotConvSAFear
  • SynPerChatbotConvSAFriendship
  • SynPerChatbotConvSAHappiness
  • SynPerChatbotConvSAJealousy
  • SynPerChatbotConvSALove
  • SynPerChatbotConvSASadness
  • SynPerChatbotConvSASatisfaction
  • SynPerChatbotConvSASurprise
  • SynPerChatbotRAGFAQPC
  • SynPerQAPC
  • SynPerTextKeywordsPC
  • TRECCOVID-Fa
  • TRECCOVID-Fa.v2
  • Touche2020-Fa
  • Touche2020-Fa.v2
sample_size
49
subset_count
1
subset_sample
  • default
aggregation_method
mean-of-task-means
model_revision_count
1
model_revision_sample
  • a0e3973064c97768e121b9b95f21adc94e0ca3fb
source model id
sbunlp/fabert
Original result ↗

MTEB

39.708%

source id
mteb
leaderboard name
MTEB Overall
dimension
embedding
metric
Average task score
rank
775
sample size
111
aggregation scope
splits
  • dev
  • test
  • validation
subsets
  • default
  • fa
  • fas
task_count
108
task_names
  • ArguAna-Fa
  • ArguAna-Fa.v2
  • BeytooteClustering
  • CExaPPC
  • CQADupstackAndroidRetrieval-Fa
  • CQADupstackEnglishRetrieval-Fa
  • CQADupstackGamingRetrieval-Fa
  • CQADupstackGisRetrieval-Fa
  • CQADupstackMathematicaRetrieval-Fa
  • CQADupstackPhysicsRetrieval-Fa
  • CQADupstackProgrammersRetrieval-Fa
  • CQADupstackRetrieval-Fa
  • CQADupstackStatsRetrieval-Fa
  • CQADupstackTexRetrieval-Fa
  • CQADupstackUnixRetrieval-Fa
  • CQADupstackWebmastersRetrieval-Fa
  • CQADupstackWordpressRetrieval-Fa
  • ClimateFEVER-Fa
  • DBPedia-Fa
  • DeepSentiPers
  • DeepSentiPers.v2
  • DigikalamagClassification
  • DigikalamagClustering
  • FEVER-FaHardNegatives
  • FarsTail
  • FarsiParaphraseDetection
  • Farsick
  • FiQA2018-Fa
  • FiQA2018-Fa.v2
  • HamshahriClustring
  • HotpotQA-Fa
  • HotpotQA-FaHardNegatives
  • MIRACLReranking
  • MIRACLRetrieval
  • MIRACLRetrievalHardNegatives
  • MSMARCO-Fa
  • MSMARCO-FaHardNegatives
  • MassiveIntentClassification
  • MassiveScenarioClassification
  • MultilingualSentimentClassification
  • NFCorpus-Fa
  • NLPTwitterAnalysisClassification
  • NLPTwitterAnalysisClassification.v2
  • NLPTwitterAnalysisClustering
  • NQ-Fa
  • NQ-FaHardNegatives
  • NeuCLIR2023Retrieval
  • NeuCLIR2023RetrievalHardNegatives
  • ParsinluEntail
  • ParsinluQueryParaphPC
  • PerShopDomainClassification
  • PerShopIntentClassification
  • PersianFoodSentimentClassification
  • PersianTextEmotion
  • PersianTextEmotion.v2
  • PersianWebDocumentRetrieval
  • Query2Query
  • QuoraRetrieval-Fa
  • QuoraRetrieval-Fa.v2
  • SAMSumFa
  • SCIDOCS-Fa
  • SCIDOCS-Fa.v2
  • SIDClassification
  • SIDClassification.v2
  • SIDClustring
  • SciFact-Fa
  • SciFact-Fa.v2
  • SentimentDKSF
  • StyleClassification
  • SynPerChatbotConvSAAnger
  • SynPerChatbotConvSAClassification
  • SynPerChatbotConvSAFear
  • SynPerChatbotConvSAFriendship
  • SynPerChatbotConvSAHappiness
  • SynPerChatbotConvSAJealousy
  • SynPerChatbotConvSALove
  • SynPerChatbotConvSASadness
  • SynPerChatbotConvSASatisfaction
  • SynPerChatbotConvSASurprise
  • SynPerChatbotConvSAToneChatbotClassification
  • SynPerChatbotConvSAToneClassification
  • SynPerChatbotConvSAToneUserClassification
  • SynPerChatbotRAGFAQPC
  • SynPerChatbotRAGFAQRetrieval
  • SynPerChatbotRAGSumSRetrieval
  • SynPerChatbotRAGToneChatbotClassification
  • SynPerChatbotRAGToneClassification
  • SynPerChatbotRAGToneUserClassification
  • SynPerChatbotRAGTopicsRetrieval
  • SynPerChatbotSatisfactionLevelClassification
  • SynPerChatbotSumSRetrieval
  • SynPerChatbotToneChatbotClassification
  • SynPerChatbotToneClassification
  • SynPerChatbotToneUserClassification
  • SynPerChatbotTopicsRetrieval
  • SynPerQAPC
  • SynPerQARetrieval
  • SynPerSTS
  • SynPerTextKeywordsPC
  • SynPerTextToneClassification
  • SynPerTextToneClassification.v3
  • TRECCOVID-Fa
  • TRECCOVID-Fa.v2
  • Touche2020-Fa
  • Touche2020-Fa.v2
  • WebFAQRetrieval
  • WikipediaRerankingMultilingual
  • WikipediaRetrievalMultilingual
sample_size
111
subset_count
3
subset_sample
  • default
  • fa
  • fas
aggregation_method
mean-of-task-means
model_revision_count
1
model_revision_sample
  • a0e3973064c97768e121b9b95f21adc94e0ca3fb
source model id
sbunlp/fabert
Original result ↗

MTEB

16.329%

source id
mteb
leaderboard name
MTEB Retrieval
dimension
embedding
metric
Average task score
rank
825
sample size
28
aggregation scope
splits
  • dev
  • test
subsets
  • default
  • fa
  • fas
task_count
28
task_names
  • CQADupstackAndroidRetrieval-Fa
  • CQADupstackEnglishRetrieval-Fa
  • CQADupstackGamingRetrieval-Fa
  • CQADupstackGisRetrieval-Fa
  • CQADupstackMathematicaRetrieval-Fa
  • CQADupstackPhysicsRetrieval-Fa
  • CQADupstackProgrammersRetrieval-Fa
  • CQADupstackRetrieval-Fa
  • CQADupstackStatsRetrieval-Fa
  • CQADupstackTexRetrieval-Fa
  • CQADupstackUnixRetrieval-Fa
  • CQADupstackWebmastersRetrieval-Fa
  • CQADupstackWordpressRetrieval-Fa
  • MIRACLRetrieval
  • MIRACLRetrievalHardNegatives
  • NeuCLIR2023Retrieval
  • NeuCLIR2023RetrievalHardNegatives
  • PersianWebDocumentRetrieval
  • QuoraRetrieval-Fa
  • QuoraRetrieval-Fa.v2
  • SynPerChatbotRAGFAQRetrieval
  • SynPerChatbotRAGSumSRetrieval
  • SynPerChatbotRAGTopicsRetrieval
  • SynPerChatbotSumSRetrieval
  • SynPerChatbotTopicsRetrieval
  • SynPerQARetrieval
  • WebFAQRetrieval
  • WikipediaRetrievalMultilingual
sample_size
28
subset_count
3
subset_sample
  • default
  • fa
  • fas
aggregation_method
mean-of-task-means
model_revision_count
1
model_revision_sample
  • a0e3973064c97768e121b9b95f21adc94e0ca3fb
source model id
sbunlp/fabert
Original result ↗

Provider specifications & access

1–1 / 1

sbunlp/fabert ↗

source model id
HuggingFace/sbunlp-fabert@1
hosting
foundry-managed-compute
platforms
  • Microsoft Foundry
license
unknown
architecture
input_modalities
    output_modalities
      capabilities

        Original sources

        1–5 / 5

        Hugging Face · sbunlp/fabert ↗

        Microsoft Foundry · Model page ↗

        Microsoft Foundry · Model documentation ↗

        Hugging Face · Catalog ↗

        Microsoft Foundry · Catalog ↗

        Source links

        1–2 / 2

        Explore more models

        Models

        Loading
        EVIDENCE / COMPARISON

        Benchmarks

        Explore every benchmark

        Loading

        These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

        Install TensorTrace