<img src="https://ollama.com/assets/library/bespoke-minicheck/640ec552-feac-4a35-8651-a85ba3f1f9bd" width="480" /> This is a grounded factuality checking model developed by [Bespoke Labs](https://bes…
<img src="https://ollama.com/assets/library/bespoke-minicheck/640ec552-feac-4a35-8651-a85ba3f1f9bd" width="480" />
This is a grounded factuality checking model developed by [Bespoke Labs](https://bespokelabs.ai).
The model takes as input a document (text) and a sentence and determines whether the sentence is supported by the document. In order to fact-check a multi-sentence claim, the claim should first be broken up into sentences. The document does not need to be chunked unless it exceeds 32K tokens.

Bespoke-MiniCheck is the SOTA fact-checking model despite its small size.
## Usage
The prompt template is as follows:
```
Document: {document}
Claim: {claim}
```
The response will either be `Yes` or `No`.
## Examples
<sub>Prompt</sub>
```
Document: A group of students gather in the school library to study for their upcoming final exams.
Claim: The students are preparing for an examination.
```
<sub>Response</sub>
```
Yes
```
<sub>Prompt</sub>
```
Document: A group of students gather in the school library to study for their upcoming final exams.
Claim: The students are on vacation.
```
<sub>Response</sub>
```
No
```
## Model performance

The performance of these models is evaluated on our new collected benchmark (unseen by our models during training), [LLM-AggreFact](https://huggingface.co/datasets/lytang/LLM-AggreFact), from 11 recent human annotated datasets on fact-checking and grounding LLM generations. **Bespoke-MiniCheck-7B is the SOTA fact-checking model despite its small size.**
## References
[Website](https://bespokelabs.ai/bespoke-minicheck)
[Paper](https://arxiv.org/pdf/2404.10774)
[LLM-AggreFact Leaderboard](https://llm-aggrefact.github.io/)