abacusai/Smaug-Mixtral-v0.1
Explore abacusai/Smaug-Mixtral-v0.1: model specifications, capabilities, benchmark results, platform pricing and original sources.
huggingface/abacusai/Smaug-Mixtral-v0.1
Specifications & capabilities
- Input
- text
- Output
- text
- Published model ID
- Smaug-Mixtral-v0.1
Evaluations
1–20 / 27
74.915%
- source id
- openLLMLeaderboardV1
- leaderboard name
- Open LLM v1 ARC
- dimension
- reasoning
- metric
- Accuracy
- rank
- 44
- aggregation scope
- suite
- Open LLM v1
- status
- archived
- precision
- bfloat16
- task_count
- 6
- metric_kind
- Accuracy
- rank_method
- Derived competition rank over published evaluations with reported scores, including variants and source flags
- weight_type
- Original
- published_at
- 2024-06-25T14:11:42Z
- chat_template
- false
- suite_version
- 1
- model_revision
- 2f83b45077479bc3f663da50c4c40372894bf92e
- source_flagged
- true
- dataset_revision
- 791ca1c841030008c6c266562c2b88c9b7196550
- metric_population
- 7260
- ranking_population
- 7260
- source_flag_anomaly
- true
- evaluated at
- 2024-02-20T02:57:50.000Z
- source model id
- abacusai/Smaug-Mixtral-v0.1
Original result ↗74.659%
- source id
- openLLMLeaderboardV1
- leaderboard name
- Open LLM v1 ARC
- dimension
- reasoning
- metric
- Accuracy
- rank
- 48
- aggregation scope
- suite
- Open LLM v1
- status
- archived
- precision
- float16
- task_count
- 6
- metric_kind
- Accuracy
- rank_method
- Derived competition rank over published evaluations with reported scores, including variants and source flags
- weight_type
- Original
- published_at
- 2024-06-25T14:11:42Z
- chat_template
- false
- suite_version
- 1
- model_revision
- 98fdc8315906b0a8b9e7f24bad89914869fcfc20
- source_flagged
- true
- dataset_revision
- 791ca1c841030008c6c266562c2b88c9b7196550
- metric_population
- 7260
- ranking_population
- 7260
- source_flag_anomaly
- true
- evaluated at
- 2024-03-02T02:40:48.000Z
- source model id
- abacusai/Smaug-Mixtral-v0.1
Original result ↗72.479%
- source id
- openLLMLeaderboardV1
- leaderboard name
- Open LLM v1 GSM8K
- dimension
- math
- metric
- Accuracy
- rank
- 152
- aggregation scope
- suite
- Open LLM v1
- status
- archived
- precision
- bfloat16
- task_count
- 6
- metric_kind
- Accuracy
- rank_method
- Derived competition rank over published evaluations with reported scores, including variants and source flags
- weight_type
- Original
- published_at
- 2024-06-25T14:11:42Z
- chat_template
- false
- suite_version
- 1
- model_revision
- 2f83b45077479bc3f663da50c4c40372894bf92e
- source_flagged
- true
- dataset_revision
- 791ca1c841030008c6c266562c2b88c9b7196550
- metric_population
- 7260
- ranking_population
- 7260
- source_flag_anomaly
- true
- evaluated at
- 2024-02-20T02:57:50.000Z
- source model id
- abacusai/Smaug-Mixtral-v0.1
Original result ↗71.948%
- source id
- openLLMLeaderboardV1
- leaderboard name
- Open LLM v1 GSM8K
- dimension
- math
- metric
- Accuracy
- rank
- 218
- aggregation scope
- suite
- Open LLM v1
- status
- archived
- precision
- float16
- task_count
- 6
- metric_kind
- Accuracy
- rank_method
- Derived competition rank over published evaluations with reported scores, including variants and source flags
- weight_type
- Original
- published_at
- 2024-06-25T14:11:42Z
- chat_template
- false
- suite_version
- 1
- model_revision
- 98fdc8315906b0a8b9e7f24bad89914869fcfc20
- source_flagged
- true
- dataset_revision
- 791ca1c841030008c6c266562c2b88c9b7196550
- metric_population
- 7260
- ranking_population
- 7260
- source_flag_anomaly
- true
- evaluated at
- 2024-03-02T02:40:48.000Z
- source model id
- abacusai/Smaug-Mixtral-v0.1
Original result ↗75.639%
- source id
- openLLMLeaderboardV1
- leaderboard name
- Open LLM v1 Average
- dimension
- general
- metric
- Six-task average
- rank
- 448
- aggregation scope
- suite
- Open LLM v1
- status
- archived
- precision
- bfloat16
- task_count
- 6
- metric_kind
- Six-task average
- rank_method
- Derived competition rank over published evaluations with reported scores, including variants and source flags
- weight_type
- Original
- published_at
- 2024-06-25T14:11:42Z
- chat_template
- false
- suite_version
- 1
- model_revision
- 2f83b45077479bc3f663da50c4c40372894bf92e
- source_flagged
- true
- dataset_revision
- 791ca1c841030008c6c266562c2b88c9b7196550
- metric_population
- 7170
- ranking_population
- 7260
- source_flag_anomaly
- true
- evaluated at
- 2024-02-20T02:57:50.000Z
- source model id
- abacusai/Smaug-Mixtral-v0.1
Original result ↗75.492%
- source id
- openLLMLeaderboardV1
- leaderboard name
- Open LLM v1 Average
- dimension
- general
- metric
- Six-task average
- rank
- 469
- aggregation scope
- suite
- Open LLM v1
- status
- archived
- precision
- float16
- task_count
- 6
- metric_kind
- Six-task average
- rank_method
- Derived competition rank over published evaluations with reported scores, including variants and source flags
- weight_type
- Original
- published_at
- 2024-06-25T14:11:42Z
- chat_template
- false
- suite_version
- 1
- model_revision
- 98fdc8315906b0a8b9e7f24bad89914869fcfc20
- source_flagged
- true
- dataset_revision
- 791ca1c841030008c6c266562c2b88c9b7196550
- metric_population
- 7170
- ranking_population
- 7260
- source_flag_anomaly
- true
- evaluated at
- 2024-03-02T02:40:48.000Z
- source model id
- abacusai/Smaug-Mixtral-v0.1
Original result ↗70.082%
- source id
- openLLMLeaderboardV1
- leaderboard name
- Open LLM v1 MMLU
- dimension
- knowledge
- metric
- Accuracy
- rank
- 488
- aggregation scope
- suite
- Open LLM v1
- status
- archived
- precision
- bfloat16
- task_count
- 6
- metric_kind
- Accuracy
- rank_method
- Derived competition rank over published evaluations with reported scores, including variants and source flags
- weight_type
- Original
- published_at
- 2024-06-25T14:11:42Z
- chat_template
- false
- suite_version
- 1
- model_revision
- 2f83b45077479bc3f663da50c4c40372894bf92e
- source_flagged
- true
- dataset_revision
- 791ca1c841030008c6c266562c2b88c9b7196550
- metric_population
- 7260
- ranking_population
- 7260
- source_flag_anomaly
- true
- evaluated at
- 2024-02-20T02:57:50.000Z
- source model id
- abacusai/Smaug-Mixtral-v0.1
Original result ↗70.064%
- source id
- openLLMLeaderboardV1
- leaderboard name
- Open LLM v1 MMLU
- dimension
- knowledge
- metric
- Accuracy
- rank
- 489
- aggregation scope
- suite
- Open LLM v1
- status
- archived
- precision
- float16
- task_count
- 6
- metric_kind
- Accuracy
- rank_method
- Derived competition rank over published evaluations with reported scores, including variants and source flags
- weight_type
- Original
- published_at
- 2024-06-25T14:11:42Z
- chat_template
- false
- suite_version
- 1
- model_revision
- 98fdc8315906b0a8b9e7f24bad89914869fcfc20
- source_flagged
- true
- dataset_revision
- 791ca1c841030008c6c266562c2b88c9b7196550
- metric_population
- 7260
- ranking_population
- 7260
- source_flag_anomaly
- true
- evaluated at
- 2024-03-02T02:40:48.000Z
- source model id
- abacusai/Smaug-Mixtral-v0.1
Original result ↗87.791%
- source id
- openLLMLeaderboardV1
- leaderboard name
- Open LLM v1 HellaSwag
- dimension
- reasoning
- metric
- Accuracy
- rank
- 878
- aggregation scope
- suite
- Open LLM v1
- status
- archived
- precision
- bfloat16
- task_count
- 6
- metric_kind
- Accuracy
- rank_method
- Derived competition rank over published evaluations with reported scores, including variants and source flags
- weight_type
- Original
- published_at
- 2024-06-25T14:11:42Z
- chat_template
- false
- suite_version
- 1
- model_revision
- 2f83b45077479bc3f663da50c4c40372894bf92e
- source_flagged
- true
- dataset_revision
- 791ca1c841030008c6c266562c2b88c9b7196550
- metric_population
- 7260
- ranking_population
- 7260
- source_flag_anomaly
- true
- evaluated at
- 2024-02-20T02:57:50.000Z
- source model id
- abacusai/Smaug-Mixtral-v0.1
Original result ↗87.722%
- source id
- openLLMLeaderboardV1
- leaderboard name
- Open LLM v1 HellaSwag
- dimension
- reasoning
- metric
- Accuracy
- rank
- 913
- aggregation scope
- suite
- Open LLM v1
- status
- archived
- precision
- float16
- task_count
- 6
- metric_kind
- Accuracy
- rank_method
- Derived competition rank over published evaluations with reported scores, including variants and source flags
- weight_type
- Original
- published_at
- 2024-06-25T14:11:42Z
- chat_template
- false
- suite_version
- 1
- model_revision
- 98fdc8315906b0a8b9e7f24bad89914869fcfc20
- source_flagged
- true
- dataset_revision
- 791ca1c841030008c6c266562c2b88c9b7196550
- metric_population
- 7260
- ranking_population
- 7260
- source_flag_anomaly
- true
- evaluated at
- 2024-03-02T02:40:48.000Z
- source model id
- abacusai/Smaug-Mixtral-v0.1
Original result ↗66.949%
- source id
- openLLMLeaderboardV1
- leaderboard name
- Open LLM v1 TruthfulQA
- dimension
- knowledge
- metric
- Accuracy
- rank
- 1059
- aggregation scope
- suite
- Open LLM v1
- status
- archived
- precision
- float16
- task_count
- 6
- metric_kind
- Accuracy
- rank_method
- Derived competition rank over published evaluations with reported scores, including variants and source flags
- weight_type
- Original
- published_at
- 2024-06-25T14:11:42Z
- chat_template
- false
- suite_version
- 1
- model_revision
- 98fdc8315906b0a8b9e7f24bad89914869fcfc20
- source_flagged
- true
- dataset_revision
- 791ca1c841030008c6c266562c2b88c9b7196550
- metric_population
- 7170
- ranking_population
- 7260
- source_flag_anomaly
- true
- evaluated at
- 2024-03-02T02:40:48.000Z
- source model id
- abacusai/Smaug-Mixtral-v0.1
Original result ↗66.879%
- source id
- openLLMLeaderboardV1
- leaderboard name
- Open LLM v1 TruthfulQA
- dimension
- knowledge
- metric
- Accuracy
- rank
- 1065
- aggregation scope
- suite
- Open LLM v1
- status
- archived
- precision
- bfloat16
- task_count
- 6
- metric_kind
- Accuracy
- rank_method
- Derived competition rank over published evaluations with reported scores, including variants and source flags
- weight_type
- Original
- published_at
- 2024-06-25T14:11:42Z
- chat_template
- false
- suite_version
- 1
- model_revision
- 2f83b45077479bc3f663da50c4c40372894bf92e
- source_flagged
- true
- dataset_revision
- 791ca1c841030008c6c266562c2b88c9b7196550
- metric_population
- 7170
- ranking_population
- 7260
- source_flag_anomaly
- true
- evaluated at
- 2024-02-20T02:57:50.000Z
- source model id
- abacusai/Smaug-Mixtral-v0.1
Original result ↗12.993%
- source id
- openLLMLeaderboardV2
- leaderboard name
- Open LLM v2 MUSR · Normalized
- dimension
- reasoning
- metric
- Normalized score
- rank
- 1299
- aggregation scope
- suite
- Open LLM v2
- co2_kg
- 7.882831004615716
- status
- archived
- precision
- bfloat16
- task_count
- 6
- metric_kind
- Normalized score
- rank_method
- Derived competition rank over published evaluations with reported scores, including variants and source flags
- weight_type
- Original
- published_at
- 2025-03-20T12:17:27Z
- chat_template
- true
- suite_version
- 2
- model_revision
- 98fdc8315906b0a8b9e7f24bad89914869fcfc20
- source_flagged
- false
- dataset_revision
- 9c09a7cae43334062a82cb164f2ef255013dafa2
- metric_population
- 4576
- ranking_population
- 4576
- source_flag_anomaly
- false
- source model id
- abacusai/Smaug-Mixtral-v0.1
Original result ↗42.981%
- source id
- openLLMLeaderboardV2
- leaderboard name
- Open LLM v2 MUSR · Raw
- dimension
- reasoning
- metric
- Raw accuracy
- rank
- 1349
- aggregation scope
- suite
- Open LLM v2
- co2_kg
- 7.882831004615716
- status
- archived
- precision
- bfloat16
- task_count
- 6
- metric_kind
- Raw accuracy
- rank_method
- Derived competition rank over published evaluations with reported scores, including variants and source flags
- weight_type
- Original
- published_at
- 2025-03-20T12:17:27Z
- chat_template
- true
- suite_version
- 2
- model_revision
- 98fdc8315906b0a8b9e7f24bad89914869fcfc20
- source_flagged
- false
- dataset_revision
- 9c09a7cae43334062a82cb164f2ef255013dafa2
- metric_population
- 4576
- ranking_population
- 4576
- source_flag_anomaly
- false
- source model id
- abacusai/Smaug-Mixtral-v0.1
Original result ↗81.689%
- source id
- openLLMLeaderboardV1
- leaderboard name
- Open LLM v1 Winogrande
- dimension
- reasoning
- metric
- Accuracy
- rank
- 1409
- aggregation scope
- suite
- Open LLM v1
- status
- archived
- precision
- bfloat16
- task_count
- 6
- metric_kind
- Accuracy
- rank_method
- Derived competition rank over published evaluations with reported scores, including variants and source flags
- weight_type
- Original
- published_at
- 2024-06-25T14:11:42Z
- chat_template
- false
- suite_version
- 1
- model_revision
- 2f83b45077479bc3f663da50c4c40372894bf92e
- source_flagged
- true
- dataset_revision
- 791ca1c841030008c6c266562c2b88c9b7196550
- metric_population
- 7260
- ranking_population
- 7260
- source_flag_anomaly
- true
- evaluated at
- 2024-02-20T02:57:50.000Z
- source model id
- abacusai/Smaug-Mixtral-v0.1
Original result ↗81.61%
- source id
- openLLMLeaderboardV1
- leaderboard name
- Open LLM v1 Winogrande
- dimension
- reasoning
- metric
- Accuracy
- rank
- 1436
- aggregation scope
- suite
- Open LLM v1
- status
- archived
- precision
- float16
- task_count
- 6
- metric_kind
- Accuracy
- rank_method
- Derived competition rank over published evaluations with reported scores, including variants and source flags
- weight_type
- Original
- published_at
- 2024-06-25T14:11:42Z
- chat_template
- false
- suite_version
- 1
- model_revision
- 98fdc8315906b0a8b9e7f24bad89914869fcfc20
- source_flagged
- true
- dataset_revision
- 791ca1c841030008c6c266562c2b88c9b7196550
- metric_population
- 7260
- ranking_population
- 7260
- source_flag_anomaly
- true
- evaluated at
- 2024-03-02T02:40:48.000Z
- source model id
- abacusai/Smaug-Mixtral-v0.1
Original result ↗55.544%
- source id
- openLLMLeaderboardV2
- leaderboard name
- Open LLM v2 IFEval · Normalized
- dimension
- instructionFollowing
- metric
- Normalized score
- rank
- 1489
- aggregation scope
- suite
- Open LLM v2
- co2_kg
- 7.882831004615716
- status
- archived
- precision
- bfloat16
- task_count
- 6
- metric_kind
- Normalized score
- rank_method
- Derived competition rank over published evaluations with reported scores, including variants and source flags
- weight_type
- Original
- published_at
- 2025-03-20T12:17:27Z
- chat_template
- true
- suite_version
- 2
- model_revision
- 98fdc8315906b0a8b9e7f24bad89914869fcfc20
- source_flagged
- false
- dataset_revision
- 9c09a7cae43334062a82cb164f2ef255013dafa2
- metric_population
- 4576
- ranking_population
- 4576
- source_flag_anomaly
- false
- source model id
- abacusai/Smaug-Mixtral-v0.1
Original result ↗55.544%
- source id
- openLLMLeaderboardV2
- leaderboard name
- Open LLM v2 IFEval · Raw
- dimension
- instructionFollowing
- metric
- Raw accuracy
- rank
- 1489
- aggregation scope
- suite
- Open LLM v2
- co2_kg
- 7.882831004615716
- status
- archived
- precision
- bfloat16
- task_count
- 6
- metric_kind
- Raw accuracy
- rank_method
- Derived competition rank over published evaluations with reported scores, including variants and source flags
- weight_type
- Original
- published_at
- 2025-03-20T12:17:27Z
- chat_template
- true
- suite_version
- 2
- model_revision
- 98fdc8315906b0a8b9e7f24bad89914869fcfc20
- source_flagged
- false
- dataset_revision
- 9c09a7cae43334062a82cb164f2ef255013dafa2
- metric_population
- 4576
- ranking_population
- 4576
- source_flag_anomaly
- false
- source model id
- abacusai/Smaug-Mixtral-v0.1
Original result ↗31.919%
- source id
- openLLMLeaderboardV2
- leaderboard name
- Open LLM v2 BBH · Normalized
- dimension
- reasoning
- metric
- Normalized score
- rank
- 1864
- aggregation scope
- suite
- Open LLM v2
- co2_kg
- 7.882831004615716
- status
- archived
- precision
- bfloat16
- task_count
- 6
- metric_kind
- Normalized score
- rank_method
- Derived competition rank over published evaluations with reported scores, including variants and source flags
- weight_type
- Original
- published_at
- 2025-03-20T12:17:27Z
- chat_template
- true
- suite_version
- 2
- model_revision
- 98fdc8315906b0a8b9e7f24bad89914869fcfc20
- source_flagged
- false
- dataset_revision
- 9c09a7cae43334062a82cb164f2ef255013dafa2
- metric_population
- 4576
- ranking_population
- 4576
- source_flag_anomaly
- false
- source model id
- abacusai/Smaug-Mixtral-v0.1
Original result ↗23.821%
- source id
- openLLMLeaderboardV2
- leaderboard name
- Open LLM v2 Average
- dimension
- general
- metric
- Normalized six-task average
- rank
- 1883
- aggregation scope
- suite
- Open LLM v2
- co2_kg
- 7.882831004615716
- status
- archived
- precision
- bfloat16
- task_count
- 6
- metric_kind
- Normalized six-task average
- rank_method
- Derived competition rank over published evaluations with reported scores, including variants and source flags
- weight_type
- Original
- published_at
- 2025-03-20T12:17:27Z
- chat_template
- true
- suite_version
- 2
- model_revision
- 98fdc8315906b0a8b9e7f24bad89914869fcfc20
- source_flagged
- false
- dataset_revision
- 9c09a7cae43334062a82cb164f2ef255013dafa2
- metric_population
- 4576
- ranking_population
- 4576
- source_flag_anomaly
- false
- source model id
- abacusai/Smaug-Mixtral-v0.1
Original result ↗Explore more models