Jeffan-31b-1710
Jeffan-31b-1710 is an open-weights decision model by Trong Vu, a full fine-tune of Google's Gemma-4-31B-it published on Hugging Face on 19 October 2026. It reads one multiple-choice question at a time about a state and returns a probability for every option. Weights and inference code are Apache 2.0.
Decision Index Full
69.8
第 6 名 · public 73.1
校准 ECE
0.046
越低越好 · Brier 0.248
模型信息
- 厂商
- Trong Vu
- 类别
- 开源权重
- 参数规模
- 31B
- 基座模型
- google/gemma-4-31B-it
- 权重
- 闭源 / 未公开
- 许可证
- 未标注 许可不明
- 输入模态
- 文本
- 决策原语
- Choice
- 延迟
- 43-121 ms
- 可微调
- 未确认
- 训练方式
- full fine-tune
- 上架状态
- Generally available
数据来源与链接
本站聚合自:
Decision Index、systemonemodels.org。
分数与链接均指向原始出处。
Decision Index 领域得分
知识与推理
74.5
语言理解
91.3
检索与分类
81.7
工具与自动化
95.7
艺术与人类品味
75.0
Decision Index · 42 个基准
| 基准 | 类型 | 该模型 | 全场最佳 | 全场均值 | 对比 | 来源 |
|---|---|---|---|---|---|---|
| Home appliances | 混合(Choice | 100.0 | 100.0 | 43.9 | 来源 ↗ | |
| ARC-Easy | 混合(Choice | 99.4 | 99.5 | 89.5 | 来源 ↗ | |
| ARC-Challenge | 混合(Choice | 98.0 | 98.2 | 82.4 | 来源 ↗ | |
| HellaSwag | 混合(Choice | 97.7 | 98.5 | 79.5 | 来源 ↗ | |
| BFCL | 混合(Choice | 97.5 | 98.8 | 86.6 | 来源 ↗ | |
| CLINC150 | 混合(Choice | 96.4 | 97.6 | 79.6 | 来源 ↗ | |
| CLadder | 混合(Choice | 95.5 | 97.7 | 70.3 | 来源 ↗ | |
| WinoGrande | 混合(Choice | 95.3 | 97.5 | 75.9 | 来源 ↗ | |
| BPoMP | 混合(Choice | 95.1 | 99.5 | 80.0 | 来源 ↗ | |
| FinEntity | 混合(Choice | 94.2 | 97.1 | 81.3 | 来源 ↗ | |
| BBH | 混合(Choice | 93.0 | 96.2 | 64.7 | 来源 ↗ | |
| GSM8K | 混合(Choice | 92.3 | 92.6 | 47.4 | 来源 ↗ | |
| When2Call | 混合(Choice | 92.1 | 93.8 | 70.1 | 来源 ↗ | |
| BANKING77 | 混合(Choice | 91.9 | 94.2 | 78.1 | 来源 ↗ | |
| CRUXEval | 混合(Choice | 90.2 | 100.0 | 57.7 | 来源 ↗ | |
| MMLU | 混合(Choice | 89.5 | 91.9 | 68.0 | 来源 ↗ | |
| HoVer | 混合(Choice | 88.6 | 90.8 | 73.2 | 来源 ↗ | |
| New Yorker | 混合(Choice | 88.3 | 88.3 | 60.8 | 来源 ↗ | |
| API-Bank | 混合(Choice | 88.0 | 93.1 | 67.4 | 来源 ↗ | |
| NLI4CT | 混合(Choice | 87.8 | 88.3 | 72.9 | 来源 ↗ | |
| PhishNChips | 混合(Choice | 84.9 | 99.9 | 69.4 | 来源 ↗ | |
| VAST | 混合(Choice | 84.2 | 84.2 | 61.5 | 来源 ↗ | |
| RAGTruth | 混合(Choice | 84.2 | 86.2 | 64.5 | 来源 ↗ | |
| ContractNLI | 混合(Choice | 84.1 | 86.7 | 69.6 | 来源 ↗ | |
| MuSR | 混合(Choice | 82.5 | 92.0 | 59.8 | 来源 ↗ | |
| POP909 | 混合(Choice | 82.2 | 86.1 | 29.7 | 来源 ↗ | |
| ANLI | 混合(Choice | 82.2 | 98.6 | 59.8 | 来源 ↗ | |
| RouterBench | 混合(Choice | 80.0 | 80.1 | 74.4 | 来源 ↗ | |
| MMLU-Pro | 混合(Choice | 79.0 | 82.1 | 47.7 | 来源 ↗ | |
| ToolRet | 混合(Choice | 75.7 | 75.7 | 59.9 | 来源 ↗ | |
| Humicroedit | 混合(Choice | 72.6 | 75.1 | 59.7 | 来源 ↗ | |
| iSarcasmEval | 混合(Choice | 68.8 | 78.5 | 45.3 | 来源 ↗ | |
| cfcolor | 混合(Choice | 66.7 | 72.9 | 60.1 | 来源 ↗ | |
| SGD | 混合(Choice | 66.1 | 85.8 | 45.5 | 来源 ↗ | |
| Amazon ESCI | 混合(Choice | 63.6 | 65.0 | 47.4 | 来源 ↗ | |
| ACOS | 混合(Choice | 63.4 | 68.0 | 24.9 | 来源 ↗ | |
| Habermas | 混合(Choice | 63.0 | 71.8 | 46.7 | 来源 ↗ | |
| GPQA Diamond | 混合(Choice | 53.1 | 75.5 | 40.1 | 来源 ↗ | |
| BRIGHT | 混合(Choice | 47.9 | 51.5 | 38.8 | 来源 ↗ | |
| SATA-Bench | 混合(Choice | 44.0 | 62.2 | 24.8 | 来源 ↗ | |
| ChessBench | 混合(Choice | 29.7 | 40.2 | 16.4 | 来源 ↗ | |
| HLE | 混合(Choice | 16.0 | 20.0 | 12.8 | 来源 ↗ |
来自 systemonemodels.org 的详细介绍
What Jeffan-31b-1710 is
Jeffan-31b-1710 is an open-weights decision model published on Hugging Face on 19 October 2026 by the account tattrongvu, whose profile gives the name Trong Vu. It is a full fine-tune of Google’s Gemma-4-31B-it. The card calls it a research model and a first release. The card links no company site, GitHub repository or hosted service.
You give it a state, such as a document, a conversation or a list of tools, and a multiple-choice question about it. It returns a probability for every option, read from one forward pass, and generates no text. The card describes Choice-style questions only.
How you run it
It does not serve Jev’s /v1/systemone request. It reads one question at a time in its own prompt layout: a fixed system line, then State:, Question: and Options: with the codes A, B, C and so on. The card shows how to call it with vLLM 0.29.0 in bf16, from Python or through vLLM’s OpenAI-compatible server, by reading the next-token probabilities of the option codes. It also ships the engine it used for its Decision Index runs, which reads an unsure question a second time. The card sizes the setup for one 96 GB GPU. Inputs can be up to 8,192 tokens.
Licence
The weights and inference code are Apache 2.0, the same licence as the Gemma 4 base. That changed on 22 October 2026: before then the weights were CC BY-NC 4.0, which bars commercial use, and the code was MIT. The Decision Index file generated that evening still lists CC BY-NC 4.0. The card says the probabilities are not validated for any production use and advises human review for high-stakes decisions.
What the benchmarks show
On the Jev Decision Index 0.4.1, generated on 22 October 2026, it scores 69.75 on the Full score, 2nd of 229 rows behind Fastino’s GLiDE at 71.52, against Jev 1.14’s 60.24. Its public score of 73.13 is also second, from a run the author submitted. Its calibration error is 0.0465 against Jev 1.14’s 0.0799. The board notes that its training data is undisclosed, so nobody outside can check what it was trained on.
The same account published a 4B sibling, Jeffan-4B-2010, on Qwen3.5-4B. It scores 52.21 on the Full score, 79th of 229.
What it is not for
The card says quality is lower on non-English inputs and on unfamiliar task types. Jeffan is not on JevBench, the Artificial Analysis Decision Index or OpenRouter, so the Decision Index is the only outside test so far.
Specifications
System One Score97.5 from 1 of 3 boardsJev Decision Index 97.5. How the score works
Question types Choice
Max Choice options255
Score levelsNot documented
Questions per callNot documented
Total context8,192 tokens
State budgetNot documented
Rate limitNone. It runs on your own hardware.
LicenceApache-2.0
SDKs
The card gives inputs of up to 8,192 tokens. The model reads one question per prompt, with options coded A, B, C and so on: the card's vLLM examples take at most 26 options, and its Decision Index engine codes up to 255. That engine reads a question a second time when the top option's probability is below 0.9. The card sizes the setup for one 96 GB GPU, checked on a B200 limited to 96 GB rather than on such a card. Image input is switched off in the card's examples.
Versions
tattrongvu/Jeffan-31b-1710, 19 Oct 2026, First release. A full fine-tune of google/gemma-4-31B-it with 31.3B parameters, bf16 weights. The licence changed from CC BY-NC 4.0 for the weights and MIT for the code to Apache 2.0 for both on 22 October 2026. The same account published Jeffan-4B-2010 on Qwen3.5-4B on 20 October. Release notes
Use cases
What people use Jeffan-31b-1710 for, one page per pattern.
Workflow controlStarter
Support inbox triage with System One models
Send a support ticket to Jev once with every question attached. Category comes back as a selected label, severity and frustration as numbers on scales you wrote, refund intent as a probability. Your code reads those values and decides what happens to the ticket.
Choice Score Noul
Workflow controlIntermediate
Intent and model routing with System One models
One Jev call reads an incoming request and returns its intent as a label plus a difficulty rating on a scale you wrote. Your router reads both numbers and picks the handler: deterministic code, a cheap model, an expensive one, or a human queue.
Choice Score
Benchmarks
Published scores, each linked to its source. A row marked vendor-run was measured by the vendor itself, so read it as one side's result. The compare hub sets these beside the other models.
BenchmarkMetricScoreRun byDate
Jev Decision Index 0.4.1Full score69.75 scoreIndependent22 Oct 2026
Jev Decision Index 0.4.1Public score73.13 scorevendor-run22 Oct 2026
Jev Decision Index 0.4.1Expected calibration error0.0465 ECEIndependent22 Oct 2026
Jev Decision Index, Full score: Row: Jeffan-31b-1710, 2nd of 229 rows by this score. Counts 60% private same-skill tests and 40% private new-domain tests; public benchmarks no longer count. 0 is random guessing, 100 is perfect. The Index ran the private tests on one RTX PRO 6000 with the author's serving. The row notes that the training data is undisclosed. Jev 1.14 scores 60.24.
Jev Decision Index, Public score: Row: Jeffan-31b-1710, 2nd of 229 rows by this score. The author submitted the public run, and the Index rescored it with its own kit. Jev 1.14 scores 60.7.
Jev Decision Index, Expected calibration error: Row: Jeffan-31b-1710. Lower is better. Jev 1.14 scores 0.0799.
VendorTrong Vu
TypeOpen source, self-hosted
StatusGenerally available
Announced19 Oct 2026
AccessOpened 19 Oct 2026
Input priceNo hosted API
Output priceNo hosted API
Latency43 to 121 ms (vendor claim)
Context8,192 tokens
Rate limitNone. It runs on your own hardware.
API docshuggingface.co
Sources
01tattrongvu/Jeffan-31b-1710 model card (Hugging Face), commit cebfa86c, checked 2026-10-23 huggingface.co
02tattrongvu/Jeffan-31b-1710 commit history (Hugging Face), checked 2026-10-23 huggingface.co
03Jeffan-31b-1710 LICENSE (Apache 2.0), checked 2026-10-23 huggingface.co
04tattrongvu (Hugging Face profile), checked 2026-10-23 huggingface.co
05Jev Decision Index leaderboard (Hugging Face Space) huggingface.co
06Jev Decision Index 0.4.1 data (index.json, generated 2026-10-22), checked 2026-10-23 huggingface.co
07OpenRouter decisions model list, checked 2026-10-23 openrouter.ai
08JevBench board (Benchmark Heaven), v1.7.47, checked 2026-10-23 (no Jeffan row) benchmarkheaven.com
09google/gemma-4-31B-it (Hugging Face) huggingface.co