Open-Source Multilingual System 1 Decision Engine Technical Whitepaper

DeepOpen is a fully open-source non-autoregressive System 1 decision engine built on Laya, purpose-built for structured decision-making scenarios. It breaks away from the conventional token-by-token text generation paradigm of large language models, completing multi-dimensional classification across over 100 languages in a single forward pass. Tested on NVIDIA T4 GPUs, it achieves latency as low as 33ms per single request and only 7.2ms for batch processing. Trained with the strictly correct reward rule RLCD reinforcement learning framework, and equipped with a built-in intelligent router that automatically matches the optimal checkpoint for every incoming request, DeepOpen thoroughly solves the longstanding pain points of traditional LLMs in classification, routing, and scoring scenarios: slow inference speed, high deployment cost, and vulnerability to hallucinations.

We provide fully reproducible training and evaluation pipelines for two widely recognized intent classification benchmarks, allowing users to replicate our state-of-the-art results with one click:

• Banking77: Full reproduction scripts, dataset configurations and pre-trained checkpoints are available at
  https://github.com/deepopen-com/deepopen/tree/main/banking77
• CLINC150: Complete end-to-end benchmark implementation for intent classification tasks can be accessed at
  https://github.com/deepopen-com/deepopen/tree/main/clinc150

Core Advantages

• Ultra-Low Latency: Non-autoregressive architecture eliminates iterative token generation, delivering millisecond-level inference for real-time decision services.
• Native Multilingual Support: Out-of-the-box classification capability for 100+ languages without additional fine-tuning for most common scenarios.
• Hallucination-Free Decision Making: The deterministic classification design ensures no arbitrary generated content, making outputs fully reliable for production routing and scoring use cases.
• Optimized GPU Efficiency: Far higher throughput than equivalent autoregressive LLMs on the same hardware, drastically reducing inference cost at scale.

Quick Start

https://github.com/deepopen-com/deepopen/tree/main/banking77

benchmark

https://github.com/deepopen-com/deepopen/tree/main/clinc150

You can then directly run the provided benchmark scripts under the banking77 and clinc150 directories to verify performance, or deploy the engine as a local decision service for your own structured scenarios.

License & Contribution DeepOpen is released under a permissive open-source license, welcoming developers, researchers and enterprise users to contribute improvements, extend supported languages, and adapt the engine for more domain-specific decision workflows.

DeepOpen 是基于laya的一款完全开源的非自回归System 1决策引擎，专为结构化类型决策场景设计。

https://github.com/deepopen-com/deepopen/tree/main/banking77

https://github.com/deepopen-com/deepopen/tree/main/clinc150

它摒弃了传统大模型逐Token生成文本的模式，在单次前向传递中即可完成100+种语言的多维度类型判断，单请求延迟低至33毫秒、批量处理仅7.2毫秒（T4显卡实测），依托严格正确评分规则RLCD完成强化学习训练，通过内置智能路由器自动为每个请求匹配最优检查点，彻底解决了传统大模型在分类、路由、打分场景下速度慢、成本高、易产生幻觉的痛点。

DeepOpen 在两个榜单打榜的初步结果： 模型： Deepopen（改进后的 Laya） 榜单： CLINC150 和 Banking77

本地测试： 对照参考成绩，分别位于第 2 位和第 5 位；

DeepOpen 基于三大独立优化的检查点构建，内置的智能路由器可在亚毫秒内完成输入内容的脚本、语言识别，自动调度对应最优模型，无需开发者手动配置切换规则：

• DeepOpen 英文检查点：基于ModernBERT-large 421M参数训练，支持512上下文窗口，在英文单语种任务中实现39.5毫秒单请求延迟，在英文意图分类、XNLI等基准测试中准确率达到0.783-0.860，专为纯英文高并发决策场景优化。
• DeepOpen-multilingual 多语言检查点：基于mmBERT-base 322M参数训练，支持1024上下文窗口，推理速度比英文模型快2倍，覆盖100+种语言，其中45种语言的准确率超过3倍随机基线，在非拉丁语种下性能远超纯英文模型，13种非英文语言的意图分类准确率达到0.451，是英文检查点的1.47倍。
• DeepOpen-typed-decisions 类型决策检查点：基于ModernBERT-large 421M参数训练，支持1024上下文窗口，专门针对结构化类型决策场景微调，在2000个决策样本的基准测试中，准确率达到0.766，超过TypeSafe Jev 1.13.0的0.727，同时Brier分数低至0.062，是目前开源决策模型中精度领先的方案。

1. 零幻觉非自回归设计：全程不生成任何文本内容，所有输出均为开发者预先定义的结构化类型结果，无需后续解析处理，从根源上杜绝了传统大模型的幻觉问题，输出结果100%符合预设的类型边界。
2. 全链路智能路由机制：在模型前向推理前，通过纯Python实现的语言检测模块，在<0.5毫秒内识别输入内容的脚本类型和所属语言，自动匹配最优检查点，彻底避免了英文模型在非拉丁语种下“高置信度错误”的致命问题——此前纯英文检查点在高棉语任务中准确率为0，却给出95.2%的错误置信度，仅靠置信度阈值完全无法规避风险。
3. 严格校准的概率输出：依托RLCD强化学习框架训练，所有输出的置信度分数具备严格的统计意义，经过域温度校准后，ECE（预期校准误差）低至0.081，远优于同类方案，可直接用于生产环境的自动置信门控流程，高置信度请求直接自动处理，低置信度请求自动流转人工审核。
4. 极致的性能表现：在特斯拉T4显卡上实测，单请求推理仅需32.8毫秒，10题批量处理仅72.3毫秒，单T4显卡最高支持每秒103-332个问题的吞吐量，实测速度是TypeSafe Jev的7.8倍。
5. 完全开源零成本部署：采用Apache 2.0开源许可，所有权重完全开放，支持本地自托管，无需调用任何付费API，不存在按Token计费的额外成本，对比TypeSafe Jev每百万Token 0.042美元的定价，长期大规模部署可节省近100%的推理成本。

快速上手部署流程

1. 安装依赖包 直接通过PyPI一键安装最新版本：

pip install deepopen

2. 推荐路由模式（开箱即用） 通过内置Router入口点，自动完成语言检测和模型调度，预加载所有检查点后即可实现亚毫秒级路由切换：

import deepopen
from deepopen import Router

将所有检查点预加载到内存，实现35毫秒以内的极速推理
router = Router(preload=True, device="cuda")

定义任意语言的输入状态
state = {
    "from": "user@acme.com",
    "subject": "Duplicate charge on invoice 4411",
    "body": "Hi, we were billed twice for March. Please refund the duplicate today or we will cancel our plan."
}

定义自定义类型决策规则
questions = {
    "department": {
        "type": "choice",
        "instructions": "Which department should handle this request?",
        "criteria": {
            "billing": "invoices, payments, refunds",
            "technical": "bugs, outages, system errors",
            "sales": "pricing, new contracts",
            "other": "everything else"
        }
    },
    "urgency": {
        "type": "score",
        "instructions": "How urgent is this request?",
        "criteria": ["not urgent", "soon", "critical deadline or blocking issue"]
    },
    "churn_risk": {
        "type": "noul",
        "instructions": "Does the user threaten to cancel or leave?"
    },
    "refund_requested": {
        "type": "noul",
        "instructions": "Does the user explicitly request a refund?"
    }
}

自动路由到最优模型完成推理
res_en = router.predict(state, questions)
print("Department :", res_en["answers"]["department"]["choice"])  输出: billing (confidence: 0.94)
print("Routing    :", res_en["routing"]["model"])                 输出: english

3. 生产环境部署优化 根据业务场景灵活调整内存策略，避免不必要的显存占用：

仅预加载业务所需的检查点，节省显存
router.preload(["english", "multilingual"])

配置LRU缓存策略，最多同时保留2个热模型
router = Router(max_loaded=2)

手动卸载模型释放内存
router.unload()

内置生产级工作流程预设 DeepOpen 提供了预先调优的开箱即用问题模板，无需从零构建规则即可快速落地核心业务场景：

• 智能模型路由：自动判断用户请求的复杂度，将简单任务路由到小模型、复杂任务路由到前沿大模型，大幅降低整体推理成本。
• 实时Prompt防护：精准识别Prompt注入、越狱指令、敏感信息泄露等风险，为大模型应用构建前置安全屏障。
• 内容安全审核：快速检测文本中的毒性内容、骚扰信息、威胁言论，适配多语言社区的内容治理需求。
• 支持工单分诊：自动完成工单意图识别、紧急度打分、用户挫败感评估、流失风险预判，大幅提升客服流转效率。

基准测试与行业对比 基于17416道题的共享基准数据集，DeepOpen与主流决策引擎的实测对比如下：

在高基数标签场景下，TypeSafe Jev在50+选项的任务中表现更优，而DeepOpen可通过调整head_max_len参数扩展选项支持能力，将选项提示预算提升至512，即可支持70+选项的精准分类，满足绝大多数业务场景需求。

开源资源与生态 DeepOpen 全链路资源完全开放，开发者可快速获取所有资料：

• Hugging Face 模型仓库：convaiinnovations/deepopen
• 在线互动演示：convaiinnovations/deepopen-demo
• 完整技术工程报告：Dev.to 专栏
• 微调教程Notebook：支持在Kaggle免费2xT4显卡上完成全流程微调，4-5小时即可完成3万道题的训练，通过RLCD强化学习将基础模型准确率从0.36提升至0.766，完全适配垂直领域的定制化决策需求。

参考资料
[1] DeepL + OpenAI integrations - connect and automate | Bardeen.ai - www.bardeen.ai
[2] Minutes.ai - Smart Government Innovation LAB - www.smartlab.gov.hk
[3] benchmark.ng — Nigeria's AI Decision Engine - benchmark.ng
[4] Phrase: AI-Powered Localization & Translation Platform - Phrase官网
[5] DeepSeek Open Web UI 安装部署全攻略：从环境配置到可视化交互 - 百度智能云
[6] Jev 是什么：前 OpenAI 研究员做的“不说话“模型，只输出带概率的结构化决策-CSDN博客 - CSDN博客
[7] 前OpenAI研究员推Jev模型，绕开文本生成直出决策 - m.counselleap.cn
[8] MCP服务器 - 开放式深度研究模型协议-MCP服务 - www.mcpworld.com
[9] SGLang开源引擎：开源创新与推理革命的融合之路 - 百度智能云
[10] GitHub - weibaohui/openDeepWiki: 完全AI驱动的 DeepWiki ,使用 Go + Eino 技术栈开发 · GitHub - GitHub
[11] 正面硬刚 OpenAI o1！DeepSeek-R1：开启 AI 自主推理新时代，现已开源！ - 知乎 - 知乎
[12] 详解多智能体架构：以 Open Deep Research 项目为例 - 知乎 - 知乎
[13] 集成DeepSeek的开源利器：3款高效应用深度解析 - 百度智能云
[14] 当开源创新遇上推理革命：SGLang如何炼就DeepSeek最强开源推理引擎？_腾讯新闻 - 腾讯网
[15] 3款集成DeepSeek的开源应用推荐：开发者高效工具指南 - 百度智能云
[16] 当开源创新遇上推理革命：SGLang如何炼就DeepSeek最强开源推理引擎？-腾讯云开发者社区-腾讯云 - 腾讯云
[17] DeepSeek-V3.2：开源大语言模型的新里程碑，在推理与智能体任务中突破性能边界 - 知乎 - 知乎
[18] Open Deep Research V2：新架构！上下文工程最新实践 - 知乎 - 知乎
[19] 深度探索DeepSeek：解锁高效开发与智能决策的新范式 - 百度智能云
[20] 【AI大模型】 Langchain+DeepSeek R1从入门到精通，收藏这一篇就够了！！_langchain从入门到实战-CSDN博客 - CSDN博客
[21] 开源×推理”双引擎驱动：SGLang打造DeepSeek开源推理新标杆 - 百度智能云

Multilingual, non-autoregressive System 1 decision engine. Typed decisions over 100+ languages in a single forward pass — 33 ms — trained with reinforcement learning against strictly proper scoring rules (RLCD), with a router that picks the right checkpoint per request.

deepopen evaluates typed questions (choice, score, noul) over any state (text, email, ticket or JSON document) in a single forward pass — 33 ms for one question, 7.2 ms/question batched, measured on a T4. No text generation, so nothing to parse and nothing to hallucinate.

Three checkpoints, and a Router that picks between them per request:

pip install deepopen

deepopen ships three checkpoints. The built-in Router is the recommended entry point: it evaluates any state in any language, automatically detects scripts and languages in sub-milliseconds, and dispatches to the optimal checkpoint in a single forward pass.

import deepopen
from deepopen import Router

# Preload checkpoints into memory for instant sub-35ms routing
router = Router(preload=True)

# 1. State in any language or schema
state = {
    "from": "user@acme.com",
    "subject": "Duplicate charge on invoice #4411",
    "body": "Hi, we were billed twice for March. Please refund the duplicate today or we will cancel our plan."
}

# 2. Define your typed questions
questions = {
    "department": {
        "type": "choice",
        "instructions": "Which department should handle this request?",
        "criteria": {
            "billing": "invoices, payments, refunds",
            "technical": "bugs, outages, system errors",
            "sales": "pricing, new contracts",
            "other": "everything else"
        }
    },
    "urgency": {
        "type": "score",
        "instructions": "How urgent is this request?",
        "criteria": ["not urgent", "soon", "critical deadline or blocking issue"]
    },
    "churn_risk": {
        "type": "noul",
        "instructions": "Does the user threaten to cancel or leave?"
    },
    "refund_requested": {
        "type": "noul",
        "instructions": "Does the user explicitly request a refund?"
    }
}

# 3. English state -> automatically routed to deepopen (ModernBERT-large, 39.5 ms)
res_en = router.predict(state, questions)
print("Department :", res_en["answers"]["department"]["choice"])  # -> billing (confidence: 0.94)
print("Routing    :", res_en["routing"]["model"])                 # -> english

# 4. Hindi state -> automatically routed to deepopen-multilingual (mmBERT-base, 32.8 ms)
res_hi = router.predict({"body": "मुझसे दो बार शुल्क लिया गया, कृपया पैसे वापस करें।"}, questions)
print("Department :", res_hi["answers"]["department"]["choice"])  # -> billing (confidence: 0.86)
print("Routing    :", res_hi["routing"]["model"])                 # -> multilingual

# 5. Explicit override when you want a specific checkpoint
res_td = router.predict(state, questions, model="typed-decisions")

Every result carries full routing metadata explaining why the choice was made:

res_hi["routing"]
# {
#   'model': 'multilingual',
#   'repo': 'convaiinnovations/deepopen/multilingual',
#   'reason': 'non-Latin script (devanagari, 100% of letters); the English checkpoint cannot read it'
# }

Inspect a routing decision without running any forward pass:

router.route({"body": "Der Kunde wurde zweimal belastet"}, questions).reason
# "Latin script but language looks like 'de', not English"

On a shared benchmark (17,416 questions, one T4 GPU, identical questions per model):

The English checkpoint collapses on non-Latin scripts (Khmer scores 0.000 accuracy at 0.952 confidence). Because the model stays confident while being wrong, confidence gating cannot save you. Router detects the script in <0.5 ms pure Python before the forward pass.

A cold checkpoint build costs seconds; language detection costs microseconds. At the default max_loaded=1, traffic that alternates languages rebuilds a model on every request (measured at a 7.4 s median reload on CPU and 10.3 s on T4).

For a server or production app, preload:

# Every checkpoint resident in memory; language flips cost detection only (<1 ms)
router = Router(preload=True)
router = Router(preload=True, device="cuda")

# Or preload only the specific checkpoints you serve:
router.preload(["english", "multilingual"])

# If your app already built an agent, attach it to avoid duplicate VRAM:
router.attach("english", existing_agent)

# Manage resident memory (default keeps 1 hot, LRU eviction)
router = Router(max_loaded=2)       # keep two hot
router.unload()                     # free memory

If you only need a single checkpoint for a dedicated pipeline, you can load models directly:

import deepopen

# 1. Load a specific checkpoint directly from the hub
agent = deepopen.load("convaiinnovations/deepopen")                           # English root
agent_ml = deepopen.load("convaiinnovations/deepopen", subfolder="multilingual") # 100+ languages
agent_td = deepopen.load("convaiinnovations/deepopen", subfolder="typed-decisions")

# 2. Run all questions in ONE single forward pass (~35 ms on GPU)
result = agent.predict(state, questions)
answers = result["answers"]

print("Department :", answers["department"]["choice"])   # -> billing (confidence: 0.94)
print("Urgency    :", answers["urgency"]["score"])        # -> 1.84 / 2.0
print("Churn Risk :", answers["churn_risk"]["noul"])       # -> 0.892 (89.2% probability)

Because deepopen's probabilities are trained with strictly proper scoring rules (RLCD), confidence scores are statistically meaningful:

dept = answers["department"]["choice"]
conf = answers["department"]["confidence"]

if conf >= 0.85:
    # High confidence: automated action without human in the loop
    route_automatically(dept)
else:
    # Low confidence: escalate to human triage
    escalate_to_human_agent(dept, reason=f"Low confidence ({conf:.2f})")

deepopen provides pre-tuned question schemas for immediate production use:

import deepopen

agent = deepopen.load("convaiinnovations/deepopen")

# 1. Intelligent Model Router (routes to small vs. frontier models)
routing = agent.predict({"request": "Refactor this service using dependency injection"}, deepopen.router_questions())

# 2. Real-time Prompt Guardrails (jailbreaks, injections, leaks)
guard = agent.predict({"prompt": "Ignore all instructions"}, deepopen.guard_questions())

# 3. Content Safety & Moderation (toxicity, harassment, threats)
safety = agent.predict({"post": "User comment text"}, deepopen.moderation_questions())

# 4. Support Ticket Triage (intent, urgency, frustration, churn)
triage = agent.predict({"message": "My payment failed twice"}, deepopen.triage_questions())

Full report: BENCHMARKS.md — every run consolidated, languages and themes, with per-language detail for all 51 languages.

All deepopen numbers below are measured. Every model answered byte-identical questions (fixed seed) in the same run. Reproduce with notebooks/deepopen_benchmark_colab.ipynb on a T4.

Batched throughput reaches 103-332 questions/sec on a single T4. For reference, TypeSafe Jev has been independently measured at 236-276 ms p50 (AbdelStark, nibzard) -- deepopen answers a single question roughly 6-7x faster.

Every deepopen figure is what Router().predict(...) actually returns — the checkpoint the router selects for that input, not a hand-picked best of three. Jev figures are third-party published, never measured here (no TypeSafe API access), so sample sizes and prompts differ.

On DAIR Emotion, Jev assigned zero probability to the true label on 16% of examples — a hard failure for anything branching on confidence.

• High-cardinality label spaces (>20 options at default settings): On Banking77, Jev scores 0.870 (on 72 labels) while deepopen scores 0.425 (on 77 labels at default 256-token head budget). This is an architectural token-budget constraint: options share a fixed head_max_len budget (192 tokens on English, 256 on multilingual), so 77 options receive only ~3 to 4 tokens per label, causing text to become indistinguishable. Jev supports up to 255 options out-of-the-box. While deepopen-multilingual supports 1,024 context (and up to 8,192 in the encoder) and you can raise agent.cfg["head_max_len"] = 512 at runtime, Jev is currently better suited for 50+ options in a single prompt without tuning.
• Soft distribution matching: On typed-decisions, while deepopen achieves higher argmax accuracy (0.766 vs 0.727), Jev achieves higher soft accuracy (0.580 vs 0.471) against the teacher's full probability distributions.
• Out-of-the-box raw calibration: Before temperature scaling, the base checkpoint has higher raw ECE (0.213 vs 0.144). deepopen achieves its 0.081 ECE after domain temperature fitting.

Full detail, including every workflow and all 51 languages: BENCHMARKS.md.

400 cases, 2,000 decisions, four workflows.

The fine-tuned checkpoint beats Jev by 3.9 points and clears the teacher ceiling, with 2.4x better Brier and 1.6x better score MAE. It wins on all four workflows: invoice processing 0.804, security incidents 0.766, customer service 0.764, agent-trace observability 0.730. By primitive: noul 0.857, choice 0.733, score 0.723.

Two places it still trails Jev: soft accuracy (0.471 vs 0.580 — its argmax is better but its distributions match the teacher less well) and ECE (0.213 vs 0.144), which temperature fitting addresses.

The base checkpoints sit below the majority-class baseline (0.362 and 0.342 against 0.461). All of the capability on this benchmark comes from fine-tuning.

Across all 51 languages the English checkpoint macro-averages 0.227 with macro ECE 0.733, and only 23 of 51 languages clear 3x random. Khmer scores 0.000 at 95.2% confidence. This is why Router exists: the model's own confidence gives no warning, so the routing decision has to be made before the forward pass.

Both checkpoints are over-confident as shipped. Refitting one temperature per (question type, option count) on held-out data moves mean ECE 0.466 -> 0.081 (deepopen) and 0.314 -> 0.106 (deepopen-multilingual). deepopen-multilingual ships with no fitted temperatures at all, so fit them before relying on its probabilities.

• The base checkpoints are near chance on typed-decisions zero-shot -- 0.362 and 0.352 against a 0.318 random baseline and a 0.461 majority-class baseline. The 0.766 figure comes from the checkpoint fine-tuned on that benchmark's own training split. deepopen is a fast base to specialise, not a zero-shot decision engine.
• High-cardinality choice questions and token budgets: Sequences split into an option prompt budget (head_max_len) and the remaining document/state budget (max_len - head_max_len):
  • deepopen (English) defaults to 512 context (head_max_len = 192, ~320 tokens for state).
  • deepopen-multilingual and deepopen-typed-decisions default to 1,024 context (head_max_len = 256, ~768 tokens for state; mmBERT-base encoder supports up to 8,192 with RoPE). At default settings, a 77-option question like Banking77 allocates only (256 - 16) // 77 ≈ 3–4 tokens per label, which causes accuracy to fall off sharply (0.425 vs Jev's 0.870). If evaluating 50+ options in a single question:
  1. Raise agent.cfg["head_max_len"] = 512 and agent.cfg["max_len"] = 1024 (or up to 2048 / 4096 / 8192) so every option has enough tokens to remain distinct.
  2. Or split large option sets into a two-step coarse-to-fine hierarchical choice.
• Ordinal score questions are the weakest primitive (SST-5 0.372).
• deepopen collapses outside English; deepopen-multilingual is weaker on English. Route, or pick deliberately.

• Hugging Face Model: convaiinnovations/deepopen
• Interactive Web Demo: convaiinnovations/deepopen-demo
• Engineering Writeup: Read the full story on Dev.to

Fine-tune deepopen on your own domain data. The notebook runs on Kaggle's free 2xT4 GPUs and does the whole loop: build the dataset, train with RLCD (proper-scoring-rule rewards, GRPO-style policy gradient), fit calibration temperatures, evaluate, and push the result to the Hub.

• notebooks/deepopen_finetune_typed_decisions_2xT4_kaggle.ipynb

Fine-tuning is where most of the value is. On the typed-decisions benchmark the base checkpoints score near chance zero-shot (0.36 and 0.35 against a 0.318 random baseline), while the fine-tuned checkpoint reaches 0.766 on the same 2,000 decisions -- above TypeSafe Jev's published 0.727 and above the 0.735 teacher self-agreement ceiling. Treat deepopen as a fast base to specialise, not as a zero-shot decision engine.

Runtime on 2xT4 is roughly 4-5 hours for 4 epochs over ~30k questions.

Apache 2.0. Developed by Deep Open