Engineering insights, architecture deep dives, and technical solutions
1 posts tagged "onpremise"
深度拆解 Edge/On-Premise 小模型與雲端大模型的雙軌路由架構:基於 Token 概率熵值的早停路由(Early-Exit Confidence Routing)、vLLM logprobs API 整合、PII 強制本地路由、冷啟動優化策略,以及三個演進階段的完整系統設計