[{"data":1,"prerenderedAt":2497},["ShallowReactive",2],{"site-info":3,"post-running-is-not-done":99,"related-zh-TW-running-is-not-done":1525},{"id":4,"company":5,"extension":16,"meta":17,"services":18,"social":96,"stem":97,"__hash__":98},"site\u002Fsite.yml",{"name":6,"nameEn":7,"email":8,"address":9,"addressEn":10,"lineUrl":11,"lineId":12,"duns":13,"youtube":14,"youtubeId":15},"銡策數位有限公司","Jecision Digital Co., Ltd.","contact@jecision.com","231 新北市新店區北新路二段 262 號 3 樓之 3","3F.-3, No. 262, Sec. 2, Beixin Rd., Xindian Dist., New Taipei City 231, Taiwan","https:\u002F\u002Fline.me\u002FR\u002Fti\u002Fp\u002F@374qnumi","@374qnumi","963905705","https:\u002F\u002Fwww.youtube.com\u002Fshorts\u002F6UE0WTmyjsg","6UE0WTmyjsg","yml",{},[19,43,69],{"key":20,"zh":21,"en":32},"audit",{"title":22,"tagline":23,"summary":24,"points":25,"deliverable":29,"detail":30,"duration":31},"企業 AI 導入健檢","先看清楚，再花錢","60 分鐘線上訪談，盤點最耗時、最依賴人力的營運環節；我們會告訴你哪些工作現在就能交給 AI、哪些還不成熟。",[26,27,28],"現況盤點：梳理日常營運中最耗時、最依賴人力的環節","痛點診斷：判斷哪些工作現在就能交給 AI，哪些建議先保留人力","可行性評估：依你的業務特性與預算，篩選適合的模型與工具","《企業 AI 導入建議報告書》（PDF）","含 1–2 個「開發成本最低、成效最明顯」的優先項目、技術棧建議、預期節省的人力時間與下一步","訪談 60 分鐘，報告書 3 個工作天內交付",{"title":33,"tagline":34,"summary":35,"points":36,"deliverable":40,"detail":41,"duration":42},"AI Readiness Audit","Look before you spend","A 60-minute remote audit of the workflows that eat the most time and people-power — and an honest answer on what AI can take over today, and what it cannot.",[37,38,39],"Audit the daily workflows that consume the most time and headcount","Triage which tasks AI can own today, and which should stay human","Shortlist the models and tools that fit your budget and constraints","AI Adoption Recommendation Report (PDF)","One or two highest-ROI entry points, a recommended tech stack, expected hours saved, and the concrete next step","60-minute call, report delivered within 3 business days",{"key":44,"zh":45,"en":56},"prompt",{"title":46,"tagline":47,"summary":48,"points":49,"deliverable":53,"detail":54,"duration":55},"提示詞工程","讓 AI 穩定產出，而不是碰運氣","針對你的實際任務設計提示詞與多步驟工作流，並做極限測試，確保換了資料、換了人操作也不會走鐘。",[50,51,52],"客製提示詞：ChatGPT、Claude、Gemini、Midjourney 等主流工具的專屬模板","除錯與優化：修掉幻覺、格式跑掉、時好時壞的問題","工作流建置：把複雜任務拆成多步驟提示詞與可重複的 SOP","提示詞文件或設定檔","附使用說明與範例，並提供 1 對 1 教學，確保你與團隊能自己維護","依任務複雜度報價，通常 3–7 個工作天",{"title":57,"tagline":58,"summary":59,"points":60,"deliverable":66,"detail":67,"duration":68},"Prompt Engineering","Make AI produce reliably, not by luck","Prompt sets and multi-step workflows designed for your actual tasks, stress-tested so they keep working when the data or the operator changes.",[61,62,64],"Custom prompts for ChatGPT, Claude, Gemini, Midjourney and friends",{"Debugging and tuning":63},"kill hallucinations, broken formats and flaky output",{"Workflow design":65},"break complex work into repeatable multi-step SOPs","Prompt library or exportable config","Ships with usage notes and examples, plus a 1:1 walkthrough so your team can maintain it","Quoted by complexity, typically 3–7 business days",{"key":70,"zh":71,"en":82},"poc",{"title":72,"tagline":73,"summary":74,"points":75,"deliverable":79,"detail":80,"duration":81},"極速 PoC","五天，看到能跑的東西","不談架構藍圖，直接做一個能動的雛形。五個工作天內交付線上可操作的智能體，用你真實的資料驗證。",[76,77,78],"Day 1：15 分鐘對焦，鎖定單一痛點與資料盤點","Day 2–4：模型串接、邏輯設計與雛形開發","Day 5：Demo 展示與 AI 擴展建議書","可線上操作的智能體雛形","交付範圍為概念驗證與展示，不含後續維運或大規模伺服器部署；要進階成完整系統會另行報價","5 個工作天",{"title":83,"tagline":84,"summary":85,"points":86,"deliverable":93,"detail":94,"duration":95},"Rapid PoC","Five days to something that runs","No architecture decks — a working prototype built in five business days, validated against your real data.",[87,89,91],{"Day 1":88},"15-minute scoping call, one pain point, data inventory",{"Day 2–4":90},"model integration, logic design, prototype build",{"Day 5":92},"live demo plus an AI expansion recommendation","A runnable online agent prototype","Proof of concept and demo scope only; ongoing operations and large-scale deployment are quoted separately","5 business days",[],"site","XTwARfVAOcrXNiKehsM6V6cdkd2HXy4d5yZVrSWrI3Q",{"zh":100,"en":898},{"id":101,"title":102,"author":103,"body":104,"cover":878,"coverAlt":879,"date":880,"description":881,"draft":882,"extension":883,"featured":882,"lang":884,"meta":885,"navigation":296,"ogImage":886,"path":887,"seo":888,"slug":889,"stem":890,"tags":891,"translationKey":889,"updated":896,"__hash__":897},"blog\u002Fblog\u002Fzh-TW\u002Frunning-is-not-done.md","「跑得起來」跟「真的幹完活」是四件事","Jecision",{"type":105,"value":106,"toc":864},"minimark",[107,114,118,121,124,131,134,137,140,143,154,157,161,164,192,195,200,203,270,273,276,279,282,285,302,306,309,312,315,321,337,340,343,346,349,352,355,358,362,365,368,371,377,380,383,386,397,400,403,406,409,412,415,418,436,439,442,445,448,451,454,457,460,463,466,469,519,522,525,528,531,534,537,540,543,546,549,552,555,558,561,564,567,612,615,618,621,630,633,636,639,642,645,656,659,662,665,668,730,733,736,739,742,745,762,765,768,771,774,806,809,820,833,841,848,852,855,858,861],[108,109,110],"blockquote",{},[111,112,113],"p",{},"數字全部來自 arXiv 2610.34241v1。那是作者自己的工程紀錄，不是受控實驗；「我認為」的部分我會標出來。",[115,116,117],"h2",{"id":117},"先講一個畫面",[111,119,120],{},"「模型有回應了。」這是最容易騙過驗收的一句話。API 回 200、文字讀得懂，但工具可能根本沒被呼叫出去。",[111,122,123],{},"2026 年 7 月，Vaibhav Jain 把幾台筆電串起來，跑一個單機放不下的模型。聊天可以，當 coding agent 用卻慢到不能用；每次重啟，32B 的權重要在 Wi-Fi 上傳五分鐘（8.7 GB）。最後他收掉分散式，改成單機。",[111,125,126,127],{},"這件事的重點不是「分散式不行」。是他從那時候開始發現：",[128,129,130],"strong",{},"模型跑得起來，跟它能不能幫你幹完活，是四件不同的事。",[111,132,133],{},"這篇要拆的就是那四層，還有驗收單上通常漏掉的那一欄。",[111,135,136],{},"一個模型權重檔（checkpoint）載得進去，丟一句話會回話，然後呢？你叫它去看一個檔案、呼叫一個工具、維持一段長 session、守住一個 repo 的介面約定。這些它可能全部做不到，而你的 API 還是回你 200。",[111,138,139],{},"問題出在「相容」這兩個字。同一個號稱相容的 endpoint，後面可能是完全不同的東西。工具呼叫的格式、一輪對話怎麼結束、context 怎麼被改寫、快取有沒有重用，四件事都可以不一樣。",[111,141,142],{},"只要有一件不一樣，你在 A 模型上驗過的流程，換到 B 模型就會壞。而且通常不會給你錯誤訊息。",[111,144,145,146,153],{},"這篇是一份單人經驗報告的導讀。",[147,148,152],"a",{"href":149,"rel":150},"https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.34241",[151],"nofollow","論文","是 arXiv 2610.34241，作者是獨立研究者，紀錄的時間橫跨 2026 年 7 月到 10 月，跑過 vLLM、SGLang、llama.cpp 和 MLX。",[111,155,156],{},"它做的事很樸素：把「能跑」拆成四層，再用 45 場有隱藏檢查的實驗，告訴你第四層要怎麼驗、代價在哪裡。",[115,158,160],{"id":159},"四層不是一個開關","四層，不是一個開關",[111,162,163],{},"作者把「模型有沒有用」拆成四層：",[165,166,167,174,180,186],"ol",{},[168,169,170,173],"li",{},[128,171,172],{},"服務得起來","：checkpoint 載得進去，送一句話有東西回來。",[168,175,176,179],{},[128,177,178],{},"文字連貫","：回得出人話，讀得懂，能回答問題。",[168,181,182,185],{},[128,183,184],{},"工具被解析並執行","：它說「我要讀這個檔案」，client 真的收到，真的去讀。",[168,187,188,191],{},[128,189,190],{},"通過獨立行為檢查","：產出的程式碼，在另一個不屬於它的檢查器裡，真的把事做對。",[111,193,194],{},"論文裡有一句話是整篇的骨架：",[108,196,197],{},[111,198,199],{},"一個 endpoint 可以滿足前三層，卻在第四層全錯。而且如果第三層過不了，你根本量不到第四層。",[111,201,202],{},"（這是作者的話，我照原意翻。）",[204,205,206,222],"table",{},[207,208,209],"thead",{},[210,211,212,216,219],"tr",{},[213,214,215],"th",{},"層級",[213,217,218],{},"你要問的問題",[213,220,221],{},"通過之後，你還不知道什麼",[223,224,225,237,248,259],"tbody",{},[210,226,227,231,234],{},[228,229,230],"td",{},"1 服務",[228,232,233],{},"載得進去、有回應嗎？",[228,235,236],{},"回的是不是人話",[210,238,239,242,245],{},[228,240,241],{},"2 連貫",[228,243,244],{},"讀得懂、講得出道理嗎？",[228,246,247],{},"它會不會真的動手",[210,249,250,253,256],{},[228,251,252],{},"3 工具",[228,254,255],{},"工具呼叫被解析、被執行了嗎？",[228,257,258],{},"執行完的結果對不對",[210,260,261,264,267],{},[228,262,263],{},"4 驗證",[228,265,266],{},"產出通過獨立的行為檢查了嗎？",[228,268,269],{},"這一層過了，才叫「幹完活」",[111,271,272],{},"這四層不是「程度」的差別，是四道不同的門。第一層過了，不代表第二層；第二層過了，也不代表它會動手。很多人停在第二層就以為快成功了，因為它講得有模有樣。",[111,274,275],{},"你平常在驗收單上看到的三個層級，大概是這樣對應。「模型有回應」是第 1 層加第 2 層；「程式跑起來」是第 3 層；「測試通過」是第 4 層。",[111,277,278],{},"但「測試通過」要看是誰的測試。廠商自己寫的測試過了，你可能還停在第 3 層；第 4 層的檢查，得是別人寫的、在驗收前就凍結的。",[111,280,281],{},"我把第 1、2 層拆開，是因為它們的失敗長得很像、原因完全不同。一個是服務沒通；一個是通了，但只會講話。",[111,283,284],{},"（如圖 1）",[286,287,290,291,290,298],"figure",{"className":288},[289],"article-figure","\n  ",[292,293],"video",{"src":294,"poster":295,"autoPlay":296,"loop":296,"muted":296,"playsInline":296,"ariaLabel":297},"\u002Fimages\u002Fblog\u002Frunning-is-not-done\u002Flevels.mp4","\u002Fimages\u002Fblog\u002Frunning-is-not-done\u002Flevels-poster.jpg",true,"四層進度：服務、連貫、工具、驗證，第三與第四層之間有斷層",[299,300,301],"figcaption",{},"圖 1：四層不是程度差別，是四道不同的門。前三層都過，第四層仍可能全錯。",[115,303,305],{"id":304},"同一個-api不同的協議","同一個 API，不同的協議",[111,307,308],{},"作者把 Claude Code 接上自己寫的 gateway，後面接不同的模型。他原本想的是「換一個 endpoint 而已」，實際上的感覺，比較像換一個工作習慣完全不同的人。",[111,310,311],{},"有的模型照工具契約做。有的用文字描述它「打算執行這個指令」。有的會把它覺得不順的介面改掉，例如把一個本來穩定的函式簽名重寫成它喜歡的樣子。這些不是模型笨，是它跟你的 client 對「工具」這件事的理解不一樣。",[111,313,314],{},"他統計了 gateway 上的真實請求。Gemma 4 有 492 個帶工具的請求，其中 294 個解析出 tool_calls。144 個以 stop 結束、沒有 tool call；還有 54 個沒有存到解析結果。",[111,316,317,318],{},"那個 144 不能全部算失敗。正常收尾也會是 stop，沒有 tool call 可能只是任務做完了。這是本文的第一個陷阱：",[128,319,320],{},"「沒有工具呼叫」本身不是錯誤訊號，要看當下該不該有。",[111,322,323,324,328,329,332,333,336],{},"還有更直接的。Gemma 4 的存檔輸出裡有 8 個 ",[325,326,327],"code",{},"\u003C|tool_call>"," 前綴的畸形呼叫，解析器完全沒接住。GPT-OSS-120B 走 SGLang（一種推論伺服器）時，",[325,330,331],{},"tool_choice=required"," 會跟它自己的工具格式（Harmony）打架，得降成 ",[325,334,335],{},"auto","。工具回合的輸出上限也要放寬，不然 reasoning 和 tool JSON 會一起被截斷。",[111,338,339],{},"回合控制是另一半。對話的識別碼（session identifier）不見、工具呼叫繞圈、thinking channel 對不上。任何一個都會讓對話「看起來還在跑，其實已經迷路」。",[111,341,342],{},"最值得記下來的是這一段：API 格式（schema）檢查全部通過，接上真的 Claude Code 之後還是壞。",[111,344,345],{},"一次是 client 的工作目錄沒有被放進 request，模型只能猜路徑。一次是 streaming 的工具參數被當成「替換」而不是「附加」，參數被切斷，模型收到一個殘缺的呼叫。",[111,347,348],{},"這兩個都不會出現在單元測試裡。它們要你真的把 client 接上去，跑一次完整流程才會出現。",[111,350,351],{},"作者的診斷問題很簡單：工具失敗的時候，是模型不會用，還是中間某個環節把呼叫弄丟了？",[111,353,354],{},"如果是後者，就修在源頭：模型的 template、parser 規則、serve flag、request 翻譯。而不是在 gateway 上不斷補丁。他後來的做法是把 gateway 縮成薄的翻譯層，只做協議與安全檢查。上游失敗就往外傳，不要讓一個失敗的 stream 看起來像正常結束。",[111,356,357],{},"「能被設定」不等於「真的跑起來、被評測過」。這句話對採購很有用：供應商說「支援」，通常只代表它能被設定。",[115,359,361],{"id":360},"對話看起來連續其實中間斷了","對話看起來連續，其實中間斷了",[111,363,364],{},"第三層過了，還有一個更隱形的問題：對話。",[111,366,367],{},"對你來說，那是一段連續的對話。對 runtime 來說，它是三樣東西：token 前綴（對話開頭的固定內容）、KV cache（算過的注意力狀態）、執行 slot。你看到的「連續」，它不一定看得到。",[111,369,370],{},"作者留下 4,165 個 gateway 回合、623 個除錯工作的紀錄。其中 85 筆是冷前綴：完全沒有重用，30,000 到 36,000 個 input token。280 筆是高重用：至少 90% 重用、少於 2,000 個新 token。",[111,372,373,374],{},"首字延遲的中位數：",[128,375,376],{},"冷前綴 69.4 秒，高重用 2.8 秒。",[111,378,379],{},"差了約 25 倍。",[111,381,382],{},"但這裡要很小心。作者自己標了「觀察性資料」：兩組工作的組成不同、不是隨機分配、沒有配對。所以不能寫成「快取讓速度快 25 倍」。能說的是：前綴有沒有被重用，跟你的體感差距很大，而且大到不像是雜訊。",[111,384,385],{},"（如圖 2）",[286,387,290,389,290,394],{"className":388},[289],[292,390],{"src":391,"poster":392,"autoPlay":296,"loop":296,"muted":296,"playsInline":296,"ariaLabel":393},"\u002Fimages\u002Fblog\u002Frunning-is-not-done\u002Fcache.mp4","\u002Fimages\u002Fblog\u002Frunning-is-not-done\u002Fcache-poster.jpg","同一段對話的兩條路徑：前綴穩定時重用快取，前綴被改寫時重算",[299,395,396],{},"圖 2：同一段對話。前綴沒變，後端重用快取；前綴被改了一點點，整段 context 重算。",[111,398,399],{},"機制很樸素。推論後端可以重用「完全相同的 token 前綴」的 KV 狀態。所以只要前綴變了，整段 context 就要重算一次 prefill（把前綴重新算過）。哪怕只是前面多一個時間戳、換一個 system banner、工具定義的順序不同、JSON 序列化方式不一樣。",[111,401,402],{},"你改的是一個 banner；你付的是整段 context。",[111,404,405],{},"還有一種更冤的：旁支請求。coding agent 的流量不是只有主對話，還有生成標題、權限確認這類小請求。如果後端只有一個可重用的執行 slot，一個旁支請求就可能把主對話的駐留狀態擠掉。",[111,407,408],{},"他的紀錄裡，65 個被歸類為 auxiliary 的請求，有 44 個沒有快取重用。在高重用的紀錄中，auxiliary 的首字延遲中位數是 12.2 秒（13 筆），主請求是 2.7 秒（267 筆）。",[111,410,411],{},"長對話超過某個長度時，系統會把它壓成摘要（compaction）。這裡也有類似的假象：他刻意把壓縮門檻調低，在 12 次測試裡，子代理壓縮觸發了 6 次。這 6 次任務全部沒有完成，而且其中一筆還被標成 PASS。",[111,413,414],{},"「壓縮發生了」不是「工作完成了」。這兩件事在 log 裡長得很像。",[111,416,417],{},"所以他把 session 連續性當成應用的責任，而不是後端的參數：",[419,420,421,424,427,430,433],"ul",{},[168,422,423],{},"訊息序列盡量 append-only，除非真的需要改前面的內容。",[168,425,426],{},"已經送出的工具訊息，不要為了美觀重新序列化。",[168,428,429],{},"streaming 的 tool JSON 和最後的完整版本要一致。",[168,431,432],{},"每回合的 metadata 不要動到被凍結的 system 前綴。",[168,434,435],{},"保住 session affinity（下一回合要回到同一個執行位置）。穩定的 prompt 不夠，slot 換了就沒有那份快取。",[111,437,438],{},"快取不是後端參數，是應用的紀律。",[115,440,441],{"id":441},"第四層才算驗收",[111,443,444],{},"接下來是這篇論文真正做實驗的地方。當部署、協議、session 都處理到可以跑範圍明確的任務之後，問題才真正出現。一個開源的小模型，能不能交出通過檢查的程式碼？",[111,446,447],{},"設計是這樣：五類 TypeScript 任務，報價、按比例分攤、客戶、拆分、零件；每類每種跑三次，總共 45 場。三種條件是：直接寫；優化委派（父層只寫規格、審查並套用工人的產出）；可分批的委派。最後一種允許父層把工作拆成連續幾包，但只有少數場次真的拆。",[111,449,450],{},"父層（負責規劃與審查的那一層）固定是 Claude Code 加 Claude Opus 5.5。工人（被委派實作的那顆模型）是量化過的 Qwen3-Coder 30B-A3B。每一場都用任務專屬的隱藏行為檢查判定，不看 agent 自己說「我完成了」。",[111,452,453],{},"結果：直接寫 15\u002F15；優化委派 12\u002F15；可分批的委派 14\u002F15。",[111,455,456],{},"這三個數字不能排名。每一格只有三次，信賴區間是重疊的；作者也明說，這些資料不足以建立一般的成功率差異。能說的是：工人真的交得出東西。30 場委派裡，26 場驗證通過，而且都保留了工人寫的程式碼。但不是每次。",[111,458,459],{},"另外，那 14\u002F15 不是「批次比較好」的證據。15 場裡只有 5 場真的用到批次呼叫，其餘是單次委派。論文自己說這是「批次標記的適應式策略」，不是批次與非批次的對照。",[111,461,462],{},"第二組更嚴格。15 個任務，各跑三次，直接與委派各 45 場。工人產出的檔案要先通過一份固定的介面契約，才能進父層審查。",[111,464,465],{},"直接寫 45\u002F45。委派給 Nemotron Cascade 2 工人 12\u002F45，其中 11 場是工人真的替換成功，1 場是父層改寫後才通過。Qwen3-Coder 工人 0\u002F45。",[111,467,468],{},"這個 45\u002F45 也要保留一點判斷空間。題目是為這份研究準備的，而且只有五類、每類三題；論文沒有做難度分層。",[204,470,471,484],{},[207,472,473],{},[210,474,475,478,481],{},[213,476,477],{},"實驗",[213,479,480],{},"直接寫",[213,482,483],{},"委派給工人",[223,485,486,497,508],{},[210,487,488,491,494],{},[228,489,490],{},"45 場試驗",[228,492,493],{},"15\u002F15",[228,495,496],{},"Qwen3-Coder 12\u002F15（優化委派）、14\u002F15（可分批）",[210,498,499,502,505],{},[228,500,501],{},"第二組（15 任務 × 3）",[228,503,504],{},"45\u002F45",[228,506,507],{},"Nemotron 12\u002F45（11 場是工人替換）；Qwen 0\u002F45",[210,509,510,513,516],{},[228,511,512],{},"第二組，中斷算失敗",[228,514,515],{},"37\u002F45",[228,517,518],{},"Nemotron 8\u002F45",[111,520,521],{},"這裡有一個必須寫出來的細節：有 16 場因為訂閱額度用完、連模型都沒有呼叫就中斷，後來被替換掉。這 16 場是直接組 8 場、Nemotron 組 8 場。替換之後，直接組多拿到 8 場成功，Nemotron 組多拿到 4 場。",[111,523,524],{},"如果把原本中斷的 16 場算成失敗，數字會變成 37\u002F45 與 8\u002F45。",[111,526,527],{},"作者兩個都寫了。這些是排程中斷，不是推論失敗。但如果你要拿這個數字去說服人，兩種算法都要看得到。",[111,529,530],{},"還有一件事要誠實講。五個任務的定義和隱藏測試，是在 AI 協助下產生的。合併後的測試檔，在第一場評估前大約 39 秒才建立，而且直到最後一場跑完才第一次進版控。",[111,532,533],{},"作者沒有把這件事藏起來。他能主張的是：45 場存下來的任務指令和可見檢查完全一致。事後比對的分數標籤，也和評分程式（scorer）對得上。但他不能宣稱這是一個有獨立事前快照的 benchmark。",[111,535,536],{},"第一組的測試流程（harness）也有限制。父層的 constraints 欄位沒有被轉給工人。gateway 會丟掉重複的 system message、截斷超過 1,800 字元的訊息。41 個重建的工人 prompt 裡，有 2 個各少了約 220 字元，都在失敗的場次。",[111,538,539],{},"所以「工人不聽指令」這個結論要打折。有些失敗是工人不會，有些是它根本沒收到完整的指令。",[111,541,542],{},"那 0\u002F45 呢？那不是「這顆模型什麼都做不到」的意思。那 45 場沒有 direct 對照組，父層接受與套用沒有重播，兩個研究的父層快取狀態也沒有重置。作者自己把它限定成「這批任務、這份契約、這個工人設定」的結果。",[111,544,545],{},"驗證是第四層，不是第三層的副產品。",[115,547,548],{"id":548},"可是帳單反過來了",[111,550,551],{},"直覺是這樣的：前沿模型的輸出 token 很貴，本地小模型便宜。那把實作委派給本地工人，不就省錢了嗎？",[111,553,554],{},"問題是委派不是免費。父層要寫規格、要看工人做了什麼、要判斷能不能用、失敗要收拾。這些都是前沿模型的 token。",[111,556,557],{},"作者把成本拆成：父層 context、寫規格、審查、收拾、工人。這篇只量到前沿模型那幾項。工人端的電費、硬體、時間都沒有算進去，而且他明講「未量測不能當成零」。",[111,559,560],{},"第一組，每次驗證成功的成本：直接寫 $0.136；委派 $0.214（優化版）與 $0.199（另一種）。大約 1.6 倍。",[111,562,563],{},"第二組：直接 $0.153；Nemotron 委派 $1.313。8.6 倍。",[111,565,566],{},"Qwen 那組的「每次成功成本」算不出來，不是零——因為它一場都沒成功。",[204,568,569,583],{},[207,570,571],{},[210,572,573,575,577,580],{},[213,574,477],{},[213,576,480],{},[213,578,579],{},"委派",[213,581,582],{},"倍數",[223,584,585,598],{},[210,586,587,589,592,595],{},[228,588,490],{},[228,590,591],{},"$0.136",[228,593,594],{},"$0.214（優化）、$0.199（另一種）",[228,596,597],{},"約 1.6×",[210,599,600,603,606,609],{},[228,601,602],{},"第二組（Nemotron）",[228,604,605],{},"$0.153",[228,607,608],{},"$1.313",[228,610,611],{},"8.6×",[111,613,614],{},"（成本是 API 列表價換算，含失敗場次，不含本地硬體、電費、工程時間。）",[111,616,617],{},"還有一個小案例。有一場收費功能的任務，工人寫的東西通過了、也被套用了，但前沿成本是 $0.1856，直接寫是 $0.1293。加上本地執行時間，實際耗時 126.4 秒對 27.4 秒。",[111,619,620],{},"成功替換，不等於省錢。",[111,622,623,624,629],{},"但這不能推成「委派一定不省錢」。作者自己引了 ",[147,625,628],{"href":626,"rel":627},"https:\u002F\u002Farxiv.org\u002Fabs\u002F2608.04804",[151],"Scrouting"," 這個反例。小模型先偵察 repo、前沿模型負責修，在 266 題上解了 159 題，最強單一模型是 158 題。只差一題，成本大約五分之一。",[111,631,632],{},"方向相反，結果就相反。他的結論只限定在自己的委派方式：前沿模型寫規格、本地工人實作。",[111,634,635],{},"成本定義也很窄：用 API 列表價換算，不含本地硬體、電費、工程時間。所以「8.6 倍」不是你的總帳，是前沿那一邊的帳。",[111,637,638],{},"便宜的是 token，貴的是協調。",[115,640,641],{"id":641},"驗收腳本自己也會誤殺",[111,643,644],{},"這篇最實用的一段，其實不是模型，是驗收。",[111,646,647,648,651,652,655],{},"他的工人產出要先過一份靜態契約：import、export、公開函式的參數列表都要對得上。問題是，任務的 stub 裡有些用不到的參數前面有底線（",[325,649,650],{},"_entries","），工人實作時很自然地把它拿掉（",[325,653,654],{},"entries","）。",[111,657,658],{},"對一般的位置參數呼叫來說，這完全不影響行為。但契約是比對「參數名的字串」，所以直接退件。父層根本沒看到，隱藏檢查也沒跑。",[111,660,661],{},"後來他做了一次事後診斷。36 個「只因底線被移除」而被退的 artifacts，跳過那一條規則、原封不動套到新的任務副本，再跑原本的隱藏檢查。",[111,663,664],{},"19 個通過，17 個還是不行。",[111,666,667],{},"所以那條規則不是全錯，它同時擋掉了好東西和壞東西。差別在於：擋掉壞東西是它的工作，擋掉好東西是它的副作用。而你的流程如果只有這道門，副作用就是最終結果。",[204,669,670,686],{},[207,671,672],{},[210,673,674,677,680,683],{},[213,675,676],{},"工人",[213,678,679],{},"只因底線被退",[213,681,682],{},"原封不動通過隱藏檢查",[213,684,685],{},"仍失敗",[223,687,688,702,716],{},[210,689,690,693,696,699],{},[228,691,692],{},"Qwen3-Coder",[228,694,695],{},"23",[228,697,698],{},"12",[228,700,701],{},"11",[210,703,704,707,710,713],{},[228,705,706],{},"Nemotron",[228,708,709],{},"13",[228,711,712],{},"7",[228,714,715],{},"6",[210,717,718,721,724,727],{},[228,719,720],{},"合計",[228,722,723],{},"36",[228,725,726],{},"19",[228,728,729],{},"17",[111,731,732],{},"作者沒有把這 19 個算成成功。它們沒有走完父層審查與套用流程，分數維持 Qwen 0\u002F45、Nemotron 12\u002F45。他也沒有因此說契約要拿掉。他提的是下一個實驗：讓工人收到具體的驗證錯誤，允許一次有界的修正。",[111,734,735],{},"那個實驗還沒做。",[111,737,738],{},"在這份契約上，比模型能力更容易出錯的，是驗收設計。",[115,740,741],{"id":741},"它不能證明什麼",[111,743,744],{},"寫到這裡應該先把邊界畫出來，不然很容易變成另一種「AI 什麼都能」或「什麼都不能」。",[419,746,747,750,753,756,759],{},[168,748,749],{},"這是單人的回溯性經驗報告，不是受控實驗。工作負載隨工程進展變動。",[168,751,752],{},"樣本小：五類任務、每格三次；單一父層；單一工人設定。",[168,754,755],{},"成本只算前沿模型、用列表價換算。",[168,757,758],{},"原始紀錄沒有公開，你沒辦法自己驗算。",[168,760,761],{},"作者明說：研究沒有合併，不替模型或執行環境排名。",[111,763,764],{},"所以這篇能給你的不是「哪顆模型比較好」，是一張驗收框架。我認為，這份報告最有價值的地方不是那些數字，是它把「完成」拆成可以逐層檢查的東西。",[111,766,767],{},"這是一份可檢查的經驗，不是一張排行榜。",[115,769,770],{"id":770},"明天就能用的檢查表",[111,772,773],{},"作者在最後整理了五條實務規則。我把它們改寫成可以直接放進 PoC 文件的形式：",[165,775,776,782,788,794,800],{},[168,777,778,781],{},[128,779,780],{},"在任務邊界定義成功。"," 不要用「有沒有回應」驗收，用「產出有沒有通過一個不屬於它的檢查」驗收。而且要用真的 client 跑，不是用自己寫的 API 測試腳本。",[168,783,784,787],{},[128,785,786],{},"把部署當成一份 recipe，不是一個模型名字。"," checkpoint revision、量化、runtime、template、context 設定、gateway、client、可用記憶體，任何一個換了，都要重驗。",[168,789,790,793],{},[128,791,792],{},"session 連續性是應用的責任。"," 前綴穩定、工具訊息不要重寫、旁支請求不要搶掉主對話的狀態。",[168,795,796,799],{},[128,797,798],{},"委派看「每次驗證成功」的成本。"," 不是 token 單價，也不是「這次呼叫花了多少」。",[168,801,802,805],{},[128,803,804],{},"能用行為驗就用行為驗。"," 靜態檢查便宜、該留，但要知道它擋掉的是什麼。",[111,807,808],{},"如果你是要買的人，三個問題就夠：",[419,810,811,814,817],{},[168,812,813],{},"這個 demo 停在哪一層？「它會回話」是第 2 層。",[168,815,816],{},"第 4 層你們怎麼驗？驗收腳本誰寫的、什麼時候凍結的？",[168,818,819],{},"成本怎麼算，每次呼叫，還是每次驗證成功？",[111,821,822,823,827,828,832],{},"我們自己的",[147,824,826],{"href":825},"\u002Fwork\u002Fvideo-transcript-pipeline","影片轉錄管線","也遇過同一類問題。模型跑起來了，流程還是卡住；而且每一個環節的失敗，看起來都像「模型有點笨」。我們沒有做這種規模的量測，所以這篇只借論文的數字。本地跑大模型的記憶體取捨，之前寫在",[147,829,831],{"href":830},"\u002Fblog\u002Fstrata-125b-on-12gb","另一篇","。",[111,834,835,836,840],{},"上一則我們談的是",[147,837,839],{"href":838},"\u002Fblog\u002Fbroken-on-arrival-quantized-models","檔案看起來對、其實不能用","；這一則是模型看起來對、其實幹不完活。",[111,842,843,844,847],{},"如果你要的是有人陪你把這張表填完，我們的",[147,845,22],{"href":846},"\u002Fservices","做的就是這件事。先判斷哪些工作現在就能交給 AI，哪些還不成熟。",[115,849,851],{"id":850},"結論你拿哪一層當完成","結論：你拿哪一層當「完成」",[111,853,854],{},"前三層可以在一個下午示範完。載一個模型、丟幾句話、接一個 client，快的話半天。",[111,856,857],{},"第四層要幾週：設計任務、寫隱藏檢查、跑足夠的次數、把失敗分類、算成本。而且做完之後，你得到的通常不是「這顆模型可以」，而是「這批任務、這份契約下，它可以」。",[111,859,860],{},"驗收單上缺的那一欄，就是差價所在。",[111,862,863],{},"明天可以先做一件事：把「測試通過」寫進你的 PoC 定義，然後要求對方出示第 4 層是怎麼驗的。",{"title":865,"searchDepth":866,"depth":866,"links":867},"",2,[868,869,870,871,872,873,874,875,876,877],{"id":117,"depth":866,"text":117},{"id":159,"depth":866,"text":160},{"id":304,"depth":866,"text":305},{"id":360,"depth":866,"text":361},{"id":441,"depth":866,"text":441},{"id":548,"depth":866,"text":548},{"id":641,"depth":866,"text":641},{"id":741,"depth":866,"text":741},{"id":770,"depth":866,"text":770},{"id":850,"depth":866,"text":851},"\u002Fimages\u002Fblog\u002Frunning-is-not-done.svg","標題「跑得起來」跟「真的幹完活」是四件事的技術筆記封面","2026-11-03","同一個 API 相容的 endpoint，在工具呼叫、回合控制、context 和快取上都不一樣。一份 4,165 個回合的單人經驗報告，把「模型有回應」「工具真的執行」「程式碼通過檢查」拆成四層，也量到委派給本地 Nemotron 工人時，每次驗證成功反而貴 8.6 倍。文末有一張能直接拿去用的驗收清單。",false,"md","zh-TW",{},"\u002Fimages\u002Fblog\u002Frunning-is-not-done.jpg","\u002Fblog\u002Fzh-tw\u002Frunning-is-not-done",{"title":102,"description":881},"running-is-not-done","blog\u002Fzh-TW\u002Frunning-is-not-done",[892,893,894,895],"local-llm","agents","serving","tool-calling",null,"Mv8FTsKM24mkF-nqqLfoFPMU-KhWVf5dDgEeAbaotSA",{"id":899,"title":900,"author":103,"body":901,"cover":1514,"coverAlt":1515,"date":880,"description":1516,"draft":882,"extension":883,"featured":882,"lang":1517,"meta":1518,"navigation":296,"ogImage":1519,"path":1520,"seo":1521,"slug":889,"stem":1522,"tags":1523,"translationKey":889,"updated":896,"__hash__":1524},"blog\u002Fblog\u002Fen\u002Frunning-is-not-done.md","A running model is not a finished job",{"type":105,"value":902,"toc":1502},[903,908,912,915,918,921,924,931,935,961,964,969,972,1033,1036,1039,1042,1045,1056,1060,1063,1066,1069,1072,1084,1087,1090,1093,1096,1099,1103,1106,1109,1112,1115,1118,1121,1124,1127,1130,1133,1150,1153,1156,1167,1171,1174,1177,1180,1183,1186,1189,1192,1195,1241,1244,1247,1250,1253,1256,1259,1262,1266,1269,1272,1275,1278,1281,1284,1287,1294,1297,1300,1304,1307,1316,1319,1322,1325,1328,1379,1382,1385,1388,1392,1406,1409,1412,1416,1419,1451,1454,1465,1472,1479,1486,1490,1493,1496,1499],[108,904,905],{},[111,906,907],{},"Every number here comes from arXiv 2610.34241v1, a single-author engineering report. It is not a controlled experiment. Where I add a judgement, I say so.",[115,909,911],{"id":910},"a-scene","A scene",[111,913,914],{},"\"The model answered.\" That is the easiest sentence to fail an acceptance test with. The API returns 200, the prose reads well, and the tool call may never have left the building.",[111,916,917],{},"In July 2026, Vaibhav Jain pooled a few laptops to serve a model too large for any one machine. Chat worked; a coding agent did not. Every restart pushed a 32B checkpoint across Wi-Fi — 8.7 GB, roughly five minutes. He went single-node and noticed the real problem: \"the model is running\" is a very weak success criterion.",[111,919,920],{},"A checkpoint loads and answers. Then you ask it to inspect a file, call a tool, hold a long session, or preserve a repository contract. It may fail all of that, and your API still returns 200.",[111,922,923],{},"\"Compatible\" hides a lot. Behind one nominally compatible endpoint, tool-call formatting, turn control, context handling and cache reuse can all differ. Change one, and a workflow you verified on model A breaks on model B — usually without an error message.",[111,925,926,930],{},[147,927,929],{"href":149,"rel":928},[151],"arXiv 2610.34241"," covers July to October 2026 across vLLM, SGLang, llama.cpp and MLX. Its move is simple: split \"it runs\" into four levels, then run 45 trials with hidden checks to see what level four costs.",[115,932,934],{"id":933},"four-levels-not-one-switch","Four levels, not one switch",[165,936,937,943,949,955],{},[168,938,939,942],{},[128,940,941],{},"Served"," — the checkpoint loads and returns something.",[168,944,945,948],{},[128,946,947],{},"Coherent"," — the text reads like language and answers the question.",[168,950,951,954],{},[128,952,953],{},"Parsed and executed tool calls"," — it says \"read this file\", the client receives it, and the read actually happens.",[168,956,957,960],{},[128,958,959],{},"Verified task completion"," — the code passes an independent behavioural check.",[111,962,963],{},"The paper's framing sentence is the spine of everything below:",[108,965,966],{},[111,967,968],{},"An endpoint can satisfy the first three and fail the fourth completely. And if it cannot satisfy the third, you cannot measure the fourth at all.",[111,970,971],{},"(My translation. The emphasis is the author's.)",[204,973,974,987],{},[207,975,976],{},[210,977,978,981,984],{},[213,979,980],{},"Level",[213,982,983],{},"The question you ask",[213,985,986],{},"What passing it still doesn't tell you",[223,988,989,1000,1011,1022],{},[210,990,991,994,997],{},[228,992,993],{},"1 Served",[228,995,996],{},"Does it load and answer?",[228,998,999],{},"Whether the answer is language",[210,1001,1002,1005,1008],{},[228,1003,1004],{},"2 Coherent",[228,1006,1007],{},"Does it read and reason?",[228,1009,1010],{},"Whether it will act",[210,1012,1013,1016,1019],{},[228,1014,1015],{},"3 Tools",[228,1017,1018],{},"Are tool calls parsed and executed?",[228,1020,1021],{},"Whether the result is right",[210,1023,1024,1027,1030],{},[228,1025,1026],{},"4 Verified",[228,1028,1029],{},"Does the output pass an independent check?",[228,1031,1032],{},"This is \"done\"",[111,1034,1035],{},"These are not degrees of the same thing. They are four different gates. Level one passing says nothing about level two; level two passing says nothing about whether the model will pick up a tool. Most people stop at level two and think they are close, because the model sounds convincing.",[111,1037,1038],{},"The three tiers on an acceptance sheet map roughly like this: \"it responds\" is levels 1 and 2; \"the code runs\" is level 3; \"the tests pass\" is level 4. I keep 1 and 2 separate because their failures look similar and have different causes — one service never came up, the other came up and only talks.",[111,1040,1041],{},"And \"the tests pass\" depends on whose tests. If the vendor wrote them, you may still be at level three. Level four needs a check written by someone else and frozen before acceptance.",[111,1043,1044],{},"(Figure 1)",[286,1046,290,1048,290,1053],{"className":1047},[289],[292,1049],{"src":1050,"poster":1051,"autoPlay":296,"loop":296,"muted":296,"playsInline":296,"ariaLabel":1052},"\u002Fimages\u002Fblog\u002Fen\u002Frunning-is-not-done\u002Flevels.mp4","\u002Fimages\u002Fblog\u002Fen\u002Frunning-is-not-done\u002Flevels-poster.jpg","Four levels of progress: served, coherent, tools, verified, with a break before verified",[299,1054,1055],{},"Figure 1: four different gates, not four degrees. The first three can pass and the fourth still fail.",[115,1057,1059],{"id":1058},"same-api-different-protocol","Same API, different protocol",[111,1061,1062],{},"He pointed Claude Code at a gateway he wrote, with different models behind it. The expectation was \"swap an endpoint\". The experience was closer to handing the work to a person with different habits.",[111,1064,1065],{},"Some models follow the tool contract. Some narrate what they intend to do instead of calling the tool. Some rewrite an interface they find awkward, including a function signature that was supposed to stay stable. None of this is stupidity; it is a different understanding of what a tool is.",[111,1067,1068],{},"The logs make the gap concrete. Of 492 Gemma 4 requests that carried tools, 294 produced parsed tool_calls. Another 144 ended with stop and no tool call; 54 had no saved parsed response.",[111,1070,1071],{},"Those 144 are not all failures. A normal finish is also a stop, and no tool call may simply mean the task was done. That is the first trap here: a missing tool call is not an error signal by itself. You have to know whether one was expected.",[111,1073,1074,1075,1077,1078,1080,1081,1083],{},"Some failures are blunter. Eight saved Gemma 4 outputs arrived with a malformed ",[325,1076,327],{}," prefix and never parsed. On GPT-OSS-120B over SGLang, ",[325,1079,331],{}," fought the model's own Harmony tool format and had to be downgraded to ",[325,1082,335],{},"; tool rounds also needed a higher output allowance, or reasoning and tool JSON were truncated together.",[111,1085,1086],{},"Turn control is the other half: a missing session identifier, a tool loop, a thinking channel that does not line up. Each is enough to make a conversation look alive while it is lost.",[111,1088,1089],{},"The episode worth memorising: every API schema check passed, and the real Claude Code client still broke. In one case the working directory never reached the model-facing request, so the model guessed paths. In another, streamed tool-argument fragments were treated as replacements instead of appends, cutting the arguments mid-call.",[111,1091,1092],{},"Neither shows up in a unit test. Both require the actual client, end to end.",[111,1094,1095],{},"His diagnostic question is the useful part: when a tool fails, did the model fail, or did something in the middle drop the call? If it is the latter, fix the source — template, parser rules, serve flags, request translation — rather than patching the gateway forever. He reduced the gateway to a thin translator doing protocol and security checks, and made upstream failures surface as errors instead of a stream that ends normally with nothing in it.",[111,1097,1098],{},"\"Configured\" is not \"served and evaluated\". When a vendor says \"supported\", it often means \"it can be configured\".",[115,1100,1102],{"id":1101},"the-conversation-looks-continuous-it-isnt","The conversation looks continuous. It isn't.",[111,1104,1105],{},"To you it is one continuous thread. To the runtime it is three things: a token prefix, a KV cache, and an execution slot. Your \"continuous\" is not necessarily its continuous.",[111,1107,1108],{},"He kept 4,165 gateway turns and 623 debug jobs. Eighty-five cold-prefix jobs — no reuse at all, 30,000 to 36,000 input tokens — had a median time to first token of 69.4 seconds. Two hundred and eighty high-reuse jobs — at least 90% reused, under 2,000 fresh tokens — had a median of 2.8 seconds.",[111,1110,1111],{},"About 25x.",[111,1113,1114],{},"The author labels this observational: the two groups are composed differently, not randomised, not matched. You cannot write \"caching makes it 25x faster\". What you can say is that prefix reuse is associated with a difference worth engineering.",[111,1116,1117],{},"The mechanism is mundane. An inference backend can reuse KV state for an identical token prefix. Change the prefix — a timestamp, a system banner, tool definitions in a different order, a different JSON serialisation — and the whole context is prefilled again.",[111,1119,1120],{},"You changed a banner. You paid for the whole context.",[111,1122,1123],{},"Auxiliary requests are the pettier version. A coding agent also generates titles and permission prompts; with one reusable execution slot, an auxiliary request can evict the main conversation's resident state. In his records, 65 requests were classified as auxiliary and 44 showed no cache reuse. Among high-reuse records, auxiliary requests had a median TTFT of about 12.2 seconds (13 observations) against 2.7 seconds for main requests (267 observations).",[111,1125,1126],{},"Compaction has the same smell. He lowered the threshold deliberately: in 12 tests, sub-agent compaction fired six times, none of those six tasks completed, and one was still labelled PASS.",[111,1128,1129],{},"\"Compaction happened\" is not \"work finished\". In a log, the two look alike.",[111,1131,1132],{},"So he treats session continuity as an application responsibility, not a backend setting:",[419,1134,1135,1138,1141,1144,1147],{},[168,1136,1137],{},"Keep the message sequence append-only unless earlier content genuinely must change.",[168,1139,1140],{},"Do not re-serialise tool messages for cosmetics.",[168,1142,1143],{},"Make streamed and final tool JSON agree.",[168,1145,1146],{},"Keep per-turn metadata out of the frozen system prefix.",[168,1148,1149],{},"Preserve session affinity. A stable prompt is not enough if the next turn lands on a slot without the cache.",[111,1151,1152],{},"Caching is not a backend flag. It is application discipline.",[111,1154,1155],{},"(Figure 2)",[286,1157,290,1159,290,1164],{"className":1158},[289],[292,1160],{"src":1161,"poster":1162,"autoPlay":296,"loop":296,"muted":296,"playsInline":296,"ariaLabel":1163},"\u002Fimages\u002Fblog\u002Fen\u002Frunning-is-not-done\u002Fcache.mp4","\u002Fimages\u002Fblog\u002Fen\u002Frunning-is-not-done\u002Fcache-poster.jpg","Two paths for one conversation: a stable prefix reuses the cache, a changed prefix refills it",[299,1165,1166],{},"Figure 2: one conversation. The prefix holds and the cache is reused; the prefix changes and the whole context is prefilled again.",[115,1168,1170],{"id":1169},"level-four-is-where-the-work-is-judged","Level four is where the work is judged",[111,1172,1173],{},"Deployment, protocol and sessions are now good enough to run bounded tasks. Can an open-weight model produce code that passes?",[111,1175,1176],{},"Five TypeScript task categories (quote, prorate, customer, split, parts), three conditions, three runs each — 45 sessions. The parent is fixed: Claude Code with Claude Opus 5.5. The worker is a quantized Qwen3-Coder 30B-A3B. Every session is judged by task-specific hidden behavioural checks, not by the agent's own claim of success.",[111,1178,1179],{},"The conditions: direct; optimised delegation, where the parent specifies, reviews and applies but does not implement; and batch-labelled delegation that may split work into sequential packages — though only a minority actually split.",[111,1181,1182],{},"Results: direct 15\u002F15; optimised delegation 12\u002F15; batch-labelled 14\u002F15.",[111,1184,1185],{},"These do not rank. Each cell has three runs, the Wilson intervals overlap, and the author says the data cannot establish general success-rate differences. It does show workers produced usable code: of 30 delegated trials, 26 verified and all 26 retained worker-produced implementations. Not every time.",[111,1187,1188],{},"And the 14\u002F15 is not evidence that batching helps. Only 5 of the 15 batch-labelled trials used a batch call; the rest used single delegation. The paper calls it an adaptive policy, not a batch-versus-single comparison.",[111,1190,1191],{},"The second suite is stricter: 15 tasks, three runs each, 45 sessions per arm, artifacts clearing a fixed interface contract before parent review.",[111,1193,1194],{},"Direct 45\u002F45. Nemotron Cascade 2 worker: 12\u002F45, of which 11 were genuine worker substitutions and one verified only after the parent rewrote the source. Qwen3-Coder worker: 0\u002F45.",[204,1196,1197,1210],{},[207,1198,1199],{},[210,1200,1201,1204,1207],{},[213,1202,1203],{},"Study",[213,1205,1206],{},"Direct",[213,1208,1209],{},"Delegated",[223,1211,1212,1222,1232],{},[210,1213,1214,1217,1219],{},[228,1215,1216],{},"45-trial suite",[228,1218,493],{},[228,1220,1221],{},"Qwen3-Coder 12\u002F15 (optimised), 14\u002F15 (batch-labelled)",[210,1223,1224,1227,1229],{},[228,1225,1226],{},"Second suite (15 tasks x 3)",[228,1228,504],{},[228,1230,1231],{},"Nemotron 12\u002F45 (11 worker substitutions); Qwen 0\u002F45",[210,1233,1234,1237,1239],{},[228,1235,1236],{},"Same, counting interruptions as failures",[228,1238,515],{},[228,1240,518],{},[111,1242,1243],{},"That second suite also has a disclosure: 16 sessions were stopped by a subscription limit before any model call and were replaced. Eight were in the direct arm and eight in the Nemotron arm. Counting the original 16 as failures gives 37\u002F45 and 8\u002F45. Those are scheduling interruptions, not inference failures — but if you cite the headline numbers, cite both.",[111,1245,1246],{},"The 45\u002F45 deserves a caveat too. The tasks were built for this study, five categories with three runs each; there is no difficulty stratification.",[111,1248,1249],{},"One more honest detail: the task definitions and hidden tests were produced with AI assistance. The combined test file was created about 39 seconds before the first evaluated session and first committed after the last of the 45. What he can claim is narrower: task instructions and visible checks were identical across the 45 saved runs, and recorded score labels agree with the scorer as later saved. He cannot claim an independently snapshotted benchmark.",[111,1251,1252],{},"The first study's harness had limits: the parent's separate constraints field was not forwarded to the worker; the gateway dropped a duplicate system message and truncated messages over 1,800 characters; two of 41 reconstructed worker prompts lost about 220 characters, both in failed trials.",[111,1254,1255],{},"So \"the worker ignored instructions\" needs a discount. Some failures were capability; some were a worker that never received the full instruction.",[111,1257,1258],{},"What about 0\u002F45? It does not mean the model can do nothing. That arm had no direct control, parent acceptance and application were not replayed, and the parent's prompt-cache state was not reset between studies. The author scopes it to these tasks, this contract, this worker configuration.",[111,1260,1261],{},"Verification is level four. It is not a by-product of level three.",[115,1263,1265],{"id":1264},"the-bill-runs-the-other-way","The bill runs the other way",[111,1267,1268],{},"Frontier output tokens are expensive; a local worker is cheap. Delegate and save money — except delegation is not free. The parent writes the spec, reads what came back, decides whether to accept it, and cleans up failures. All of that is frontier tokens.",[111,1270,1271],{},"The cost model has five terms: parent context, specification, review, recovery, worker. The study measured the frontier terms. Electricity, hardware and engineering time were not measured — and unmeasured is not zero.",[111,1273,1274],{},"First suite, frontier cost per verified success: $0.136 direct; $0.214 optimised delegation; $0.199 batch-labelled. About 1.6x.",[111,1276,1277],{},"Second suite: $0.153 direct; $1.313 for Nemotron delegation. 8.6x.",[111,1279,1280],{},"Qwen's per-success cost is undefined, not zero — it had no successes.",[111,1282,1283],{},"One small case is worth keeping: a fee task where the worker's implementation passed and was applied, yet the frontier cost was $0.1856 versus $0.1293 direct, and wall time 126.4 seconds versus 27.4.",[111,1285,1286],{},"A successful substitution is not a saving.",[111,1288,1289,1290,1293],{},"This does not generalise to \"delegation never pays\". The author cites ",[147,1291,628],{"href":626,"rel":1292},[151],": a small model scouts the repository, frontier models fix, and it solved 159 of 266 tasks against 158 for the strongest single model — one task apart — at roughly one fifth the cost. The direction is reversed: scout small, fix big. He limits his finding to his own contract, frontier-specifies \u002F local-implements.",[111,1295,1296],{},"The cost definition is narrow too: API list prices, no hardware, electricity or engineering time. \"8.6x\" is the frontier side of the bill, not your total.",[111,1298,1299],{},"Tokens are cheap. Coordination is not.",[115,1301,1303],{"id":1302},"the-acceptance-script-will-reject-good-code","The acceptance script will reject good code",[111,1305,1306],{},"The most practical part of the report is not about models. It is about acceptance.",[111,1308,1309,1310,1312,1313,1315],{},"Worker artifacts had to clear a static contract: imports, exports, and exported function parameter lists had to match. The stubs used underscored parameters for unused arguments (",[325,1311,650],{},"); workers naturally dropped the underscore (",[325,1314,654],{},").",[111,1317,1318],{},"For ordinary positional callers this changes nothing. But the contract compared parameter-name strings, so the artifact was rejected before parent review, before the hidden checks ran.",[111,1320,1321],{},"The post-hoc diagnostic: 36 valid-JSON artifacts rejected solely for dropping the underscore were applied unchanged to fresh task copies, with that rule bypassed, and re-run against the original sealed checks.",[111,1323,1324],{},"19 passed. 17 still failed.",[111,1326,1327],{},"So the rule was not simply wrong. It screened out both good and bad output. Rejecting bad output is its job; rejecting good output is its side effect — and if that gate is your only gate, the side effect is the final result.",[204,1329,1330,1346],{},[207,1331,1332],{},[210,1333,1334,1337,1340,1343],{},[213,1335,1336],{},"Worker",[213,1338,1339],{},"Rejected for underscores",[213,1341,1342],{},"Passed sealed checks unchanged",[213,1344,1345],{},"Still failed",[223,1347,1348,1358,1368],{},[210,1349,1350,1352,1354,1356],{},[228,1351,692],{},[228,1353,695],{},[228,1355,698],{},[228,1357,701],{},[210,1359,1360,1362,1364,1366],{},[228,1361,706],{},[228,1363,709],{},[228,1365,712],{},[228,1367,715],{},[210,1369,1370,1373,1375,1377],{},[228,1371,1372],{},"Total",[228,1374,723],{},[228,1376,726],{},[228,1378,729],{},[111,1380,1381],{},"He did not count the 19 as successes. They never went through parent review and application; the frozen scores stay Qwen 0\u002F45 and Nemotron 12\u002F45. He also did not conclude the contract should go. He proposed the next experiment: give the worker the specific validation error and allow one bounded repair.",[111,1383,1384],{},"That experiment has not been run.",[111,1386,1387],{},"On this contract, acceptance design failed more easily than model capability did.",[115,1389,1391],{"id":1390},"what-this-cannot-prove","What this cannot prove",[419,1393,1394,1397,1400,1403],{},[168,1395,1396],{},"A single-author retrospective, not a controlled experiment. The workload changed as the engineering changed.",[168,1398,1399],{},"Small sample: five task categories, three runs per cell, one fixed parent, one worker configuration.",[168,1401,1402],{},"Cost covers frontier tokens at list prices only.",[168,1404,1405],{},"The raw records are private. You cannot recompute the numbers.",[111,1407,1408],{},"The studies are not pooled and do not rank models or runtimes.",[111,1410,1411],{},"So this gives you an acceptance framework, not a leaderboard. The most valuable part, in my reading, is not the numbers — it is that \"done\" becomes something you can check level by level.",[115,1413,1415],{"id":1414},"a-checklist-you-can-use-tomorrow","A checklist you can use tomorrow",[111,1417,1418],{},"The author's five rules, rewritten for a PoC document:",[165,1420,1421,1427,1433,1439,1445],{},[168,1422,1423,1426],{},[128,1424,1425],{},"Define success at the task boundary."," Accept on \"the output passed a check it does not own\", not on \"it responded\". Exercise the real client, not a harness you wrote.",[168,1428,1429,1432],{},[128,1430,1431],{},"Treat deployment as a recipe, not a model name."," Checkpoint revision, quantisation, runtime, template, context settings, gateway, client, resident memory. Change one, re-verify.",[168,1434,1435,1438],{},[128,1436,1437],{},"Make session continuity an application responsibility."," Stable prefix, no cosmetic re-serialisation, auxiliary requests must not evict the main conversation.",[168,1440,1441,1444],{},[128,1442,1443],{},"Price delegation per verified success."," Not per token, not per call.",[168,1446,1447,1450],{},[128,1448,1449],{},"Verify by behaviour where you can."," Static gates are cheap and worth keeping, but know what they reject.",[111,1452,1453],{},"If you are the buyer, three questions are enough:",[419,1455,1456,1459,1462],{},[168,1457,1458],{},"Which level does this demo stop at? \"It answers\" is level two.",[168,1460,1461],{},"How do you verify level four, and who wrote the checks, and when were they frozen?",[168,1463,1464],{},"Is the cost per call or per verified success?",[111,1466,1467,1468,1471],{},"Our own ",[147,1469,1470],{"href":825},"video transcription pipeline"," ran into the same class of problem: the model ran, the pipeline still stalled, and every failure looked like \"the model is a bit dumb\". We have not run measurements at this scale, so this essay borrows the paper's numbers.",[111,1473,1474,1475,1478],{},"The previous essay was about ",[147,1476,1477],{"href":838},"files that look right and cannot be used","; this one is about models that look right and cannot finish the job.",[111,1480,1481,1482,1485],{},"If you want help filling in the table, our ",[147,1483,1484],{"href":846},"AI readiness audit"," does exactly that: what can be handed to AI now, and what should wait.",[115,1487,1489],{"id":1488},"conclusion-which-level-counts-as-done","Conclusion: which level counts as done",[111,1491,1492],{},"The first three levels can be demonstrated in an afternoon. Load a model, send a few prompts, connect a client.",[111,1494,1495],{},"Level four takes weeks: design tasks, write hidden checks, run enough trials, classify failures, price the work. What you get is rarely \"this model is good\" — it is \"on these tasks, under this contract, it is good\".",[111,1497,1498],{},"The missing column on your acceptance sheet is where the price difference lives.",[111,1500,1501],{},"Tomorrow: write \"passes the tests\" into your PoC definition, and ask who wrote the tests and when they froze them.",{"title":865,"searchDepth":866,"depth":866,"links":1503},[1504,1505,1506,1507,1508,1509,1510,1511,1512,1513],{"id":910,"depth":866,"text":911},{"id":933,"depth":866,"text":934},{"id":1058,"depth":866,"text":1059},{"id":1101,"depth":866,"text":1102},{"id":1169,"depth":866,"text":1170},{"id":1264,"depth":866,"text":1265},{"id":1302,"depth":866,"text":1303},{"id":1390,"depth":866,"text":1391},{"id":1414,"depth":866,"text":1415},{"id":1488,"depth":866,"text":1489},"\u002Fimages\u002Fblog\u002Fen\u002Frunning-is-not-done.svg","Cover for an essay on the four levels between a served model and finished work","Same endpoint, different behaviour: tool calls, turn control, context, cache. A 4,165-turn solo report splits 'it runs' into four levels, finding local delegation costs 8.6x more per verified success.","en",{},"\u002Fimages\u002Fblog\u002Fen\u002Frunning-is-not-done.jpg","\u002Fblog\u002Fen\u002Frunning-is-not-done",{"title":900,"description":1516},"blog\u002Fen\u002Frunning-is-not-done",[892,893,894,895],"0DdevvjhzLCzG5RvWM1GlSfrg6_pwOm2oIXJq0yz_YU",[1526,1999],{"id":1527,"title":1528,"author":103,"body":1529,"cover":1985,"coverAlt":1986,"date":1987,"description":1988,"draft":882,"extension":883,"featured":882,"lang":884,"meta":1989,"navigation":296,"ogImage":1990,"path":1991,"seo":1992,"slug":1993,"stem":1994,"tags":1995,"translationKey":1993,"updated":896,"__hash__":1998},"blog\u002Fblog\u002Fzh-TW\u002Fbroken-on-arrival-quantized-models.md","你下載的量化模型可能是壞的，而且 checksum 驗不出來",{"type":105,"value":1530,"toc":1974},[1531,1540,1542,1549,1552,1555,1562,1566,1569,1576,1582,1593,1596,1599,1606,1616,1623,1627,1630,1688,1691,1697,1701,1704,1711,1718,1721,1725,1728,1833,1843,1854,1857,1870,1880,1894,1897,1915,1918,1923,1929,1936,1939,1961],[108,1532,1533],{},[111,1534,1535,1536,1539],{},"數字來自 arXiv 2609.05881v3 與作者的 ",[325,1537,1538],{},"quantcheck"," 稽核資料；「今天還在不在」是我們自己查的，腳本在文末。",[115,1541,117],{"id":117},[111,1543,1544,1545,1548],{},"你在筆電上 ",[325,1546,1547],{},"ollama pull qwen2.5-coder:3b","，選了一個 2-bit 的版本，因為記憶體不夠。",[111,1550,1551],{},"模型跑起來了，有回應，速度也不錯。但你叫它寫一個排序函式，它寫出來的東西編譯不過；再試一題，還是不過。",[111,1553,1554],{},"你會先懷疑自己：prompt 寫錯？溫度太高？還是這顆模型本來就這麼弱？",[111,1556,1557,1558,1561],{},"有一篇論文把這個懷疑變成了普查：",[128,1559,1560],{},"Ollama 官方庫裡 305 個「能寫程式」的模型 tag，有 7 個在 164 題 HumanEval+ 上幾乎全錯。"," 其中六個一題都沒對。",[115,1563,1565],{"id":1564},"最反直覺的地方檔案是好的","最反直覺的地方：檔案是好的",[111,1567,1568],{},"如果模型壞了，你會想「是不是下載壞了、檔案損毀」。",[111,1570,1571,1572,1575],{},"作者把其中五個檔案從原始權重重新量化回來，",[128,1573,1574],{},"tensor 對 tensor 比對，完全一致","。Qwen2.5-Coder-3B 那四個 tag，434 個 tensor 全部相同。",[111,1577,1578,1579],{},"也就是說：",[128,1580,1581],{},"這些檔案是「正確的壞檔案」。",[111,1583,1584,1585,1588,1589,1592],{},"checksum 會過、檔案大小會對、",[325,1586,1587],{},"ollama show"," 看起來一切正常——因為檔案真的沒問題。壞的是",[128,1590,1591],{},"量化的配方","：同樣的權重、同樣的工具鏈、同樣的 tensor 組合，少了校準，就把模型壓成了不會寫程式的形狀。",[115,1594,1595],{"id":1595},"為什麼會這樣",[111,1597,1598],{},"量化是「把權重從 16 bit 壓到 2–4 bit」。壓的過程要不斷決定「這個誤差要丟給哪個權重」。",[111,1600,1601,1602,1605],{},"預設的做法是均勻地丟；比較好的做法是",[128,1603,1604],{},"看校準資料","，把誤差集中在不重要的權重上——llama.cpp 的 importance matrix（imatrix）就是做這件事。",[111,1607,1608,1609,1612,1613,832],{},"沒有 imatrix，小模型在低 bit 特別容易崩。這次的七個 suspects 全部落在 ",[128,1610,1611],{},"q2_K 到 q3_K_L、模型 ≤4B"," 這個範圍；同一個範圍有 48 個 tag，等於 ",[128,1614,1615],{},"14.6%",[111,1617,1618,1619,1622],{},"但要注意：",[128,1620,1621],{},"配方本身不能預測成敗","。同樣的無校準配方，Qwen2.5-Coder-1.5B 的 q2_K–q3_K_L 全部正常（13–15 題）。所以不是「低 bit 一定壞」，是「壞了你不會知道」。",[115,1624,1626],{"id":1625},"加回校準救回大部分","加回校準，救回大部分",[111,1628,1629],{},"作者固定權重、工具鏈與 tensor 組合，只補上 imatrix：",[204,1631,1632,1645],{},[207,1633,1634],{},[210,1635,1636,1639,1642],{},[213,1637,1638],{},"Tag",[213,1640,1641],{},"原本",[213,1643,1644],{},"補 imatrix 後（滿分 164）",[223,1646,1647,1658,1668,1678],{},[210,1648,1649,1652,1655],{},[228,1650,1651],{},"Qwen2.5-Coder-3B q2_K",[228,1653,1654],{},"0",[228,1656,1657],{},"106–113",[210,1659,1660,1663,1665],{},[228,1661,1662],{},"Qwen2.5-Coder-3B q3_K_M",[228,1664,1654],{},[228,1666,1667],{},"76–89",[210,1669,1670,1673,1675],{},[228,1671,1672],{},"Qwen2.5-Coder-3B q3_K_L",[228,1674,1654],{},[228,1676,1677],{},"80–86",[210,1679,1680,1683,1685],{},[228,1681,1682],{},"Qwen2.5-Coder-3B q3_K_S",[228,1684,1654],{},[228,1686,1687],{},"13–30（救不回）",[111,1689,1690],{},"（論文摘要寫「0 → 76–113」；上面是 v3.4 稽核資料的分級數字，我以資料為準。）",[111,1692,1693,1696],{},[128,1694,1695],{},"同一顆模型、同樣的位元數，只差一張校準表，從 0 題變成三位數。"," 這就是「量化不只是壓縮，是配方」的意思。",[115,1698,1700],{"id":1699},"另一種壞法少了-chat-template","另一種壞法：少了 chat template",[111,1702,1703],{},"七個 suspects 之外，作者還測了 22 個 Hugging Face 社群檔案，其中兩個（TheBloke 的 deepseek-coder-6.7B Q3_K_M／Q4_K_M）在普查時只對 5、6 題。",[111,1705,1706,1707,1710],{},"原因不是量化，是",[128,1708,1709],{},"檔案沒有附 chat template","。Ollama 找不到 template，就用預設的 ChatML 去套，跟模型原本的指令格式不合——30 個回答裡有 28 個自己接出幻覺的對話輪。",[111,1712,1713,1714,1717],{},"補上正確 template 之後：",[128,1715,1716],{},"98 與 101 題","（滿分 164）。",[111,1719,1720],{},"所以「跑得起來」跟「它能好好回答」之間，還隔著一個 template。",[115,1722,1724],{"id":1723},"我們今天查了這些-tag-還在","我們今天查了：這些 tag 還在",[111,1726,1727],{},"文章不能只講九月的事。我們拿稽核資料裡的 blob digest，去比對 Ollama registry 今天的 manifest：",[204,1729,1730,1745],{},[207,1731,1732],{},[210,1733,1734,1736,1739,1742],{},[213,1735,1638],{},[213,1737,1738],{},"稽核時的 digest",[213,1740,1741],{},"今天",[213,1743,1744],{},"狀態",[223,1746,1747,1761,1773,1785,1797,1809,1821],{},[210,1748,1749,1752,1755,1758],{},[228,1750,1751],{},"qwen2.5-coder:3b-instruct-q2_K",[228,1753,1754],{},"14c4247b8ce4…",[228,1756,1757],{},"相同",[228,1759,1760],{},"還在",[210,1762,1763,1766,1769,1771],{},[228,1764,1765],{},"qwen2.5-coder:3b-instruct-q3_K_S",[228,1767,1768],{},"81d167af7fa3…",[228,1770,1757],{},[228,1772,1760],{},[210,1774,1775,1778,1781,1783],{},[228,1776,1777],{},"qwen2.5-coder:3b-instruct-q3_K_M",[228,1779,1780],{},"9bb5d28b842a…",[228,1782,1757],{},[228,1784,1760],{},[210,1786,1787,1790,1793,1795],{},[228,1788,1789],{},"qwen2.5-coder:3b-instruct-q3_K_L",[228,1791,1792],{},"7a88dc11faae…",[228,1794,1757],{},[228,1796,1760],{},[210,1798,1799,1802,1805,1807],{},[228,1800,1801],{},"phi3.5:3.8b-mini-instruct-q2_K",[228,1803,1804],{},"54d47caa8bf3…",[228,1806,1757],{},[228,1808,1760],{},[210,1810,1811,1814,1817,1819],{},[228,1812,1813],{},"llama3.2:1b-instruct-q2_K",[228,1815,1816],{},"d1ede02a2111…",[228,1818,1757],{},[228,1820,1760],{},[210,1822,1823,1826,1829,1831],{},[228,1824,1825],{},"llama3.2:1b-instruct-q3_K_S",[228,1827,1828],{},"81881dcc12ec…",[228,1830,1757],{},[228,1832,1760],{},[111,1834,1835,1838,1839,1842],{},[128,1836,1837],{},"七個都還在，digest 跟當時一模一樣。"," 這不是歷史事件，是你今天 ",[325,1840,1841],{},"ollama pull"," 就可能拿到的檔案。",[111,1844,1845,1846,1849,1850,1853],{},"（查法：",[325,1847,1848],{},"https:\u002F\u002Fregistry.ollama.ai\u002Fv2\u002Flibrary\u002F\u003Cmodel>\u002Fmanifests\u002F\u003Ctag>","，比對 ",[325,1851,1852],{},"application\u002Fvnd.ollama.image.model"," 那層的 digest。腳本在文末。）",[115,1855,1856],{"id":1856},"你現在該做的三件事",[111,1858,1859,1862,1863,1865,1866,1869],{},[128,1860,1861],{},"一、篩選。"," 作者釋出了 ",[325,1864,1538],{},"：對每個 tag 跑 15 題螢幕測試。成本大約 ",[128,1867,1868],{},"30 秒生成","（4090、不含下載）。66 個通過螢幕測試的重測，沒有一個是壞的——所以它是個堪用的預檢。",[111,1871,1872,1875,1876,1879],{},[128,1873,1874],{},"二、如果你自己發佈量化模型","：補上 imatrix，然後",[128,1877,1878],{},"在發佈前跑一次功能測試","。registry 不會幫你測。",[111,1881,1882,1885,1886,1889,1890,1893],{},[128,1883,1884],{},"三、檢查 chat template。"," ",[325,1887,1888],{},"ollama show \u003Cmodel> --template","；如果印出 ",[325,1891,1892],{},"{{ .Prompt }}","，代表沒有 template，模型可能用錯格式回答。",[115,1895,1896],{"id":1896},"別過度解讀",[419,1898,1899,1902,1905,1912],{},[168,1900,1901],{},"2.3% 是「Ollama 官方庫、能寫程式、8GB 以下」這個子集的比例，不是所有量化模型",[168,1903,1904],{},"只測程式碼能力（HumanEval+）；perplexity 只能當預檢，不能當功能證明",[168,1906,1907,1908,1911],{},"七個是",[128,1909,1910],{},"下限","：deepseek-coder-1.3B 全系列（含 fp16）都只對 1–6 題，無法用群組基線判定，作者沒有把它們算進去",[168,1913,1914],{},"作者是獨立研究者，資料與工具都公開，但這是單一 registry、單一 runtime 版本的結果",[115,1916,1917],{"id":1917},"結論",[111,1919,1920],{},[128,1921,1922],{},"可重現，不等於可用。",[111,1924,1925,1926,1928],{},"檔案一致、checksum 通過、",[325,1927,1587],{}," 正常——這些都只證明「檔案是你以為的那個檔案」，不證明「模型會做事」。",[111,1930,1931,1932,1935],{},"所以問題不是「量化能不能用」，是它",[128,1933,1934],{},"有沒有被測過","。目前答案是：registries 沒有，所以你得自己測。",[1937,1938],"hr",{},[111,1940,1941],{},[1942,1943,1944,1945,1950,1951,1950,1956,832],"em",{},"原始資料：",[147,1946,1949],{"href":1947,"rel":1948},"https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.05881",[151],"arXiv 2609.05881v3","、",[147,1952,1955],{"href":1953,"rel":1954},"https:\u002F\u002Fgithub.com\u002Faditi-p31\u002Fquantcheck",[151],"quantcheck（稽核資料與工具）",[147,1957,1960],{"href":1958,"rel":1959},"https:\u002F\u002Fgithub.com\u002Follama\u002Follama\u002Fissues\u002F18252",[151],"Ollama issue #18252",[111,1962,1963],{},[1942,1964,1965,1966,1969,1970,1973],{},"我們比對 digest 的腳本：",[325,1967,1968],{},"scripts\u002Fcheck-ollama-tags.ts","。跑 ",[325,1971,1972],{},"npm run check:ollama -- --list"," 會列出這七個 tag 今天的 digest 與稽核值是否一致。",{"title":865,"searchDepth":866,"depth":866,"links":1975},[1976,1977,1978,1979,1980,1981,1982,1983,1984],{"id":117,"depth":866,"text":117},{"id":1564,"depth":866,"text":1565},{"id":1595,"depth":866,"text":1595},{"id":1625,"depth":866,"text":1626},{"id":1699,"depth":866,"text":1700},{"id":1723,"depth":866,"text":1724},{"id":1856,"depth":866,"text":1856},{"id":1896,"depth":866,"text":1896},{"id":1917,"depth":866,"text":1917},"\u002Fimages\u002Fblog\u002Fbroken-on-arrival-quantized-models.svg","量化模型檔案完整但功能失效的示意圖，checksum 打勾、程式碼全錯","2026-11-02","有人把 Ollama 官方庫 305 個程式碼模型全跑了一遍，發現 7 個低 bit 量化版幾乎全錯。但檔案沒有壞——重建回來 tensor 完全一致，所以 checksum 抓不到。這篇講它怎麼發生的、你現在能做的三件事。",{},"\u002Fimages\u002Fblog\u002Fbroken-on-arrival-quantized-models.jpg","\u002Fblog\u002Fzh-tw\u002Fbroken-on-arrival-quantized-models",{"title":1528,"description":1988},"broken-on-arrival-quantized-models","blog\u002Fzh-TW\u002Fbroken-on-arrival-quantized-models",[1996,892,1997],"quantization","gguf","iM5l1m1nXRGY-pGG2MmAYggyxY9V2uAe132PQbEixw8",{"id":2000,"title":2001,"author":103,"body":2002,"cover":2483,"coverAlt":2484,"date":2485,"description":2486,"draft":882,"extension":883,"featured":882,"lang":884,"meta":2487,"navigation":296,"ogImage":2488,"path":2489,"seo":2490,"slug":2491,"stem":2492,"tags":2493,"translationKey":2491,"updated":896,"__hash__":2496},"blog\u002Fblog\u002Fzh-TW\u002Fstrata-125b-on-12gb.md","125B 模型，12GB 顯卡，94 tok\u002Fs",{"type":105,"value":2003,"toc":2471},[2004,2021,2024,2027,2030,2033,2039,2046,2050,2053,2067,2070,2074,2081,2084,2087,2091,2097,2100,2150,2153,2156,2162,2169,2173,2176,2179,2182,2185,2246,2253,2258,2265,2268,2274,2280,2286,2289,2292,2298,2305,2312,2316,2319,2326,2329,2377,2380,2387,2394,2397,2417,2420,2423,2426,2433,2445,2448,2468],[108,2005,2006],{},[111,2007,2008,2009,2014,2015,2020],{},"這篇沒有我們的實測。數據來自 Qwen 官方發布、",[147,2010,2013],{"href":2011,"rel":2012},"https:\u002F\u002Fgithub.com\u002FBlockYugaLabs\u002FStrata",[151],"Strata 開源專案"," 的說明，以及一位對岸開發者的",[147,2016,2019],{"href":2017,"rel":2018},"https:\u002F\u002Fwww.bilibili.com\u002Fvideo\u002FBV1WwJy6EEuN\u002F",[151],"公開實測影片","。文中會標明哪個數字是誰給的。",[115,2022,2023],{"id":2023},"三個數字",[111,2025,2026],{},"125B 的模型。12GB 的顯示卡。94 tok\u002Fs。",[111,2028,2029],{},"半年前，這三個數字放在一起是笑話。當時想在家用電腦跑 27B 的模型，量化後 20GB，加上 KV cache，你得有 24GB 的卡才勉強開到 128K 上下文。",[111,2031,2032],{},"現在 125B 跑在 12GB 上，而且不是「能跑」，是「比你讀字還快」。",[111,2034,2035,2036,832],{},"真正變的不是模型變聰明，是",[128,2037,2038],{},"模型有多大，和硬體要多好，已經開始脫鉤",[111,2040,2041,2042,2045],{},"先講清楚前提：94 tok\u002Fs 是 RTX 5070（12GB）搭配 ",[128,2043,2044],{},"64GB 系統記憶體","的成績。只有 32GB 記憶體的話，只能跑砍半的 Coder 版本。這不是「插一張卡就好」的故事。",[115,2047,2049],{"id":2048},"兩個限制被分別繞過","兩個限制，被分別繞過",[111,2051,2052],{},"大模型跑不動，卡在兩件事：",[165,2054,2055,2061],{},[168,2056,2057,2060],{},[128,2058,2059],{},"裝不下","：權重是硬的。這顆模型加上 51B 的 n-gram 查表，量化後檔案還是 70–80GB。",[168,2062,2063,2066],{},[128,2064,2065],{},"算不動","：每個 token 都要讓所有參數參與計算。",[111,2068,2069],{},"Qwen3.8-Flash-Next 處理了第二個，Strata 處理了第一個。",[115,2071,2073],{"id":2072},"第一個限制不是每個專家都要上班","第一個限制：不是每個專家都要上班",[111,2075,2076,2077,2080],{},"Flash-Next 是 MoE（混合專家）架構。官方數字是 125B 總參數、",[128,2078,2079],{},"6B 激活","：每個 token 的計算量，大約等於一個 6B 模型。",[111,2082,2083],{},"打個比方：傳統稠密模型是「公司裡每個部門都得參與每一個案子」，MoE 是「只叫相關的幾個部門進來開會」。",[111,2085,2086],{},"所以「算不動」這件事，先解決了一半。",[115,2088,2090],{"id":2089},"第二個限制把整台電腦當成一台機器","第二個限制：把整台電腦當成一台機器",[111,2092,2093,2094,832],{},"但 6B 的計算量，不代表 6B 的記憶體——",[128,2095,2096],{},"125B 的權重還是得存在某個地方",[111,2098,2099],{},"Strata 的做法是不要全部塞進顯存，按「多常用」分三層放（見圖 1）：",[204,2101,2102,2115],{},[207,2103,2104],{},[210,2105,2106,2109,2112],{},[213,2107,2108],{},"層",[213,2110,2111],{},"放什麼",[213,2113,2114],{},"佔用",[223,2116,2117,2128,2139],{},[210,2118,2119,2122,2125],{},[228,2120,2121],{},"顯存 VRAM",[228,2123,2124],{},"最常命中的專家",[228,2126,2127],{},"12–24 GB",[210,2129,2130,2133,2136],{},[228,2131,2132],{},"系統記憶體",[228,2134,2135],{},"全部專家（開機時載入）",[228,2137,2138],{},"約 47 GB",[210,2140,2141,2144,2147],{},[228,2142,2143],{},"SSD",[228,2145,2146],{},"其餘權重與 n-gram 查表",[228,2148,2149],{},"檔案總量 70–80 GB",[111,2151,2152],{},"（47GB 是那位開發者在 IQ3_S 量化下的實測；官方建議 64GB 記憶體就是為了裝下這一層。）",[111,2154,2155],{},"跑起來的時候，GPU 算它手上的專家；沒命中的，要嘛走 PCIe 搬進顯存，要嘛直接讓 CPU 用記憶體裡的權重算。",[111,2157,2158,2159,832],{},"這就是為什麼 12GB 的卡能跑——",[128,2160,2161],{},"顯存不再是「裝不裝得下」的容器，而是「快取多大」的問題",[111,2163,2164],{},[2165,2166],"img",{"alt":2167,"src":2168},"如圖 1：三層記憶體階層——顯存放熱門專家、記憶體放全部專家、SSD 放冷門權重與 n-gram 查表","\u002Fimages\u002Fblog\u002Fstrata-125b-on-12gb\u002F01-memory-tiers.svg",[115,2170,2172],{"id":2171},"還有一個加速器猜然後驗證","還有一個加速器：猜，然後驗證",[111,2174,2175],{},"除了省記憶體，Strata 還用了投機解碼：小模型先猜接下來幾個 token，大模型一次驗證。",[111,2177,2178],{},"猜對就省下好幾步，官方數字大約快 1.6–1.8 倍。",[115,2180,2181],{"id":2181},"實際多快",[111,2183,2184],{},"官方在 RTX 5070（12GB 顯存、64GB RAM）上測到的：",[204,2186,2187,2200],{},[207,2188,2189],{},[210,2190,2191,2194,2197],{},[213,2192,2193],{},"量化",[213,2195,2196],{},"寫答案",[213,2198,2199],{},"讀 prompt（32K）",[223,2201,2202,2213,2224,2235],{},[210,2203,2204,2207,2210],{},[228,2205,2206],{},"Q2_0",[228,2208,2209],{},"94 tok\u002Fs",[228,2211,2212],{},"2,650 tok\u002Fs",[210,2214,2215,2218,2221],{},[228,2216,2217],{},"IQ2_XS",[228,2219,2220],{},"79 tok\u002Fs",[228,2222,2223],{},"2,090 tok\u002Fs",[210,2225,2226,2229,2232],{},[228,2227,2228],{},"IQ3_XXS",[228,2230,2231],{},"62 tok\u002Fs",[228,2233,2234],{},"1,750 tok\u002Fs",[210,2236,2237,2240,2243],{},[228,2238,2239],{},"IQ3_S",[228,2241,2242],{},"53 tok\u002Fs",[228,2244,2245],{},"1,620 tok\u002Fs",[111,2247,2248,2249,2252],{},"24GB 的卡更快。那位開發者在他的 4090 + 64GB RAM 上實測：",[128,2250,2251],{},"解碼約 140 tok\u002Fs、prefill 約 4,400 tok\u002Fs","，256K 上下文開滿也還跑得動。",[108,2254,2255],{},[111,2256,2257],{},"他第一次測到 203 tok\u002Fs，後來發現是 benchmark 沒把 reasoning 時間與快取命中算進去。修正後才是 140。這種「先懷疑自己的量測」比數字本身更有價值。",[111,2259,2260,2261,2264],{},"要注意上面那張表的「讀 prompt 2,650 tok\u002Fs」是快取熱起來之後的數字。",[128,2262,2263],{},"冷啟動第一次讀長文會慢很多","——官方文件寫的是每 30,000 tokens 約一分鐘，大約 500 tok\u002Fs。",[115,2266,2267],{"id":2267},"代價",[111,2269,2270,2273],{},[128,2271,2272],{},"一、量化到 2-bit。"," 94 tok\u002Fs 是 Q2_0 的數字，那是很兇的壓縮。要品質就得犧牲速度——IQ3_S 剩 53 tok\u002Fs。這是取捨，不是免費的。",[111,2275,2276,2279],{},[128,2277,2278],{},"二、記憶體與啟動。"," 開機要把 35–55GB 載進記憶體，電腦會卡 1–3 分鐘；整個程序佔用超過 50GB。32GB 的機器只能跑 Coder 變體（砍掉一半專家），官方也提醒它在中文本表現較弱。",[111,2281,2282,2285],{},[128,2283,2284],{},"三、一次一個請求。"," 預設不支援併發，第二個請求要排隊。開併發得調參數，12GB 的卡上會讓每個回答變慢。",[115,2287,2288],{"id":2288},"調參的迷思",[111,2290,2291],{},"Strata 有不少參數可調：PCIE_FRAC（GPU 與 CPU 怎麼分攤計算）、CPU Workers、MTP 深度、KVGROW。",[111,2293,2294,2295,832],{},"那位開發者花了大量時間做矩陣測試，他的結論是：",[128,2296,2297],{},"官方自動校準就夠了",[111,2299,2300,2301,2304],{},"他自己測出的「快 5%、快 9%」，多跑幾輪就消失，有時候還反過來。原因是速度受當下生成的內容、專家路由狀態、MTP 命中率、GPU 快取影響——這些變異比參數本身還大。這是",[128,2302,2303],{},"在他那台機器上","的結論，換硬體要重新驗證。",[111,2306,2307,2308,2311],{},"有一個參數值得單獨提：",[128,2309,2310],{},"KVGROW","。預設開 256K 上下文時，引擎會先把整段 KV cache 的顯存預留起來，結果短對話也沒位置放熱專家，速度掉到最慢檔。KVGROW 讓 cache 跟著實際用量長大：短對話快，長對話才讓位。",[115,2313,2315],{"id":2314},"所以該不該自架","所以，該不該自架？",[111,2317,2318],{},"先把興奮放一邊，算一算。匯率抓 1 美元 ≈ NT$31.5。",[111,2320,2321,2322,2325],{},"Qwen3.8-Flash 的 API 價格是 ",[128,2323,2324],{},"$0.16 \u002F 百萬 input token、$0.47 \u002F 百萬 output","——output 是 input 的三倍，這段先只算 input，方便比較。",[111,2327,2328],{},"一台跑得舒服的機器（4090 或同級 + 64GB RAM）大約台幣 8–10 萬。",[204,2330,2331,2344],{},[207,2332,2333],{},[210,2334,2335,2338,2341],{},[213,2336,2337],{},"每月 input 用量",[213,2339,2340],{},"API 成本",[213,2342,2343],{},"自架電費",[223,2345,2346,2357,2367],{},[210,2347,2348,2351,2354],{},[228,2349,2350],{},"1,000 萬 token",[228,2352,2353],{},"約 NT$50",[228,2355,2356],{},"NT$250–370",[210,2358,2359,2362,2365],{},[228,2360,2361],{},"1 億 token",[228,2363,2364],{},"約 NT$500",[228,2366,2356],{},[210,2368,2369,2372,2375],{},[228,2370,2371],{},"10 億 token",[228,2373,2374],{},"約 NT$5,000",[228,2376,2356],{},[111,2378,2379],{},"電費怎麼算的：推論時約 350W，每天用兩小時、其餘時間閒置 80W，一個月約 74 度；台電累進電價抓每度 NT$3–5，就是每個月 250–370 元。24 小時滿載的話是 250 度左右，大約 NT$750–1,250。",[111,2381,2382,2383,2386],{},"看表就知道結論：",[128,2384,2385],{},"每月十億 token 以下，API 連電費都比較便宜","，還沒算機器折舊。",[111,2388,2389,2390,2393],{},"粗算回本點：",[325,2391,2392],{},"10 萬 ÷ (5,000 − 370) ≈ 22 個月","——那是每月十億 token 的量才成立。對絕大多數中小企業，這個量級不會出現。",[111,2395,2396],{},"那自架還有什麼意義？三個：",[419,2398,2399,2405,2411],{},[168,2400,2401,2404],{},[128,2402,2403],{},"隱私","：資料不出門。這是最難用錢換的",[168,2406,2407,2410],{},[128,2408,2409],{},"離線","：工廠、船上、沒有網路的現場",[168,2412,2413,2416],{},[128,2414,2415],{},"不受限","：不用擔心費率調整、政策變更、API 下架",[111,2418,2419],{},"如果你不屬於這三種，用 API 就好，而且會用得比自架開心。",[115,2421,2422],{"id":2422},"真正的重點",[111,2424,2425],{},"這篇想說的不是「125B 好強」。",[111,2427,2428,2429,2432],{},"是",[128,2430,2431],{},"模型大小已經不能拿來判斷硬體需求了","。同樣 125B，你可以用 360GB 的伺服器跑，也可以用 12GB 的遊戲卡跑——差別只在於有沒有人幫你把記憶體階層調度好。",[111,2434,2435,2436,2439,2440,2444],{},"這也是我們做",[147,2437,2438],{"href":846},"地端部署","時真正在處理的問題：不是「這顆模型跑不跑得動」，而是「你的瓶頸在哪一層」。我們在",[147,2441,2443],{"href":2442},"\u002Fwork\u002Fembedded-vision-background","資源受限裝置","上累積的經驗，換到 LLM 上其實是同一件事。",[111,2446,2447],{},"如果你正在評估要不要自架，先問三個問題：",[165,2449,2450,2456,2462],{},[168,2451,2452,2455],{},[128,2453,2454],{},"每月 token 量有沒有到億級？"," 沒有，API 幾乎一定更划算。",[168,2457,2458,2461],{},[128,2459,2460],{},"資料能不能出門？"," 不能，才輪到自架。",[168,2463,2464,2467],{},[128,2465,2466],{},"有沒有 64GB 記憶體與一張 12GB 以上的卡？"," 沒有，連門檻都不到。",[111,2469,2470],{},"三題有兩題是「否」，答案就是 API。",{"title":865,"searchDepth":866,"depth":866,"links":2472},[2473,2474,2475,2476,2477,2478,2479,2480,2481,2482],{"id":2023,"depth":866,"text":2023},{"id":2048,"depth":866,"text":2049},{"id":2072,"depth":866,"text":2073},{"id":2089,"depth":866,"text":2090},{"id":2171,"depth":866,"text":2172},{"id":2181,"depth":866,"text":2181},{"id":2267,"depth":866,"text":2267},{"id":2288,"depth":866,"text":2288},{"id":2314,"depth":866,"text":2315},{"id":2422,"depth":866,"text":2422},"\u002Fimages\u002Fblog\u002Fstrata-125b-on-12gb.svg","125B 模型跑在 12GB 顯卡上的示意圖","2026-10-24","Strata 讓 125B 的 Qwen3.8-Flash-Next 跑在 12GB 顯卡的遊戲機上，解碼 94 tok\u002Fs。拆解它靠什麼做到、代價在哪裡，以及中小企業該不該自架。",{},"\u002Fimages\u002Fblog\u002Fstrata-125b-on-12gb.jpg","\u002Fblog\u002Fzh-tw\u002Fstrata-125b-on-12gb",{"title":2001,"description":2486},"strata-125b-on-12gb","blog\u002Fzh-TW\u002Fstrata-125b-on-12gb",[892,2494,2495],"qwen","inference","AzAFYoV9JKM6X1-qHOAhv64-PaZVkfvITS_J8_39_Oc",1793708018241]