[{"data":1,"prerenderedAt":2682},["ShallowReactive",2],{"post-running-is-not-done":3,"site-info":1430,"related-en-running-is-not-done":1525},{"zh":4,"en":803},{"id":5,"title":6,"author":7,"body":8,"cover":783,"coverAlt":784,"date":785,"description":786,"draft":787,"extension":788,"featured":787,"lang":789,"meta":790,"navigation":200,"ogImage":791,"path":792,"seo":793,"slug":794,"stem":795,"tags":796,"translationKey":794,"updated":801,"__hash__":802},"blog\u002Fblog\u002Fzh-TW\u002Frunning-is-not-done.md","「跑得起來」跟「真的幹完活」是四件事","Jecision",{"type":9,"value":10,"toc":769},"minimark",[11,18,22,25,28,35,38,41,44,47,58,61,65,68,96,99,104,107,174,177,180,183,186,189,206,210,213,216,219,225,241,244,247,250,253,256,259,262,266,269,272,275,281,284,287,290,301,304,307,310,313,316,319,322,340,343,346,349,352,355,358,361,364,367,370,373,423,426,429,432,435,438,441,444,447,450,453,456,459,462,465,468,471,516,519,522,525,534,537,540,543,546,549,560,563,566,569,572,634,637,640,643,646,649,666,669,672,675,678,710,713,724,737,745,753,757,760,763,766],[12,13,14],"blockquote",{},[15,16,17],"p",{},"數字全部來自 arXiv 2610.34241v1。那是作者自己的工程紀錄，不是受控實驗；「我認為」的部分我會標出來。",[19,20,21],"h2",{"id":21},"先講一個畫面",[15,23,24],{},"「模型有回應了。」這是最容易騙過驗收的一句話。API 回 200、文字讀得懂，但工具可能根本沒被呼叫出去。",[15,26,27],{},"2026 年 7 月，Vaibhav Jain 把幾台筆電串起來，跑一個單機放不下的模型。聊天可以，當 coding agent 用卻慢到不能用；每次重啟，32B 的權重要在 Wi-Fi 上傳五分鐘（8.7 GB）。最後他收掉分散式，改成單機。",[15,29,30,31],{},"這件事的重點不是「分散式不行」。是他從那時候開始發現：",[32,33,34],"strong",{},"模型跑得起來，跟它能不能幫你幹完活，是四件不同的事。",[15,36,37],{},"這篇要拆的就是那四層，還有驗收單上通常漏掉的那一欄。",[15,39,40],{},"一個模型權重檔（checkpoint）載得進去，丟一句話會回話，然後呢？你叫它去看一個檔案、呼叫一個工具、維持一段長 session、守住一個 repo 的介面約定。這些它可能全部做不到，而你的 API 還是回你 200。",[15,42,43],{},"問題出在「相容」這兩個字。同一個號稱相容的 endpoint，後面可能是完全不同的東西。工具呼叫的格式、一輪對話怎麼結束、context 怎麼被改寫、快取有沒有重用，四件事都可以不一樣。",[15,45,46],{},"只要有一件不一樣，你在 A 模型上驗過的流程，換到 B 模型就會壞。而且通常不會給你錯誤訊息。",[15,48,49,50,57],{},"這篇是一份單人經驗報告的導讀。",[51,52,56],"a",{"href":53,"rel":54},"https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.34241",[55],"nofollow","論文","是 arXiv 2610.34241，作者是獨立研究者，紀錄的時間橫跨 2026 年 7 月到 10 月，跑過 vLLM、SGLang、llama.cpp 和 MLX。",[15,59,60],{},"它做的事很樸素：把「能跑」拆成四層，再用 45 場有隱藏檢查的實驗，告訴你第四層要怎麼驗、代價在哪裡。",[19,62,64],{"id":63},"四層不是一個開關","四層，不是一個開關",[15,66,67],{},"作者把「模型有沒有用」拆成四層：",[69,70,71,78,84,90],"ol",{},[72,73,74,77],"li",{},[32,75,76],{},"服務得起來","：checkpoint 載得進去，送一句話有東西回來。",[72,79,80,83],{},[32,81,82],{},"文字連貫","：回得出人話，讀得懂，能回答問題。",[72,85,86,89],{},[32,87,88],{},"工具被解析並執行","：它說「我要讀這個檔案」，client 真的收到，真的去讀。",[72,91,92,95],{},[32,93,94],{},"通過獨立行為檢查","：產出的程式碼，在另一個不屬於它的檢查器裡，真的把事做對。",[15,97,98],{},"論文裡有一句話是整篇的骨架：",[12,100,101],{},[15,102,103],{},"一個 endpoint 可以滿足前三層，卻在第四層全錯。而且如果第三層過不了，你根本量不到第四層。",[15,105,106],{},"（這是作者的話，我照原意翻。）",[108,109,110,126],"table",{},[111,112,113],"thead",{},[114,115,116,120,123],"tr",{},[117,118,119],"th",{},"層級",[117,121,122],{},"你要問的問題",[117,124,125],{},"通過之後，你還不知道什麼",[127,128,129,141,152,163],"tbody",{},[114,130,131,135,138],{},[132,133,134],"td",{},"1 服務",[132,136,137],{},"載得進去、有回應嗎？",[132,139,140],{},"回的是不是人話",[114,142,143,146,149],{},[132,144,145],{},"2 連貫",[132,147,148],{},"讀得懂、講得出道理嗎？",[132,150,151],{},"它會不會真的動手",[114,153,154,157,160],{},[132,155,156],{},"3 工具",[132,158,159],{},"工具呼叫被解析、被執行了嗎？",[132,161,162],{},"執行完的結果對不對",[114,164,165,168,171],{},[132,166,167],{},"4 驗證",[132,169,170],{},"產出通過獨立的行為檢查了嗎？",[132,172,173],{},"這一層過了，才叫「幹完活」",[15,175,176],{},"這四層不是「程度」的差別，是四道不同的門。第一層過了，不代表第二層；第二層過了，也不代表它會動手。很多人停在第二層就以為快成功了，因為它講得有模有樣。",[15,178,179],{},"你平常在驗收單上看到的三個層級，大概是這樣對應。「模型有回應」是第 1 層加第 2 層；「程式跑起來」是第 3 層；「測試通過」是第 4 層。",[15,181,182],{},"但「測試通過」要看是誰的測試。廠商自己寫的測試過了，你可能還停在第 3 層；第 4 層的檢查，得是別人寫的、在驗收前就凍結的。",[15,184,185],{},"我把第 1、2 層拆開，是因為它們的失敗長得很像、原因完全不同。一個是服務沒通；一個是通了，但只會講話。",[15,187,188],{},"（如圖 1）",[190,191,194,195,194,202],"figure",{"className":192},[193],"article-figure","\n  ",[196,197],"video",{"src":198,"poster":199,"autoPlay":200,"loop":200,"muted":200,"playsInline":200,"ariaLabel":201},"\u002Fimages\u002Fblog\u002Frunning-is-not-done\u002Flevels.mp4","\u002Fimages\u002Fblog\u002Frunning-is-not-done\u002Flevels-poster.jpg",true,"四層進度：服務、連貫、工具、驗證，第三與第四層之間有斷層",[203,204,205],"figcaption",{},"圖 1：四層不是程度差別，是四道不同的門。前三層都過，第四層仍可能全錯。",[19,207,209],{"id":208},"同一個-api不同的協議","同一個 API，不同的協議",[15,211,212],{},"作者把 Claude Code 接上自己寫的 gateway，後面接不同的模型。他原本想的是「換一個 endpoint 而已」，實際上的感覺，比較像換一個工作習慣完全不同的人。",[15,214,215],{},"有的模型照工具契約做。有的用文字描述它「打算執行這個指令」。有的會把它覺得不順的介面改掉，例如把一個本來穩定的函式簽名重寫成它喜歡的樣子。這些不是模型笨，是它跟你的 client 對「工具」這件事的理解不一樣。",[15,217,218],{},"他統計了 gateway 上的真實請求。Gemma 4 有 492 個帶工具的請求，其中 294 個解析出 tool_calls。144 個以 stop 結束、沒有 tool call；還有 54 個沒有存到解析結果。",[15,220,221,222],{},"那個 144 不能全部算失敗。正常收尾也會是 stop，沒有 tool call 可能只是任務做完了。這是本文的第一個陷阱：",[32,223,224],{},"「沒有工具呼叫」本身不是錯誤訊號，要看當下該不該有。",[15,226,227,228,232,233,236,237,240],{},"還有更直接的。Gemma 4 的存檔輸出裡有 8 個 ",[229,230,231],"code",{},"\u003C|tool_call>"," 前綴的畸形呼叫，解析器完全沒接住。GPT-OSS-120B 走 SGLang（一種推論伺服器）時，",[229,234,235],{},"tool_choice=required"," 會跟它自己的工具格式（Harmony）打架，得降成 ",[229,238,239],{},"auto","。工具回合的輸出上限也要放寬，不然 reasoning 和 tool JSON 會一起被截斷。",[15,242,243],{},"回合控制是另一半。對話的識別碼（session identifier）不見、工具呼叫繞圈、thinking channel 對不上。任何一個都會讓對話「看起來還在跑，其實已經迷路」。",[15,245,246],{},"最值得記下來的是這一段：API 格式（schema）檢查全部通過，接上真的 Claude Code 之後還是壞。",[15,248,249],{},"一次是 client 的工作目錄沒有被放進 request，模型只能猜路徑。一次是 streaming 的工具參數被當成「替換」而不是「附加」，參數被切斷，模型收到一個殘缺的呼叫。",[15,251,252],{},"這兩個都不會出現在單元測試裡。它們要你真的把 client 接上去，跑一次完整流程才會出現。",[15,254,255],{},"作者的診斷問題很簡單：工具失敗的時候，是模型不會用，還是中間某個環節把呼叫弄丟了？",[15,257,258],{},"如果是後者，就修在源頭：模型的 template、parser 規則、serve flag、request 翻譯。而不是在 gateway 上不斷補丁。他後來的做法是把 gateway 縮成薄的翻譯層，只做協議與安全檢查。上游失敗就往外傳，不要讓一個失敗的 stream 看起來像正常結束。",[15,260,261],{},"「能被設定」不等於「真的跑起來、被評測過」。這句話對採購很有用：供應商說「支援」，通常只代表它能被設定。",[19,263,265],{"id":264},"對話看起來連續其實中間斷了","對話看起來連續，其實中間斷了",[15,267,268],{},"第三層過了，還有一個更隱形的問題：對話。",[15,270,271],{},"對你來說，那是一段連續的對話。對 runtime 來說，它是三樣東西：token 前綴（對話開頭的固定內容）、KV cache（算過的注意力狀態）、執行 slot。你看到的「連續」，它不一定看得到。",[15,273,274],{},"作者留下 4,165 個 gateway 回合、623 個除錯工作的紀錄。其中 85 筆是冷前綴：完全沒有重用，30,000 到 36,000 個 input token。280 筆是高重用：至少 90% 重用、少於 2,000 個新 token。",[15,276,277,278],{},"首字延遲的中位數：",[32,279,280],{},"冷前綴 69.4 秒，高重用 2.8 秒。",[15,282,283],{},"差了約 25 倍。",[15,285,286],{},"但這裡要很小心。作者自己標了「觀察性資料」：兩組工作的組成不同、不是隨機分配、沒有配對。所以不能寫成「快取讓速度快 25 倍」。能說的是：前綴有沒有被重用，跟你的體感差距很大，而且大到不像是雜訊。",[15,288,289],{},"（如圖 2）",[190,291,194,293,194,298],{"className":292},[193],[196,294],{"src":295,"poster":296,"autoPlay":200,"loop":200,"muted":200,"playsInline":200,"ariaLabel":297},"\u002Fimages\u002Fblog\u002Frunning-is-not-done\u002Fcache.mp4","\u002Fimages\u002Fblog\u002Frunning-is-not-done\u002Fcache-poster.jpg","同一段對話的兩條路徑：前綴穩定時重用快取，前綴被改寫時重算",[203,299,300],{},"圖 2：同一段對話。前綴沒變，後端重用快取；前綴被改了一點點，整段 context 重算。",[15,302,303],{},"機制很樸素。推論後端可以重用「完全相同的 token 前綴」的 KV 狀態。所以只要前綴變了，整段 context 就要重算一次 prefill（把前綴重新算過）。哪怕只是前面多一個時間戳、換一個 system banner、工具定義的順序不同、JSON 序列化方式不一樣。",[15,305,306],{},"你改的是一個 banner；你付的是整段 context。",[15,308,309],{},"還有一種更冤的：旁支請求。coding agent 的流量不是只有主對話，還有生成標題、權限確認這類小請求。如果後端只有一個可重用的執行 slot，一個旁支請求就可能把主對話的駐留狀態擠掉。",[15,311,312],{},"他的紀錄裡，65 個被歸類為 auxiliary 的請求，有 44 個沒有快取重用。在高重用的紀錄中，auxiliary 的首字延遲中位數是 12.2 秒（13 筆），主請求是 2.7 秒（267 筆）。",[15,314,315],{},"長對話超過某個長度時，系統會把它壓成摘要（compaction）。這裡也有類似的假象：他刻意把壓縮門檻調低，在 12 次測試裡，子代理壓縮觸發了 6 次。這 6 次任務全部沒有完成，而且其中一筆還被標成 PASS。",[15,317,318],{},"「壓縮發生了」不是「工作完成了」。這兩件事在 log 裡長得很像。",[15,320,321],{},"所以他把 session 連續性當成應用的責任，而不是後端的參數：",[323,324,325,328,331,334,337],"ul",{},[72,326,327],{},"訊息序列盡量 append-only，除非真的需要改前面的內容。",[72,329,330],{},"已經送出的工具訊息，不要為了美觀重新序列化。",[72,332,333],{},"streaming 的 tool JSON 和最後的完整版本要一致。",[72,335,336],{},"每回合的 metadata 不要動到被凍結的 system 前綴。",[72,338,339],{},"保住 session affinity（下一回合要回到同一個執行位置）。穩定的 prompt 不夠，slot 換了就沒有那份快取。",[15,341,342],{},"快取不是後端參數，是應用的紀律。",[19,344,345],{"id":345},"第四層才算驗收",[15,347,348],{},"接下來是這篇論文真正做實驗的地方。當部署、協議、session 都處理到可以跑範圍明確的任務之後，問題才真正出現。一個開源的小模型，能不能交出通過檢查的程式碼？",[15,350,351],{},"設計是這樣：五類 TypeScript 任務，報價、按比例分攤、客戶、拆分、零件；每類每種跑三次，總共 45 場。三種條件是：直接寫；優化委派（父層只寫規格、審查並套用工人的產出）；可分批的委派。最後一種允許父層把工作拆成連續幾包，但只有少數場次真的拆。",[15,353,354],{},"父層（負責規劃與審查的那一層）固定是 Claude Code 加 Claude Opus 5.5。工人（被委派實作的那顆模型）是量化過的 Qwen3-Coder 30B-A3B。每一場都用任務專屬的隱藏行為檢查判定，不看 agent 自己說「我完成了」。",[15,356,357],{},"結果：直接寫 15\u002F15；優化委派 12\u002F15；可分批的委派 14\u002F15。",[15,359,360],{},"這三個數字不能排名。每一格只有三次，信賴區間是重疊的；作者也明說，這些資料不足以建立一般的成功率差異。能說的是：工人真的交得出東西。30 場委派裡，26 場驗證通過，而且都保留了工人寫的程式碼。但不是每次。",[15,362,363],{},"另外，那 14\u002F15 不是「批次比較好」的證據。15 場裡只有 5 場真的用到批次呼叫，其餘是單次委派。論文自己說這是「批次標記的適應式策略」，不是批次與非批次的對照。",[15,365,366],{},"第二組更嚴格。15 個任務，各跑三次，直接與委派各 45 場。工人產出的檔案要先通過一份固定的介面契約，才能進父層審查。",[15,368,369],{},"直接寫 45\u002F45。委派給 Nemotron Cascade 2 工人 12\u002F45，其中 11 場是工人真的替換成功，1 場是父層改寫後才通過。Qwen3-Coder 工人 0\u002F45。",[15,371,372],{},"這個 45\u002F45 也要保留一點判斷空間。題目是為這份研究準備的，而且只有五類、每類三題；論文沒有做難度分層。",[108,374,375,388],{},[111,376,377],{},[114,378,379,382,385],{},[117,380,381],{},"實驗",[117,383,384],{},"直接寫",[117,386,387],{},"委派給工人",[127,389,390,401,412],{},[114,391,392,395,398],{},[132,393,394],{},"45 場試驗",[132,396,397],{},"15\u002F15",[132,399,400],{},"Qwen3-Coder 12\u002F15（優化委派）、14\u002F15（可分批）",[114,402,403,406,409],{},[132,404,405],{},"第二組（15 任務 × 3）",[132,407,408],{},"45\u002F45",[132,410,411],{},"Nemotron 12\u002F45（11 場是工人替換）；Qwen 0\u002F45",[114,413,414,417,420],{},[132,415,416],{},"第二組，中斷算失敗",[132,418,419],{},"37\u002F45",[132,421,422],{},"Nemotron 8\u002F45",[15,424,425],{},"這裡有一個必須寫出來的細節：有 16 場因為訂閱額度用完、連模型都沒有呼叫就中斷，後來被替換掉。這 16 場是直接組 8 場、Nemotron 組 8 場。替換之後，直接組多拿到 8 場成功，Nemotron 組多拿到 4 場。",[15,427,428],{},"如果把原本中斷的 16 場算成失敗，數字會變成 37\u002F45 與 8\u002F45。",[15,430,431],{},"作者兩個都寫了。這些是排程中斷，不是推論失敗。但如果你要拿這個數字去說服人，兩種算法都要看得到。",[15,433,434],{},"還有一件事要誠實講。五個任務的定義和隱藏測試，是在 AI 協助下產生的。合併後的測試檔，在第一場評估前大約 39 秒才建立，而且直到最後一場跑完才第一次進版控。",[15,436,437],{},"作者沒有把這件事藏起來。他能主張的是：45 場存下來的任務指令和可見檢查完全一致。事後比對的分數標籤，也和評分程式（scorer）對得上。但他不能宣稱這是一個有獨立事前快照的 benchmark。",[15,439,440],{},"第一組的測試流程（harness）也有限制。父層的 constraints 欄位沒有被轉給工人。gateway 會丟掉重複的 system message、截斷超過 1,800 字元的訊息。41 個重建的工人 prompt 裡，有 2 個各少了約 220 字元，都在失敗的場次。",[15,442,443],{},"所以「工人不聽指令」這個結論要打折。有些失敗是工人不會，有些是它根本沒收到完整的指令。",[15,445,446],{},"那 0\u002F45 呢？那不是「這顆模型什麼都做不到」的意思。那 45 場沒有 direct 對照組，父層接受與套用沒有重播，兩個研究的父層快取狀態也沒有重置。作者自己把它限定成「這批任務、這份契約、這個工人設定」的結果。",[15,448,449],{},"驗證是第四層，不是第三層的副產品。",[19,451,452],{"id":452},"可是帳單反過來了",[15,454,455],{},"直覺是這樣的：前沿模型的輸出 token 很貴，本地小模型便宜。那把實作委派給本地工人，不就省錢了嗎？",[15,457,458],{},"問題是委派不是免費。父層要寫規格、要看工人做了什麼、要判斷能不能用、失敗要收拾。這些都是前沿模型的 token。",[15,460,461],{},"作者把成本拆成：父層 context、寫規格、審查、收拾、工人。這篇只量到前沿模型那幾項。工人端的電費、硬體、時間都沒有算進去，而且他明講「未量測不能當成零」。",[15,463,464],{},"第一組，每次驗證成功的成本：直接寫 $0.136；委派 $0.214（優化版）與 $0.199（另一種）。大約 1.6 倍。",[15,466,467],{},"第二組：直接 $0.153；Nemotron 委派 $1.313。8.6 倍。",[15,469,470],{},"Qwen 那組的「每次成功成本」算不出來，不是零——因為它一場都沒成功。",[108,472,473,487],{},[111,474,475],{},[114,476,477,479,481,484],{},[117,478,381],{},[117,480,384],{},[117,482,483],{},"委派",[117,485,486],{},"倍數",[127,488,489,502],{},[114,490,491,493,496,499],{},[132,492,394],{},[132,494,495],{},"$0.136",[132,497,498],{},"$0.214（優化）、$0.199（另一種）",[132,500,501],{},"約 1.6×",[114,503,504,507,510,513],{},[132,505,506],{},"第二組（Nemotron）",[132,508,509],{},"$0.153",[132,511,512],{},"$1.313",[132,514,515],{},"8.6×",[15,517,518],{},"（成本是 API 列表價換算，含失敗場次，不含本地硬體、電費、工程時間。）",[15,520,521],{},"還有一個小案例。有一場收費功能的任務，工人寫的東西通過了、也被套用了，但前沿成本是 $0.1856，直接寫是 $0.1293。加上本地執行時間，實際耗時 126.4 秒對 27.4 秒。",[15,523,524],{},"成功替換，不等於省錢。",[15,526,527,528,533],{},"但這不能推成「委派一定不省錢」。作者自己引了 ",[51,529,532],{"href":530,"rel":531},"https:\u002F\u002Farxiv.org\u002Fabs\u002F2608.04804",[55],"Scrouting"," 這個反例。小模型先偵察 repo、前沿模型負責修，在 266 題上解了 159 題，最強單一模型是 158 題。只差一題，成本大約五分之一。",[15,535,536],{},"方向相反，結果就相反。他的結論只限定在自己的委派方式：前沿模型寫規格、本地工人實作。",[15,538,539],{},"成本定義也很窄：用 API 列表價換算，不含本地硬體、電費、工程時間。所以「8.6 倍」不是你的總帳，是前沿那一邊的帳。",[15,541,542],{},"便宜的是 token，貴的是協調。",[19,544,545],{"id":545},"驗收腳本自己也會誤殺",[15,547,548],{},"這篇最實用的一段，其實不是模型，是驗收。",[15,550,551,552,555,556,559],{},"他的工人產出要先過一份靜態契約：import、export、公開函式的參數列表都要對得上。問題是，任務的 stub 裡有些用不到的參數前面有底線（",[229,553,554],{},"_entries","），工人實作時很自然地把它拿掉（",[229,557,558],{},"entries","）。",[15,561,562],{},"對一般的位置參數呼叫來說，這完全不影響行為。但契約是比對「參數名的字串」，所以直接退件。父層根本沒看到，隱藏檢查也沒跑。",[15,564,565],{},"後來他做了一次事後診斷。36 個「只因底線被移除」而被退的 artifacts，跳過那一條規則、原封不動套到新的任務副本，再跑原本的隱藏檢查。",[15,567,568],{},"19 個通過，17 個還是不行。",[15,570,571],{},"所以那條規則不是全錯，它同時擋掉了好東西和壞東西。差別在於：擋掉壞東西是它的工作，擋掉好東西是它的副作用。而你的流程如果只有這道門，副作用就是最終結果。",[108,573,574,590],{},[111,575,576],{},[114,577,578,581,584,587],{},[117,579,580],{},"工人",[117,582,583],{},"只因底線被退",[117,585,586],{},"原封不動通過隱藏檢查",[117,588,589],{},"仍失敗",[127,591,592,606,620],{},[114,593,594,597,600,603],{},[132,595,596],{},"Qwen3-Coder",[132,598,599],{},"23",[132,601,602],{},"12",[132,604,605],{},"11",[114,607,608,611,614,617],{},[132,609,610],{},"Nemotron",[132,612,613],{},"13",[132,615,616],{},"7",[132,618,619],{},"6",[114,621,622,625,628,631],{},[132,623,624],{},"合計",[132,626,627],{},"36",[132,629,630],{},"19",[132,632,633],{},"17",[15,635,636],{},"作者沒有把這 19 個算成成功。它們沒有走完父層審查與套用流程，分數維持 Qwen 0\u002F45、Nemotron 12\u002F45。他也沒有因此說契約要拿掉。他提的是下一個實驗：讓工人收到具體的驗證錯誤，允許一次有界的修正。",[15,638,639],{},"那個實驗還沒做。",[15,641,642],{},"在這份契約上，比模型能力更容易出錯的，是驗收設計。",[19,644,645],{"id":645},"它不能證明什麼",[15,647,648],{},"寫到這裡應該先把邊界畫出來，不然很容易變成另一種「AI 什麼都能」或「什麼都不能」。",[323,650,651,654,657,660,663],{},[72,652,653],{},"這是單人的回溯性經驗報告，不是受控實驗。工作負載隨工程進展變動。",[72,655,656],{},"樣本小：五類任務、每格三次；單一父層；單一工人設定。",[72,658,659],{},"成本只算前沿模型、用列表價換算。",[72,661,662],{},"原始紀錄沒有公開，你沒辦法自己驗算。",[72,664,665],{},"作者明說：研究沒有合併，不替模型或執行環境排名。",[15,667,668],{},"所以這篇能給你的不是「哪顆模型比較好」，是一張驗收框架。我認為，這份報告最有價值的地方不是那些數字，是它把「完成」拆成可以逐層檢查的東西。",[15,670,671],{},"這是一份可檢查的經驗，不是一張排行榜。",[19,673,674],{"id":674},"明天就能用的檢查表",[15,676,677],{},"作者在最後整理了五條實務規則。我把它們改寫成可以直接放進 PoC 文件的形式：",[69,679,680,686,692,698,704],{},[72,681,682,685],{},[32,683,684],{},"在任務邊界定義成功。"," 不要用「有沒有回應」驗收，用「產出有沒有通過一個不屬於它的檢查」驗收。而且要用真的 client 跑，不是用自己寫的 API 測試腳本。",[72,687,688,691],{},[32,689,690],{},"把部署當成一份 recipe，不是一個模型名字。"," checkpoint revision、量化、runtime、template、context 設定、gateway、client、可用記憶體，任何一個換了，都要重驗。",[72,693,694,697],{},[32,695,696],{},"session 連續性是應用的責任。"," 前綴穩定、工具訊息不要重寫、旁支請求不要搶掉主對話的狀態。",[72,699,700,703],{},[32,701,702],{},"委派看「每次驗證成功」的成本。"," 不是 token 單價，也不是「這次呼叫花了多少」。",[72,705,706,709],{},[32,707,708],{},"能用行為驗就用行為驗。"," 靜態檢查便宜、該留，但要知道它擋掉的是什麼。",[15,711,712],{},"如果你是要買的人，三個問題就夠：",[323,714,715,718,721],{},[72,716,717],{},"這個 demo 停在哪一層？「它會回話」是第 2 層。",[72,719,720],{},"第 4 層你們怎麼驗？驗收腳本誰寫的、什麼時候凍結的？",[72,722,723],{},"成本怎麼算，每次呼叫，還是每次驗證成功？",[15,725,726,727,731,732,736],{},"我們自己的",[51,728,730],{"href":729},"\u002Fwork\u002Fvideo-transcript-pipeline","影片轉錄管線","也遇過同一類問題。模型跑起來了，流程還是卡住；而且每一個環節的失敗，看起來都像「模型有點笨」。我們沒有做這種規模的量測，所以這篇只借論文的數字。本地跑大模型的記憶體取捨，之前寫在",[51,733,735],{"href":734},"\u002Fblog\u002Fstrata-125b-on-12gb","另一篇","。",[15,738,739,740,744],{},"上一則我們談的是",[51,741,743],{"href":742},"\u002Fblog\u002Fbroken-on-arrival-quantized-models","檔案看起來對、其實不能用","；這一則是模型看起來對、其實幹不完活。",[15,746,747,748,752],{},"如果你要的是有人陪你把這張表填完，我們的",[51,749,751],{"href":750},"\u002Fservices","企業 AI 導入健檢","做的就是這件事。先判斷哪些工作現在就能交給 AI，哪些還不成熟。",[19,754,756],{"id":755},"結論你拿哪一層當完成","結論：你拿哪一層當「完成」",[15,758,759],{},"前三層可以在一個下午示範完。載一個模型、丟幾句話、接一個 client，快的話半天。",[15,761,762],{},"第四層要幾週：設計任務、寫隱藏檢查、跑足夠的次數、把失敗分類、算成本。而且做完之後，你得到的通常不是「這顆模型可以」，而是「這批任務、這份契約下，它可以」。",[15,764,765],{},"驗收單上缺的那一欄，就是差價所在。",[15,767,768],{},"明天可以先做一件事：把「測試通過」寫進你的 PoC 定義，然後要求對方出示第 4 層是怎麼驗的。",{"title":770,"searchDepth":771,"depth":771,"links":772},"",2,[773,774,775,776,777,778,779,780,781,782],{"id":21,"depth":771,"text":21},{"id":63,"depth":771,"text":64},{"id":208,"depth":771,"text":209},{"id":264,"depth":771,"text":265},{"id":345,"depth":771,"text":345},{"id":452,"depth":771,"text":452},{"id":545,"depth":771,"text":545},{"id":645,"depth":771,"text":645},{"id":674,"depth":771,"text":674},{"id":755,"depth":771,"text":756},"\u002Fimages\u002Fblog\u002Frunning-is-not-done.svg","標題「跑得起來」跟「真的幹完活」是四件事的技術筆記封面","2026-11-03","同一個 API 相容的 endpoint，在工具呼叫、回合控制、context 和快取上都不一樣。一份 4,165 個回合的單人經驗報告，把「模型有回應」「工具真的執行」「程式碼通過檢查」拆成四層，也量到委派給本地 Nemotron 工人時，每次驗證成功反而貴 8.6 倍。文末有一張能直接拿去用的驗收清單。",false,"md","zh-TW",{},"\u002Fimages\u002Fblog\u002Frunning-is-not-done.jpg","\u002Fblog\u002Fzh-tw\u002Frunning-is-not-done",{"title":6,"description":786},"running-is-not-done","blog\u002Fzh-TW\u002Frunning-is-not-done",[797,798,799,800],"local-llm","agents","serving","tool-calling",null,"Mv8FTsKM24mkF-nqqLfoFPMU-KhWVf5dDgEeAbaotSA",{"id":804,"title":805,"author":7,"body":806,"cover":1419,"coverAlt":1420,"date":785,"description":1421,"draft":787,"extension":788,"featured":787,"lang":1422,"meta":1423,"navigation":200,"ogImage":1424,"path":1425,"seo":1426,"slug":794,"stem":1427,"tags":1428,"translationKey":794,"updated":801,"__hash__":1429},"blog\u002Fblog\u002Fen\u002Frunning-is-not-done.md","A running model is not a finished job",{"type":9,"value":807,"toc":1407},[808,813,817,820,823,826,829,836,840,866,869,874,877,938,941,944,947,950,961,965,968,971,974,977,989,992,995,998,1001,1004,1008,1011,1014,1017,1020,1023,1026,1029,1032,1035,1038,1055,1058,1061,1072,1076,1079,1082,1085,1088,1091,1094,1097,1100,1146,1149,1152,1155,1158,1161,1164,1167,1171,1174,1177,1180,1183,1186,1189,1192,1199,1202,1205,1209,1212,1221,1224,1227,1230,1233,1284,1287,1290,1293,1297,1311,1314,1317,1321,1324,1356,1359,1370,1377,1384,1391,1395,1398,1401,1404],[12,809,810],{},[15,811,812],{},"Every number here comes from arXiv 2610.34241v1, a single-author engineering report. It is not a controlled experiment. Where I add a judgement, I say so.",[19,814,816],{"id":815},"a-scene","A scene",[15,818,819],{},"\"The model answered.\" That is the easiest sentence to fail an acceptance test with. The API returns 200, the prose reads well, and the tool call may never have left the building.",[15,821,822],{},"In July 2026, Vaibhav Jain pooled a few laptops to serve a model too large for any one machine. Chat worked; a coding agent did not. Every restart pushed a 32B checkpoint across Wi-Fi — 8.7 GB, roughly five minutes. He went single-node and noticed the real problem: \"the model is running\" is a very weak success criterion.",[15,824,825],{},"A checkpoint loads and answers. Then you ask it to inspect a file, call a tool, hold a long session, or preserve a repository contract. It may fail all of that, and your API still returns 200.",[15,827,828],{},"\"Compatible\" hides a lot. Behind one nominally compatible endpoint, tool-call formatting, turn control, context handling and cache reuse can all differ. Change one, and a workflow you verified on model A breaks on model B — usually without an error message.",[15,830,831,835],{},[51,832,834],{"href":53,"rel":833},[55],"arXiv 2610.34241"," covers July to October 2026 across vLLM, SGLang, llama.cpp and MLX. Its move is simple: split \"it runs\" into four levels, then run 45 trials with hidden checks to see what level four costs.",[19,837,839],{"id":838},"four-levels-not-one-switch","Four levels, not one switch",[69,841,842,848,854,860],{},[72,843,844,847],{},[32,845,846],{},"Served"," — the checkpoint loads and returns something.",[72,849,850,853],{},[32,851,852],{},"Coherent"," — the text reads like language and answers the question.",[72,855,856,859],{},[32,857,858],{},"Parsed and executed tool calls"," — it says \"read this file\", the client receives it, and the read actually happens.",[72,861,862,865],{},[32,863,864],{},"Verified task completion"," — the code passes an independent behavioural check.",[15,867,868],{},"The paper's framing sentence is the spine of everything below:",[12,870,871],{},[15,872,873],{},"An endpoint can satisfy the first three and fail the fourth completely. And if it cannot satisfy the third, you cannot measure the fourth at all.",[15,875,876],{},"(My translation. The emphasis is the author's.)",[108,878,879,892],{},[111,880,881],{},[114,882,883,886,889],{},[117,884,885],{},"Level",[117,887,888],{},"The question you ask",[117,890,891],{},"What passing it still doesn't tell you",[127,893,894,905,916,927],{},[114,895,896,899,902],{},[132,897,898],{},"1 Served",[132,900,901],{},"Does it load and answer?",[132,903,904],{},"Whether the answer is language",[114,906,907,910,913],{},[132,908,909],{},"2 Coherent",[132,911,912],{},"Does it read and reason?",[132,914,915],{},"Whether it will act",[114,917,918,921,924],{},[132,919,920],{},"3 Tools",[132,922,923],{},"Are tool calls parsed and executed?",[132,925,926],{},"Whether the result is right",[114,928,929,932,935],{},[132,930,931],{},"4 Verified",[132,933,934],{},"Does the output pass an independent check?",[132,936,937],{},"This is \"done\"",[15,939,940],{},"These are not degrees of the same thing. They are four different gates. Level one passing says nothing about level two; level two passing says nothing about whether the model will pick up a tool. Most people stop at level two and think they are close, because the model sounds convincing.",[15,942,943],{},"The three tiers on an acceptance sheet map roughly like this: \"it responds\" is levels 1 and 2; \"the code runs\" is level 3; \"the tests pass\" is level 4. I keep 1 and 2 separate because their failures look similar and have different causes — one service never came up, the other came up and only talks.",[15,945,946],{},"And \"the tests pass\" depends on whose tests. If the vendor wrote them, you may still be at level three. Level four needs a check written by someone else and frozen before acceptance.",[15,948,949],{},"(Figure 1)",[190,951,194,953,194,958],{"className":952},[193],[196,954],{"src":955,"poster":956,"autoPlay":200,"loop":200,"muted":200,"playsInline":200,"ariaLabel":957},"\u002Fimages\u002Fblog\u002Fen\u002Frunning-is-not-done\u002Flevels.mp4","\u002Fimages\u002Fblog\u002Fen\u002Frunning-is-not-done\u002Flevels-poster.jpg","Four levels of progress: served, coherent, tools, verified, with a break before verified",[203,959,960],{},"Figure 1: four different gates, not four degrees. The first three can pass and the fourth still fail.",[19,962,964],{"id":963},"same-api-different-protocol","Same API, different protocol",[15,966,967],{},"He pointed Claude Code at a gateway he wrote, with different models behind it. The expectation was \"swap an endpoint\". The experience was closer to handing the work to a person with different habits.",[15,969,970],{},"Some models follow the tool contract. Some narrate what they intend to do instead of calling the tool. Some rewrite an interface they find awkward, including a function signature that was supposed to stay stable. None of this is stupidity; it is a different understanding of what a tool is.",[15,972,973],{},"The logs make the gap concrete. Of 492 Gemma 4 requests that carried tools, 294 produced parsed tool_calls. Another 144 ended with stop and no tool call; 54 had no saved parsed response.",[15,975,976],{},"Those 144 are not all failures. A normal finish is also a stop, and no tool call may simply mean the task was done. That is the first trap here: a missing tool call is not an error signal by itself. You have to know whether one was expected.",[15,978,979,980,982,983,985,986,988],{},"Some failures are blunter. Eight saved Gemma 4 outputs arrived with a malformed ",[229,981,231],{}," prefix and never parsed. On GPT-OSS-120B over SGLang, ",[229,984,235],{}," fought the model's own Harmony tool format and had to be downgraded to ",[229,987,239],{},"; tool rounds also needed a higher output allowance, or reasoning and tool JSON were truncated together.",[15,990,991],{},"Turn control is the other half: a missing session identifier, a tool loop, a thinking channel that does not line up. Each is enough to make a conversation look alive while it is lost.",[15,993,994],{},"The episode worth memorising: every API schema check passed, and the real Claude Code client still broke. In one case the working directory never reached the model-facing request, so the model guessed paths. In another, streamed tool-argument fragments were treated as replacements instead of appends, cutting the arguments mid-call.",[15,996,997],{},"Neither shows up in a unit test. Both require the actual client, end to end.",[15,999,1000],{},"His diagnostic question is the useful part: when a tool fails, did the model fail, or did something in the middle drop the call? If it is the latter, fix the source — template, parser rules, serve flags, request translation — rather than patching the gateway forever. He reduced the gateway to a thin translator doing protocol and security checks, and made upstream failures surface as errors instead of a stream that ends normally with nothing in it.",[15,1002,1003],{},"\"Configured\" is not \"served and evaluated\". When a vendor says \"supported\", it often means \"it can be configured\".",[19,1005,1007],{"id":1006},"the-conversation-looks-continuous-it-isnt","The conversation looks continuous. It isn't.",[15,1009,1010],{},"To you it is one continuous thread. To the runtime it is three things: a token prefix, a KV cache, and an execution slot. Your \"continuous\" is not necessarily its continuous.",[15,1012,1013],{},"He kept 4,165 gateway turns and 623 debug jobs. Eighty-five cold-prefix jobs — no reuse at all, 30,000 to 36,000 input tokens — had a median time to first token of 69.4 seconds. Two hundred and eighty high-reuse jobs — at least 90% reused, under 2,000 fresh tokens — had a median of 2.8 seconds.",[15,1015,1016],{},"About 25x.",[15,1018,1019],{},"The author labels this observational: the two groups are composed differently, not randomised, not matched. You cannot write \"caching makes it 25x faster\". What you can say is that prefix reuse is associated with a difference worth engineering.",[15,1021,1022],{},"The mechanism is mundane. An inference backend can reuse KV state for an identical token prefix. Change the prefix — a timestamp, a system banner, tool definitions in a different order, a different JSON serialisation — and the whole context is prefilled again.",[15,1024,1025],{},"You changed a banner. You paid for the whole context.",[15,1027,1028],{},"Auxiliary requests are the pettier version. A coding agent also generates titles and permission prompts; with one reusable execution slot, an auxiliary request can evict the main conversation's resident state. In his records, 65 requests were classified as auxiliary and 44 showed no cache reuse. Among high-reuse records, auxiliary requests had a median TTFT of about 12.2 seconds (13 observations) against 2.7 seconds for main requests (267 observations).",[15,1030,1031],{},"Compaction has the same smell. He lowered the threshold deliberately: in 12 tests, sub-agent compaction fired six times, none of those six tasks completed, and one was still labelled PASS.",[15,1033,1034],{},"\"Compaction happened\" is not \"work finished\". In a log, the two look alike.",[15,1036,1037],{},"So he treats session continuity as an application responsibility, not a backend setting:",[323,1039,1040,1043,1046,1049,1052],{},[72,1041,1042],{},"Keep the message sequence append-only unless earlier content genuinely must change.",[72,1044,1045],{},"Do not re-serialise tool messages for cosmetics.",[72,1047,1048],{},"Make streamed and final tool JSON agree.",[72,1050,1051],{},"Keep per-turn metadata out of the frozen system prefix.",[72,1053,1054],{},"Preserve session affinity. A stable prompt is not enough if the next turn lands on a slot without the cache.",[15,1056,1057],{},"Caching is not a backend flag. It is application discipline.",[15,1059,1060],{},"(Figure 2)",[190,1062,194,1064,194,1069],{"className":1063},[193],[196,1065],{"src":1066,"poster":1067,"autoPlay":200,"loop":200,"muted":200,"playsInline":200,"ariaLabel":1068},"\u002Fimages\u002Fblog\u002Fen\u002Frunning-is-not-done\u002Fcache.mp4","\u002Fimages\u002Fblog\u002Fen\u002Frunning-is-not-done\u002Fcache-poster.jpg","Two paths for one conversation: a stable prefix reuses the cache, a changed prefix refills it",[203,1070,1071],{},"Figure 2: one conversation. The prefix holds and the cache is reused; the prefix changes and the whole context is prefilled again.",[19,1073,1075],{"id":1074},"level-four-is-where-the-work-is-judged","Level four is where the work is judged",[15,1077,1078],{},"Deployment, protocol and sessions are now good enough to run bounded tasks. Can an open-weight model produce code that passes?",[15,1080,1081],{},"Five TypeScript task categories (quote, prorate, customer, split, parts), three conditions, three runs each — 45 sessions. The parent is fixed: Claude Code with Claude Opus 5.5. The worker is a quantized Qwen3-Coder 30B-A3B. Every session is judged by task-specific hidden behavioural checks, not by the agent's own claim of success.",[15,1083,1084],{},"The conditions: direct; optimised delegation, where the parent specifies, reviews and applies but does not implement; and batch-labelled delegation that may split work into sequential packages — though only a minority actually split.",[15,1086,1087],{},"Results: direct 15\u002F15; optimised delegation 12\u002F15; batch-labelled 14\u002F15.",[15,1089,1090],{},"These do not rank. Each cell has three runs, the Wilson intervals overlap, and the author says the data cannot establish general success-rate differences. It does show workers produced usable code: of 30 delegated trials, 26 verified and all 26 retained worker-produced implementations. Not every time.",[15,1092,1093],{},"And the 14\u002F15 is not evidence that batching helps. Only 5 of the 15 batch-labelled trials used a batch call; the rest used single delegation. The paper calls it an adaptive policy, not a batch-versus-single comparison.",[15,1095,1096],{},"The second suite is stricter: 15 tasks, three runs each, 45 sessions per arm, artifacts clearing a fixed interface contract before parent review.",[15,1098,1099],{},"Direct 45\u002F45. Nemotron Cascade 2 worker: 12\u002F45, of which 11 were genuine worker substitutions and one verified only after the parent rewrote the source. Qwen3-Coder worker: 0\u002F45.",[108,1101,1102,1115],{},[111,1103,1104],{},[114,1105,1106,1109,1112],{},[117,1107,1108],{},"Study",[117,1110,1111],{},"Direct",[117,1113,1114],{},"Delegated",[127,1116,1117,1127,1137],{},[114,1118,1119,1122,1124],{},[132,1120,1121],{},"45-trial suite",[132,1123,397],{},[132,1125,1126],{},"Qwen3-Coder 12\u002F15 (optimised), 14\u002F15 (batch-labelled)",[114,1128,1129,1132,1134],{},[132,1130,1131],{},"Second suite (15 tasks x 3)",[132,1133,408],{},[132,1135,1136],{},"Nemotron 12\u002F45 (11 worker substitutions); Qwen 0\u002F45",[114,1138,1139,1142,1144],{},[132,1140,1141],{},"Same, counting interruptions as failures",[132,1143,419],{},[132,1145,422],{},[15,1147,1148],{},"That second suite also has a disclosure: 16 sessions were stopped by a subscription limit before any model call and were replaced. Eight were in the direct arm and eight in the Nemotron arm. Counting the original 16 as failures gives 37\u002F45 and 8\u002F45. Those are scheduling interruptions, not inference failures — but if you cite the headline numbers, cite both.",[15,1150,1151],{},"The 45\u002F45 deserves a caveat too. The tasks were built for this study, five categories with three runs each; there is no difficulty stratification.",[15,1153,1154],{},"One more honest detail: the task definitions and hidden tests were produced with AI assistance. The combined test file was created about 39 seconds before the first evaluated session and first committed after the last of the 45. What he can claim is narrower: task instructions and visible checks were identical across the 45 saved runs, and recorded score labels agree with the scorer as later saved. He cannot claim an independently snapshotted benchmark.",[15,1156,1157],{},"The first study's harness had limits: the parent's separate constraints field was not forwarded to the worker; the gateway dropped a duplicate system message and truncated messages over 1,800 characters; two of 41 reconstructed worker prompts lost about 220 characters, both in failed trials.",[15,1159,1160],{},"So \"the worker ignored instructions\" needs a discount. Some failures were capability; some were a worker that never received the full instruction.",[15,1162,1163],{},"What about 0\u002F45? It does not mean the model can do nothing. That arm had no direct control, parent acceptance and application were not replayed, and the parent's prompt-cache state was not reset between studies. The author scopes it to these tasks, this contract, this worker configuration.",[15,1165,1166],{},"Verification is level four. It is not a by-product of level three.",[19,1168,1170],{"id":1169},"the-bill-runs-the-other-way","The bill runs the other way",[15,1172,1173],{},"Frontier output tokens are expensive; a local worker is cheap. Delegate and save money — except delegation is not free. The parent writes the spec, reads what came back, decides whether to accept it, and cleans up failures. All of that is frontier tokens.",[15,1175,1176],{},"The cost model has five terms: parent context, specification, review, recovery, worker. The study measured the frontier terms. Electricity, hardware and engineering time were not measured — and unmeasured is not zero.",[15,1178,1179],{},"First suite, frontier cost per verified success: $0.136 direct; $0.214 optimised delegation; $0.199 batch-labelled. About 1.6x.",[15,1181,1182],{},"Second suite: $0.153 direct; $1.313 for Nemotron delegation. 8.6x.",[15,1184,1185],{},"Qwen's per-success cost is undefined, not zero — it had no successes.",[15,1187,1188],{},"One small case is worth keeping: a fee task where the worker's implementation passed and was applied, yet the frontier cost was $0.1856 versus $0.1293 direct, and wall time 126.4 seconds versus 27.4.",[15,1190,1191],{},"A successful substitution is not a saving.",[15,1193,1194,1195,1198],{},"This does not generalise to \"delegation never pays\". The author cites ",[51,1196,532],{"href":530,"rel":1197},[55],": a small model scouts the repository, frontier models fix, and it solved 159 of 266 tasks against 158 for the strongest single model — one task apart — at roughly one fifth the cost. The direction is reversed: scout small, fix big. He limits his finding to his own contract, frontier-specifies \u002F local-implements.",[15,1200,1201],{},"The cost definition is narrow too: API list prices, no hardware, electricity or engineering time. \"8.6x\" is the frontier side of the bill, not your total.",[15,1203,1204],{},"Tokens are cheap. Coordination is not.",[19,1206,1208],{"id":1207},"the-acceptance-script-will-reject-good-code","The acceptance script will reject good code",[15,1210,1211],{},"The most practical part of the report is not about models. It is about acceptance.",[15,1213,1214,1215,1217,1218,1220],{},"Worker artifacts had to clear a static contract: imports, exports, and exported function parameter lists had to match. The stubs used underscored parameters for unused arguments (",[229,1216,554],{},"); workers naturally dropped the underscore (",[229,1219,558],{},").",[15,1222,1223],{},"For ordinary positional callers this changes nothing. But the contract compared parameter-name strings, so the artifact was rejected before parent review, before the hidden checks ran.",[15,1225,1226],{},"The post-hoc diagnostic: 36 valid-JSON artifacts rejected solely for dropping the underscore were applied unchanged to fresh task copies, with that rule bypassed, and re-run against the original sealed checks.",[15,1228,1229],{},"19 passed. 17 still failed.",[15,1231,1232],{},"So the rule was not simply wrong. It screened out both good and bad output. Rejecting bad output is its job; rejecting good output is its side effect — and if that gate is your only gate, the side effect is the final result.",[108,1234,1235,1251],{},[111,1236,1237],{},[114,1238,1239,1242,1245,1248],{},[117,1240,1241],{},"Worker",[117,1243,1244],{},"Rejected for underscores",[117,1246,1247],{},"Passed sealed checks unchanged",[117,1249,1250],{},"Still failed",[127,1252,1253,1263,1273],{},[114,1254,1255,1257,1259,1261],{},[132,1256,596],{},[132,1258,599],{},[132,1260,602],{},[132,1262,605],{},[114,1264,1265,1267,1269,1271],{},[132,1266,610],{},[132,1268,613],{},[132,1270,616],{},[132,1272,619],{},[114,1274,1275,1278,1280,1282],{},[132,1276,1277],{},"Total",[132,1279,627],{},[132,1281,630],{},[132,1283,633],{},[15,1285,1286],{},"He did not count the 19 as successes. They never went through parent review and application; the frozen scores stay Qwen 0\u002F45 and Nemotron 12\u002F45. He also did not conclude the contract should go. He proposed the next experiment: give the worker the specific validation error and allow one bounded repair.",[15,1288,1289],{},"That experiment has not been run.",[15,1291,1292],{},"On this contract, acceptance design failed more easily than model capability did.",[19,1294,1296],{"id":1295},"what-this-cannot-prove","What this cannot prove",[323,1298,1299,1302,1305,1308],{},[72,1300,1301],{},"A single-author retrospective, not a controlled experiment. The workload changed as the engineering changed.",[72,1303,1304],{},"Small sample: five task categories, three runs per cell, one fixed parent, one worker configuration.",[72,1306,1307],{},"Cost covers frontier tokens at list prices only.",[72,1309,1310],{},"The raw records are private. You cannot recompute the numbers.",[15,1312,1313],{},"The studies are not pooled and do not rank models or runtimes.",[15,1315,1316],{},"So this gives you an acceptance framework, not a leaderboard. The most valuable part, in my reading, is not the numbers — it is that \"done\" becomes something you can check level by level.",[19,1318,1320],{"id":1319},"a-checklist-you-can-use-tomorrow","A checklist you can use tomorrow",[15,1322,1323],{},"The author's five rules, rewritten for a PoC document:",[69,1325,1326,1332,1338,1344,1350],{},[72,1327,1328,1331],{},[32,1329,1330],{},"Define success at the task boundary."," Accept on \"the output passed a check it does not own\", not on \"it responded\". Exercise the real client, not a harness you wrote.",[72,1333,1334,1337],{},[32,1335,1336],{},"Treat deployment as a recipe, not a model name."," Checkpoint revision, quantisation, runtime, template, context settings, gateway, client, resident memory. Change one, re-verify.",[72,1339,1340,1343],{},[32,1341,1342],{},"Make session continuity an application responsibility."," Stable prefix, no cosmetic re-serialisation, auxiliary requests must not evict the main conversation.",[72,1345,1346,1349],{},[32,1347,1348],{},"Price delegation per verified success."," Not per token, not per call.",[72,1351,1352,1355],{},[32,1353,1354],{},"Verify by behaviour where you can."," Static gates are cheap and worth keeping, but know what they reject.",[15,1357,1358],{},"If you are the buyer, three questions are enough:",[323,1360,1361,1364,1367],{},[72,1362,1363],{},"Which level does this demo stop at? \"It answers\" is level two.",[72,1365,1366],{},"How do you verify level four, and who wrote the checks, and when were they frozen?",[72,1368,1369],{},"Is the cost per call or per verified success?",[15,1371,1372,1373,1376],{},"Our own ",[51,1374,1375],{"href":729},"video transcription pipeline"," ran into the same class of problem: the model ran, the pipeline still stalled, and every failure looked like \"the model is a bit dumb\". We have not run measurements at this scale, so this essay borrows the paper's numbers.",[15,1378,1379,1380,1383],{},"The previous essay was about ",[51,1381,1382],{"href":742},"files that look right and cannot be used","; this one is about models that look right and cannot finish the job.",[15,1385,1386,1387,1390],{},"If you want help filling in the table, our ",[51,1388,1389],{"href":750},"AI readiness audit"," does exactly that: what can be handed to AI now, and what should wait.",[19,1392,1394],{"id":1393},"conclusion-which-level-counts-as-done","Conclusion: which level counts as done",[15,1396,1397],{},"The first three levels can be demonstrated in an afternoon. Load a model, send a few prompts, connect a client.",[15,1399,1400],{},"Level four takes weeks: design tasks, write hidden checks, run enough trials, classify failures, price the work. What you get is rarely \"this model is good\" — it is \"on these tasks, under this contract, it is good\".",[15,1402,1403],{},"The missing column on your acceptance sheet is where the price difference lives.",[15,1405,1406],{},"Tomorrow: write \"passes the tests\" into your PoC definition, and ask who wrote the tests and when they froze them.",{"title":770,"searchDepth":771,"depth":771,"links":1408},[1409,1410,1411,1412,1413,1414,1415,1416,1417,1418],{"id":815,"depth":771,"text":816},{"id":838,"depth":771,"text":839},{"id":963,"depth":771,"text":964},{"id":1006,"depth":771,"text":1007},{"id":1074,"depth":771,"text":1075},{"id":1169,"depth":771,"text":1170},{"id":1207,"depth":771,"text":1208},{"id":1295,"depth":771,"text":1296},{"id":1319,"depth":771,"text":1320},{"id":1393,"depth":771,"text":1394},"\u002Fimages\u002Fblog\u002Fen\u002Frunning-is-not-done.svg","Cover for an essay on the four levels between a served model and finished work","Same endpoint, different behaviour: tool calls, turn control, context, cache. A 4,165-turn solo report splits 'it runs' into four levels, finding local delegation costs 8.6x more per verified success.","en",{},"\u002Fimages\u002Fblog\u002Fen\u002Frunning-is-not-done.jpg","\u002Fblog\u002Fen\u002Frunning-is-not-done",{"title":805,"description":1421},"blog\u002Fen\u002Frunning-is-not-done",[797,798,799,800],"0DdevvjhzLCzG5RvWM1GlSfrg6_pwOm2oIXJq0yz_YU",{"id":1431,"company":1432,"extension":1443,"meta":1444,"services":1445,"social":1522,"stem":1523,"__hash__":1524},"site\u002Fsite.yml",{"name":1433,"nameEn":1434,"email":1435,"address":1436,"addressEn":1437,"lineUrl":1438,"lineId":1439,"duns":1440,"youtube":1441,"youtubeId":1442},"銡策數位有限公司","Jecision Digital Co., Ltd.","contact@jecision.com","231 新北市新店區北新路二段 262 號 3 樓之 3","3F.-3, No. 262, Sec. 2, Beixin Rd., Xindian Dist., New Taipei City 231, Taiwan","https:\u002F\u002Fline.me\u002FR\u002Fti\u002Fp\u002F@374qnumi","@374qnumi","963905705","https:\u002F\u002Fwww.youtube.com\u002Fshorts\u002F6UE0WTmyjsg","6UE0WTmyjsg","yml",{},[1446,1469,1495],{"key":1447,"zh":1448,"en":1458},"audit",{"title":751,"tagline":1449,"summary":1450,"points":1451,"deliverable":1455,"detail":1456,"duration":1457},"先看清楚，再花錢","60 分鐘線上訪談，盤點最耗時、最依賴人力的營運環節；我們會告訴你哪些工作現在就能交給 AI、哪些還不成熟。",[1452,1453,1454],"現況盤點：梳理日常營運中最耗時、最依賴人力的環節","痛點診斷：判斷哪些工作現在就能交給 AI，哪些建議先保留人力","可行性評估：依你的業務特性與預算，篩選適合的模型與工具","《企業 AI 導入建議報告書》（PDF）","含 1–2 個「開發成本最低、成效最明顯」的優先項目、技術棧建議、預期節省的人力時間與下一步","訪談 60 分鐘，報告書 3 個工作天內交付",{"title":1459,"tagline":1460,"summary":1461,"points":1462,"deliverable":1466,"detail":1467,"duration":1468},"AI Readiness Audit","Look before you spend","A 60-minute remote audit of the workflows that eat the most time and people-power — and an honest answer on what AI can take over today, and what it cannot.",[1463,1464,1465],"Audit the daily workflows that consume the most time and headcount","Triage which tasks AI can own today, and which should stay human","Shortlist the models and tools that fit your budget and constraints","AI Adoption Recommendation Report (PDF)","One or two highest-ROI entry points, a recommended tech stack, expected hours saved, and the concrete next step","60-minute call, report delivered within 3 business days",{"key":1470,"zh":1471,"en":1482},"prompt",{"title":1472,"tagline":1473,"summary":1474,"points":1475,"deliverable":1479,"detail":1480,"duration":1481},"提示詞工程","讓 AI 穩定產出，而不是碰運氣","針對你的實際任務設計提示詞與多步驟工作流，並做極限測試，確保換了資料、換了人操作也不會走鐘。",[1476,1477,1478],"客製提示詞：ChatGPT、Claude、Gemini、Midjourney 等主流工具的專屬模板","除錯與優化：修掉幻覺、格式跑掉、時好時壞的問題","工作流建置：把複雜任務拆成多步驟提示詞與可重複的 SOP","提示詞文件或設定檔","附使用說明與範例，並提供 1 對 1 教學，確保你與團隊能自己維護","依任務複雜度報價，通常 3–7 個工作天",{"title":1483,"tagline":1484,"summary":1485,"points":1486,"deliverable":1492,"detail":1493,"duration":1494},"Prompt Engineering","Make AI produce reliably, not by luck","Prompt sets and multi-step workflows designed for your actual tasks, stress-tested so they keep working when the data or the operator changes.",[1487,1488,1490],"Custom prompts for ChatGPT, Claude, Gemini, Midjourney and friends",{"Debugging and tuning":1489},"kill hallucinations, broken formats and flaky output",{"Workflow design":1491},"break complex work into repeatable multi-step SOPs","Prompt library or exportable config","Ships with usage notes and examples, plus a 1:1 walkthrough so your team can maintain it","Quoted by complexity, typically 3–7 business days",{"key":1496,"zh":1497,"en":1508},"poc",{"title":1498,"tagline":1499,"summary":1500,"points":1501,"deliverable":1505,"detail":1506,"duration":1507},"極速 PoC","五天，看到能跑的東西","不談架構藍圖，直接做一個能動的雛形。五個工作天內交付線上可操作的智能體，用你真實的資料驗證。",[1502,1503,1504],"Day 1：15 分鐘對焦，鎖定單一痛點與資料盤點","Day 2–4：模型串接、邏輯設計與雛形開發","Day 5：Demo 展示與 AI 擴展建議書","可線上操作的智能體雛形","交付範圍為概念驗證與展示，不含後續維運或大規模伺服器部署；要進階成完整系統會另行報價","5 個工作天",{"title":1509,"tagline":1510,"summary":1511,"points":1512,"deliverable":1519,"detail":1520,"duration":1521},"Rapid PoC","Five days to something that runs","No architecture decks — a working prototype built in five business days, validated against your real data.",[1513,1515,1517],{"Day 1":1514},"15-minute scoping call, one pain point, data inventory",{"Day 2–4":1516},"model integration, logic design, prototype build",{"Day 5":1518},"live demo plus an AI expansion recommendation","A runnable online agent prototype","Proof of concept and demo scope only; ongoing operations and large-scale deployment are quoted separately","5 business days",[],"site","XTwARfVAOcrXNiKehsM6V6cdkd2HXy4d5yZVrSWrI3Q",[1526,1987],{"id":1527,"title":1528,"author":7,"body":1529,"cover":1973,"coverAlt":1974,"date":1975,"description":1976,"draft":787,"extension":788,"featured":787,"lang":1422,"meta":1977,"navigation":200,"ogImage":1978,"path":1979,"seo":1980,"slug":1981,"stem":1982,"tags":1983,"translationKey":1981,"updated":801,"__hash__":1986},"blog\u002Fblog\u002Fen\u002Fbroken-on-arrival-quantized-models.md","Your quantized model might be broken",{"type":9,"value":1530,"toc":1962},[1531,1540,1542,1549,1552,1555,1561,1565,1568,1575,1582,1593,1597,1600,1607,1617,1623,1627,1630,1688,1691,1697,1701,1704,1711,1717,1720,1724,1727,1832,1838,1849,1853,1866,1872,1886,1890,1908,1912,1917,1923,1930,1933],[12,1532,1533],{},[15,1534,1535,1536,1539],{},"Numbers come from arXiv 2609.05881v3 and the author's ",[229,1537,1538],{},"quantcheck"," audit data. Whether the bad files are still being served is our own check; the method is at the end.",[19,1541,816],{"id":815},[15,1543,1544,1545,1548],{},"You are on a laptop. You run ",[229,1546,1547],{},"ollama pull qwen2.5-coder:3b",", choosing a 2-bit tag because memory is tight.",[15,1550,1551],{},"The model starts. It answers, and it is fast. Then you ask for a sorting function and the code does not compile. You try another task; same thing.",[15,1553,1554],{},"Your first instinct is to blame yourself: the prompt? the temperature? maybe this model is just weak?",[15,1556,1557,1558],{},"A paper turned that doubt into a census: ",[32,1559,1560],{},"of 305 code-capable tags in the official Ollama library, seven solved at most six of 164 HumanEval+ tasks. Six solved none.",[19,1562,1564],{"id":1563},"the-counter-intuitive-part-the-files-are-fine","The counter-intuitive part: the files are fine",[15,1566,1567],{},"When a model misbehaves, you suspect a corrupt download.",[15,1569,1570,1571,1574],{},"The author rebuilt five of the seven from the original weights with llama.cpp. Comparing tensor by tensor, ",[32,1572,1573],{},"all five matched exactly"," — 434 tensors for the Qwen2.5-Coder-3B tags.",[15,1576,1577,1578,1581],{},"So these are not corrupt files. They are ",[32,1579,1580],{},"correctly reproduced broken artifacts",".",[15,1583,1584,1585,1588,1589,1592],{},"Checksums pass. File sizes match. ",[229,1586,1587],{},"ollama show"," looks normal. Because the file really is exactly what it claims to be. What is broken is the ",[32,1590,1591],{},"quantization recipe",": same weights, same toolchain, same tensor mix, but without calibration the model collapsed into a shape that cannot write code.",[19,1594,1596],{"id":1595},"why-this-happens","Why this happens",[15,1598,1599],{},"Quantization squeezes weights from 16 bits down to 2–4 bits. Every step has to decide where the rounding error lands.",[15,1601,1602,1603,1606],{},"The default approach spreads it evenly. The better approach uses ",[32,1604,1605],{},"calibration data"," — llama.cpp's importance matrix (imatrix) — to push error away from the weights that matter, based on activation statistics.",[15,1608,1609,1610,1613,1614,1581],{},"Without imatrix, small models at low bit are the ones that fall apart. All seven suspects sit in ",[32,1611,1612],{},"q2_K through q3_K_L, in models of at most 4B parameters",". That subgroup contains 48 tags, so the failure rate inside it is ",[32,1615,1616],{},"14.6%",[15,1618,1619,1620,1581],{},"But the recipe alone does not predict failure: the same uncalibrated recipe left Qwen2.5-Coder-1.5B functional (13–15 of 15). The problem is not \"low bit always breaks\". It is that ",[32,1621,1622],{},"when it breaks, you cannot tell",[19,1624,1626],{"id":1625},"calibration-rescues-most-of-them","Calibration rescues most of them",[15,1628,1629],{},"Holding weights, toolchain and tensor mix fixed, adding only an imatrix:",[108,1631,1632,1645],{},[111,1633,1634],{},[114,1635,1636,1639,1642],{},[117,1637,1638],{},"Tag",[117,1640,1641],{},"Before",[117,1643,1644],{},"After imatrix (of 164)",[127,1646,1647,1658,1668,1678],{},[114,1648,1649,1652,1655],{},[132,1650,1651],{},"Qwen2.5-Coder-3B q2_K",[132,1653,1654],{},"0",[132,1656,1657],{},"106–113",[114,1659,1660,1663,1665],{},[132,1661,1662],{},"Qwen2.5-Coder-3B q3_K_M",[132,1664,1654],{},[132,1666,1667],{},"76–89",[114,1669,1670,1673,1675],{},[132,1671,1672],{},"Qwen2.5-Coder-3B q3_K_L",[132,1674,1654],{},[132,1676,1677],{},"80–86",[114,1679,1680,1683,1685],{},[132,1681,1682],{},"Qwen2.5-Coder-3B q3_K_S",[132,1684,1654],{},[132,1686,1687],{},"13–30 (still weak)",[15,1689,1690],{},"(The paper's abstract says \"0 → 76–113\"; the per-level numbers above come from the v3.4 audit data, which we treat as authoritative.)",[15,1692,1693,1694],{},"Same model, same bit width, one calibration table: ",[32,1695,1696],{},"from zero tasks to three digits.",[19,1698,1700],{"id":1699},"another-failure-mode-a-missing-chat-template","Another failure mode: a missing chat template",[15,1702,1703],{},"Beyond the seven, the author also tested 22 popular Hugging Face community files. Two of them (TheBloke's deepseek-coder-6.7B Q3_K_M and Q4_K_M) scored 5 and 6 of 15 in the census.",[15,1705,1706,1707,1710],{},"The cause was not quantization but a ",[32,1708,1709],{},"missing chat template",". With no template, Ollama falls back to a default ChatML format that does not match the model's instruction format: 28 of 30 answers ran on into invented chat turns.",[15,1712,1713,1714,1581],{},"With the right template: ",[32,1715,1716],{},"98 and 101 of 164",[15,1718,1719],{},"So \"it runs\" and \"it answers properly\" are separated by a template.",[19,1721,1723],{"id":1722},"we-checked-the-bad-tags-are-still-being-served","We checked: the bad tags are still being served",[15,1725,1726],{},"A September incident is only interesting if it is still downloadable. We compared the blob digests in the audit data against the current Ollama registry manifests:",[108,1728,1729,1744],{},[111,1730,1731],{},[114,1732,1733,1735,1738,1741],{},[117,1734,1638],{},[117,1736,1737],{},"Audited digest",[117,1739,1740],{},"Today",[117,1742,1743],{},"Status",[127,1745,1746,1760,1772,1784,1796,1808,1820],{},[114,1747,1748,1751,1754,1757],{},[132,1749,1750],{},"qwen2.5-coder:3b-instruct-q2_K",[132,1752,1753],{},"14c4247b8ce4…",[132,1755,1756],{},"same",[132,1758,1759],{},"still served",[114,1761,1762,1765,1768,1770],{},[132,1763,1764],{},"qwen2.5-coder:3b-instruct-q3_K_S",[132,1766,1767],{},"81d167af7fa3…",[132,1769,1756],{},[132,1771,1759],{},[114,1773,1774,1777,1780,1782],{},[132,1775,1776],{},"qwen2.5-coder:3b-instruct-q3_K_M",[132,1778,1779],{},"9bb5d28b842a…",[132,1781,1756],{},[132,1783,1759],{},[114,1785,1786,1789,1792,1794],{},[132,1787,1788],{},"qwen2.5-coder:3b-instruct-q3_K_L",[132,1790,1791],{},"7a88dc11faae…",[132,1793,1756],{},[132,1795,1759],{},[114,1797,1798,1801,1804,1806],{},[132,1799,1800],{},"phi3.5:3.8b-mini-instruct-q2_K",[132,1802,1803],{},"54d47caa8bf3…",[132,1805,1756],{},[132,1807,1759],{},[114,1809,1810,1813,1816,1818],{},[132,1811,1812],{},"llama3.2:1b-instruct-q2_K",[132,1814,1815],{},"d1ede02a2111…",[132,1817,1756],{},[132,1819,1759],{},[114,1821,1822,1825,1828,1830],{},[132,1823,1824],{},"llama3.2:1b-instruct-q3_K_S",[132,1826,1827],{},"81881dcc12ec…",[132,1829,1756],{},[132,1831,1759],{},[15,1833,1834,1837],{},[32,1835,1836],{},"Seven out of seven are still there, with identical digests."," This is not history; it is something you can pull today.",[15,1839,1840,1841,1844,1845,1848],{},"(Method: fetch ",[229,1842,1843],{},"https:\u002F\u002Fregistry.ollama.ai\u002Fv2\u002Flibrary\u002F\u003Cmodel>\u002Fmanifests\u002F\u003Ctag>"," and compare the ",[229,1846,1847],{},"application\u002Fvnd.ollama.image.model"," layer digest. Script at the end.)",[19,1850,1852],{"id":1851},"three-things-you-can-do-now","Three things you can do now",[15,1854,1855,1858,1859,1861,1862,1865],{},[32,1856,1857],{},"Screen."," The author released ",[229,1860,1538],{},", a 15-task screen per tag. It costs about ",[32,1863,1864],{},"30 seconds of generation"," (on a 4090, excluding download). None of 66 sampled tags that passed the screen turned out to be non-functional, so it works as a pre-filter.",[15,1867,1868,1871],{},[32,1869,1870],{},"If you publish quantizations:"," add an imatrix, and run a functional test before publishing. Registries will not do it for you.",[15,1873,1874,1877,1878,1881,1882,1885],{},[32,1875,1876],{},"Check the chat template."," ",[229,1879,1880],{},"ollama show \u003Cmodel> --template","; if it prints ",[229,1883,1884],{},"{{ .Prompt }}",", there is no template and the model may be answering in the wrong format.",[19,1887,1889],{"id":1888},"do-not-over-read-this","Do not over-read this",[323,1891,1892,1895,1898,1905],{},[72,1893,1894],{},"2.3% is the share of official, code-capable, ≤8 GB tags — not of all quantized models",[72,1896,1897],{},"Only coding ability was tested (HumanEval+); perplexity is a pre-check, not proof of function",[72,1899,1900,1901,1904],{},"Seven is a ",[32,1902,1903],{},"lower bound",": all 12 deepseek-coder-1.3B tags, including fp16, score 1–6 of 15, so a group baseline cannot judge them and the author does not count them",[72,1906,1907],{},"The author is an independent researcher; data and tools are public, but this is one registry and one runtime version",[19,1909,1911],{"id":1910},"conclusion","Conclusion",[15,1913,1914],{},[32,1915,1916],{},"Exact reproducibility is not function.",[15,1918,1919,1920,1922],{},"Identical tensors, a passing checksum, a normal ",[229,1921,1587],{}," — all of that proves the file is the file. None of it proves the model can do the job.",[15,1924,1925,1926,1929],{},"So the question is not whether quantized models are usable. It is ",[32,1927,1928],{},"whether anyone tested whether this one is",". Right now the registries have not, which leaves it to you.",[1931,1932],"hr",{},[15,1934,1935],{},[1936,1937,1938,1939,1944,1945,1944,1949,1954,1955,1958,1959,1581],"em",{},"Sources: ",[51,1940,1943],{"href":1941,"rel":1942},"https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.05881",[55],"arXiv 2609.05881v3",", ",[51,1946,1538],{"href":1947,"rel":1948},"https:\u002F\u002Fgithub.com\u002Faditi-p31\u002Fquantcheck",[55],[51,1950,1953],{"href":1951,"rel":1952},"https:\u002F\u002Fgithub.com\u002Follama\u002Follama\u002Fissues\u002F18252",[55],"Ollama issue #18252",". Our digest check: ",[229,1956,1957],{},"scripts\u002Fcheck-ollama-tags.ts","; run ",[229,1960,1961],{},"npm run check:ollama -- --list",{"title":770,"searchDepth":771,"depth":771,"links":1963},[1964,1965,1966,1967,1968,1969,1970,1971,1972],{"id":815,"depth":771,"text":816},{"id":1563,"depth":771,"text":1564},{"id":1595,"depth":771,"text":1596},{"id":1625,"depth":771,"text":1626},{"id":1699,"depth":771,"text":1700},{"id":1722,"depth":771,"text":1723},{"id":1851,"depth":771,"text":1852},{"id":1888,"depth":771,"text":1889},{"id":1910,"depth":771,"text":1911},"\u002Fimages\u002Fblog\u002Fen\u002Fbroken-on-arrival-quantized-models.svg","A quantized model file that passes its checksum but fails every coding task","2026-11-02","Someone tested every code-capable tag in Ollama's library. Seven low-bit quantizations failed nearly all tasks, yet the files were exact rebuilds. Checksums pass; the recipe is broken.",{},"\u002Fimages\u002Fblog\u002Fen\u002Fbroken-on-arrival-quantized-models.jpg","\u002Fblog\u002Fen\u002Fbroken-on-arrival-quantized-models",{"title":1528,"description":1976},"broken-on-arrival-quantized-models","blog\u002Fen\u002Fbroken-on-arrival-quantized-models",[1984,797,1985],"quantization","gguf","tOVVVr1HmeG95b9UhxSxh_uy4dZQvyUwEripBKmKjww",{"id":1988,"title":1989,"author":7,"body":1990,"cover":2667,"coverAlt":2668,"date":2669,"description":2670,"draft":787,"extension":788,"featured":787,"lang":1422,"meta":2671,"navigation":200,"ogImage":2672,"path":2673,"seo":2674,"slug":2675,"stem":2676,"tags":2677,"translationKey":2675,"updated":801,"__hash__":2681},"blog\u002Fblog\u002Fen\u002Fshared-trunk-four-modalities.md","Shared dimensions: a new direction for going multimodal",{"type":9,"value":1991,"toc":2648},[1992,1997,2001,2004,2007,2010,2023,2029,2033,2036,2042,2048,2054,2060,2064,2067,2076,2079,2082,2087,2090,2101,2104,2107,2112,2116,2119,2124,2127,2134,2140,2144,2147,2153,2156,2160,2163,2169,2172,2175,2186,2189,2192,2198,2202,2205,2209,2212,2215,2221,2228,2232,2235,2241,2244,2248,2251,2254,2257,2263,2267,2270,2330,2333,2499,2502,2505,2511,2517,2523,2529,2532,2536,2539,2545,2551,2557,2567,2578,2584,2588,2591,2597,2600,2613,2619,2625,2627],[12,1993,1994],{},[15,1995,1996],{},"No benchmarks of our own here. The numbers come from the paper and the model card, linked at the end; the judgments are ours and labelled.",[19,1998,2000],{"id":1999},"start-with-a-scene","Start with a scene",[15,2002,2003],{},"Suppose you want a system that can search everything inside a company.",[15,2005,2006],{},"Contracts are PDFs. Invoices are scans. Meetings are audio. Product photos are images.",[15,2008,2009],{},"Multimodal used to offer two paths. One was to run four models — text embeddings, image embeddings, speech-to-text, audio classification — with four vector spaces that do not talk to each other, plus an alignment layer on top. The other was to make one model big enough to swallow every modality.",[15,2011,2012,2013,2018,2019,2022],{},"In October 2026, Google's ",[51,2014,2017],{"href":2015,"rel":2016},"https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.37364",[55],"EmbeddingGemma 2"," demonstrated a third path: ",[32,2020,2021],{},"share the semantic backbone, and isolate capacity for each modality in branches",". Text-only runs at 271M; turn on vision when you need images, turn on audio when you need sound.",[15,2024,2025,2026],{},"What is actually valuable here is not \"one model handles four data types\". It is that ",[32,2027,2028],{},"going multimodal does not require making the whole model fatter.",[19,2030,2032],{"id":2031},"the-old-bill-for-scaling-up","The old bill for scaling up",[15,2034,2035],{},"To see why the third path matters, look at what the old paths cost.",[15,2037,2038,2041],{},[32,2039,2040],{},"The first is multiplication."," A dense model's bill is parameters times tokens. Want one more capability, and you scale everything up together.",[15,2043,2044,2047],{},[32,2045,2046],{},"The second is gradient conflict."," The same weights must satisfy \"this pixel is a cat\" and \"this word is a subject\". The gradients pull in different directions — negative transfer, in multitask learning terms.",[15,2049,2050,2053],{},[32,2051,2052],{},"The third is input geometry."," Text is discrete tokens, images are pixel grids, audio is waveforms. They do not even agree on what one unit is.",[15,2055,2056,2057],{},"These three costs are not an argument that multimodal is impossible. They explain why the old path stops paying off past a certain size. ",[32,2058,2059],{},"The value of the new direction is a different way to pay.",[19,2061,2063],{"id":2062},"the-new-direction-shared-dimensions-isolated-capacity","The new direction: shared dimensions, isolated capacity",[15,2065,2066],{},"EmbeddingGemma 2 splits at the entrance and merges in semantics.",[2068,2069,2074],"pre",{"className":2070,"code":2072,"language":2073},[2071],"language-text","image\u002Fvideo\u002Faudio ──[per-modality encoders]──> soft tokens ──[one 24-layer transformer]──> vector\n                        ↑ split                                    ↑ shared\n","text",[229,2075,2072],{"__ignoreMap":770},[15,2077,2078],{},"Each modality encoder handles low-level features (pixels, waveforms) and turns them into \"soft tokens\" — continuous vectors that sit in the sequence like text tokens but need not correspond to any word. High-level meaning goes through one shared transformer.",[15,2080,2081],{},"The parameters split like this: a 271M trunk (text also runs here), a 167M vision encoder, and a 305M audio encoder.",[15,2083,2084],{},[32,2085,2086],{},"Note that 305M: the audio encoder is bigger than the entire text trunk.",[15,2088,2089],{},"That is the point of the new direction. Capacity lives in branches, so it can be switched off:",[190,2091,194,2093,194,2098],{"className":2092},[193],[196,2094],{"src":2095,"poster":2096,"autoPlay":200,"loop":200,"muted":200,"playsInline":200,"ariaLabel":2097},"\u002Fimages\u002Fblog\u002Fshared-trunk-four-modalities\u002Fsplit.mp4","\u002Fimages\u002Fblog\u002Fshared-trunk-four-modalities\u002Fsplit-poster.jpg","Parameter split and actual loaded memory",[203,2099,2100],{},"Parameters on the left, actual load on the right. Branches open on demand — that is how scale stops meaning bloat.",[15,2102,2103],{},"On a Pixel 11 Pro at Q4_C, peak memory (including runtime compilation overhead): 191 MB for text, 447 MB for text plus images, 567 MB for everything.",[15,2105,2106],{},"And text quality does not pay for it. Its 271M text path scores 61.36 on multilingual MTEB; the previous generation at 308M scored 61.15. Fewer parameters, three extra modalities, no text regression.",[15,2108,2109],{},[32,2110,2111],{},"That is the most promising part: scale can be opened selectively instead of expanded universally.",[19,2113,2115],{"id":2114},"two-ways-to-split-two-bills","Two ways to split, two bills",[15,2117,2118],{},"\"Share the trunk, isolate capacity\" can be done at different places. Two current answers:",[2120,2121,2123],"h3",{"id":2122},"moe-split-in-the-middle-of-the-model","MoE: split in the middle of the model",[15,2125,2126],{},"Many experts sit inside a layer; each token visits a few, chosen by a router.",[15,2128,2129,2130,2133],{},"What it buys: ",[32,2131,2132],{},"capacity decouples from compute per token."," Parameters can be plentiful while each token touches a small slice.",[15,2135,2136,2137],{},"The cost is training difficulty: router collapse, uneven expert load, sparse gradients. ",[32,2138,2139],{},"MoE trades training difficulty for inference cost.",[2120,2141,2143],{"id":2142},"embeddinggemma-2-split-at-the-entrance","EmbeddingGemma 2: split at the entrance",[15,2145,2146],{},"Heterogeneous modalities go to designed encoders; homogeneous meaning goes to the shared trunk.",[15,2148,2129,2149,2152],{},[32,2150,2151],{},"capacity decouples from resident memory."," You do not keep the whole model resident for the occasional video search.",[15,2154,2155],{},"These are not the same problem, and they are not mutually exclusive. One saves compute; the other saves memory.",[19,2157,2159],{"id":2158},"early-fusion-late-fusion-where-alignment-starts","Early fusion, late fusion: where alignment starts",[15,2161,2162],{},"If you know CLIP, this looks like \"multimodal, again\".",[15,2164,2165,2166,1581],{},"The difference is ",[32,2167,2168],{},"where the merge happens",[15,2170,2171],{},"Late fusion (CLIP, SigLIP): two full encoders run independently, then a contrastive loss aligns them. The towers never exchange information.",[15,2173,2174],{},"Early fusion (EmbeddingGemma 2): encoders produce soft tokens that enter one transformer, where all modalities mix.",[190,2176,194,2178,194,2183],{"className":2177},[193],[196,2179],{"src":2180,"poster":2181,"autoPlay":200,"loop":200,"muted":200,"playsInline":200,"ariaLabel":2182},"\u002Fimages\u002Fblog\u002Fshared-trunk-four-modalities\u002Fflow.mp4","\u002Fimages\u002Fblog\u002Fshared-trunk-four-modalities\u002Fflow-poster.jpg","Late fusion versus early fusion data flow",[203,2184,2185],{},"The difference is not whether alignment exists — it is which layer starts aligning. That decides whether cross-modal interaction can be learned.",[15,2187,2188],{},"Late fusion lets each tower be strong on its own; the cost is that it cannot learn cross-modal interaction — text inside images, the original audio of a video, the relationship between a chart and its heading.",[15,2190,2191],{},"Early fusion costs something too: every modality must fit through one narrow channel. The model is 512-dimensional and 24 layers deep. (Our reading: deliberately narrow.)",[15,2193,2194,2195],{},"Its bet: ",[32,2196,2197],{},"only a narrow semantic space forces a genuinely shared representation.",[19,2199,2201],{"id":2200},"three-pieces-of-evidence-for-this-direction","Three pieces of evidence for this direction",[15,2203,2204],{},"The paper ran ablations; three of them speak directly to the \"shared trunk\" idea.",[2120,2206,2208],{"id":2207},"removing-a-text-only-mechanism-improves-everything","Removing a text-only mechanism improves everything",[15,2210,2211],{},"Gemma's per-layer embeddings used an extra lookup table to inject token information into intermediate layers. EmbeddingGemma 2 drops the lookup table and keeps only the projection.",[15,2213,2214],{},"Overall goes from 55.07 to 55.46; image 54.57 → 54.67, video 47.13 → 47.82, visual documents 61.76 → 62.37. These ablations are intermediate training recipes, not the final model (which reaches 59.01 MMEB-v2 overall).",[15,2216,2217,2220],{},[32,2218,2219],{},"Removing it made everything better",", text included. The paper's explanation: the lookup table carries \"no direct signal\" for non-text modalities, and costs memory.",[15,2222,2223,2224,2227],{},"Our reading: ",[32,2225,2226],{},"a shared space is achieved by removing things, not adding them."," Any mechanism that lets one modality bypass the shared space drags the whole down.",[2120,2229,2231],{"id":2230},"the-vision-encoder-cannot-be-frozen","The vision encoder cannot be frozen",[15,2233,2234],{},"Trainable: 52.27 overall, 50.10 image, 43.26 video. Frozen: 48.18, 44.35, 39.24 — a 4.1 point gap.",[15,2236,2237,2238],{},"Many multimodal models freeze modality encoders to save cost and preserve prior ability. This result says otherwise: ",[32,2239,2240],{},"encoders have to be trained to adapt to the shared space.",[15,2242,2243],{},"For this direction that is both good news and a reminder — the shared trunk is not free; it asks every branch to converge somewhere, and that convergence can be trained.",[2120,2245,2247],{"id":2246},"the-audio-encoder-is-frozen-and-audio-merely-ties","The audio encoder is frozen, and audio merely ties",[15,2249,2250],{},"The paper is explicit: the audio encoder is identical to Gemma 4 E2B\u002FE4B and frozen during training.",[15,2252,2253],{},"Audio is also where it is closest to its rival: MAEB Mean(Task) 49.87 versus Jina Omni Nano 50.14, a 0.27 gap. Mean(Type) is 48.65 versus 55.31 — not a tie. By contrast, image is 57.28 versus 55.43 and video 50.67 versus 35.87, both clearly ahead.",[15,2255,2256],{},"The authors leave one line: further training of the audio encoder \"may also yield similar gains\".",[15,2258,2259,2262],{},[32,2260,2261],{},"This is our speculation, not the paper's conclusion",": unfreezing audio could open that gap. They did not do it, so we can only say it is possible — and that is exactly the part of this path that is not finished.",[19,2264,2266],{"id":2265},"how-to-use-it-selection-is-not-a-ranking","How to use it: selection is not a ranking",[15,2268,2269],{},"What matters is what your use case needs, not who is first.",[108,2271,2272,2285],{},[111,2273,2274],{},[114,2275,2276,2279,2282],{},[117,2277,2278],{},"Your use case",[117,2280,2281],{},"A reasonable choice",[117,2283,2284],{},"Why",[127,2286,2287,2298,2309,2319],{},[114,2288,2289,2292,2295],{},[132,2290,2291],{},"Text-only retrieval \u002F RAG",[132,2293,2294],{},"Qwen3 or Jina text models",[132,2296,2297],{},"Higher scores at the same size, smaller models",[114,2299,2300,2303,2306],{},[132,2301,2302],{},"Text plus image search",[132,2304,2305],{},"EmbeddingGemma 2 or a SigLIP-style specialist",[132,2307,2308],{},"Depends on whether you want one checkpoint for everything",[114,2310,2311,2314,2316],{},[132,2312,2313],{},"Video, audio, mixed modality",[132,2315,2017],{},[132,2317,2318],{},"15–20 points ahead on video; native audio beats ASR by about 12 points",[114,2320,2321,2324,2327],{},[132,2322,2323],{},"Phone \u002F offline, tight memory",[132,2325,2326],{},"EmbeddingGemma 2 text-only",[132,2328,2329],{},"191 MB, and only the encoders you need",[15,2331,2332],{},"Some numbers worth reading together:",[108,2334,2335,2363],{},[111,2336,2337],{},[114,2338,2339,2342,2345,2348,2351,2354,2357,2360],{},[117,2340,2341],{},"Model",[117,2343,2344],{},"Params",[117,2346,2347],{},"Multilingual text",[117,2349,2350],{},"English text",[117,2352,2353],{},"Code",[117,2355,2356],{},"Multimodal overall",[117,2358,2359],{},"Image",[117,2361,2362],{},"Video",[127,2364,2365,2400,2424,2453,2476],{},[114,2366,2367,2371,2374,2377,2380,2385,2390,2395],{},[132,2368,2369],{},[32,2370,2017],{},[132,2372,2373],{},"744M",[132,2375,2376],{},"61.36",[132,2378,2379],{},"68.46",[132,2381,2382],{},[32,2383,2384],{},"78.68",[132,2386,2387],{},[32,2388,2389],{},"59.01",[132,2391,2392],{},[32,2393,2394],{},"57.28",[132,2396,2397],{},[32,2398,2399],{},"50.67",[114,2401,2402,2405,2408,2411,2414,2417,2420,2422],{},[132,2403,2404],{},"EmbeddingGemma 1",[132,2406,2407],{},"308M",[132,2409,2410],{},"61.15",[132,2412,2413],{},"69.67",[132,2415,2416],{},"68.76",[132,2418,2419],{},"—",[132,2421,2419],{},[132,2423,2419],{},[114,2425,2426,2429,2432,2437,2442,2444,2447,2450],{},[132,2427,2428],{},"Jina Embed v5 Omni Nano",[132,2430,2431],{},"986M",[132,2433,2434],{},[32,2435,2436],{},"65.52",[132,2438,2439],{},[32,2440,2441],{},"71.11",[132,2443,2419],{},[132,2445,2446],{},"43.61",[132,2448,2449],{},"31.56",[132,2451,2452],{},"30.73",[114,2454,2455,2458,2461,2464,2467,2470,2472,2474],{},[132,2456,2457],{},"Qwen3-Embedding-0.6B",[132,2459,2460],{},"596M",[132,2462,2463],{},"64.34",[132,2465,2466],{},"70.47",[132,2468,2469],{},"75.42",[132,2471,2419],{},[132,2473,2419],{},[132,2475,2419],{},[114,2477,2478,2481,2484,2486,2488,2490,2493,2496],{},[132,2479,2480],{},"InterestFM UIR CAFe 0.5B",[132,2482,2483],{},"894M",[132,2485,2419],{},[132,2487,2419],{},[132,2489,2419],{},[132,2491,2492],{},"49.33",[132,2494,2495],{},"55.43",[132,2497,2498],{},"35.87",[15,2500,2501],{},"(— means the paper reports no score for that cell; the paper uses ∗ for \"not supported\" and – for \"score unavailable\", which are not the same thing. All numbers from Table 6.)",[15,2503,2504],{},"Three things stand out.",[15,2506,2507,2510],{},[32,2508,2509],{},"First, versus the previous generation, text barely moved — but three modalities appeared."," The real difference is not the score; it is that the extra capability did not eat the existing one.",[15,2512,2513,2516],{},[32,2514,2515],{},"Second, text-only it loses to specialists."," Jina leads by 4.2 on multilingual and 2.7 on English; Qwen3 by 3.0. And 744M is the \"everything loaded\" number — text-only runs at 271M. The appendix gives the fair comparison: harrier-oss-v1-270m at 268M scores 66.5 and jina-embed-v5-text-nano at 212M scores 65.5, both higher. Both are text specialists.",[15,2518,2519,2520],{},"That is not a flaw, it is a division of labour. ",[32,2521,2522],{},"Wanting one checkpoint that can grow into other modalities, and wanting the strongest text model, are two different requirements.",[15,2524,2525,2528],{},[32,2526,2527],{},"Third, multimodal it leads clearly."," Video 50.67 versus Jina 30.73 and InterestFM 35.87 — a 15 to 20 point gap. Image 57.28 versus 55.43 is modest but consistent.",[15,2530,2531],{},"On audio: same model, native audio 69.54 versus 57.59 for ASR — about 12 points. What is lost in transcription is the non-linguistic part of sound. (The previous generation's ASR scored 50.95; comparing that to 69.54 gives 18.6, but that is a generational comparison, not a modality gap.)",[19,2533,2535],{"id":2534},"what-is-not-finished","What is not finished",[15,2537,2538],{},"To judge whether a new path is worth betting on, look at what it still lacks. Four things.",[15,2540,2541,2544],{},[32,2542,2543],{},"Text is not the strongest."," The Pareto frontier the paper claims is a quality-versus-parameters efficiency frontier (beating models 2–12× larger), not a first place in every column.",[15,2546,2547,2550],{},[32,2548,2549],{},"Visual documents still have room."," At a 280-token budget, VisDoc is 67.84 versus Jina Omni Nano's 71.34. Scans, tables and dense layouts depend on fine visual detail, and the paper notes that raising the token budget still improves VisDoc — this looks more like \"not tuned yet\" than a ceiling.",[15,2552,2553,2556],{},[32,2554,2555],{},"Quantization hurts audio most."," Moving from bfloat16 to int4, text barely notices (61.36 → 60.84) while audio drops 5.6 points (69.54 → 63.92). The paper notes, however, that float and quantized runs use different evaluation harnesses, so the drop includes harness differences and cannot be blamed on quantization alone.",[15,2558,2559,2562,2563,2566],{},[32,2560,2561],{},"The same token budget moves three tasks in three directions."," Images are trained at 280 tokens per frame, video at 140; raising inference from 280 to 560 improves visual documents sharply (63.38 → 67.62), saturates images, and ",[32,2564,2565],{},"makes video worse"," (48.51 → 46.52). The paper attributes this to excessive sequence length diluting attention and mean-pooled representations.",[190,2568,194,2570,194,2575],{"className":2569},[193],[196,2571],{"src":2572,"poster":2573,"autoPlay":200,"loop":200,"muted":200,"playsInline":200,"ariaLabel":2574},"\u002Fimages\u002Fblog\u002Fshared-trunk-four-modalities\u002Fbudget.mp4","\u002Fimages\u002Fblog\u002Fshared-trunk-four-modalities\u002Fbudget-poster.jpg","Vision token budget effects on image, video and visual documents",[203,2576,2577],{},"One knob, three directions. Images train at 280, video at 140; video is still improving at 280 and drops from 280 to 560.",[15,2579,2580,2581],{},"So the knob has to be tuned per task — ",[32,2582,2583],{},"not a defect, but the control this direction gives you.",[19,2585,2587],{"id":2586},"conclusion-a-new-direction-for-scaling-up","Conclusion: a new direction for scaling up",[15,2589,2590],{},"Back to the scene. You no longer have to choose between \"four models\" and \"one big model\".",[15,2592,2593,2596],{},[32,2594,2595],{},"Shared dimensions, capacity isolated in branches",": one semantic space, open the branch you need.",[15,2598,2599],{},"Two ways to split, two different bills:",[12,2601,2602,2608],{},[15,2603,2604,2607],{},[32,2605,2606],{},"MoE"," splits inside the model. Homogeneous experts, a learned router decides who works. It buys decoupling of capacity from compute per token.",[15,2609,2610,2612],{},[32,2611,2017],{}," splits at the entrance. Heterogeneous modalities, designed encoders decide who handles what. It buys decoupling of capacity from resident memory.",[15,2614,2615,2616],{},"The shared lesson: ",[32,2617,2618],{},"splitting is not about getting bigger; it is about making some parts switchable off.",[15,2620,2621,2622],{},"And the most counter-intuitive lesson of all — ",[32,2623,2624],{},"a shared space comes from removing specialised machinery, not from adding more modules.",[1931,2626],{},[15,2628,2629],{},[1936,2630,2631,2632,1944,2636,2641,2642,2647],{},"We have not measured this model's deployment difficulty, tooling support or real latency, so this is not a recommendation to migrate. If you do, measure three things first: how big your text trunk actually is, whether the branches can be switched off, and what share of your queries are cross-modal. Sources: ",[51,2633,2635],{"href":2015,"rel":2634},[55],"arXiv 2610.37364",[51,2637,2640],{"href":2638,"rel":2639},"https:\u002F\u002Fai.google.dev\u002Fgemma\u002Fdocs",[55],"Gemma docs",". Related MoE work: ",[51,2643,2646],{"href":2644,"rel":2645},"https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.37386",[55],"arXiv 2610.37386"," (channel selection, not a MoE survey).",{"title":770,"searchDepth":771,"depth":771,"links":2649},[2650,2651,2652,2653,2658,2659,2664,2665,2666],{"id":1999,"depth":771,"text":2000},{"id":2031,"depth":771,"text":2032},{"id":2062,"depth":771,"text":2063},{"id":2114,"depth":771,"text":2115,"children":2654},[2655,2657],{"id":2122,"depth":2656,"text":2123},3,{"id":2142,"depth":2656,"text":2143},{"id":2158,"depth":771,"text":2159},{"id":2200,"depth":771,"text":2201,"children":2660},[2661,2662,2663],{"id":2207,"depth":2656,"text":2208},{"id":2230,"depth":2656,"text":2231},{"id":2246,"depth":2656,"text":2247},{"id":2265,"depth":771,"text":2266},{"id":2534,"depth":771,"text":2535},{"id":2586,"depth":771,"text":2587},"\u002Fimages\u002Fblog\u002Fen\u002Fshared-trunk-four-modalities.svg","Parameter split showing a 271M text trunk, 167M vision encoder and 305M audio encoder isolated in branches that can be loaded on demand","2026-11-01","Multimodal used to mean four models, or one big enough to swallow everything. EmbeddingGemma 2 shows a third path: share the semantic backbone, isolate capacity in branches, load only what you need.",{},"\u002Fimages\u002Fblog\u002Fen\u002Fshared-trunk-four-modalities.jpg","\u002Fblog\u002Fen\u002Fshared-trunk-four-modalities",{"title":1989,"description":2670},"shared-trunk-four-modalities","blog\u002Fen\u002Fshared-trunk-four-modalities",[2678,2679,2680],"multimodal","embedding","model-design","wi1YnHGolk3e0hWo1T4Qw9oxzCu8vI22647doO01M-Q",1793708019255]