[{"data":1,"prerenderedAt":6149},["ShallowReactive",2],{"post-kmeans-fewer-epochs":3,"site-info":4741,"related-en-kmeans-fewer-epochs":4837},{"zh":4,"en":2496},{"id":5,"title":6,"author":7,"body":8,"cover":2476,"coverAlt":2477,"date":2478,"description":2479,"draft":2480,"extension":2481,"featured":2480,"lang":2482,"meta":2483,"navigation":159,"ogImage":2484,"path":2485,"seo":2486,"slug":2487,"stem":2488,"tags":2489,"translationKey":2487,"updated":2494,"__hash__":2495},"blog\u002Fblog\u002Fzh-TW\u002Fkmeans-fewer-epochs.md","k-means 的 25 遍，可以只讀 2 遍","Jecision",{"type":9,"value":10,"toc":2466},"minimark",[11,38,42,45,48,51,54,57,60,138,145,148,165,168,172,175,178,181,1063,1079,1082,1085,1146,1150,1153,1156,1161,1883,1893,1900,1906,1912,1984,1987,1993,1996,1999,2002,2005,2012,2015,2018,2021,2032,2035,2039,2042,2048,2051,2138,2141,2148,2155,2158,2161,2178,2212,2215,2218,2221,2228,2231,2285,2288,2291,2294,2297,2303,2379,2385,2425,2431,2434,2437,2462],[12,13,14],"blockquote",{},[15,16,17,18,25,26,31,32,37],"p",{},"數字來自 ",[19,20,24],"a",{"href":21,"rel":22},"https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.09501",[23],"nofollow","arXiv 2611.09501v1","（2026-11-06，FAIR at Meta）、",[19,27,30],{"href":28,"rel":29},"https:\u002F\u002Fe-means.github.io",[23],"官方專案頁","與 ",[19,33,36],{"href":34,"rel":35},"https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Ffaiss\u002Fblob\u002Fmain\u002Ffaiss\u002FClustering.h",[23],"faiss 原始碼","。我們沒有獨立重跑論文實驗；文中那個 120 點的小模擬是我們自己跑的，設定寫在圖 2 前面。",[39,40,41],"h2",{"id":41},"先確認這筆錢你付過了",[15,43,44],{},"你在做 RAG，或做資料去重。文件進來、切成 chunk、算成向量，然後某一支排程把它們聚成叢集、寫成索引檔。",[15,46,47],{},"重建一次索引，最慢的往往就是這一步。它用的還是 1957 年的演算法，預設要你把資料掃 25 遍。這篇要講的是，同樣的損失，有沒有機會只掃 2 遍。",[15,49,50],{},"有，但只在某些條件下成立。FAIR 的 e-means 用 1.4–2.6 輪追平同樣的損失，也已經進了 faiss 正式版。它加速的是建置、不是查詢，而且有規模門檻。這篇會把條件畫清楚。",[15,52,53],{},"讀完你會多一個問題可以問：「那你們重建索引要多久？」",[39,55,56],{"id":56},"它藏在建置流程的中間",[15,58,59],{},"k-means 不是冷門演算法，它是這一堆東西的底層：",[61,62,63,79],"table",{},[64,65,66],"thead",{},[67,68,69,73,76],"tr",{},[70,71,72],"th",{},"應用",[70,74,75],{},"k-means 在哪",[70,77,78],{},"你什麼時候付這筆錢",[80,81,82,94,105,116,127],"tbody",{},[67,83,84,88,91],{},[85,86,87],"td",{},"IVF 向量索引",[85,89,90],{},"粗量化：先分成大類，查詢時只進對應的桶",[85,92,93],{},"建索引、換模型、補資料",[67,95,96,99,102],{},[85,97,98],{},"乘積量化（PQ）",[85,100,101],{},"把向量切段，每一段各自分群一次",[85,103,104],{},"同上",[67,106,107,110,113],{},[85,108,109],{},"VQ-VAE、神經音訊編碼",[85,111,112],{},"碼簿學習（每個叢集配一個代表向量）",[85,114,115],{},"訓練、換資料集",[67,117,118,121,124],{},[85,119,120],{},"資料去重、異常偵測",[85,122,123],{},"聚類找重複與離群",[85,125,126],{},"匯入、清洗",[67,128,129,132,135],{},[85,130,131],{},"客戶分群、影像分群",[85,133,134],{},"就是它本身",[85,136,137],{},"每次分析",[15,139,140,141],{},"共同的形狀是：",[142,143,144],"strong",{},"它在離線，不在查詢路徑上。",[15,146,147],{},"如圖 1，一支索引的壽命裡，左半是建置、右半是查詢。查詢只讀一次索引；建置要把向量掃過很多遍。你每天感覺到的延遲在右半，真正吃掉時間的在左半。",[149,150,153,154,153,161],"figure",{"className":151},[152],"article-figure","\n  ",[155,156],"video",{"src":157,"poster":158,"autoPlay":159,"loop":159,"muted":159,"playsInline":159,"ariaLabel":160},"\u002Fimages\u002Fblog\u002Fkmeans-fewer-epochs\u002Fpipeline.mp4","\u002Fimages\u002Fblog\u002Fkmeans-fewer-epochs\u002Fpipeline-poster.jpg",true,"索引建置管線：原始資料、嵌入、k-means、索引檔，最後才是查詢；k-means 那格標著 25 遍掃描",[162,163,164],"figcaption",{},"圖 1：左半是建置、右半是查詢；k-means 那格標著 25 遍掃描。",[15,166,167],{},"粗算一下：以 100 萬筆、768 維、k = 4,096 為例（fp32），一趟掃描大約 3.1 GB。Faiss 的 k-means 預設跑 25 輪，就是 77 GB 上下的讀取，還不含距離計算與寫入。這個數字會隨維度、資料型別變動，但量級就是這樣。",[39,169,171],{"id":170},"它還是-1957-年那支-lloyd","它還是 1957 年那支 Lloyd",[15,173,174],{},"Lloyd 的 k-means 只有兩步：把每個點分給最近的質心；把每個質心移到它那群點的平均位置。重複到不再變動。",[15,176,177],{},"它保證收斂——收到局部最小值，不是全域。這個保證從 1957 年貝爾實驗室的技術備忘錄一路用到現在，正式發表是 1982 年。",[15,179,180],{},"要最小化的東西長這樣：",[182,183,186],"span",{"className":184},[185],"katex-display",[182,187,190,369],{"className":188},[189],"katex",[182,191,194],{"className":192},[193],"katex-mathml",[195,196,199],"math",{"xmlns":197,"display":198},"http:\u002F\u002Fwww.w3.org\u002F1998\u002FMath\u002FMathML","block",[200,201,202,364],"semantics",{},[203,204,205,218,223,226,230,233,236,239,249,266,305,307,311,313,315,317,319,321,324,327,340],"mrow",{},[203,206,207,212,215],{},[208,209,211],"mi",{"mathvariant":210},"normal","M",[208,213,214],{"mathvariant":210},"S",[208,216,217],{"mathvariant":210},"E",[219,220,222],"mo",{"stretchy":221},"false","(",[208,224,225],{},"X",[219,227,229],{"separator":228},"true",",",[208,231,232],{},"C",[219,234,235],{"stretchy":221},")",[219,237,238],{},"=",[240,241,242,246],"mfrac",{},[243,244,245],"mn",{},"1",[208,247,248],{},"n",[250,251,252,255,264],"munderover",{},[219,253,254],{},"∑",[203,256,257,260,262],{},[208,258,259],{},"i",[219,261,238],{},[243,263,245],{},[208,265,248],{},[267,268,269,302],"msup",{},[203,270,271,274,282,285,300],{},[219,272,273],{"fence":228},"∥",[275,276,277,280],"msub",{},[208,278,279],{},"x",[208,281,259],{},[219,283,284],{},"−",[275,286,287,290],{},[208,288,289],{},"c",[203,291,292,294,296,298],{},[208,293,19],{},[219,295,222],{"stretchy":221},[208,297,259],{},[219,299,235],{"stretchy":221},[219,301,273],{"fence":228},[243,303,304],{},"2",[219,306,229],{"separator":228},[308,309],"mspace",{"width":310},"2em",[208,312,19],{},[219,314,222],{"stretchy":221},[208,316,259],{},[219,318,235],{"stretchy":221},[219,320,238],{},[208,322,323],{},"arg",[219,325,326],{},"⁡",[328,329,330,337],"munder",{},[203,331,332,335],{},[208,333,334],{},"min",[219,336,326],{},[208,338,339],{},"j",[267,341,342,362],{},[203,343,344,346,352,354,360],{},[219,345,273],{"fence":228},[275,347,348,350],{},[208,349,279],{},[208,351,259],{},[219,353,284],{},[275,355,356,358],{},[208,357,289],{},[208,359,339],{},[219,361,273],{"fence":228},[243,363,304],{},[365,366,368],"annotation",{"encoding":367},"application\u002Fx-tex","\\mathrm{MSE}(X, C) = \\frac{1}{n} \\sum_{i=1}^{n} \\left\\| x_i - c_{a(i)} \\right\\|^2,\n\\qquad a(i) = \\arg\\min_j \\left\\| x_i - c_j \\right\\|^2",[182,370,373,427,864],{"className":371,"ariaHidden":228},[372],"katex-html",[182,374,377,382,391,395,400,404,408,412,416,420,424],{"className":375},[376],"base",[182,378],{"className":379,"style":381},[380],"strut","height:1em;vertical-align:-0.25em;",[182,383,386],{"className":384},[385],"mord",[182,387,390],{"className":388},[385,389],"mathrm","MSE",[182,392,222],{"className":393},[394],"mopen",[182,396,225],{"className":397,"style":399},[385,398],"mathnormal","margin-right:0.0785em;",[182,401,229],{"className":402},[403],"mpunct",[182,405],{"className":406,"style":407},[308],"margin-right:0.1667em;",[182,409,232],{"className":410,"style":411},[385,398],"margin-right:0.0715em;",[182,413,235],{"className":414},[415],"mclose",[182,417],{"className":418,"style":419},[308],"margin-right:0.2778em;",[182,421,238],{"className":422},[423],"mrel",[182,425],{"className":426,"style":419},[308],[182,428,430,434,512,515,596,599,830,833,836,840,843,846,849,852,855,858,861],{"className":429},[376],[182,431],{"className":432,"style":433},[380],"height:2.9291em;vertical-align:-1.2777em;",[182,435,437,441,509],{"className":436},[385],[182,438],{"className":439},[394,440],"nulldelimiter",[182,442,444],{"className":443},[240],[182,445,449,500],{"className":446},[447,448],"vlist-t","vlist-t2",[182,450,453,495],{"className":451},[452],"vlist-r",[182,454,458,472,483],{"className":455,"style":457},[456],"vlist","height:1.3214em;",[182,459,461,466],{"style":460},"top:-2.314em;",[182,462],{"className":463,"style":465},[464],"pstrut","height:3em;",[182,467,469],{"className":468},[385],[182,470,248],{"className":471},[385,398],[182,473,475,478],{"style":474},"top:-3.23em;",[182,476],{"className":477,"style":465},[464],[182,479],{"className":480,"style":482},[481],"frac-line","border-bottom-width:0.04em;",[182,484,486,489],{"style":485},"top:-3.677em;",[182,487],{"className":488,"style":465},[464],[182,490,492],{"className":491},[385],[182,493,245],{"className":494},[385],[182,496,499],{"className":497},[498],"vlist-s","​",[182,501,503],{"className":502},[452],[182,504,507],{"className":505,"style":506},[456],"height:0.686em;",[182,508],{},[182,510],{"className":511},[415,440],[182,513],{"className":514,"style":407},[308],[182,516,520],{"className":517},[518,519],"mop","op-limits",[182,521,523,587],{"className":522},[447,448],[182,524,526,584],{"className":525},[452],[182,527,530,556,569],{"className":528,"style":529},[456],"height:1.6514em;",[182,531,533,537],{"style":532},"top:-1.8723em;margin-left:0em;",[182,534],{"className":535,"style":536},[464],"height:3.05em;",[182,538,544],{"className":539},[540,541,542,543],"sizing","reset-size6","size3","mtight",[182,545,547,550,553],{"className":546},[385,543],[182,548,259],{"className":549},[385,398,543],[182,551,238],{"className":552},[423,543],[182,554,245],{"className":555},[385,543],[182,557,559,562],{"style":558},"top:-3.05em;",[182,560],{"className":561,"style":536},[464],[182,563,564],{},[182,565,254],{"className":566},[518,567,568],"op-symbol","large-op",[182,570,572,575],{"style":571},"top:-4.3em;margin-left:0em;",[182,573],{"className":574,"style":536},[464],[182,576,578],{"className":577},[540,541,542,543],[182,579,581],{"className":580},[385,543],[182,582,248],{"className":583},[385,398,543],[182,585,499],{"className":586},[498],[182,588,590],{"className":589},[452],[182,591,594],{"className":592,"style":593},[456],"height:1.2777em;",[182,595],{},[182,597],{"className":598,"style":407},[308],[182,600,603,805],{"className":601},[602],"minner",[182,604,606,657,702,706,710,713,768],{"className":605},[602],[182,607,609],{"className":608},[394],[182,610,614],{"className":611},[612,613],"delimsizing","mult",[182,615,617,648],{"className":616},[447,448],[182,618,620,645],{"className":619},[452],[182,621,624],{"className":622,"style":623},[456],"height:0.85em;",[182,625,627,631],{"style":626},"top:-2.85em;",[182,628],{"className":629,"style":630},[464],"height:3.2em;",[182,632,634],{"style":633},"width:0.556em;height:1.2em;",[635,636,641],"svg",{"xmlns":637,"width":638,"height":639,"viewBox":640},"http:\u002F\u002Fwww.w3.org\u002F2000\u002Fsvg","0.556em","1.2em","0 0 556 1200",[642,643],"path",{"d":644},"M145 15 v585 v0 v585 c2.667,10,9.667,15,21,15\nc10,0,16.667,-5,20,-15 v-585 v0 v-585 c-2.667,-10,-9.667,-15,-21,-15\nc-10,0,-16.667,5,-20,15z M188 15 H145 v585 v0 v585 h43z\nM367 15 v585 v0 v585 c2.667,10,9.667,15,21,15\nc10,0,16.667,-5,20,-15 v-585 v0 v-585 c-2.667,-10,-9.667,-15,-21,-15\nc-10,0,-16.667,5,-20,15z M410 15 H367 v585 v0 v585 h43z",[182,646,499],{"className":647},[498],[182,649,651],{"className":650},[452],[182,652,655],{"className":653,"style":654},[456],"height:0.35em;",[182,656],{},[182,658,660,663],{"className":659},[385],[182,661,279],{"className":662},[385,398],[182,664,667],{"className":665},[666],"msupsub",[182,668,670,693],{"className":669},[447,448],[182,671,673,690],{"className":672},[452],[182,674,677],{"className":675,"style":676},[456],"height:0.3117em;",[182,678,680,684],{"style":679},"top:-2.55em;margin-left:0em;margin-right:0.05em;",[182,681],{"className":682,"style":683},[464],"height:2.7em;",[182,685,687],{"className":686},[540,541,542,543],[182,688,259],{"className":689},[385,398,543],[182,691,499],{"className":692},[498],[182,694,696],{"className":695},[452],[182,697,700],{"className":698,"style":699},[456],"height:0.15em;",[182,701],{},[182,703],{"className":704,"style":705},[308],"margin-right:0.2222em;",[182,707,284],{"className":708},[709],"mbin",[182,711],{"className":712,"style":705},[308],[182,714,716,719],{"className":715},[385],[182,717,289],{"className":718},[385,398],[182,720,722],{"className":721},[666],[182,723,725,759],{"className":724},[447,448],[182,726,728,756],{"className":727},[452],[182,729,732],{"className":730,"style":731},[456],"height:0.3448em;",[182,733,735,738],{"style":734},"top:-2.5198em;margin-left:0em;margin-right:0.05em;",[182,736],{"className":737,"style":683},[464],[182,739,741],{"className":740},[540,541,542,543],[182,742,744,747,750,753],{"className":743},[385,543],[182,745,19],{"className":746},[385,398,543],[182,748,222],{"className":749},[394,543],[182,751,259],{"className":752},[385,398,543],[182,754,235],{"className":755},[415,543],[182,757,499],{"className":758},[498],[182,760,762],{"className":761},[452],[182,763,766],{"className":764,"style":765},[456],"height:0.3552em;",[182,767],{},[182,769,771],{"className":770},[415],[182,772,774],{"className":773},[612,613],[182,775,777,797],{"className":776},[447,448],[182,778,780,794],{"className":779},[452],[182,781,783],{"className":782,"style":623},[456],[182,784,785,788],{"style":626},[182,786],{"className":787,"style":630},[464],[182,789,790],{"style":633},[635,791,792],{"xmlns":637,"width":638,"height":639,"viewBox":640},[642,793],{"d":644},[182,795,499],{"className":796},[498],[182,798,800],{"className":799},[452],[182,801,803],{"className":802,"style":654},[456],[182,804],{},[182,806,808],{"className":807},[666],[182,809,811],{"className":810},[447],[182,812,814],{"className":813},[452],[182,815,818],{"className":816,"style":817},[456],"height:1.054em;",[182,819,821,824],{"style":820},"top:-3.3029em;margin-right:0.05em;",[182,822],{"className":823,"style":683},[464],[182,825,827],{"className":826},[540,541,542,543],[182,828,304],{"className":829},[385,543],[182,831],{"className":832,"style":407},[308],[182,834,229],{"className":835},[403],[182,837],{"className":838,"style":839},[308],"margin-right:2em;",[182,841],{"className":842,"style":407},[308],[182,844,19],{"className":845},[385,398],[182,847,222],{"className":848},[394],[182,850,259],{"className":851},[385,398],[182,853,235],{"className":854},[415],[182,856],{"className":857,"style":419},[308],[182,859,238],{"className":860},[423],[182,862],{"className":863,"style":419},[308],[182,865,867,871,879,882,931,934],{"className":866},[376],[182,868],{"className":869,"style":870},[380],"height:1.8178em;vertical-align:-0.8638em;",[182,872,874,875],{"className":873},[518],"ar",[182,876,878],{"style":877},"margin-right:0.0139em;","g",[182,880],{"className":881,"style":407},[308],[182,883,885],{"className":884},[518,519],[182,886,888,922],{"className":887},[447,448],[182,889,891,919],{"className":890},[452],[182,892,895,908],{"className":893,"style":894},[456],"height:0.6679em;",[182,896,898,901],{"style":897},"top:-2.3723em;margin-left:0em;",[182,899],{"className":900,"style":465},[464],[182,902,904],{"className":903},[540,541,542,543],[182,905,339],{"className":906,"style":907},[385,398,543],"margin-right:0.0572em;",[182,909,911,914],{"style":910},"top:-3em;",[182,912],{"className":913,"style":465},[464],[182,915,916],{},[182,917,334],{"className":918},[518],[182,920,499],{"className":921},[498],[182,923,925],{"className":924},[452],[182,926,929],{"className":927,"style":928},[456],"height:0.8638em;",[182,930],{},[182,932],{"className":933,"style":407},[308],[182,935,937,1038],{"className":936},[602],[182,938,940,945,985,988,991,994,1035],{"className":939},[602],[182,941,273],{"className":942,"style":944},[394,943],"delimcenter","top:0em;",[182,946,948,951],{"className":947},[385],[182,949,279],{"className":950},[385,398],[182,952,954],{"className":953},[666],[182,955,957,977],{"className":956},[447,448],[182,958,960,974],{"className":959},[452],[182,961,963],{"className":962,"style":676},[456],[182,964,965,968],{"style":679},[182,966],{"className":967,"style":683},[464],[182,969,971],{"className":970},[540,541,542,543],[182,972,259],{"className":973},[385,398,543],[182,975,499],{"className":976},[498],[182,978,980],{"className":979},[452],[182,981,983],{"className":982,"style":699},[456],[182,984],{},[182,986],{"className":987,"style":705},[308],[182,989,284],{"className":990},[709],[182,992],{"className":993,"style":705},[308],[182,995,997,1000],{"className":996},[385],[182,998,289],{"className":999},[385,398],[182,1001,1003],{"className":1002},[666],[182,1004,1006,1026],{"className":1005},[447,448],[182,1007,1009,1023],{"className":1008},[452],[182,1010,1012],{"className":1011,"style":676},[456],[182,1013,1014,1017],{"style":679},[182,1015],{"className":1016,"style":683},[464],[182,1018,1020],{"className":1019},[540,541,542,543],[182,1021,339],{"className":1022,"style":907},[385,398,543],[182,1024,499],{"className":1025},[498],[182,1027,1029],{"className":1028},[452],[182,1030,1033],{"className":1031,"style":1032},[456],"height:0.2861em;",[182,1034],{},[182,1036,273],{"className":1037,"style":944},[415,943],[182,1039,1041],{"className":1040},[666],[182,1042,1044],{"className":1043},[447],[182,1045,1047],{"className":1046},[452],[182,1048,1051],{"className":1049,"style":1050},[456],"height:0.954em;",[182,1052,1054,1057],{"style":1053},"top:-3.2029em;margin-right:0.05em;",[182,1055],{"className":1056,"style":683},[464],[182,1058,1060],{"className":1059},[540,541,542,543],[182,1061,304],{"className":1062},[385,543],[15,1064,1065,1066,1070,1071,1074,1075,1078],{},"Faiss 的 ",[1067,1068,1069],"code",{},"Kmeans"," 預設 ",[1067,1072,1073],{},"niter = 25","，每一輪都是一次完整分配。原始碼裡還有一個常被忽略的角落：",[1067,1076,1077],{},"max_points_per_centroid = 256","。當 n 遠大於 k 時，Faiss 會先抽樣到每個叢集最多 256 點。前面那個例子（100 萬點配 4,096 個叢集）剛好還在門檻內（256 × 4,096 ≈ 105 萬）。所以你掃的就是全部。",[15,1080,1081],{},"每一輪的成本是 O(n · k · d)，而且是隨機讀取。這是記憶體頻寬在吃時間，不是浮點運算。25 輪不是幻覺，是真的把資料從頭到尾走了 25 次。這裡的「輪」就是 epoch：資料完整跑過一遍。",[15,1083,1084],{},"那為什麼沒被換掉？小批次版本 1967 年就有（MacQueen）。2010 年 Sculley 的 web-scale 版本也在跑。它們每輪只讀一小批、快得多，代價是誤差偏高，而且你得自己調批次大小與學習率。對大部分場景，「慢，但不用想」是一個很合理的預設。",[61,1086,1087,1103],{},[64,1088,1089],{},[67,1090,1091,1094,1097,1100],{},[70,1092,1093],{},"做法",[70,1095,1096],{},"每輪讀多少",[70,1098,1099],{},"統計量怎麼更新",[70,1101,1102],{},"代價",[80,1104,1105,1119,1133],{},[67,1106,1107,1110,1113,1116],{},[85,1108,1109],{},"Lloyd（全量，Faiss 預設）",[85,1111,1112],{},"全部 n 點",[85,1114,1115],{},"每輪重算精確均值",[85,1117,1118],{},"慢；預設 25 輪",[67,1120,1121,1124,1127,1130],{},[85,1122,1123],{},"MacQueen、Sculley（小批次）",[85,1125,1126],{},"一小批",[85,1128,1129],{},"累加，從不忘記",[85,1131,1132],{},"快，但早期錯誤的分配會一直投票",[67,1134,1135,1138,1140,1143],{},[85,1136,1137],{},"e-means",[85,1139,1126],{},[85,1141,1142],{},"指數移動平均，舊分配衰減",[85,1144,1145],{},"快；參數由標度律算",[39,1147,1149],{"id":1148},"e-means-換掉的四件事","e-means 換掉的四件事",[15,1151,1152],{},"論文的做法叫 e-means，來自 Meta FAIR。論文標題直接寫著「one-tenth the epochs」。作者名單裡有 Matthijs Douze 與 Hervé Jégou，兩人是 Faiss 的作者；再加上 Hugo Touvron。這是同一個團隊在挑戰自己的預設值。",[15,1154,1155],{},"它沒有發明新的損失函數——目標還是上面那條 MSE，改的是「質心怎麼更新」。",[15,1157,1158],{},[142,1159,1160],{},"第一件事，統計量改成指數移動平均。",[182,1162,1164],{"className":1163},[185],[182,1165,1167,1288],{"className":1166},[189],[182,1168,1170],{"className":1169},[193],[195,1171,1172],{"xmlns":197,"display":198},[200,1173,1174,1285],{},[203,1175,1176,1183,1186,1188,1190,1192,1195,1197,1203,1206,1208,1212,1218,1220,1222,1228,1230,1232,1234,1236,1238,1240,1246,1248,1250,1252,1259,1261,1263,1269,1271],{},[275,1177,1178,1181],{},[208,1179,1180],{},"N",[208,1182,339],{},[219,1184,1185],{},"←",[219,1187,222],{"stretchy":221},[243,1189,245],{},[219,1191,284],{},[208,1193,1194],{},"α",[219,1196,235],{"stretchy":221},[275,1198,1199,1201],{},[208,1200,1180],{},[208,1202,339],{},[219,1204,1205],{},"+",[208,1207,1194],{},[1209,1210,1211],"mtext",{}," ",[275,1213,1214,1216],{},[208,1215,248],{},[208,1217,339],{},[219,1219,229],{"separator":228},[308,1221],{"width":310},[275,1223,1224,1226],{},[208,1225,214],{},[208,1227,339],{},[219,1229,1185],{},[219,1231,222],{"stretchy":221},[243,1233,245],{},[219,1235,284],{},[208,1237,1194],{},[219,1239,235],{"stretchy":221},[275,1241,1242,1244],{},[208,1243,214],{},[208,1245,339],{},[219,1247,1205],{},[208,1249,1194],{},[1209,1251,1211],{},[275,1253,1254,1257],{},[208,1255,1256],{},"s",[208,1258,339],{},[219,1260,229],{"separator":228},[308,1262],{"width":310},[275,1264,1265,1267],{},[208,1266,289],{},[208,1268,339],{},[219,1270,238],{},[240,1272,1273,1279],{},[275,1274,1275,1277],{},[208,1276,214],{},[208,1278,339],{},[275,1280,1281,1283],{},[208,1282,1180],{},[208,1284,339],{},[365,1286,1287],{"encoding":367},"N_j \\leftarrow (1-\\alpha) N_j + \\alpha\\, n_j, \\qquad\nS_j \\leftarrow (1-\\alpha) S_j + \\alpha\\, s_j, \\qquad\nc_j = \\frac{S_j}{N_j}",[182,1289,1291,1349,1370,1433,1545,1566,1627,1738],{"className":1290,"ariaHidden":228},[372],[182,1292,1294,1298,1340,1343,1346],{"className":1293},[376],[182,1295],{"className":1296,"style":1297},[380],"height:0.9694em;vertical-align:-0.2861em;",[182,1299,1301,1305],{"className":1300},[385],[182,1302,1180],{"className":1303,"style":1304},[385,398],"margin-right:0.109em;",[182,1306,1308],{"className":1307},[666],[182,1309,1311,1332],{"className":1310},[447,448],[182,1312,1314,1329],{"className":1313},[452],[182,1315,1317],{"className":1316,"style":676},[456],[182,1318,1320,1323],{"style":1319},"top:-2.55em;margin-left:-0.109em;margin-right:0.05em;",[182,1321],{"className":1322,"style":683},[464],[182,1324,1326],{"className":1325},[540,541,542,543],[182,1327,339],{"className":1328,"style":907},[385,398,543],[182,1330,499],{"className":1331},[498],[182,1333,1335],{"className":1334},[452],[182,1336,1338],{"className":1337,"style":1032},[456],[182,1339],{},[182,1341],{"className":1342,"style":419},[308],[182,1344,1185],{"className":1345},[423],[182,1347],{"className":1348,"style":419},[308],[182,1350,1352,1355,1358,1361,1364,1367],{"className":1351},[376],[182,1353],{"className":1354,"style":381},[380],[182,1356,222],{"className":1357},[394],[182,1359,245],{"className":1360},[385],[182,1362],{"className":1363,"style":705},[308],[182,1365,284],{"className":1366},[709],[182,1368],{"className":1369,"style":705},[308],[182,1371,1373,1377,1381,1384,1424,1427,1430],{"className":1372},[376],[182,1374],{"className":1375,"style":1376},[380],"height:1.0361em;vertical-align:-0.2861em;",[182,1378,1194],{"className":1379,"style":1380},[385,398],"margin-right:0.0037em;",[182,1382,235],{"className":1383},[415],[182,1385,1387,1390],{"className":1386},[385],[182,1388,1180],{"className":1389,"style":1304},[385,398],[182,1391,1393],{"className":1392},[666],[182,1394,1396,1416],{"className":1395},[447,448],[182,1397,1399,1413],{"className":1398},[452],[182,1400,1402],{"className":1401,"style":676},[456],[182,1403,1404,1407],{"style":1319},[182,1405],{"className":1406,"style":683},[464],[182,1408,1410],{"className":1409},[540,541,542,543],[182,1411,339],{"className":1412,"style":907},[385,398,543],[182,1414,499],{"className":1415},[498],[182,1417,1419],{"className":1418},[452],[182,1420,1422],{"className":1421,"style":1032},[456],[182,1423],{},[182,1425],{"className":1426,"style":705},[308],[182,1428,1205],{"className":1429},[709],[182,1431],{"className":1432,"style":705},[308],[182,1434,1436,1439,1442,1445,1485,1488,1491,1494,1536,1539,1542],{"className":1435},[376],[182,1437],{"className":1438,"style":1297},[380],[182,1440,1194],{"className":1441,"style":1380},[385,398],[182,1443],{"className":1444,"style":407},[308],[182,1446,1448,1451],{"className":1447},[385],[182,1449,248],{"className":1450},[385,398],[182,1452,1454],{"className":1453},[666],[182,1455,1457,1477],{"className":1456},[447,448],[182,1458,1460,1474],{"className":1459},[452],[182,1461,1463],{"className":1462,"style":676},[456],[182,1464,1465,1468],{"style":679},[182,1466],{"className":1467,"style":683},[464],[182,1469,1471],{"className":1470},[540,541,542,543],[182,1472,339],{"className":1473,"style":907},[385,398,543],[182,1475,499],{"className":1476},[498],[182,1478,1480],{"className":1479},[452],[182,1481,1483],{"className":1482,"style":1032},[456],[182,1484],{},[182,1486,229],{"className":1487},[403],[182,1489],{"className":1490,"style":839},[308],[182,1492],{"className":1493,"style":407},[308],[182,1495,1497,1501],{"className":1496},[385],[182,1498,214],{"className":1499,"style":1500},[385,398],"margin-right:0.0576em;",[182,1502,1504],{"className":1503},[666],[182,1505,1507,1528],{"className":1506},[447,448],[182,1508,1510,1525],{"className":1509},[452],[182,1511,1513],{"className":1512,"style":676},[456],[182,1514,1516,1519],{"style":1515},"top:-2.55em;margin-left:-0.0576em;margin-right:0.05em;",[182,1517],{"className":1518,"style":683},[464],[182,1520,1522],{"className":1521},[540,541,542,543],[182,1523,339],{"className":1524,"style":907},[385,398,543],[182,1526,499],{"className":1527},[498],[182,1529,1531],{"className":1530},[452],[182,1532,1534],{"className":1533,"style":1032},[456],[182,1535],{},[182,1537],{"className":1538,"style":419},[308],[182,1540,1185],{"className":1541},[423],[182,1543],{"className":1544,"style":419},[308],[182,1546,1548,1551,1554,1557,1560,1563],{"className":1547},[376],[182,1549],{"className":1550,"style":381},[380],[182,1552,222],{"className":1553},[394],[182,1555,245],{"className":1556},[385],[182,1558],{"className":1559,"style":705},[308],[182,1561,284],{"className":1562},[709],[182,1564],{"className":1565,"style":705},[308],[182,1567,1569,1572,1575,1578,1618,1621,1624],{"className":1568},[376],[182,1570],{"className":1571,"style":1376},[380],[182,1573,1194],{"className":1574,"style":1380},[385,398],[182,1576,235],{"className":1577},[415],[182,1579,1581,1584],{"className":1580},[385],[182,1582,214],{"className":1583,"style":1500},[385,398],[182,1585,1587],{"className":1586},[666],[182,1588,1590,1610],{"className":1589},[447,448],[182,1591,1593,1607],{"className":1592},[452],[182,1594,1596],{"className":1595,"style":676},[456],[182,1597,1598,1601],{"style":1515},[182,1599],{"className":1600,"style":683},[464],[182,1602,1604],{"className":1603},[540,541,542,543],[182,1605,339],{"className":1606,"style":907},[385,398,543],[182,1608,499],{"className":1609},[498],[182,1611,1613],{"className":1612},[452],[182,1614,1616],{"className":1615,"style":1032},[456],[182,1617],{},[182,1619],{"className":1620,"style":705},[308],[182,1622,1205],{"className":1623},[709],[182,1625],{"className":1626,"style":705},[308],[182,1628,1630,1634,1637,1640,1680,1683,1686,1689,1729,1732,1735],{"className":1629},[376],[182,1631],{"className":1632,"style":1633},[380],"height:0.7167em;vertical-align:-0.2861em;",[182,1635,1194],{"className":1636,"style":1380},[385,398],[182,1638],{"className":1639,"style":407},[308],[182,1641,1643,1646],{"className":1642},[385],[182,1644,1256],{"className":1645},[385,398],[182,1647,1649],{"className":1648},[666],[182,1650,1652,1672],{"className":1651},[447,448],[182,1653,1655,1669],{"className":1654},[452],[182,1656,1658],{"className":1657,"style":676},[456],[182,1659,1660,1663],{"style":679},[182,1661],{"className":1662,"style":683},[464],[182,1664,1666],{"className":1665},[540,541,542,543],[182,1667,339],{"className":1668,"style":907},[385,398,543],[182,1670,499],{"className":1671},[498],[182,1673,1675],{"className":1674},[452],[182,1676,1678],{"className":1677,"style":1032},[456],[182,1679],{},[182,1681,229],{"className":1682},[403],[182,1684],{"className":1685,"style":839},[308],[182,1687],{"className":1688,"style":407},[308],[182,1690,1692,1695],{"className":1691},[385],[182,1693,289],{"className":1694},[385,398],[182,1696,1698],{"className":1697},[666],[182,1699,1701,1721],{"className":1700},[447,448],[182,1702,1704,1718],{"className":1703},[452],[182,1705,1707],{"className":1706,"style":676},[456],[182,1708,1709,1712],{"style":679},[182,1710],{"className":1711,"style":683},[464],[182,1713,1715],{"className":1714},[540,541,542,543],[182,1716,339],{"className":1717,"style":907},[385,398,543],[182,1719,499],{"className":1720},[498],[182,1722,1724],{"className":1723},[452],[182,1725,1727],{"className":1726,"style":1032},[456],[182,1728],{},[182,1730],{"className":1731,"style":419},[308],[182,1733,238],{"className":1734},[423],[182,1736],{"className":1737,"style":419},[308],[182,1739,1741,1745],{"className":1740},[376],[182,1742],{"className":1743,"style":1744},[380],"height:2.3324em;vertical-align:-0.9721em;",[182,1746,1748,1751,1880],{"className":1747},[385],[182,1749],{"className":1750},[394,440],[182,1752,1754],{"className":1753},[240],[182,1755,1757,1871],{"className":1756},[447,448],[182,1758,1760,1868],{"className":1759},[452],[182,1761,1764,1812,1820],{"className":1762,"style":1763},[456],"height:1.3603em;",[182,1765,1766,1769],{"style":460},[182,1767],{"className":1768,"style":465},[464],[182,1770,1772],{"className":1771},[385],[182,1773,1775,1778],{"className":1774},[385],[182,1776,1180],{"className":1777,"style":1304},[385,398],[182,1779,1781],{"className":1780},[666],[182,1782,1784,1804],{"className":1783},[447,448],[182,1785,1787,1801],{"className":1786},[452],[182,1788,1790],{"className":1789,"style":676},[456],[182,1791,1792,1795],{"style":1319},[182,1793],{"className":1794,"style":683},[464],[182,1796,1798],{"className":1797},[540,541,542,543],[182,1799,339],{"className":1800,"style":907},[385,398,543],[182,1802,499],{"className":1803},[498],[182,1805,1807],{"className":1806},[452],[182,1808,1810],{"className":1809,"style":1032},[456],[182,1811],{},[182,1813,1814,1817],{"style":474},[182,1815],{"className":1816,"style":465},[464],[182,1818],{"className":1819,"style":482},[481],[182,1821,1822,1825],{"style":485},[182,1823],{"className":1824,"style":465},[464],[182,1826,1828],{"className":1827},[385],[182,1829,1831,1834],{"className":1830},[385],[182,1832,214],{"className":1833,"style":1500},[385,398],[182,1835,1837],{"className":1836},[666],[182,1838,1840,1860],{"className":1839},[447,448],[182,1841,1843,1857],{"className":1842},[452],[182,1844,1846],{"className":1845,"style":676},[456],[182,1847,1848,1851],{"style":1515},[182,1849],{"className":1850,"style":683},[464],[182,1852,1854],{"className":1853},[540,541,542,543],[182,1855,339],{"className":1856,"style":907},[385,398,543],[182,1858,499],{"className":1859},[498],[182,1861,1863],{"className":1862},[452],[182,1864,1866],{"className":1865,"style":1032},[456],[182,1867],{},[182,1869,499],{"className":1870},[498],[182,1872,1874],{"className":1873},[452],[182,1875,1878],{"className":1876,"style":1877},[456],"height:0.9721em;",[182,1879],{},[182,1881],{"className":1882},[415,440],[15,1884,1885,1888,1889,1892],{},[1067,1886,1887],{},"n_j","、",[1067,1890,1891],{},"s_j"," 是這一小批點裡分到第 j 叢集的數量與總和。α 決定「忘得多快」：α 大，最近幾批說話；α 小，歷史平均。舊的分配不是被刪掉，是被打折。",[15,1894,1895,1896,1899],{},"MacQueen 的更新是 ",[1067,1897,1898],{},"S ← S + s","，永不忘記。訓練早期質心位置很差，那時候分出來的標籤多半是錯的；但它們在後面每一輪都還有完整的一票，會把質心拖住。e-means 給它們一個衰減係數，等於「慢慢忘掉早期的錯誤判斷」。",[15,1901,1902,1905],{},[142,1903,1904],{},"第二件事，α 不是常數。"," 它從較大的值開始，沿著 cosine 曲線降到初始值的十分之一。先降快、後降慢，像一條降溫曲線，後期就越來越穩。",[15,1907,1908,1911],{},[142,1909,1910],{},"第三件事，死叢集要救。"," 小批次更新很容易出現「沒有任何點分給它」的質心。叢集數看起來是 k，實際能用的不到 k。e-means 每五批檢查一次。把低於平均計數 10% 的叢集重新指派：挑一顆大的，拆成兩顆，補上那個空位。Faiss 本來就有這個做法。影響有多大？論文自己的 ablation（一次關掉一個元件，看損失掉多少）拆得很細：",[61,1913,1914,1933],{},[64,1915,1916],{},[67,1917,1918,1921,1924,1927,1930],{},[70,1919,1920],{},"資料集",[70,1922,1923],{},"復活策略",[70,1925,1926],{},"死叢集",[70,1928,1929],{},"佔碼簿",[70,1931,1932],{},"測試 MSE",[80,1934,1935,1952,1968],{},[67,1936,1937,1940,1943,1946,1949],{},[85,1938,1939],{},"DEEP",[85,1941,1942],{},"按大小分裂",[85,1944,1945],{},"47",[85,1947,1948],{},"0.3%",[85,1950,1951],{},"0.3876",[67,1953,1954,1956,1959,1962,1965],{},[85,1955,1939],{},[85,1957,1958],{},"從批次隨機抽點",[85,1960,1961],{},"2,343",[85,1963,1964],{},"14.3%",[85,1966,1967],{},"0.3912",[67,1969,1970,1972,1975,1978,1981],{},[85,1971,1939],{},[85,1973,1974],{},"不救",[85,1976,1977],{},"4,313",[85,1979,1980],{},"26.3%",[85,1982,1983],{},"0.3945",[15,1985,1986],{},"100 萬個向量、k = 16,384、5 個 epoch、5 個種子平均；SIFT 上的差距類似。關掉復活，四分之一的叢集是空的。其他幾個元件（初始化、cosine、finalization）只差零點幾個百分點。四件事不是平權的，影響最大的是這一件。",[15,1988,1989,1992],{},[142,1990,1991],{},"第四件事，參數不要你猜。"," 批次大小與初始步長由 (n, k, T) 的標度律（用問題規模反推參數的經驗公式）算出來。論文量到的超參 regret 約 0.07%，你不用掃。",[15,1994,1995],{},"這裡要小心一個字面陷阱：「不用調參」指的是相對於 Lloyd 沒有多出新的旋鈕。k 還是你選的，epoch 數也是（預設 5），只是那兩個原本最難調的內部參數被公式接走了。",[15,1997,1998],{},"還有一個設計選擇值得講：他們不用梯度下降。專案頁的說法是，學習率一大，梯度更新會衝過頭。EMA 則保證質心一定落在「曾經被分配到這個叢集的點」的凸包裡面——不是當前那一批，是歷史。",[15,2000,2001],{},"而且 Lloyd 其實是 e-means 的特例。批次設成全部 n、α 設成 1，一次 e-means 更新就等於一次 Lloyd。所以兩者不是兩個世界，是同一條公式的兩端——論文發現，那一端剛好是最差的參數組合。",[15,2003,2004],{},"還有一個實務上的小設計。訓練結束時，最後一批的分配可以拿來重算一次精確均值（last-epoch-mean）。這會讓訓練集損失再降一點。如果你在意的是新資料上的表現，就用另一組慢速 EMA 收尾。兩者成本都可忽略，等於一次訓練能產兩組質心，看你拿質心去服務誰。",[15,2006,2007,2008,2011],{},"最後是這篇最反直覺的地方——",[142,2009,2010],{},"雜訊有用","。",[15,2013,2014],{},"Lloyd 每輪的損失只會下降，所以它一旦走進一個局部最小值，就沒有動力出來。小批次更新每次只看一小部分，梯度有雜訊。論文自己的解釋是：正是這個雜訊，會把質心推離那些「看起來收斂了、其實分錯了」的配置。他們用 200 個種子在玩具資料上重跑，e-means 掉進病態配置的機率明顯較低。",[15,2016,2017],{},"如圖 2，這是我們自己跑的小型模擬，不是論文原圖。設定是三個高斯團、每團 40 點、固定亂數種子、k = 3、同一組初始中心（刻意放壞）。e-means 用批次 12、初始 α = 0.35 加 cosine 衰減、2.5 個 epoch；Lloyd 則是每輪全量。",[15,2019,2020],{},"左邊的 Lloyd 第 8 輪就幾乎不動，再跑到 33 輪仍停在 MSE 5.39。兩顆質心擠在同一團，另一顆落在兩團中間。右邊的 e-means 用 25 次更新收到 1.28，一顆對一團。",[149,2022,153,2024,153,2029],{"className":2023},[152],[155,2025],{"src":2026,"poster":2027,"autoPlay":159,"loop":159,"muted":159,"playsInline":159,"ariaLabel":2028},"\u002Fimages\u002Fblog\u002Fkmeans-fewer-epochs\u002Fcentroids.mp4","\u002Fimages\u002Fblog\u002Fkmeans-fewer-epochs\u002Fcentroids-poster.jpg","同一組資料、同一組初始中心：左邊 Lloyd 的兩顆質心擠在同一團，右邊 e-means 的三顆各對到一團",[162,2030,2031],{},"圖 2：同一組資料、同一組初始中心。左邊的兩顆擠在同一團，右邊的三顆各對到一團（我們自己的模擬）。",[15,2033,2034],{},"這個例子只說明「同一組初始值下，兩種更新方式會走到不同的地方」，不是保證。論文的結論是機率，不是每次都能逃出來。",[39,2036,2038],{"id":2037},"快-10-倍是在比什麼","「快 10 倍」是在比什麼",[15,2040,2041],{},"先講對照條件，這是全文最容易被誤讀的地方：",[15,2043,2044,2047],{},[142,2045,2046],{},"基準是「e-means 跑 1.4–2.6 輪」對「Lloyd 跑 25 輪」的損失。"," 不是「Lloyd 跑到完全收斂」。",[15,2049,2050],{},"換句話說，基準是「大家平常就跑 25 輪」的那個點。Lloyd 再跑下去損失還會降一點，差距會縮小。",[61,2052,2053,2066],{},[64,2054,2055],{},[67,2056,2057,2060,2063],{},[70,2058,2059],{},"項目",[70,2061,2062],{},"數字",[70,2064,2065],{},"條件與出處",[80,2067,2068,2083,2097,2111,2122],{},[67,2069,2070,2073,2076],{},[85,2071,2072],{},"追平 Lloyd 跑 25 輪所需輪數",[85,2074,2075],{},"1.4–2.6",[85,2077,2078,2082],{},[19,2079,2081],{"href":21,"rel":2080},[23],"論文摘要","；多個嵌入資料集",[67,2084,2085,2088,2091],{},[85,2086,2087],{},"GPU 加速",[85,2089,2090],{},"10–45×",[85,2092,2093,2094],{},"對 Flash-KMeans、H100；",[19,2095,2081],{"href":21,"rel":2096},[23],[67,2098,2099,2102,2105],{},[85,2100,2101],{},"CPU 加速",[85,2103,2104],{},"8–15×",[85,2106,2107,2108],{},"對 Faiss；",[19,2109,30],{"href":28,"rel":2110},[23],[67,2112,2113,2116,2119],{},[85,2114,2115],{},"最大規模",[85,2117,2118],{},"100 億點、1,600 萬叢集，2.5 輪",[85,2120,2121],{},"DINO10B：論文的 100 億筆 DINO 特徵資料集",[67,2123,2124,2127,2130],{},[85,2125,2126],{},"社群量到的 Lloyd 迭代次數",[85,2128,2129],{},"19–229 輪",[85,2131,2132,2137],{},[19,2133,2136],{"href":2134,"rel":2135},"https:\u002F\u002Fgithub.com\u002Fscikit-learn\u002Fscikit-learn\u002Fissues\u002F35383",[23],"scikit-learn issue #35383","；社群實測，不是論文",[15,2139,2140],{},"最後一列不是在說「25 輪不夠」。它是在說，Lloyd 的收斂輪數本來就浮動——社群那邊量到 19 到 229 輪。所以重點不是 25 這個數字，是「每一輪都是一次全量掃描」。25 只是預設值，不是物理常數。",[15,2142,2143,2144,2147],{},"還有一個陷阱：",[142,2145,2146],{},"輪數不等於時間。"," 如果只算資料掃過幾遍，2.5 輪對 25 輪大約是 10 倍。實際的 wall-clock 卻是 10–45 倍。我的理解是，差距有一部分來自小批次更容易吃滿 GPU；論文沒有把 10–45 倍拆到「哪一段省下來」這一層。",[15,2149,2150,2151,2154],{},"保守的講法：",[142,2152,2153],{},"大約 10 倍","。45 倍是特定設定、特定 GPU、對特定的對照實作。任何寫成「k-means 快 45 倍」的句子，都要帶著這三個條件。",[15,2156,2157],{},"順帶說明一個容易混掉的細節。摘要寫的是 GPU 10–45 倍（對 Flash-KMeans）。專案頁另外寫 CPU 8–15 倍（對 Faiss）。兩個數字量的對象不同，不要互換、也不要相加。",[15,2159,2160],{},"我認為 GPU 的倍數比 CPU 大，是因為差距有一部分來自「小批次能不能吃滿硬體」。GPU 核心多、記憶體頻寬高，同樣的批次更容易被排滿。CPU 那組（8–15 倍）反而對一般讀者更有參考價值——多數人手上沒有 H100。",[15,2162,2163,2164,2169,2170,2173,2174,2177],{},"它進正式版了嗎？進了。faiss ",[19,2165,2168],{"href":2166,"rel":2167},"https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Ffaiss\u002Freleases\u002Ftag\u002Fv1.16.0",[23],"1.16.0","（2026-11-10 發布）包含 ",[1067,2171,2172],{},"faiss.contrib.e_means","；PyPI 的 ",[1067,2175,2176],{},"faiss-cpu"," 也已經是 1.16.0。用法五行：",[2179,2180,2185],"pre",{"className":2181,"code":2182,"language":2183,"meta":2184,"style":2184},"language-python shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","from faiss.contrib.e_means import Emeans\n\nem = Emeans(d, k, nepochs=5)\ncentroids, labels = em.train(x)   # x 是 PyTorch tensor (n, d)\n","python","",[1067,2186,2187,2194,2200,2206],{"__ignoreMap":2184},[182,2188,2191],{"class":2189,"line":2190},"line",1,[182,2192,2193],{},"from faiss.contrib.e_means import Emeans\n",[182,2195,2197],{"class":2189,"line":2196},2,[182,2198,2199],{"emptyLinePlaceholder":159},"\n",[182,2201,2203],{"class":2189,"line":2202},3,[182,2204,2205],{},"em = Emeans(d, k, nepochs=5)\n",[182,2207,2209],{"class":2189,"line":2208},4,[182,2210,2211],{},"centroids, labels = em.train(x)   # x 是 PyTorch tensor (n, d)\n",[15,2213,2214],{},"還沒有的東西：獨立複現。論文很新（11 月 6 日），有釋出複現腳本與基準資料，但目前只有作者自己的數字。",[15,2216,2217],{},"另外它終究是隨機演算法：同一個資料集、不同的種子，收斂到的解會不一樣。論文的數字是多個資料集、多個設定的結果，不是每一次執行都會剛好落在 1.4 輪。",[39,2219,2220],{"id":2220},"所以你的索引會變快嗎",[15,2222,2223,2224,2227],{},"先說結論：",[142,2225,2226],{},"它加速的是索引建置，不是查詢。"," 如果你的問題是「檢索太慢」，這篇幫不上忙。那個要換 HNSW（一種不用先分群的圖索引）、量化，或換一顆更小的模型。",[15,2229,2230],{},"有感的條件大概長這樣：",[61,2232,2233,2243],{},[64,2234,2235],{},[67,2236,2237,2240],{},[70,2238,2239],{},"你的情況",[70,2241,2242],{},"建議",[80,2244,2245,2253,2261,2269,2277],{},[67,2246,2247,2250],{},[85,2248,2249],{},"幾千份文件的 RAG，重建只要幾秒",[85,2251,2252],{},"先別動，快 10 倍對你沒差",[67,2254,2255,2258],{},[85,2256,2257],{},"10 萬點以上、k 上千、每次重建要等",[85,2259,2260],{},"值得做一次 A\u002FB",[67,2262,2263,2266],{},[85,2264,2265],{},"重建頻繁（每天或每次換模型）",[85,2267,2268],{},"值得換，先看 CPU 路徑",[67,2270,2271,2274],{},[85,2272,2273],{},"用 HNSW、不需要粗量化",[85,2275,2276],{},"這一步不存在，跳過",[67,2278,2279,2282],{},[85,2280,2281],{},"想加速的是推論",[85,2283,2284],{},"換錯題目了",[15,2286,2287],{},"還有一個常被忽略的前提：GPU 那條路要 H100 或 H200 等級的卡。還要對應的 GPU 核心函式庫（CuTe 或 Triton 後端），以及對得上版本的 CUTLASS、QuACK。CPU 那條路只要 faiss 本身，反而是多數人實際能跑的。專案頁的 8–15 倍就是 CPU 對 Faiss。",[15,2289,2290],{},"代價也要寫清楚：新 API 要驗，換初始化之後 recall 要重測。近似解本來就不保證全域最優（Lloyd 也不保證）。快 10 倍換到的是重建時間，不是免費的品質。",[15,2292,2293],{},"要驗的話，準備一份查詢集，跟暴力搜尋（Flat）的 top-10 對答案算 recall@10。只看 MSE 會漏掉檢索品質這一塊。",[39,2295,2296],{"id":2296},"明天可以做的三件事",[15,2298,2299,2302],{},[142,2300,2301],{},"第一，量一次重建。"," 記五個數字：n、k、niter、wall-clock、最終 MSE。沒有這五個，任何「要不要換」都是猜的。",[2179,2304,2306],{"className":2181,"code":2305,"language":2183,"meta":2184,"style":2184},"import numpy as np, faiss, time\n\nx = np.ascontiguousarray(x, dtype='float32')\nflat = faiss.IndexFlatL2(x.shape[1])   # 只用來算最終損失\n\nt0 = time.perf_counter()\nkm = faiss.Kmeans(x.shape[1], k, niter=25)\nkm.train(x)\nsecs = time.perf_counter() - t0\n\nflat.add(km.centroids)\nD, _ = flat.search(x, 1)\nprint(f\"{secs:.1f}s  MSE={D.mean():.3f}\")\n",[1067,2307,2308,2313,2317,2322,2327,2332,2338,2344,2350,2356,2361,2367,2373],{"__ignoreMap":2184},[182,2309,2310],{"class":2189,"line":2190},[182,2311,2312],{},"import numpy as np, faiss, time\n",[182,2314,2315],{"class":2189,"line":2196},[182,2316,2199],{"emptyLinePlaceholder":159},[182,2318,2319],{"class":2189,"line":2202},[182,2320,2321],{},"x = np.ascontiguousarray(x, dtype='float32')\n",[182,2323,2324],{"class":2189,"line":2208},[182,2325,2326],{},"flat = faiss.IndexFlatL2(x.shape[1])   # 只用來算最終損失\n",[182,2328,2330],{"class":2189,"line":2329},5,[182,2331,2199],{"emptyLinePlaceholder":159},[182,2333,2335],{"class":2189,"line":2334},6,[182,2336,2337],{},"t0 = time.perf_counter()\n",[182,2339,2341],{"class":2189,"line":2340},7,[182,2342,2343],{},"km = faiss.Kmeans(x.shape[1], k, niter=25)\n",[182,2345,2347],{"class":2189,"line":2346},8,[182,2348,2349],{},"km.train(x)\n",[182,2351,2353],{"class":2189,"line":2352},9,[182,2354,2355],{},"secs = time.perf_counter() - t0\n",[182,2357,2359],{"class":2189,"line":2358},10,[182,2360,2199],{"emptyLinePlaceholder":159},[182,2362,2364],{"class":2189,"line":2363},11,[182,2365,2366],{},"flat.add(km.centroids)\n",[182,2368,2370],{"class":2189,"line":2369},12,[182,2371,2372],{},"D, _ = flat.search(x, 1)\n",[182,2374,2376],{"class":2189,"line":2375},13,[182,2377,2378],{},"print(f\"{secs:.1f}s  MSE={D.mean():.3f}\")\n",[15,2380,2381,2384],{},[142,2382,2383],{},"第二，把 niter 降下來看看。"," 這是最便宜的實驗：同一份資料，niter 跑 25、10、5，看 MSE 差多少。如果 10 輪的損失只差一點，你的預設值就多付了 1.5 倍的掃描。",[2179,2386,2388],{"className":2181,"code":2387,"language":2183,"meta":2184,"style":2184},"for niter in (25, 10, 5):\n    km = faiss.Kmeans(x.shape[1], k, niter=niter)\n    km.train(x)\n    flat.reset()\n    flat.add(km.centroids)\n    D, _ = flat.search(x, 1)\n    print(niter, f\"{D.mean():.3f}\")\n",[1067,2389,2390,2395,2400,2405,2410,2415,2420],{"__ignoreMap":2184},[182,2391,2392],{"class":2189,"line":2190},[182,2393,2394],{},"for niter in (25, 10, 5):\n",[182,2396,2397],{"class":2189,"line":2196},[182,2398,2399],{},"    km = faiss.Kmeans(x.shape[1], k, niter=niter)\n",[182,2401,2402],{"class":2189,"line":2202},[182,2403,2404],{},"    km.train(x)\n",[182,2406,2407],{"class":2189,"line":2208},[182,2408,2409],{},"    flat.reset()\n",[182,2411,2412],{"class":2189,"line":2329},[182,2413,2414],{},"    flat.add(km.centroids)\n",[182,2416,2417],{"class":2189,"line":2334},[182,2418,2419],{},"    D, _ = flat.search(x, 1)\n",[182,2421,2422],{"class":2189,"line":2340},[182,2423,2424],{},"    print(niter, f\"{D.mean():.3f}\")\n",[15,2426,2427,2430],{},[142,2428,2429],{},"第三，規模夠大才換 e-means。"," 用同一份資料、同一組初始化，比 wall-clock 與最終 MSE；有索引的話再比 recall@10。CPU 那條路不用 GPU，適合先做。",[15,2432,2433],{},"最後回到那個 25。它不是物理常數，是一個預設值。這篇的重點不是叫你換掉 k-means，而是提醒你：那個 25 可以量、可以調，也可以被換掉。而提案換掉它的，是 Faiss 自己的作者。",[39,2435,2436],{"id":2436},"延伸閱讀",[2438,2439,2440,2448,2455],"ul",{},[2441,2442,2443,2447],"li",{},[19,2444,2446],{"href":2445},"\u002Fblog\u002Frunning-is-not-done","跑得起來跟真的幹完活是四件事","：同一種拆法，拆的是「模型可用」的四層",[2441,2449,2450,2454],{},[19,2451,2453],{"href":2452},"\u002Fblog\u002Fquantization-kernel-dispatch","量化會不會加速，取決於你怎麼寫 kernel","：同一類「實作方式吃掉十倍差距」",[2441,2456,2457,2461],{},[19,2458,2460],{"href":2459},"\u002Fblog\u002Fstrata-125b-on-12gb","125B 模型跑在 12GB 顯卡","：地端部署的記憶體取捨",[2463,2464,2465],"style",{},"html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"title":2184,"searchDepth":2196,"depth":2196,"links":2467},[2468,2469,2470,2471,2472,2473,2474,2475],{"id":41,"depth":2196,"text":41},{"id":56,"depth":2196,"text":56},{"id":170,"depth":2196,"text":171},{"id":1148,"depth":2196,"text":1149},{"id":2037,"depth":2196,"text":2038},{"id":2220,"depth":2196,"text":2220},{"id":2296,"depth":2196,"text":2296},{"id":2436,"depth":2196,"text":2436},"\u002Fimages\u002Fblog\u002Fkmeans-fewer-epochs.svg","標題「k-means 的 25 遍，可以只讀 2 遍」的技術筆記封面","2026-11-14","Faiss 的 k-means 預設跑 25 輪、每輪把訓練向量完整掃一遍；Meta FAIR 的 e-means 改用指數移動平均加小批次更新，1.4–2.6 輪就追平同樣的損失。這篇講它換掉了哪幾件事、10 到 45 倍是在比什麼，以及你的索引什麼時候才真的會有感。",false,"md","zh-TW",{},"\u002Fimages\u002Fblog\u002Fkmeans-fewer-epochs.jpg","\u002Fblog\u002Fzh-tw\u002Fkmeans-fewer-epochs",{"title":6,"description":2479},"kmeans-fewer-epochs","blog\u002Fzh-TW\u002Fkmeans-fewer-epochs",[2490,2491,2492,2493],"clustering","faiss","vector-search","performance",null,"om7iB4cAehVtmN9CyzhnlvaE9_LZBmcSxRJuFmeOjmM",{"id":2497,"title":2498,"author":7,"body":2499,"cover":4730,"coverAlt":4731,"date":2478,"description":4732,"draft":2480,"extension":2481,"featured":2480,"lang":4733,"meta":4734,"navigation":159,"ogImage":4735,"path":4736,"seo":4737,"slug":2487,"stem":4738,"tags":4739,"translationKey":2487,"updated":2494,"__hash__":4740},"blog\u002Fblog\u002Fen\u002Fkmeans-fewer-epochs.md","k-means Reads Your Data 25 Times. Two Would Do.",{"type":9,"value":2500,"toc":4720},[2501,2531,2535,2538,2541,2544,2548,2551,2623,2629,2632,2643,2646,2650,2653,2656,2659,3406,3418,3421,3424,3484,3488,3496,3499,3504,4207,4215,4221,4227,4233,4295,4298,4309,4312,4315,4318,4324,4332,4335,4338,4349,4352,4356,4359,4365,4447,4450,4457,4464,4467,4480,4502,4505,4508,4512,4519,4522,4576,4579,4582,4585,4589,4595,4653,4659,4691,4697,4700,4704,4718],[12,2502,2503],{},[15,2504,2505,2506,2510,2511,2515,2516,2520,2521,2525,2526,2530],{},"Four kinds of source. The ",[19,2507,2509],{"href":21,"rel":2508},[23],"paper"," (arXiv 2611.09501, Nov 6, 2026; FAIR at Meta) supplies the headline numbers; the ",[19,2512,2514],{"href":28,"rel":2513},[23],"project page"," supplies the CPU range and the ablation; the ",[19,2517,2519],{"href":34,"rel":2518},[23],"faiss source"," and its ",[19,2522,2524],{"href":2166,"rel":2523},[23],"1.16.0 release"," supply the defaults and the shipping status; one ",[19,2527,2529],{"href":2134,"rel":2528},[23],"GitHub issue"," supplies the 19–229 iteration range, and it is community data, not the paper. The 3.1 GB \u002F 77 GB arithmetic and the Figure 2 simulation are ours. We did not rerun the paper's experiments.",[39,2532,2534],{"id":2533},"the-bill-you-already-paid","The bill you already paid",[15,2536,2537],{},"You are deduplicating a corpus, or building RAG (retrieval-augmented generation: search your own documents, then put the hits in the prompt). Files arrive, get chunked, get embedded. Then a scheduled job clusters the vectors and writes an index; only after that can queries run.",[15,2539,2540],{},"That last step is usually the slowest part of a rebuild, and it still runs the 1957 algorithm: by default, 25 passes over your data. Meta FAIR's e-means matches the same loss in about two, and it ships in faiss.",[15,2542,2543],{},"It speeds up the build, not the query, and it has a size threshold. This essay draws those lines, and ends with three things you can measure tomorrow.",[39,2545,2547],{"id":2546},"it-hides-inside-the-build","It hides inside the build",[15,2549,2550],{},"k-means is not exotic. It is the layer under all of this:",[61,2552,2553,2566],{},[64,2554,2555],{},[67,2556,2557,2560,2563],{},[70,2558,2559],{},"Where",[70,2561,2562],{},"What k-means does",[70,2564,2565],{},"When you pay",[80,2567,2568,2579,2590,2601,2612],{},[67,2569,2570,2573,2576],{},[85,2571,2572],{},"IVF indexes",[85,2574,2575],{},"coarse quantizer: assign each vector to one of k buckets",[85,2577,2578],{},"build, re-embed, new data",[67,2580,2581,2584,2587],{},[85,2582,2583],{},"Product quantization",[85,2585,2586],{},"one clustering per sub-space, after splitting the vector",[85,2588,2589],{},"same",[67,2591,2592,2595,2598],{},[85,2593,2594],{},"VQ-VAE, neural audio codecs",[85,2596,2597],{},"codebook learning (one reference vector per cluster)",[85,2599,2600],{},"training, dataset change",[67,2602,2603,2606,2609],{},[85,2604,2605],{},"Dedup, anomaly detection",[85,2607,2608],{},"cluster to find duplicates and outliers",[85,2610,2611],{},"ingest, cleanup",[67,2613,2614,2617,2620],{},[85,2615,2616],{},"Customer or image segmentation",[85,2618,2619],{},"the algorithm itself",[85,2621,2622],{},"every analysis",[15,2624,2625,2626],{},"The common shape: ",[142,2627,2628],{},"it runs offline, off the query path.",[15,2630,2631],{},"Figure 1 shows the life of an index. The left half is the build; the right half is the query. A query reads the index once. The build sweeps the vectors many times. The latency you feel every day is on the right; the time that actually disappears is on the left.",[149,2633,153,2635,153,2640],{"className":2634},[152],[155,2636],{"src":2637,"poster":2638,"autoPlay":159,"loop":159,"muted":159,"playsInline":159,"ariaLabel":2639},"\u002Fimages\u002Fblog\u002Fen\u002Fkmeans-fewer-epochs\u002Fpipeline.mp4","\u002Fimages\u002Fblog\u002Fen\u002Fkmeans-fewer-epochs\u002Fpipeline-poster.jpg","Index build pipeline: raw data, embeddings, k-means, index; only then a query. The k-means box is tagged 25 passes.",[162,2641,2642],{},"Figure 1: build on the left, query on the right; the k-means box carries 25 passes.",[15,2644,2645],{},"Some arithmetic. Take 1M vectors, 768 dimensions, fp32, k = 4,096: one pass is about 3.1 GB. Faiss's default 25 passes is roughly 77 GB of reads, before distance arithmetic and index writes. The numbers move with dimension, dtype and nredo (Faiss can rerun clustering and keep the best run), but the order of magnitude does not.",[39,2647,2649],{"id":2648},"still-lloyds-1957-algorithm","Still Lloyd's 1957 algorithm",[15,2651,2652],{},"Lloyd's loop is two steps: assign each point to its nearest centroid; move each centroid to the mean of its points. Repeat until nothing moves.",[15,2654,2655],{},"It converges, to a local minimum rather than a global one. That guarantee comes from a 1957 Bell Labs memo, formally published in 1982.",[15,2657,2658],{},"The objective:",[182,2660,2662],{"className":2661},[185],[182,2663,2665,2801],{"className":2664},[189],[182,2666,2668],{"className":2667},[193],[195,2669,2670],{"xmlns":197,"display":198},[200,2671,2672,2798],{},[203,2673,2674,2682,2684,2686,2688,2690,2692,2694,2700,2714,2746,2748,2750,2752,2754,2756,2758,2760,2762,2764,2774],{},[203,2675,2676,2678,2680],{},[208,2677,211],{"mathvariant":210},[208,2679,214],{"mathvariant":210},[208,2681,217],{"mathvariant":210},[219,2683,222],{"stretchy":221},[208,2685,225],{},[219,2687,229],{"separator":228},[208,2689,232],{},[219,2691,235],{"stretchy":221},[219,2693,238],{},[240,2695,2696,2698],{},[243,2697,245],{},[208,2699,248],{},[250,2701,2702,2704,2712],{},[219,2703,254],{},[203,2705,2706,2708,2710],{},[208,2707,259],{},[219,2709,238],{},[243,2711,245],{},[208,2713,248],{},[267,2715,2716,2744],{},[203,2717,2718,2720,2726,2728,2742],{},[219,2719,273],{"fence":228},[275,2721,2722,2724],{},[208,2723,279],{},[208,2725,259],{},[219,2727,284],{},[275,2729,2730,2732],{},[208,2731,289],{},[203,2733,2734,2736,2738,2740],{},[208,2735,19],{},[219,2737,222],{"stretchy":221},[208,2739,259],{},[219,2741,235],{"stretchy":221},[219,2743,273],{"fence":228},[243,2745,304],{},[219,2747,229],{"separator":228},[308,2749],{"width":310},[208,2751,19],{},[219,2753,222],{"stretchy":221},[208,2755,259],{},[219,2757,235],{"stretchy":221},[219,2759,238],{},[208,2761,323],{},[219,2763,326],{},[328,2765,2766,2772],{},[203,2767,2768,2770],{},[208,2769,334],{},[219,2771,326],{},[208,2773,339],{},[267,2775,2776,2796],{},[203,2777,2778,2780,2786,2788,2794],{},[219,2779,273],{"fence":228},[275,2781,2782,2784],{},[208,2783,279],{},[208,2785,259],{},[219,2787,284],{},[275,2789,2790,2792],{},[208,2791,289],{},[208,2793,339],{},[219,2795,273],{"fence":228},[243,2797,304],{},[365,2799,2800],{"encoding":367},"\\mathrm{MSE}(X, C) = \\frac{1}{n}\\sum_{i=1}^{n}\\left\\| x_i - c_{a(i)} \\right\\|^2,\n\\qquad a(i) = \\arg\\min_j \\left\\| x_i - c_j \\right\\|^2",[182,2802,2804,2843,3221],{"className":2803,"ariaHidden":228},[372],[182,2805,2807,2810,2816,2819,2822,2825,2828,2831,2834,2837,2840],{"className":2806},[376],[182,2808],{"className":2809,"style":381},[380],[182,2811,2813],{"className":2812},[385],[182,2814,390],{"className":2815},[385,389],[182,2817,222],{"className":2818},[394],[182,2820,225],{"className":2821,"style":399},[385,398],[182,2823,229],{"className":2824},[403],[182,2826],{"className":2827,"style":407},[308],[182,2829,232],{"className":2830,"style":411},[385,398],[182,2832,235],{"className":2833},[415],[182,2835],{"className":2836,"style":419},[308],[182,2838,238],{"className":2839},[423],[182,2841],{"className":2842,"style":419},[308],[182,2844,2846,2849,2911,2914,2981,2984,3188,3191,3194,3197,3200,3203,3206,3209,3212,3215,3218],{"className":2845},[376],[182,2847],{"className":2848,"style":433},[380],[182,2850,2852,2855,2908],{"className":2851},[385],[182,2853],{"className":2854},[394,440],[182,2856,2858],{"className":2857},[240],[182,2859,2861,2900],{"className":2860},[447,448],[182,2862,2864,2897],{"className":2863},[452],[182,2865,2867,2878,2886],{"className":2866,"style":457},[456],[182,2868,2869,2872],{"style":460},[182,2870],{"className":2871,"style":465},[464],[182,2873,2875],{"className":2874},[385],[182,2876,248],{"className":2877},[385,398],[182,2879,2880,2883],{"style":474},[182,2881],{"className":2882,"style":465},[464],[182,2884],{"className":2885,"style":482},[481],[182,2887,2888,2891],{"style":485},[182,2889],{"className":2890,"style":465},[464],[182,2892,2894],{"className":2893},[385],[182,2895,245],{"className":2896},[385],[182,2898,499],{"className":2899},[498],[182,2901,2903],{"className":2902},[452],[182,2904,2906],{"className":2905,"style":506},[456],[182,2907],{},[182,2909],{"className":2910},[415,440],[182,2912],{"className":2913,"style":407},[308],[182,2915,2917],{"className":2916},[518,519],[182,2918,2920,2973],{"className":2919},[447,448],[182,2921,2923,2970],{"className":2922},[452],[182,2924,2926,2946,2956],{"className":2925,"style":529},[456],[182,2927,2928,2931],{"style":532},[182,2929],{"className":2930,"style":536},[464],[182,2932,2934],{"className":2933},[540,541,542,543],[182,2935,2937,2940,2943],{"className":2936},[385,543],[182,2938,259],{"className":2939},[385,398,543],[182,2941,238],{"className":2942},[423,543],[182,2944,245],{"className":2945},[385,543],[182,2947,2948,2951],{"style":558},[182,2949],{"className":2950,"style":536},[464],[182,2952,2953],{},[182,2954,254],{"className":2955},[518,567,568],[182,2957,2958,2961],{"style":571},[182,2959],{"className":2960,"style":536},[464],[182,2962,2964],{"className":2963},[540,541,542,543],[182,2965,2967],{"className":2966},[385,543],[182,2968,248],{"className":2969},[385,398,543],[182,2971,499],{"className":2972},[498],[182,2974,2976],{"className":2975},[452],[182,2977,2979],{"className":2978,"style":593},[456],[182,2980],{},[182,2982],{"className":2983,"style":407},[308],[182,2985,2987,3165],{"className":2986},[602],[182,2988,2990,3027,3067,3070,3073,3076,3128],{"className":2989},[602],[182,2991,2993],{"className":2992},[394],[182,2994,2996],{"className":2995},[612,613],[182,2997,2999,3019],{"className":2998},[447,448],[182,3000,3002,3016],{"className":3001},[452],[182,3003,3005],{"className":3004,"style":623},[456],[182,3006,3007,3010],{"style":626},[182,3008],{"className":3009,"style":630},[464],[182,3011,3012],{"style":633},[635,3013,3014],{"xmlns":637,"width":638,"height":639,"viewBox":640},[642,3015],{"d":644},[182,3017,499],{"className":3018},[498],[182,3020,3022],{"className":3021},[452],[182,3023,3025],{"className":3024,"style":654},[456],[182,3026],{},[182,3028,3030,3033],{"className":3029},[385],[182,3031,279],{"className":3032},[385,398],[182,3034,3036],{"className":3035},[666],[182,3037,3039,3059],{"className":3038},[447,448],[182,3040,3042,3056],{"className":3041},[452],[182,3043,3045],{"className":3044,"style":676},[456],[182,3046,3047,3050],{"style":679},[182,3048],{"className":3049,"style":683},[464],[182,3051,3053],{"className":3052},[540,541,542,543],[182,3054,259],{"className":3055},[385,398,543],[182,3057,499],{"className":3058},[498],[182,3060,3062],{"className":3061},[452],[182,3063,3065],{"className":3064,"style":699},[456],[182,3066],{},[182,3068],{"className":3069,"style":705},[308],[182,3071,284],{"className":3072},[709],[182,3074],{"className":3075,"style":705},[308],[182,3077,3079,3082],{"className":3078},[385],[182,3080,289],{"className":3081},[385,398],[182,3083,3085],{"className":3084},[666],[182,3086,3088,3120],{"className":3087},[447,448],[182,3089,3091,3117],{"className":3090},[452],[182,3092,3094],{"className":3093,"style":731},[456],[182,3095,3096,3099],{"style":734},[182,3097],{"className":3098,"style":683},[464],[182,3100,3102],{"className":3101},[540,541,542,543],[182,3103,3105,3108,3111,3114],{"className":3104},[385,543],[182,3106,19],{"className":3107},[385,398,543],[182,3109,222],{"className":3110},[394,543],[182,3112,259],{"className":3113},[385,398,543],[182,3115,235],{"className":3116},[415,543],[182,3118,499],{"className":3119},[498],[182,3121,3123],{"className":3122},[452],[182,3124,3126],{"className":3125,"style":765},[456],[182,3127],{},[182,3129,3131],{"className":3130},[415],[182,3132,3134],{"className":3133},[612,613],[182,3135,3137,3157],{"className":3136},[447,448],[182,3138,3140,3154],{"className":3139},[452],[182,3141,3143],{"className":3142,"style":623},[456],[182,3144,3145,3148],{"style":626},[182,3146],{"className":3147,"style":630},[464],[182,3149,3150],{"style":633},[635,3151,3152],{"xmlns":637,"width":638,"height":639,"viewBox":640},[642,3153],{"d":644},[182,3155,499],{"className":3156},[498],[182,3158,3160],{"className":3159},[452],[182,3161,3163],{"className":3162,"style":654},[456],[182,3164],{},[182,3166,3168],{"className":3167},[666],[182,3169,3171],{"className":3170},[447],[182,3172,3174],{"className":3173},[452],[182,3175,3177],{"className":3176,"style":817},[456],[182,3178,3179,3182],{"style":820},[182,3180],{"className":3181,"style":683},[464],[182,3183,3185],{"className":3184},[540,541,542,543],[182,3186,304],{"className":3187},[385,543],[182,3189],{"className":3190,"style":407},[308],[182,3192,229],{"className":3193},[403],[182,3195],{"className":3196,"style":839},[308],[182,3198],{"className":3199,"style":407},[308],[182,3201,19],{"className":3202},[385,398],[182,3204,222],{"className":3205},[394],[182,3207,259],{"className":3208},[385,398],[182,3210,235],{"className":3211},[415],[182,3213],{"className":3214,"style":419},[308],[182,3216,238],{"className":3217},[423],[182,3219],{"className":3220,"style":419},[308],[182,3222,3224,3227,3232,3235,3279,3282],{"className":3223},[376],[182,3225],{"className":3226,"style":870},[380],[182,3228,874,3230],{"className":3229},[518],[182,3231,878],{"style":877},[182,3233],{"className":3234,"style":407},[308],[182,3236,3238],{"className":3237},[518,519],[182,3239,3241,3271],{"className":3240},[447,448],[182,3242,3244,3268],{"className":3243},[452],[182,3245,3247,3258],{"className":3246,"style":894},[456],[182,3248,3249,3252],{"style":897},[182,3250],{"className":3251,"style":465},[464],[182,3253,3255],{"className":3254},[540,541,542,543],[182,3256,339],{"className":3257,"style":907},[385,398,543],[182,3259,3260,3263],{"style":910},[182,3261],{"className":3262,"style":465},[464],[182,3264,3265],{},[182,3266,334],{"className":3267},[518],[182,3269,499],{"className":3270},[498],[182,3272,3274],{"className":3273},[452],[182,3275,3277],{"className":3276,"style":928},[456],[182,3278],{},[182,3280],{"className":3281,"style":407},[308],[182,3283,3285,3383],{"className":3284},[602],[182,3286,3288,3291,3331,3334,3337,3340,3380],{"className":3287},[602],[182,3289,273],{"className":3290,"style":944},[394,943],[182,3292,3294,3297],{"className":3293},[385],[182,3295,279],{"className":3296},[385,398],[182,3298,3300],{"className":3299},[666],[182,3301,3303,3323],{"className":3302},[447,448],[182,3304,3306,3320],{"className":3305},[452],[182,3307,3309],{"className":3308,"style":676},[456],[182,3310,3311,3314],{"style":679},[182,3312],{"className":3313,"style":683},[464],[182,3315,3317],{"className":3316},[540,541,542,543],[182,3318,259],{"className":3319},[385,398,543],[182,3321,499],{"className":3322},[498],[182,3324,3326],{"className":3325},[452],[182,3327,3329],{"className":3328,"style":699},[456],[182,3330],{},[182,3332],{"className":3333,"style":705},[308],[182,3335,284],{"className":3336},[709],[182,3338],{"className":3339,"style":705},[308],[182,3341,3343,3346],{"className":3342},[385],[182,3344,289],{"className":3345},[385,398],[182,3347,3349],{"className":3348},[666],[182,3350,3352,3372],{"className":3351},[447,448],[182,3353,3355,3369],{"className":3354},[452],[182,3356,3358],{"className":3357,"style":676},[456],[182,3359,3360,3363],{"style":679},[182,3361],{"className":3362,"style":683},[464],[182,3364,3366],{"className":3365},[540,541,542,543],[182,3367,339],{"className":3368,"style":907},[385,398,543],[182,3370,499],{"className":3371},[498],[182,3373,3375],{"className":3374},[452],[182,3376,3378],{"className":3377,"style":1032},[456],[182,3379],{},[182,3381,273],{"className":3382,"style":944},[415,943],[182,3384,3386],{"className":3385},[666],[182,3387,3389],{"className":3388},[447],[182,3390,3392],{"className":3391},[452],[182,3393,3395],{"className":3394,"style":1050},[456],[182,3396,3397,3400],{"style":1053},[182,3398],{"className":3399,"style":683},[464],[182,3401,3403],{"className":3402},[540,541,542,543],[182,3404,304],{"className":3405},[385,543],[15,3407,3408,3409,3411,3412,3414,3415,3417],{},"Faiss's ",[1067,3410,1069],{}," defaults to ",[1067,3413,1073],{},", and each iteration is a full assignment pass — one epoch in the usual sense: the data is seen once. This essay uses pass and epoch interchangeably. One more default gets less attention: ",[1067,3416,1077],{},". When n is much larger than k, Faiss subsamples to at most 256 points per centroid. The 1M-point, 4,096-centroid example sits just inside that threshold (256 × 4,096 ≈ 1.05M), so all of it is scanned.",[15,3419,3420],{},"Each pass costs O(n · k · d) and is memory-bound, random-access. Twenty-five passes are not a metaphor; they are 25 full sweeps of the array.",[15,3422,3423],{},"Why has it survived? Mini-batch versions date back to 1967 (MacQueen), and Sculley's web-scale version has been around since 2010. They read one batch per update, which is much faster, at the cost of a higher error and two knobs you must tune. For most teams, \"slow but no decisions\" is a reasonable default.",[61,3425,3426,3442],{},[64,3427,3428],{},[67,3429,3430,3433,3436,3439],{},[70,3431,3432],{},"Variant",[70,3434,3435],{},"Data per update",[70,3437,3438],{},"Statistics",[70,3440,3441],{},"Cost",[80,3443,3444,3458,3472],{},[67,3445,3446,3449,3452,3455],{},[85,3447,3448],{},"Lloyd (Faiss default)",[85,3450,3451],{},"all n points",[85,3453,3454],{},"exact mean, recomputed",[85,3456,3457],{},"slow; 25 passes",[67,3459,3460,3463,3466,3469],{},[85,3461,3462],{},"MacQueen \u002F Sculley",[85,3464,3465],{},"one mini-batch",[85,3467,3468],{},"cumulative, never forgotten",[85,3470,3471],{},"fast; early mistakes keep voting",[67,3473,3474,3476,3478,3481],{},[85,3475,1137],{},[85,3477,3465],{},[85,3479,3480],{},"exponential moving average",[85,3482,3483],{},"fast; parameters from a scaling law",[39,3485,3487],{"id":3486},"what-e-means-changed","What e-means changed",[15,3489,3490,3491,3495],{},"The paper's algorithm is e-means, from Meta FAIR. The title says it outright: ",[19,3492,3494],{"href":21,"rel":3493},[23],"\"one-tenth the epochs\"",". The author list includes Matthijs Douze and Hervé Jégou, the authors of faiss; Hugo Touvron is on it too. This is a team arguing against its own default.",[15,3497,3498],{},"It does not invent a new loss. The objective is the MSE above; what changes is how a centroid moves.",[15,3500,3501],{},[142,3502,3503],{},"One: the statistics become an exponential moving average.",[182,3505,3507],{"className":3506},[185],[182,3508,3510,3623],{"className":3509},[189],[182,3511,3513],{"className":3512},[193],[195,3514,3515],{"xmlns":197,"display":198},[200,3516,3517,3621],{},[203,3518,3519,3525,3527,3529,3531,3533,3535,3537,3543,3545,3547,3549,3555,3557,3559,3565,3567,3569,3571,3573,3575,3577,3583,3585,3587,3589,3595,3597,3599,3605,3607],{},[275,3520,3521,3523],{},[208,3522,1180],{},[208,3524,339],{},[219,3526,1185],{},[219,3528,222],{"stretchy":221},[243,3530,245],{},[219,3532,284],{},[208,3534,1194],{},[219,3536,235],{"stretchy":221},[275,3538,3539,3541],{},[208,3540,1180],{},[208,3542,339],{},[219,3544,1205],{},[208,3546,1194],{},[1209,3548,1211],{},[275,3550,3551,3553],{},[208,3552,248],{},[208,3554,339],{},[219,3556,229],{"separator":228},[308,3558],{"width":310},[275,3560,3561,3563],{},[208,3562,214],{},[208,3564,339],{},[219,3566,1185],{},[219,3568,222],{"stretchy":221},[243,3570,245],{},[219,3572,284],{},[208,3574,1194],{},[219,3576,235],{"stretchy":221},[275,3578,3579,3581],{},[208,3580,214],{},[208,3582,339],{},[219,3584,1205],{},[208,3586,1194],{},[1209,3588,1211],{},[275,3590,3591,3593],{},[208,3592,1256],{},[208,3594,339],{},[219,3596,229],{"separator":228},[308,3598],{"width":310},[275,3600,3601,3603],{},[208,3602,289],{},[208,3604,339],{},[219,3606,238],{},[240,3608,3609,3615],{},[275,3610,3611,3613],{},[208,3612,214],{},[208,3614,339],{},[275,3616,3617,3619],{},[208,3618,1180],{},[208,3620,339],{},[365,3622,1287],{"encoding":367},[182,3624,3626,3681,3702,3763,3873,3894,3955,4065],{"className":3625,"ariaHidden":228},[372],[182,3627,3629,3632,3672,3675,3678],{"className":3628},[376],[182,3630],{"className":3631,"style":1297},[380],[182,3633,3635,3638],{"className":3634},[385],[182,3636,1180],{"className":3637,"style":1304},[385,398],[182,3639,3641],{"className":3640},[666],[182,3642,3644,3664],{"className":3643},[447,448],[182,3645,3647,3661],{"className":3646},[452],[182,3648,3650],{"className":3649,"style":676},[456],[182,3651,3652,3655],{"style":1319},[182,3653],{"className":3654,"style":683},[464],[182,3656,3658],{"className":3657},[540,541,542,543],[182,3659,339],{"className":3660,"style":907},[385,398,543],[182,3662,499],{"className":3663},[498],[182,3665,3667],{"className":3666},[452],[182,3668,3670],{"className":3669,"style":1032},[456],[182,3671],{},[182,3673],{"className":3674,"style":419},[308],[182,3676,1185],{"className":3677},[423],[182,3679],{"className":3680,"style":419},[308],[182,3682,3684,3687,3690,3693,3696,3699],{"className":3683},[376],[182,3685],{"className":3686,"style":381},[380],[182,3688,222],{"className":3689},[394],[182,3691,245],{"className":3692},[385],[182,3694],{"className":3695,"style":705},[308],[182,3697,284],{"className":3698},[709],[182,3700],{"className":3701,"style":705},[308],[182,3703,3705,3708,3711,3714,3754,3757,3760],{"className":3704},[376],[182,3706],{"className":3707,"style":1376},[380],[182,3709,1194],{"className":3710,"style":1380},[385,398],[182,3712,235],{"className":3713},[415],[182,3715,3717,3720],{"className":3716},[385],[182,3718,1180],{"className":3719,"style":1304},[385,398],[182,3721,3723],{"className":3722},[666],[182,3724,3726,3746],{"className":3725},[447,448],[182,3727,3729,3743],{"className":3728},[452],[182,3730,3732],{"className":3731,"style":676},[456],[182,3733,3734,3737],{"style":1319},[182,3735],{"className":3736,"style":683},[464],[182,3738,3740],{"className":3739},[540,541,542,543],[182,3741,339],{"className":3742,"style":907},[385,398,543],[182,3744,499],{"className":3745},[498],[182,3747,3749],{"className":3748},[452],[182,3750,3752],{"className":3751,"style":1032},[456],[182,3753],{},[182,3755],{"className":3756,"style":705},[308],[182,3758,1205],{"className":3759},[709],[182,3761],{"className":3762,"style":705},[308],[182,3764,3766,3769,3772,3775,3815,3818,3821,3824,3864,3867,3870],{"className":3765},[376],[182,3767],{"className":3768,"style":1297},[380],[182,3770,1194],{"className":3771,"style":1380},[385,398],[182,3773],{"className":3774,"style":407},[308],[182,3776,3778,3781],{"className":3777},[385],[182,3779,248],{"className":3780},[385,398],[182,3782,3784],{"className":3783},[666],[182,3785,3787,3807],{"className":3786},[447,448],[182,3788,3790,3804],{"className":3789},[452],[182,3791,3793],{"className":3792,"style":676},[456],[182,3794,3795,3798],{"style":679},[182,3796],{"className":3797,"style":683},[464],[182,3799,3801],{"className":3800},[540,541,542,543],[182,3802,339],{"className":3803,"style":907},[385,398,543],[182,3805,499],{"className":3806},[498],[182,3808,3810],{"className":3809},[452],[182,3811,3813],{"className":3812,"style":1032},[456],[182,3814],{},[182,3816,229],{"className":3817},[403],[182,3819],{"className":3820,"style":839},[308],[182,3822],{"className":3823,"style":407},[308],[182,3825,3827,3830],{"className":3826},[385],[182,3828,214],{"className":3829,"style":1500},[385,398],[182,3831,3833],{"className":3832},[666],[182,3834,3836,3856],{"className":3835},[447,448],[182,3837,3839,3853],{"className":3838},[452],[182,3840,3842],{"className":3841,"style":676},[456],[182,3843,3844,3847],{"style":1515},[182,3845],{"className":3846,"style":683},[464],[182,3848,3850],{"className":3849},[540,541,542,543],[182,3851,339],{"className":3852,"style":907},[385,398,543],[182,3854,499],{"className":3855},[498],[182,3857,3859],{"className":3858},[452],[182,3860,3862],{"className":3861,"style":1032},[456],[182,3863],{},[182,3865],{"className":3866,"style":419},[308],[182,3868,1185],{"className":3869},[423],[182,3871],{"className":3872,"style":419},[308],[182,3874,3876,3879,3882,3885,3888,3891],{"className":3875},[376],[182,3877],{"className":3878,"style":381},[380],[182,3880,222],{"className":3881},[394],[182,3883,245],{"className":3884},[385],[182,3886],{"className":3887,"style":705},[308],[182,3889,284],{"className":3890},[709],[182,3892],{"className":3893,"style":705},[308],[182,3895,3897,3900,3903,3906,3946,3949,3952],{"className":3896},[376],[182,3898],{"className":3899,"style":1376},[380],[182,3901,1194],{"className":3902,"style":1380},[385,398],[182,3904,235],{"className":3905},[415],[182,3907,3909,3912],{"className":3908},[385],[182,3910,214],{"className":3911,"style":1500},[385,398],[182,3913,3915],{"className":3914},[666],[182,3916,3918,3938],{"className":3917},[447,448],[182,3919,3921,3935],{"className":3920},[452],[182,3922,3924],{"className":3923,"style":676},[456],[182,3925,3926,3929],{"style":1515},[182,3927],{"className":3928,"style":683},[464],[182,3930,3932],{"className":3931},[540,541,542,543],[182,3933,339],{"className":3934,"style":907},[385,398,543],[182,3936,499],{"className":3937},[498],[182,3939,3941],{"className":3940},[452],[182,3942,3944],{"className":3943,"style":1032},[456],[182,3945],{},[182,3947],{"className":3948,"style":705},[308],[182,3950,1205],{"className":3951},[709],[182,3953],{"className":3954,"style":705},[308],[182,3956,3958,3961,3964,3967,4007,4010,4013,4016,4056,4059,4062],{"className":3957},[376],[182,3959],{"className":3960,"style":1633},[380],[182,3962,1194],{"className":3963,"style":1380},[385,398],[182,3965],{"className":3966,"style":407},[308],[182,3968,3970,3973],{"className":3969},[385],[182,3971,1256],{"className":3972},[385,398],[182,3974,3976],{"className":3975},[666],[182,3977,3979,3999],{"className":3978},[447,448],[182,3980,3982,3996],{"className":3981},[452],[182,3983,3985],{"className":3984,"style":676},[456],[182,3986,3987,3990],{"style":679},[182,3988],{"className":3989,"style":683},[464],[182,3991,3993],{"className":3992},[540,541,542,543],[182,3994,339],{"className":3995,"style":907},[385,398,543],[182,3997,499],{"className":3998},[498],[182,4000,4002],{"className":4001},[452],[182,4003,4005],{"className":4004,"style":1032},[456],[182,4006],{},[182,4008,229],{"className":4009},[403],[182,4011],{"className":4012,"style":839},[308],[182,4014],{"className":4015,"style":407},[308],[182,4017,4019,4022],{"className":4018},[385],[182,4020,289],{"className":4021},[385,398],[182,4023,4025],{"className":4024},[666],[182,4026,4028,4048],{"className":4027},[447,448],[182,4029,4031,4045],{"className":4030},[452],[182,4032,4034],{"className":4033,"style":676},[456],[182,4035,4036,4039],{"style":679},[182,4037],{"className":4038,"style":683},[464],[182,4040,4042],{"className":4041},[540,541,542,543],[182,4043,339],{"className":4044,"style":907},[385,398,543],[182,4046,499],{"className":4047},[498],[182,4049,4051],{"className":4050},[452],[182,4052,4054],{"className":4053,"style":1032},[456],[182,4055],{},[182,4057],{"className":4058,"style":419},[308],[182,4060,238],{"className":4061},[423],[182,4063],{"className":4064,"style":419},[308],[182,4066,4068,4071],{"className":4067},[376],[182,4069],{"className":4070,"style":1744},[380],[182,4072,4074,4077,4204],{"className":4073},[385],[182,4075],{"className":4076},[394,440],[182,4078,4080],{"className":4079},[240],[182,4081,4083,4196],{"className":4082},[447,448],[182,4084,4086,4193],{"className":4085},[452],[182,4087,4089,4137,4145],{"className":4088,"style":1763},[456],[182,4090,4091,4094],{"style":460},[182,4092],{"className":4093,"style":465},[464],[182,4095,4097],{"className":4096},[385],[182,4098,4100,4103],{"className":4099},[385],[182,4101,1180],{"className":4102,"style":1304},[385,398],[182,4104,4106],{"className":4105},[666],[182,4107,4109,4129],{"className":4108},[447,448],[182,4110,4112,4126],{"className":4111},[452],[182,4113,4115],{"className":4114,"style":676},[456],[182,4116,4117,4120],{"style":1319},[182,4118],{"className":4119,"style":683},[464],[182,4121,4123],{"className":4122},[540,541,542,543],[182,4124,339],{"className":4125,"style":907},[385,398,543],[182,4127,499],{"className":4128},[498],[182,4130,4132],{"className":4131},[452],[182,4133,4135],{"className":4134,"style":1032},[456],[182,4136],{},[182,4138,4139,4142],{"style":474},[182,4140],{"className":4141,"style":465},[464],[182,4143],{"className":4144,"style":482},[481],[182,4146,4147,4150],{"style":485},[182,4148],{"className":4149,"style":465},[464],[182,4151,4153],{"className":4152},[385],[182,4154,4156,4159],{"className":4155},[385],[182,4157,214],{"className":4158,"style":1500},[385,398],[182,4160,4162],{"className":4161},[666],[182,4163,4165,4185],{"className":4164},[447,448],[182,4166,4168,4182],{"className":4167},[452],[182,4169,4171],{"className":4170,"style":676},[456],[182,4172,4173,4176],{"style":1515},[182,4174],{"className":4175,"style":683},[464],[182,4177,4179],{"className":4178},[540,541,542,543],[182,4180,339],{"className":4181,"style":907},[385,398,543],[182,4183,499],{"className":4184},[498],[182,4186,4188],{"className":4187},[452],[182,4189,4191],{"className":4190,"style":1032},[456],[182,4192],{},[182,4194,499],{"className":4195},[498],[182,4197,4199],{"className":4198},[452],[182,4200,4202],{"className":4201,"style":1877},[456],[182,4203],{},[182,4205],{"className":4206},[415,440],[15,4208,4209,4211,4212,4214],{},[1067,4210,1887],{}," and ",[1067,4213,1891],{}," are the count and sum the current batch contributes to cluster j. α sets how fast to forget: a large α listens to recent batches, a small one averages history. Old assignments are not deleted; they are discounted.",[15,4216,4217,4218,4220],{},"MacQueen's rule is ",[1067,4219,1898],{},", and it never forgets. Early in training the centroids are bad, so the labels are mostly wrong, and those wrong labels keep a full vote forever, dragging the centroid. e-means gives them a decay factor: slowly forget the early mistakes.",[15,4222,4223,4226],{},[142,4224,4225],{},"Two: α is not constant."," It starts larger and follows a cosine schedule down to a tenth of its initial value. Fast at first, slow at the end.",[15,4228,4229,4232],{},[142,4230,4231],{},"Three: dead clusters must be revived."," Mini-batch updates easily produce centroids with no points assigned. The cluster count says k; the usable count is lower. Every five batches, e-means reassigns clusters below 10% of the mean count by splitting a large donor cluster in two. Faiss already had this trick. The paper's ablation (turn one component off, see what the loss does) is unusually clean:",[61,4234,4235,4254],{},[64,4236,4237],{},[67,4238,4239,4242,4245,4248,4251],{},[70,4240,4241],{},"Dataset",[70,4243,4244],{},"Revival",[70,4246,4247],{},"Dead clusters",[70,4249,4250],{},"Of codebook",[70,4252,4253],{},"Test MSE",[80,4255,4256,4269,4282],{},[67,4257,4258,4260,4263,4265,4267],{},[85,4259,1939],{},[85,4261,4262],{},"split by size",[85,4264,1945],{},[85,4266,1948],{},[85,4268,1951],{},[67,4270,4271,4273,4276,4278,4280],{},[85,4272,1939],{},[85,4274,4275],{},"resample from batch",[85,4277,1961],{},[85,4279,1964],{},[85,4281,1967],{},[67,4283,4284,4286,4289,4291,4293],{},[85,4285,1939],{},[85,4287,4288],{},"off",[85,4290,1977],{},[85,4292,1980],{},[85,4294,1983],{},[15,4296,4297],{},"1M vectors, k = 16,384, 5 epochs, averaged over 5 seeds; SIFT behaves the same way. With revival off, a quarter of the codebook is empty. The other components (initialization, cosine, finalization) move the loss by a fraction of a percent. The four changes are not equal; this is the one that matters.",[15,4299,4300,4303,4304,4308],{},[142,4301,4302],{},"Four: you do not tune the parameters."," Batch size and initial step come from scaling laws fitted on (n, k, T); the reported regret is about ",[19,4305,4307],{"href":28,"rel":4306},[23],"0.07%",". There is a trap in \"no hyperparameters\": it means no new knobs relative to Lloyd. You still pick k and the number of epochs T (5 by default). The two hardest internal parameters are handled by a formula.",[15,4310,4311],{},"A design note: they do not use gradient descent. The project page says large learning rates make gradient updates overshoot. EMA keeps the centroid inside the convex hull of the points ever assigned to that cluster — not the current batch, the history.",[15,4313,4314],{},"Lloyd is a special case of e-means: set the batch to all n and α to 1, and one e-means update equals one Lloyd update. The two are ends of one formula, and the paper finds that end is the worst parameter setting.",[15,4316,4317],{},"One practical detail: at the end of training, the last batch's assignments can be used to recompute an exact mean (last-epoch-mean), lowering the train loss a little more. If what you care about is unseen data, finish with a second, slower EMA. Both cost little, so one run can produce two sets of centroids.",[15,4319,4320,4321],{},"Then the counterintuitive part: ",[142,4322,4323],{},"noise helps.",[15,4325,4326,4327,4331],{},"Lloyd's loss only decreases, so once it settles into a local minimum it has no reason to leave. A mini-batch sees a slice at a time, so its updates carry noise. The paper's explanation is that this noise pushes centroids out of configurations that look converged but are mispartitioned. Across ",[19,4328,4330],{"href":28,"rel":4329},[23],"200 seeds"," on toy data, e-means lands in pathological configurations far less often.",[15,4333,4334],{},"Figure 2 is our own small simulation, not the paper's figure: three Gaussian blobs of 40 points each, fixed seed, k = 3, and the same deliberately bad initial centroids. e-means uses batch 12, an initial α of 0.35 with cosine decay, and 2.5 epochs; Lloyd is full-batch.",[15,4336,4337],{},"Lloyd stops moving by pass 8 and is still at MSE 5.39 after 33 passes: two centroids in one blob, one between the other two. e-means reaches 1.28 in 25 updates, one centroid per blob.",[149,4339,153,4341,153,4346],{"className":4340},[152],[155,4342],{"src":4343,"poster":4344,"autoPlay":159,"loop":159,"muted":159,"playsInline":159,"ariaLabel":4345},"\u002Fimages\u002Fblog\u002Fen\u002Fkmeans-fewer-epochs\u002Fcentroids.mp4","\u002Fimages\u002Fblog\u002Fen\u002Fkmeans-fewer-epochs\u002Fcentroids-poster.jpg","Same data, same initial centroids: on the left, two Lloyd centroids share a blob; on the right, three e-means centroids each take one.",[162,4347,4348],{},"Figure 2: our simulation, settings above. Left: two centroids share a blob. Right: one each.",[15,4350,4351],{},"This example shows only that, from one initialization, two update rules end up in different places. It is not a guarantee. The paper's claim is probabilistic: not every run escapes.",[39,4353,4355],{"id":4354},"what-the-10-compares","What the \"10×\" compares",[15,4357,4358],{},"The comparison is the easiest thing to misread:",[15,4360,4361,4364],{},[142,4362,4363],{},"It is e-means at 1.4–2.6 epochs against Lloyd at 25 epochs. It is not Lloyd run to convergence."," Lloyd would keep improving a little; the gap would narrow.",[61,4366,4367,4380],{},[64,4368,4369],{},[67,4370,4371,4374,4377],{},[70,4372,4373],{},"Item",[70,4375,4376],{},"Number",[70,4378,4379],{},"Condition and source",[80,4381,4382,4396,4409,4422,4433],{},[67,4383,4384,4387,4389],{},[85,4385,4386],{},"Epochs to match Lloyd at 25",[85,4388,2075],{},[85,4390,4391,4395],{},[19,4392,4394],{"href":21,"rel":4393},[23],"abstract","; several embedding datasets",[67,4397,4398,4401,4403],{},[85,4399,4400],{},"GPU speedup",[85,4402,2090],{},[85,4404,4405,4406],{},"against Flash-KMeans, an alternative GPU k-means implementation, on H100; ",[19,4407,4394],{"href":21,"rel":4408},[23],[67,4410,4411,4414,4416],{},[85,4412,4413],{},"CPU speedup",[85,4415,2104],{},[85,4417,4418,4419],{},"against Faiss; ",[19,4420,2514],{"href":28,"rel":4421},[23],[67,4423,4424,4427,4430],{},[85,4425,4426],{},"Largest run",[85,4428,4429],{},"10B points, 16M clusters, 2.5 epochs",[85,4431,4432],{},"DINO10B, the paper's 10B DINO-feature dataset",[67,4434,4435,4438,4441],{},[85,4436,4437],{},"Lloyd iterations in the wild",[85,4439,4440],{},"19–229",[85,4442,4443,4446],{},[19,4444,2136],{"href":2134,"rel":4445},[23],"; community, not the paper",[15,4448,4449],{},"The last row is not an argument that 25 is too few. It says Lloyd's iteration count varies with the data. The point is not the number 25; it is that every pass is a full sweep. 25 is a default, not a law.",[15,4451,4452,4453,4456],{},"Another trap: ",[142,4454,4455],{},"epochs are not time."," Counting data sweeps alone, 2.5 against 25 is about 10×. The measured wall-clock is 10–45×. My reading is that part of the gap is mini-batches saturating a GPU better; the paper does not break the 10–45× down to that level.",[15,4458,4459,4460,4463],{},"The conservative statement: ",[142,4461,4462],{},"about 10×."," The 45× end is a specific setting, a specific GPU, against a specific reference implementation.",[15,4465,4466],{},"One detail worth separating: the abstract says 10–45× on GPU against Flash-KMeans. The project page reports 8–15× on CPU against Faiss. Different baselines; do not swap or add them.",[15,4468,4469,4470,4473,4474,4476,4477,4479],{},"It shipped: faiss ",[19,4471,2168],{"href":2166,"rel":4472},[23]," (Nov 10, 2026) includes ",[1067,4475,2172],{},", and ",[1067,4478,2176],{}," on PyPI is 1.16.0. Four lines:",[2179,4481,4483],{"className":2181,"code":4482,"language":2183,"meta":2184,"style":2184},"from faiss.contrib.e_means import Emeans\n\nem = Emeans(d, k, nepochs=5)\ncentroids, labels = em.train(x)   # x is a PyTorch tensor (n, d)\n",[1067,4484,4485,4489,4493,4497],{"__ignoreMap":2184},[182,4486,4487],{"class":2189,"line":2190},[182,4488,2193],{},[182,4490,4491],{"class":2189,"line":2196},[182,4492,2199],{"emptyLinePlaceholder":159},[182,4494,4495],{"class":2189,"line":2202},[182,4496,2205],{},[182,4498,4499],{"class":2189,"line":2208},[182,4500,4501],{},"centroids, labels = em.train(x)   # x is a PyTorch tensor (n, d)\n",[15,4503,4504],{},"What it does not have yet: independent replication. The paper is days old, with reproduction scripts and benchmark data, but only the authors' numbers.",[15,4506,4507],{},"It is also stochastic: different seeds land on different solutions. The reported numbers come from many datasets and settings, not a guarantee that every run finishes in 1.4 epochs.",[39,4509,4511],{"id":4510},"will-your-index-get-faster","Will your index get faster?",[15,4513,4514,4515,4518],{},"The headline: ",[142,4516,4517],{},"it speeds up index construction, not queries."," If your problem is \"search is slow,\" this is the wrong essay. That is HNSW (a graph index without clustering), quantization, or a smaller model.",[15,4520,4521],{},"Where it pays:",[61,4523,4524,4534],{},[64,4525,4526],{},[67,4527,4528,4531],{},[70,4529,4530],{},"Your situation",[70,4532,4533],{},"Advice",[80,4535,4536,4544,4552,4560,4568],{},[67,4537,4538,4541],{},[85,4539,4540],{},"A few thousand documents; rebuilds take seconds",[85,4542,4543],{},"leave it; 10× is invisible",[67,4545,4546,4549],{},[85,4547,4548],{},"100k+ vectors, k in the thousands, rebuilds hurt",[85,4550,4551],{},"worth an A\u002FB",[67,4553,4554,4557],{},[85,4555,4556],{},"Frequent rebuilds (daily, or every model swap)",[85,4558,4559],{},"worth switching; start with the CPU path",[67,4561,4562,4565],{},[85,4563,4564],{},"HNSW, no coarse quantizer",[85,4566,4567],{},"this step does not exist; skip",[67,4569,4570,4573],{},[85,4571,4572],{},"You want inference faster",[85,4574,4575],{},"wrong problem",[15,4577,4578],{},"One prerequisite is easy to miss: the GPU path wants an H100 or H200 and matching CUDA and kernel builds; the CPU path is the one most readers can actually run. The CPU path needs only faiss itself, and the 8–15× number is exactly that path.",[15,4580,4581],{},"The costs: a new API to validate, recall to re-measure after changing initialization. An approximate solution does not guarantee a global optimum (Lloyd does not either). The 10× buys build time, not free quality.",[15,4583,4584],{},"To check it: keep a query set and compare recall@10 — the share of queries whose top 10 contain a true nearest neighbour — against exact (Flat) search. MSE alone will miss retrieval quality.",[39,4586,4588],{"id":4587},"three-things-to-do-tomorrow","Three things to do tomorrow",[15,4590,4591,4594],{},[142,4592,4593],{},"First, measure one rebuild."," Record five numbers: n, k, niter, wall-clock, final MSE. Without them, \"switch or not\" is a guess.",[2179,4596,4598],{"className":2181,"code":4597,"language":2183,"meta":2184,"style":2184},"import numpy as np, faiss, time\n\nx = np.ascontiguousarray(x, dtype='float32')\nflat = faiss.IndexFlatL2(x.shape[1])   # only to score the final loss\n\nt0 = time.perf_counter()\nkm = faiss.Kmeans(x.shape[1], k, niter=25)\nkm.train(x)\nsecs = time.perf_counter() - t0\n\nflat.add(km.centroids)\nD, _ = flat.search(x, 1)\nprint(f\"{secs:.1f}s  MSE={D.mean():.3f}\")\n",[1067,4599,4600,4604,4608,4612,4617,4621,4625,4629,4633,4637,4641,4645,4649],{"__ignoreMap":2184},[182,4601,4602],{"class":2189,"line":2190},[182,4603,2312],{},[182,4605,4606],{"class":2189,"line":2196},[182,4607,2199],{"emptyLinePlaceholder":159},[182,4609,4610],{"class":2189,"line":2202},[182,4611,2321],{},[182,4613,4614],{"class":2189,"line":2208},[182,4615,4616],{},"flat = faiss.IndexFlatL2(x.shape[1])   # only to score the final loss\n",[182,4618,4619],{"class":2189,"line":2329},[182,4620,2199],{"emptyLinePlaceholder":159},[182,4622,4623],{"class":2189,"line":2334},[182,4624,2337],{},[182,4626,4627],{"class":2189,"line":2340},[182,4628,2343],{},[182,4630,4631],{"class":2189,"line":2346},[182,4632,2349],{},[182,4634,4635],{"class":2189,"line":2352},[182,4636,2355],{},[182,4638,4639],{"class":2189,"line":2358},[182,4640,2199],{"emptyLinePlaceholder":159},[182,4642,4643],{"class":2189,"line":2363},[182,4644,2366],{},[182,4646,4647],{"class":2189,"line":2369},[182,4648,2372],{},[182,4650,4651],{"class":2189,"line":2375},[182,4652,2378],{},[15,4654,4655,4658],{},[142,4656,4657],{},"Second, lower niter and look."," The cheapest experiment: the same data at niter 25, 10 and 5. If 10 is close enough, your default was paying 1.5× the scans for nothing.",[2179,4660,4661],{"className":2181,"code":2387,"language":2183,"meta":2184,"style":2184},[1067,4662,4663,4667,4671,4675,4679,4683,4687],{"__ignoreMap":2184},[182,4664,4665],{"class":2189,"line":2190},[182,4666,2394],{},[182,4668,4669],{"class":2189,"line":2196},[182,4670,2399],{},[182,4672,4673],{"class":2189,"line":2202},[182,4674,2404],{},[182,4676,4677],{"class":2189,"line":2208},[182,4678,2409],{},[182,4680,4681],{"class":2189,"line":2329},[182,4682,2414],{},[182,4684,4685],{"class":2189,"line":2334},[182,4686,2419],{},[182,4688,4689],{"class":2189,"line":2340},[182,4690,2424],{},[15,4692,4693,4696],{},[142,4694,4695],{},"Third, switch only at scale."," Use the same data and the same initialization; compare wall-clock, final MSE, and recall@10 if you have an index. The CPU path needs no GPU, so it is the right place to start.",[15,4698,4699],{},"The number 25 is not a physical constant. It is a default. The point is not to replace k-means; it is that 25 can be measured, tuned, or replaced -- and the people proposing to replace it are the authors of faiss.",[39,4701,4703],{"id":4702},"further-reading","Further reading",[2438,4705,4706,4712],{},[2441,4707,4708,4711],{},[19,4709,4710],{"href":2445},"A running model is not a finished job",": the same kind of teardown, applied to \"the model works\"",[2441,4713,4714,4717],{},[19,4715,4716],{"href":2452},"Quantization speed is often a kernel problem",": another case where implementation choices decide a 10× gap",[2463,4719,2465],{},{"title":2184,"searchDepth":2196,"depth":2196,"links":4721},[4722,4723,4724,4725,4726,4727,4728,4729],{"id":2533,"depth":2196,"text":2534},{"id":2546,"depth":2196,"text":2547},{"id":2648,"depth":2196,"text":2649},{"id":3486,"depth":2196,"text":3487},{"id":4354,"depth":2196,"text":4355},{"id":4510,"depth":2196,"text":4511},{"id":4587,"depth":2196,"text":4588},{"id":4702,"depth":2196,"text":4703},"\u002Fimages\u002Fblog\u002Fen\u002Fkmeans-fewer-epochs.svg","Cover for the English essay, k-means Reads Your Data 25 Times","Faiss's k-means defaults to 25 full passes over the training set; Meta FAIR's e-means matches that loss in 1.4–2.6 with EMA updates and dead-cluster revival. What the 10× really compares.","en",{},"\u002Fimages\u002Fblog\u002Fen\u002Fkmeans-fewer-epochs.jpg","\u002Fblog\u002Fen\u002Fkmeans-fewer-epochs",{"title":2498,"description":4732},"blog\u002Fen\u002Fkmeans-fewer-epochs",[2490,2491,2492,2493],"p0-rCBuSmcTSAtS0HEnwAqCdgrJz-0Ah0cXj_OTmTvA",{"id":4742,"company":4743,"extension":4754,"meta":4755,"services":4756,"social":4834,"stem":4835,"__hash__":4836},"site\u002Fsite.yml",{"name":4744,"nameEn":4745,"email":4746,"address":4747,"addressEn":4748,"lineUrl":4749,"lineId":4750,"duns":4751,"youtube":4752,"youtubeId":4753},"銡策數位有限公司","Jecision Digital Co., Ltd.","contact@jecision.com","231 新北市新店區北新路二段 262 號 3 樓之 3","3F.-3, No. 262, Sec. 2, Beixin Rd., Xindian Dist., New Taipei City 231, Taiwan","https:\u002F\u002Fline.me\u002FR\u002Fti\u002Fp\u002F@374qnumi","@374qnumi","963905705","https:\u002F\u002Fwww.youtube.com\u002Fshorts\u002F6UE0WTmyjsg","6UE0WTmyjsg","yml",{},[4757,4781,4807],{"key":4758,"zh":4759,"en":4770},"audit",{"title":4760,"tagline":4761,"summary":4762,"points":4763,"deliverable":4767,"detail":4768,"duration":4769},"企業 AI 導入健檢","先看清楚，再花錢","60 分鐘線上訪談，盤點最耗時、最依賴人力的營運環節；我們會告訴你哪些工作現在就能交給 AI、哪些還不成熟。",[4764,4765,4766],"現況盤點：梳理日常營運中最耗時、最依賴人力的環節","痛點診斷：判斷哪些工作現在就能交給 AI，哪些建議先保留人力","可行性評估：依你的業務特性與預算，篩選適合的模型與工具","《企業 AI 導入建議報告書》（PDF）","含 1–2 個「開發成本最低、成效最明顯」的優先項目、技術棧建議、預期節省的人力時間與下一步","訪談 60 分鐘，報告書 3 個工作天內交付",{"title":4771,"tagline":4772,"summary":4773,"points":4774,"deliverable":4778,"detail":4779,"duration":4780},"AI Readiness Audit","Look before you spend","A 60-minute remote audit of the workflows that eat the most time and people-power — and an honest answer on what AI can take over today, and what it cannot.",[4775,4776,4777],"Audit the daily workflows that consume the most time and headcount","Triage which tasks AI can own today, and which should stay human","Shortlist the models and tools that fit your budget and constraints","AI Adoption Recommendation Report (PDF)","One or two highest-ROI entry points, a recommended tech stack, expected hours saved, and the concrete next step","60-minute call, report delivered within 3 business days",{"key":4782,"zh":4783,"en":4794},"prompt",{"title":4784,"tagline":4785,"summary":4786,"points":4787,"deliverable":4791,"detail":4792,"duration":4793},"提示詞工程","讓 AI 穩定產出，而不是碰運氣","針對你的實際任務設計提示詞與多步驟工作流，並做極限測試，確保換了資料、換了人操作也不會走鐘。",[4788,4789,4790],"客製提示詞：ChatGPT、Claude、Gemini、Midjourney 等主流工具的專屬模板","除錯與優化：修掉幻覺、格式跑掉、時好時壞的問題","工作流建置：把複雜任務拆成多步驟提示詞與可重複的 SOP","提示詞文件或設定檔","附使用說明與範例，並提供 1 對 1 教學，確保你與團隊能自己維護","依任務複雜度報價，通常 3–7 個工作天",{"title":4795,"tagline":4796,"summary":4797,"points":4798,"deliverable":4804,"detail":4805,"duration":4806},"Prompt Engineering","Make AI produce reliably, not by luck","Prompt sets and multi-step workflows designed for your actual tasks, stress-tested so they keep working when the data or the operator changes.",[4799,4800,4802],"Custom prompts for ChatGPT, Claude, Gemini, Midjourney and friends",{"Debugging and tuning":4801},"kill hallucinations, broken formats and flaky output",{"Workflow design":4803},"break complex work into repeatable multi-step SOPs","Prompt library or exportable config","Ships with usage notes and examples, plus a 1:1 walkthrough so your team can maintain it","Quoted by complexity, typically 3–7 business days",{"key":4808,"zh":4809,"en":4820},"poc",{"title":4810,"tagline":4811,"summary":4812,"points":4813,"deliverable":4817,"detail":4818,"duration":4819},"極速 PoC","五天，看到能跑的東西","不談架構藍圖，直接做一個能動的雛形。五個工作天內交付線上可操作的智能體，用你真實的資料驗證。",[4814,4815,4816],"Day 1：15 分鐘對焦，鎖定單一痛點與資料盤點","Day 2–4：模型串接、邏輯設計與雛形開發","Day 5：Demo 展示與 AI 擴展建議書","可線上操作的智能體雛形","交付範圍為概念驗證與展示，不含後續維運或大規模伺服器部署；要進階成完整系統會另行報價","5 個工作天",{"title":4821,"tagline":4822,"summary":4823,"points":4824,"deliverable":4831,"detail":4832,"duration":4833},"Rapid PoC","Five days to something that runs","No architecture decks — a working prototype built in five business days, validated against your real data.",[4825,4827,4829],{"Day 1":4826},"15-minute scoping call, one pain point, data inventory",{"Day 2–4":4828},"model integration, logic design, prototype build",{"Day 5":4830},"live demo plus an AI expansion recommendation","A runnable online agent prototype","Proof of concept and demo scope only; ongoing operations and large-scale deployment are quoted separately","5 business days",[],"site","XTwARfVAOcrXNiKehsM6V6cdkd2HXy4d5yZVrSWrI3Q",[4838,5494],{"id":4839,"title":4840,"author":7,"body":4841,"cover":5478,"coverAlt":5479,"date":5480,"description":5481,"draft":2480,"extension":2481,"featured":2480,"lang":4733,"meta":5482,"navigation":159,"ogImage":5483,"path":5484,"seo":5485,"slug":5486,"stem":5487,"tags":5488,"translationKey":5486,"updated":2494,"__hash__":5493},"blog\u002Fblog\u002Fen\u002Fquantization-kernel-dispatch.md","Quantization Speed Is Often a Kernel Problem",{"type":9,"value":4842,"toc":5468},[4843,4848,4852,4855,4858,4861,4864,4868,4871,4874,4877,4880,4883,4886,4889,4894,4897,4908,4912,4915,4918,4980,4983,4986,4989,4992,4998,5004,5010,5013,5109,5112,5115,5118,5122,5125,5128,5131,5201,5204,5207,5210,5213,5216,5219,5223,5226,5229,5278,5281,5284,5287,5290,5294,5297,5303,5309,5315,5321,5382,5386,5406,5410,5413,5416,5430,5433,5441,5449,5452],[12,4844,4845],{},[15,4846,4847],{},"Every number here comes from arXiv 2605.05699v1 (May 7, 2026). The author is Mohamed Amine Bergach at Illumina; this is a single-author preprint, not peer-reviewed. We did not rerun it on an Apple M1. Where I add a judgement, I say so.",[39,4849,4851],{"id":4850},"a-scene","A scene",[15,4853,4854],{},"You are running a 1B model on a laptop; push the context from 256 to 4K and decoding slows as the KV cache (the K\u002FV state kept during decoding) grows with every token. This essay is about two ledgers hidden in the usual advice — \"quantize, but it slows you down\" — and when to keep an fp16 fallback.",[15,4856,4857],{},"The implementation tax: the same algorithm written as four eager dispatches or two fused kernels. The deployment bill: an int4 cache against fp16, end to end, including where it breaks. By the end you should know which number is being compared to what, and four questions to ask of any benchmark that claims quantization is faster.",[15,4859,4860],{},"The paper runs the same int4 KV-cache method both ways. The eager version is 17% slower on a 360M model, measured against SRHT (a sign-randomized Hadamard rotation used as the control). Fuse the four steps into two kernels and SmolLM comes out 1% faster; on a 1.7B model it is 5% faster. Qwen is still 1% slower. End to end against fp16, Gemma-3 1B at a 256-token prefix has 8.2% lower latency.",[15,4862,4863],{},"Then the warning. On Qwen2.5-1.5B at a 2K prefix, the int4 cache emits two tokens and stops. fp16 emits 64.",[39,4865,4867],{"id":4866},"first-the-17","First, the 17%",[15,4869,4870],{},"The method is rotate-then-quantize. Each K and V vector in the KV cache is rotated, compressed to int4 (a 4-bit integer), then dequantized and rotated back when attention reads it.",[15,4872,4873],{},"The rotation here is SRFT (sign-randomized FFT): multiply by a fixed random ±1 pattern, then take an FFT. The point is to spread out outliers. In a K\u002FV vector, a few large values stretch the quantization scale and squash the rest toward zero. Rotate first and the coordinates look close to Gaussian; uniform quantization no longer has to survive a handful of spikes.",[15,4875,4876],{},"The problem is how the four steps are written. The eager version calls rotate, quantize, dequantize and inverse-rotate as four separate PyTorch\u002FMPS operations. Every layer, every decoding step, pays four kernel dispatches.",[15,4878,4879],{},"Dispatch is the queueing and data movement around a GPU kernel launch. The arithmetic may take microseconds; four launches and four round trips through memory can cost more than the arithmetic itself.",[15,4881,4882],{},"The paper's measured tax: 17% higher latency on SmolLM2-360M, 16% on 1.7B, 12% on Qwen2.5-1.5B. The baseline is SRHT (sign-randomized Hadamard transform), another rotation, not fp16.",[15,4884,4885],{},"Why SRHT? Because SRFT and SRHT produce quantization quality within noise of each other; the difference is whether the rotation is an FFT or a Hadamard transform. Using SRHT as the control isolates the cost of the implementation from the cost of the rotation.",[15,4887,4888],{},"The paper puts the mechanism in one line, which I paraphrase:",[12,4890,4891],{},[15,4892,4893],{},"The cost is dispatch, not compute. The underlying rfft and irfft calls take single-digit microseconds, but the four-step pipeline multiplies that overhead.",[15,4895,4896],{},"That is why the same algorithm turns from +17% into −1% — not because quantization improved, but because four launches became two (Figure 1).",[149,4898,153,4900,153,4905],{"className":4899},[152],[155,4901],{"src":4902,"poster":4903,"autoPlay":159,"loop":159,"muted":159,"playsInline":159,"ariaLabel":4904},"\u002Fimages\u002Fblog\u002Fen\u002Fquantization-kernel-dispatch\u002Fkernel-fusion.mp4","\u002Fimages\u002Fblog\u002Fen\u002Fquantization-kernel-dispatch\u002Fkernel-fusion-poster.jpg","Left: four eager kernel dispatches. Right: two fused kernels, forward and inverse.",[162,4906,4907],{},"Figure 1: The same quantization, eager with four dispatches per layer per step; fused into a forward and an inverse kernel.",[39,4909,4911],{"id":4910},"fusing-four-steps-into-two-kernels-is-not-enough","Fusing four steps into two kernels is not enough",[15,4913,4914],{},"The fusion puts the first two steps and the last two into one Metal kernel each: forward does rotate plus quantize; inverse does dequantize plus inverse-rotate. Data stays in fast on-chip memory instead of round-tripping to device memory.",[15,4916,4917],{},"The same comparison flips for SmolLM, but Qwen stays 1% slower:",[61,4919,4920,4936],{},[64,4921,4922],{},[67,4923,4924,4927,4930,4933],{},[70,4925,4926],{},"Model",[70,4928,4929],{},"SRHT baseline",[70,4931,4932],{},"Eager Δ",[70,4934,4935],{},"Fused SRFT (Δ)",[80,4937,4938,4952,4966],{},[67,4939,4940,4943,4946,4949],{},[85,4941,4942],{},"SmolLM2-360M",[85,4944,4945],{},"37.4 ms\u002Ftok",[85,4947,4948],{},"+17%",[85,4950,4951],{},"37.0 ms\u002Ftok (−1%)",[67,4953,4954,4957,4960,4963],{},[85,4955,4956],{},"SmolLM2-1.7B",[85,4958,4959],{},"222.2 ms\u002Ftok",[85,4961,4962],{},"+16%",[85,4964,4965],{},"211.9 ms\u002Ftok (−5%)",[67,4967,4968,4971,4974,4977],{},[85,4969,4970],{},"Qwen2.5-1.5B",[85,4972,4973],{},"84.9 ms\u002Ftok",[85,4975,4976],{},"+12%",[85,4978,4979],{},"85.7 ms\u002Ftok (+1%)",[15,4981,4982],{},"(Δ against SRHT; percentages rounded; 256-token prompt plus 128 generated tokens; Apple M1. Not against fp16.)",[15,4984,4985],{},"Why is the gap larger on 1.7B? The eager penalty is roughly the same (+17% vs +16%). The difference appears after fusion: 1.7B has a 2048-dim KV per layer, 360M only 320, so the fused kernel's fixed launch cost is spread across more work. That is why it moves from −1% to −5%.",[15,4987,4988],{},"At this point the story looks clean: four steps become two kernels, and 17% slower becomes 1% faster (on 360M). But if you stop here and conclude \"quantization is always faster\", you will miss the next section.",[15,4990,4991],{},"The end-to-end −8.2% against fp16 is not fusion alone. It needs three conditions.",[15,4993,4994,4997],{},[142,4995,4996],{},"First, the quantizer."," With a single per-token scale (one scale per token), Qwen's 4-bit ΔPPL explodes to +7975. The author switches to per-channel λ plus per-group abs-max (one scale per channel, one abs-max per 32 values) and pulls the same number down to +638.6. That recipe runs inside the fused kernel.",[15,4999,5000,5003],{},[142,5001,5002],{},"Second, a dequant-prefix cache."," The dequantized prefix is cached between steps instead of recomputed; the last 16 tokens stay in fp16 and the cache is invalidated when that window fills. Without it, Qwen at a 1024-token prefix is 2.4% slower than fp16; with it, it is 1.6% faster. The paper's text says −0.7%; its own ms\u002Ftok, 137.1 → 134.9, implies −1.6%. We use the table's numbers and note the discrepancy.",[15,5005,5006,5009],{},[142,5007,5008],{},"Third, the model architecture."," Gemma-3 1B has sliding-window layers: most layers see a fixed window and stay in fp16; only a few full-attention layers (which see the whole prefix) pay the dequant cost. That is why its advantage survives to 4096 tokens. Qwen has 28 full-attention layers, so the cost grows linearly with prefix length.",[15,5011,5012],{},"Put those three back into the end-to-end table:",[61,5014,5015,5030],{},[64,5016,5017],{},[67,5018,5019,5021,5024,5027],{},[70,5020,4926],{},[70,5022,5023],{},"Prefix",[70,5025,5026],{},"fp16 → int4 (ms\u002Ftok)",[70,5028,5029],{},"Latency Δ",[80,5031,5032,5046,5059,5072,5085,5097],{},[67,5033,5034,5037,5040,5043],{},[85,5035,5036],{},"Gemma-3 1B",[85,5038,5039],{},"256",[85,5041,5042],{},"71.2 → 65.4",[85,5044,5045],{},"−8.2%",[67,5047,5048,5050,5053,5056],{},[85,5049,5036],{},[85,5051,5052],{},"1024",[85,5054,5055],{},"83.3 → 82.5",[85,5057,5058],{},"−0.9%",[67,5060,5061,5063,5066,5069],{},[85,5062,5036],{},[85,5064,5065],{},"2048",[85,5067,5068],{},"129.5 → 121.3",[85,5070,5071],{},"−6.3%",[67,5073,5074,5076,5079,5082],{},[85,5075,5036],{},[85,5077,5078],{},"4096",[85,5080,5081],{},"188.5 → 182.4",[85,5083,5084],{},"−3.2%",[67,5086,5087,5089,5091,5094],{},[85,5088,4970],{},[85,5090,5039],{},[85,5092,5093],{},"90.7 → 88.3",[85,5095,5096],{},"−2.6%",[67,5098,5099,5101,5103,5106],{},[85,5100,4970],{},[85,5102,5052],{},[85,5104,5105],{},"137.1 → 134.9",[85,5107,5108],{},"−1.6%",[15,5110,5111],{},"(Δ against fp16; Apple M1; 256-token prompt plus 128 generated tokens. Qwen at 2048 is not in the table; see the next section. Qwen 1024's −1.6% is the arithmetic from 137.1 → 134.9; the paper's text says −0.7%. We use the table's numbers.)",[15,5113,5114],{},"On memory: Qwen is 3.0–3.1×; Gemma's 19.5× is not a like-for-like comparison (an apples-to-apples comparison inside the full-attention layers is about 3.2×), so do not compare it directly with Qwen.",[15,5116,5117],{},"The −8.2% is the best case, at a 256-token prefix. The same model at 4096 is −3.2%; Qwen at 1024 is −1.6%. \"Quantization is 8% faster\" is not a constant; it is a curve that moves with prefix length.",[39,5119,5121],{"id":5120},"before-you-celebrate-check-the-quality-bill","Before you celebrate, check the quality bill",[15,5123,5124],{},"Speed is one ledger. Quality is the other.",[15,5126,5127],{},"Quantization quality is usually measured as perplexity (PPL). ΔPPL is the change from the fp16 baseline; lower is better.",[15,5129,5130],{},"The 4-bit per-token ΔPPL on Cosmopedia (8192 held-out tokens):",[61,5132,5133,5151],{},[64,5134,5135],{},[67,5136,5137,5139,5142,5145,5148],{},[70,5138,4926],{},[70,5140,5141],{},"FP16 PPL",[70,5143,5144],{},"No rotation (identity)",[70,5146,5147],{},"SRHT",[70,5149,5150],{},"SRFT",[80,5152,5153,5170,5186],{},[67,5154,5155,5158,5161,5164,5167],{},[85,5156,5157],{},"SmolLM2-135M",[85,5159,5160],{},"6.61",[85,5162,5163],{},"+1.96 ± 0.15",[85,5165,5166],{},"+0.34 ± 0.01",[85,5168,5169],{},"+0.38 ± 0.06",[67,5171,5172,5174,5177,5180,5183],{},[85,5173,4942],{},[85,5175,5176],{},"4.87",[85,5178,5179],{},"+0.97 ± 0.08",[85,5181,5182],{},"+0.20 ± 0.03",[85,5184,5185],{},"+0.21 ± 0.02",[67,5187,5188,5190,5193,5196,5199],{},[85,5189,4956],{},[85,5191,5192],{},"3.77",[85,5194,5195],{},"+1.04",[85,5197,5198],{},"+0.16",[85,5200,5198],{},[15,5202,5203],{},"(The first two rows are the mean ± 1σ over three seeds; 1.7B ran one seed because of compute cost. 6-bit and 8-bit are lossless in this test.)",[15,5205,5206],{},"Rotation is not optional. At 135M, no rotation is +1.96; with rotation it is +0.38, roughly a 5× reduction. That is also why \"quantization breaks\" and \"quantization is just slower\" are often two sides of the same problem: if the rotation is wrong, the 4-bit error lands directly in PPL.",[15,5208,5209],{},"6-bit and 8-bit are lossless; 4-bit is the trade-off. 3-bit is worse in absolute terms; the paper does not sell 3-bit, and I would not start there.",[15,5211,5212],{},"Qwen's 4-bit is a different story. The Python reference's per-token ΔPPL is +7975.3; the Metal per-token version is +7727.9. With per-channel λ plus per-group g32, the fused version lands at +638.6 — a 12.5× improvement, in the same range as the Python reference's +654.6.",[15,5214,5215],{},"But +638.6 is still +638.6. A 12.5× improvement is not the same as deployable. The paper calls this \"closing the catastrophe\"; I think it is closer to moving a fire from the whole house into one room. If you enable 4-bit on Qwen, validate with task metrics, not just the relative PPL gain.",[15,5217,5218],{},"Gemma looks better: per-token +18.7, fused +3.6. The Python reference is −5.2; the +8.8 gap comes from the kernel's fp32 reduction and rounding order, not the algorithm. The author rules that out with two ablations.",[39,5220,5222],{"id":5221},"it-really-breaks-qwen-at-2k-emits-two-tokens","It really breaks: Qwen at 2K emits two tokens",[15,5224,5225],{},"Back to the warning.",[15,5227,5228],{},"Qwen2.5-1.5B at a 2048-token prefix, int4, emits two tokens and hits EOS (end-of-sequence); fp16 emits 64.",[61,5230,5231,5243],{},[64,5232,5233],{},[67,5234,5235,5237,5240],{},[70,5236,4373],{},[70,5238,5239],{},"fp16",[70,5241,5242],{},"int4",[80,5244,5245,5256,5267],{},[67,5246,5247,5250,5253],{},[85,5248,5249],{},"Generated tokens (2048 prefix)",[85,5251,5252],{},"64",[85,5254,5255],{},"2 (EOS)",[67,5257,5258,5261,5264],{},[85,5259,5260],{},"Peak memory",[85,5262,5263],{},"4447 MB",[85,5265,5266],{},"5527 MB",[67,5268,5269,5272,5275],{},[85,5270,5271],{},"Author's fallback",[85,5273,5274],{},"—",[85,5276,5277],{},"fp16 above about 1.5K",[15,5279,5280],{},"There is more than one cause. The dequant-prefix cache plus int4 storage pushes peak memory from 4447 to 5527 MB (+24%); combined with quantization noise across 28 full-attention layers, the logits tip into an early EOS.",[15,5282,5283],{},"I think the design that makes 1024 faster is also part of what makes 2048 break. The prefix cache turns dequantization from \"slower as the prefix grows\" into a fixed cost, but it also occupies memory; on a full-attention stack like Qwen, both happen at once.",[15,5285,5286],{},"Why is Gemma fine? Its sliding-window layers stay in fp16; only a few full-attention layers pay the dequant cost. Longer prefixes grow the cost more slowly. Same cache, different architecture, different curve.",[15,5288,5289],{},"If you want to enable this cache on your own model, test 2K first. Not \"does it generate\", but \"is the output still correct, and does it stop early\". The author's fallback is explicit: above about 1.5K, go back to fp16.",[39,5291,5293],{"id":5292},"four-questions-for-any-quantization-benchmark","Four questions for any quantization benchmark",[15,5295,5296],{},"The most useful part of the paper is not \"Apple Silicon is fast\". It is a way to split a benchmark into questions.",[15,5298,5299,5302],{},[142,5300,5301],{},"First, the platform."," Unified memory or separate HBM? The paper says the conventional trade-off returns on GPUs with separate HBM and high dispatch overhead. The same kernel does not necessarily stay faster when you move it. Ask where the number was measured.",[15,5304,5305,5308],{},[142,5306,5307],{},"Second, the kernel."," Fused, or four separate steps? If the benchmark does not say, the number may be dominated by dispatch tax. Ask: how many dispatches per layer per step?",[15,5310,5311,5314],{},[142,5312,5313],{},"Third, the prefix and the architecture."," The −8% at 256 tokens is not the −3.2% at 4K; full-attention and sliding-window stacks curve differently. Ask for 256, 1K and 2K, not just the best chart.",[15,5316,5317,5320],{},[142,5318,5319],{},"Fourth, the quality."," What is the ΔPPL baseline, and which quantizer? +638.6 is better than +7975, but it is still +638.6. Per-token and per-channel+group are different things; do not compare them as if they were the same.",[61,5322,5323,5336],{},[64,5324,5325],{},[67,5326,5327,5330,5333],{},[70,5328,5329],{},"Question",[70,5331,5332],{},"Why it matters",[70,5334,5335],{},"How to check",[80,5337,5338,5349,5360,5371],{},[67,5339,5340,5343,5346],{},[85,5341,5342],{},"Unified memory or separate HBM?",[85,5344,5345],{},"Where the trade-off flips",[85,5347,5348],{},"Hardware spec; ask for per-platform numbers",[67,5350,5351,5354,5357],{},[85,5352,5353],{},"Fused kernel or four dispatches?",[85,5355,5356],{},"Dispatch tax can be 12–17% vs SRHT",[85,5358,5359],{},"Does the benchmark say \"4 dispatches\" or \"fused\"?",[67,5361,5362,5365,5368],{},[85,5363,5364],{},"Prefix length and attention pattern?",[85,5366,5367],{},"Long context may break",[85,5369,5370],{},"Ask for 256 \u002F 1K \u002F 2K; check full-attention vs sliding-window",[67,5372,5373,5376,5379],{},[85,5374,5375],{},"ΔPPL baseline and quantizer?",[85,5377,5378],{},"+638.6 is better than +7975 but still +638.6",[85,5380,5381],{},"Check the baseline; per-token or per-channel+group",[39,5383,5385],{"id":5384},"what-this-does-not-prove","What this does not prove",[2438,5387,5388,5391,5394,5397,5400,5403],{},[2441,5389,5390],{},"One platform: Apple M1 only; no discrete-HBM GPU.",[2441,5392,5393],{},"Small models: 135M–1.7B; nothing above 7B.",[2441,5395,5396],{},"A single-author preprint, not peer-reviewed; we did not reproduce it.",[2441,5398,5399],{},"Gemma's 19.5× is not like-for-like; an apples-to-apples comparison inside the full-attention layers is about 3.2×.",[2441,5401,5402],{},"Qwen at 2K breaks; the author's own fallback is 1.5K.",[2441,5404,5405],{},"This is not the only path. It only addresses the cost of writing the four steps separately.",[39,5407,5409],{"id":5408},"conclusion-the-question-is-not-is-quantization-fast","Conclusion: the question is not \"is quantization fast?\"",[15,5411,5412],{},"Quantization is not a switch. It is a stack: platform, kernel, quantizer, prefix length, architecture. Change one and the answer changes.",[15,5414,5415],{},"The two ledgers:",[2438,5417,5418,5424],{},[2441,5419,5420,5423],{},[142,5421,5422],{},"Implementation tax."," The same method, eager four steps vs two fused kernels: 11–21 percentage points.",[2441,5425,5426,5429],{},[142,5427,5428],{},"Deployment bill."," int4 cache vs fp16: up to −8.2% at a short prefix, single digits at long prefixes (Qwen 1024 is −1.6%, Gemma 4096 is −3.2%), and it can stop early.",[15,5431,5432],{},"One thing to do tomorrow: run your model at 256, 1K and 2K, record ms\u002Ftok and output quality. If it is a full-attention stack, keep an fp16 fallback above 1.5K; 2K is the test point.",[15,5434,5435,5436,5440],{},"If you want help running those three prefixes on your own model, our ",[19,5437,5439],{"href":5438},"\u002Fen\u002Fservices","AI readiness audit"," is exactly that: what can be handed to AI now, and what should wait.",[15,5442,5443,5444,5448],{},"The series so far: ",[19,5445,5447],{"href":5446},"\u002Fen\u002Fblog\u002Fbroken-on-arrival-quantized-models","broken-on-arrival"," on the recipe, and this essay on the implementation. The earlier memory-trade-off essay is available in Chinese. The question moves from \"should we quantize?\" to \"how is that kernel written?\"",[5450,5451],"hr",{},[15,5453,5454],{},[5455,5456,5457,5458,5463,5464,5467],"em",{},"Paper: ",[19,5459,5462],{"href":5460,"rel":5461},"https:\u002F\u002Farxiv.org\u002Fabs\u002F2605.05699",[23],"arXiv 2605.05699v1"," (Mohamed Amine Bergach, Illumina). Previous essay: ",[19,5465,5466],{"href":5446},"the quantized model may be broken",".",{"title":2184,"searchDepth":2196,"depth":2196,"links":5469},[5470,5471,5472,5473,5474,5475,5476,5477],{"id":4850,"depth":2196,"text":4851},{"id":4866,"depth":2196,"text":4867},{"id":4910,"depth":2196,"text":4911},{"id":5120,"depth":2196,"text":5121},{"id":5221,"depth":2196,"text":5222},{"id":5292,"depth":2196,"text":5293},{"id":5384,"depth":2196,"text":5385},{"id":5408,"depth":2196,"text":5409},"\u002Fimages\u002Fblog\u002Fen\u002Fquantization-kernel-dispatch.svg","Cover for the English essay, Quantization Speed Is Often a Kernel Problem","2026-11-05","Against SRHT, eager int4 KV-cache quantization ran 12–17% slower; fusing the four steps into Metal kernels cut most of the tax. The 8.2% win at 256 tokens has conditions—and Qwen breaks at 2K.",{},"\u002Fimages\u002Fblog\u002Fen\u002Fquantization-kernel-dispatch.jpg","\u002Fblog\u002Fen\u002Fquantization-kernel-dispatch",{"title":4840,"description":5481},"quantization-kernel-dispatch","blog\u002Fen\u002Fquantization-kernel-dispatch",[5489,5490,5491,5492],"quantization","kv-cache","apple-silicon","inference","VrMGy2tQ__ko07oo7IbI-0WNfg5nBCIyvoC2Snzani0",{"id":5495,"title":4710,"author":7,"body":5496,"cover":6133,"coverAlt":6134,"date":6135,"description":6136,"draft":2480,"extension":2481,"featured":2480,"lang":4733,"meta":6137,"navigation":159,"ogImage":6138,"path":6139,"seo":6140,"slug":6141,"stem":6142,"tags":6143,"translationKey":6141,"updated":2494,"__hash__":6148},"blog\u002Fblog\u002Fen\u002Frunning-is-not-done.md",{"type":9,"value":5497,"toc":6121},[5498,5503,5505,5508,5511,5514,5517,5525,5529,5556,5559,5564,5567,5628,5631,5634,5637,5640,5651,5655,5658,5661,5664,5667,5682,5685,5688,5691,5694,5697,5701,5704,5707,5710,5713,5716,5719,5722,5725,5728,5731,5748,5751,5754,5765,5769,5772,5775,5778,5781,5784,5787,5790,5793,5843,5846,5849,5852,5855,5858,5861,5864,5868,5871,5874,5877,5880,5883,5886,5889,5898,5901,5904,5908,5911,5922,5925,5928,5931,5934,5996,5999,6002,6005,6009,6023,6026,6029,6033,6036,6068,6071,6082,6090,6098,6105,6109,6112,6115,6118],[12,5499,5500],{},[15,5501,5502],{},"Every number here comes from arXiv 2610.34241v1, a single-author engineering report. It is not a controlled experiment. Where I add a judgement, I say so.",[39,5504,4851],{"id":4850},[15,5506,5507],{},"\"The model answered.\" That is the easiest sentence to fail an acceptance test with. The API returns 200, the prose reads well, and the tool call may never have left the building.",[15,5509,5510],{},"In July 2026, Vaibhav Jain pooled a few laptops to serve a model too large for any one machine. Chat worked; a coding agent did not. Every restart pushed a 32B checkpoint across Wi-Fi — 8.7 GB, roughly five minutes. He went single-node and noticed the real problem: \"the model is running\" is a very weak success criterion.",[15,5512,5513],{},"A checkpoint loads and answers. Then you ask it to inspect a file, call a tool, hold a long session, or preserve a repository contract. It may fail all of that, and your API still returns 200.",[15,5515,5516],{},"\"Compatible\" hides a lot. Behind one nominally compatible endpoint, tool-call formatting, turn control, context handling and cache reuse can all differ. Change one, and a workflow you verified on model A breaks on model B — usually without an error message.",[15,5518,5519,5524],{},[19,5520,5523],{"href":5521,"rel":5522},"https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.34241",[23],"arXiv 2610.34241"," covers July to October 2026 across vLLM, SGLang, llama.cpp and MLX. Its move is simple: split \"it runs\" into four levels, then run 45 trials with hidden checks to see what level four costs.",[39,5526,5528],{"id":5527},"four-levels-not-one-switch","Four levels, not one switch",[5530,5531,5532,5538,5544,5550],"ol",{},[2441,5533,5534,5537],{},[142,5535,5536],{},"Served"," — the checkpoint loads and returns something.",[2441,5539,5540,5543],{},[142,5541,5542],{},"Coherent"," — the text reads like language and answers the question.",[2441,5545,5546,5549],{},[142,5547,5548],{},"Parsed and executed tool calls"," — it says \"read this file\", the client receives it, and the read actually happens.",[2441,5551,5552,5555],{},[142,5553,5554],{},"Verified task completion"," — the code passes an independent behavioural check.",[15,5557,5558],{},"The paper's framing sentence is the spine of everything below:",[12,5560,5561],{},[15,5562,5563],{},"An endpoint can satisfy the first three and fail the fourth completely. And if it cannot satisfy the third, you cannot measure the fourth at all.",[15,5565,5566],{},"(My translation. The emphasis is the author's.)",[61,5568,5569,5582],{},[64,5570,5571],{},[67,5572,5573,5576,5579],{},[70,5574,5575],{},"Level",[70,5577,5578],{},"The question you ask",[70,5580,5581],{},"What passing it still doesn't tell you",[80,5583,5584,5595,5606,5617],{},[67,5585,5586,5589,5592],{},[85,5587,5588],{},"1 Served",[85,5590,5591],{},"Does it load and answer?",[85,5593,5594],{},"Whether the answer is language",[67,5596,5597,5600,5603],{},[85,5598,5599],{},"2 Coherent",[85,5601,5602],{},"Does it read and reason?",[85,5604,5605],{},"Whether it will act",[67,5607,5608,5611,5614],{},[85,5609,5610],{},"3 Tools",[85,5612,5613],{},"Are tool calls parsed and executed?",[85,5615,5616],{},"Whether the result is right",[67,5618,5619,5622,5625],{},[85,5620,5621],{},"4 Verified",[85,5623,5624],{},"Does the output pass an independent check?",[85,5626,5627],{},"This is \"done\"",[15,5629,5630],{},"These are not degrees of the same thing. They are four different gates. Level one passing says nothing about level two; level two passing says nothing about whether the model will pick up a tool. Most people stop at level two and think they are close, because the model sounds convincing.",[15,5632,5633],{},"The three tiers on an acceptance sheet map roughly like this: \"it responds\" is levels 1 and 2; \"the code runs\" is level 3; \"the tests pass\" is level 4. I keep 1 and 2 separate because their failures look similar and have different causes — one service never came up, the other came up and only talks.",[15,5635,5636],{},"And \"the tests pass\" depends on whose tests. If the vendor wrote them, you may still be at level three. Level four needs a check written by someone else and frozen before acceptance.",[15,5638,5639],{},"(Figure 1)",[149,5641,153,5643,153,5648],{"className":5642},[152],[155,5644],{"src":5645,"poster":5646,"autoPlay":159,"loop":159,"muted":159,"playsInline":159,"ariaLabel":5647},"\u002Fimages\u002Fblog\u002Fen\u002Frunning-is-not-done\u002Flevels.mp4","\u002Fimages\u002Fblog\u002Fen\u002Frunning-is-not-done\u002Flevels-poster.jpg","Four levels of progress: served, coherent, tools, verified, with a break before verified",[162,5649,5650],{},"Figure 1: four different gates, not four degrees. The first three can pass and the fourth still fail.",[39,5652,5654],{"id":5653},"same-api-different-protocol","Same API, different protocol",[15,5656,5657],{},"He pointed Claude Code at a gateway he wrote, with different models behind it. The expectation was \"swap an endpoint\". The experience was closer to handing the work to a person with different habits.",[15,5659,5660],{},"Some models follow the tool contract. Some narrate what they intend to do instead of calling the tool. Some rewrite an interface they find awkward, including a function signature that was supposed to stay stable. None of this is stupidity; it is a different understanding of what a tool is.",[15,5662,5663],{},"The logs make the gap concrete. Of 492 Gemma 4 requests that carried tools, 294 produced parsed tool_calls. Another 144 ended with stop and no tool call; 54 had no saved parsed response.",[15,5665,5666],{},"Those 144 are not all failures. A normal finish is also a stop, and no tool call may simply mean the task was done. That is the first trap here: a missing tool call is not an error signal by itself. You have to know whether one was expected.",[15,5668,5669,5670,5673,5674,5677,5678,5681],{},"Some failures are blunter. Eight saved Gemma 4 outputs arrived with a malformed ",[1067,5671,5672],{},"\u003C|tool_call>"," prefix and never parsed. On GPT-OSS-120B over SGLang, ",[1067,5675,5676],{},"tool_choice=required"," fought the model's own Harmony tool format and had to be downgraded to ",[1067,5679,5680],{},"auto","; tool rounds also needed a higher output allowance, or reasoning and tool JSON were truncated together.",[15,5683,5684],{},"Turn control is the other half: a missing session identifier, a tool loop, a thinking channel that does not line up. Each is enough to make a conversation look alive while it is lost.",[15,5686,5687],{},"The episode worth memorising: every API schema check passed, and the real Claude Code client still broke. In one case the working directory never reached the model-facing request, so the model guessed paths. In another, streamed tool-argument fragments were treated as replacements instead of appends, cutting the arguments mid-call.",[15,5689,5690],{},"Neither shows up in a unit test. Both require the actual client, end to end.",[15,5692,5693],{},"His diagnostic question is the useful part: when a tool fails, did the model fail, or did something in the middle drop the call? If it is the latter, fix the source — template, parser rules, serve flags, request translation — rather than patching the gateway forever. He reduced the gateway to a thin translator doing protocol and security checks, and made upstream failures surface as errors instead of a stream that ends normally with nothing in it.",[15,5695,5696],{},"\"Configured\" is not \"served and evaluated\". When a vendor says \"supported\", it often means \"it can be configured\".",[39,5698,5700],{"id":5699},"the-conversation-looks-continuous-it-isnt","The conversation looks continuous. It isn't.",[15,5702,5703],{},"To you it is one continuous thread. To the runtime it is three things: a token prefix, a KV cache, and an execution slot. Your \"continuous\" is not necessarily its continuous.",[15,5705,5706],{},"He kept 4,165 gateway turns and 623 debug jobs. Eighty-five cold-prefix jobs — no reuse at all, 30,000 to 36,000 input tokens — had a median time to first token of 69.4 seconds. Two hundred and eighty high-reuse jobs — at least 90% reused, under 2,000 fresh tokens — had a median of 2.8 seconds.",[15,5708,5709],{},"About 25x.",[15,5711,5712],{},"The author labels this observational: the two groups are composed differently, not randomised, not matched. You cannot write \"caching makes it 25x faster\". What you can say is that prefix reuse is associated with a difference worth engineering.",[15,5714,5715],{},"The mechanism is mundane. An inference backend can reuse KV state for an identical token prefix. Change the prefix — a timestamp, a system banner, tool definitions in a different order, a different JSON serialisation — and the whole context is prefilled again.",[15,5717,5718],{},"You changed a banner. You paid for the whole context.",[15,5720,5721],{},"Auxiliary requests are the pettier version. A coding agent also generates titles and permission prompts; with one reusable execution slot, an auxiliary request can evict the main conversation's resident state. In his records, 65 requests were classified as auxiliary and 44 showed no cache reuse. Among high-reuse records, auxiliary requests had a median TTFT of about 12.2 seconds (13 observations) against 2.7 seconds for main requests (267 observations).",[15,5723,5724],{},"Compaction has the same smell. He lowered the threshold deliberately: in 12 tests, sub-agent compaction fired six times, none of those six tasks completed, and one was still labelled PASS.",[15,5726,5727],{},"\"Compaction happened\" is not \"work finished\". In a log, the two look alike.",[15,5729,5730],{},"So he treats session continuity as an application responsibility, not a backend setting:",[2438,5732,5733,5736,5739,5742,5745],{},[2441,5734,5735],{},"Keep the message sequence append-only unless earlier content genuinely must change.",[2441,5737,5738],{},"Do not re-serialise tool messages for cosmetics.",[2441,5740,5741],{},"Make streamed and final tool JSON agree.",[2441,5743,5744],{},"Keep per-turn metadata out of the frozen system prefix.",[2441,5746,5747],{},"Preserve session affinity. A stable prompt is not enough if the next turn lands on a slot without the cache.",[15,5749,5750],{},"Caching is not a backend flag. It is application discipline.",[15,5752,5753],{},"(Figure 2)",[149,5755,153,5757,153,5762],{"className":5756},[152],[155,5758],{"src":5759,"poster":5760,"autoPlay":159,"loop":159,"muted":159,"playsInline":159,"ariaLabel":5761},"\u002Fimages\u002Fblog\u002Fen\u002Frunning-is-not-done\u002Fcache.mp4","\u002Fimages\u002Fblog\u002Fen\u002Frunning-is-not-done\u002Fcache-poster.jpg","Two paths for one conversation: a stable prefix reuses the cache, a changed prefix refills it",[162,5763,5764],{},"Figure 2: one conversation. The prefix holds and the cache is reused; the prefix changes and the whole context is prefilled again.",[39,5766,5768],{"id":5767},"level-four-is-where-the-work-is-judged","Level four is where the work is judged",[15,5770,5771],{},"Deployment, protocol and sessions are now good enough to run bounded tasks. Can an open-weight model produce code that passes?",[15,5773,5774],{},"Five TypeScript task categories (quote, prorate, customer, split, parts), three conditions, three runs each — 45 sessions. The parent is fixed: Claude Code with Claude Opus 5.5. The worker is a quantized Qwen3-Coder 30B-A3B. Every session is judged by task-specific hidden behavioural checks, not by the agent's own claim of success.",[15,5776,5777],{},"The conditions: direct; optimised delegation, where the parent specifies, reviews and applies but does not implement; and batch-labelled delegation that may split work into sequential packages — though only a minority actually split.",[15,5779,5780],{},"Results: direct 15\u002F15; optimised delegation 12\u002F15; batch-labelled 14\u002F15.",[15,5782,5783],{},"These do not rank. Each cell has three runs, the Wilson intervals overlap, and the author says the data cannot establish general success-rate differences. It does show workers produced usable code: of 30 delegated trials, 26 verified and all 26 retained worker-produced implementations. Not every time.",[15,5785,5786],{},"And the 14\u002F15 is not evidence that batching helps. Only 5 of the 15 batch-labelled trials used a batch call; the rest used single delegation. The paper calls it an adaptive policy, not a batch-versus-single comparison.",[15,5788,5789],{},"The second suite is stricter: 15 tasks, three runs each, 45 sessions per arm, artifacts clearing a fixed interface contract before parent review.",[15,5791,5792],{},"Direct 45\u002F45. Nemotron Cascade 2 worker: 12\u002F45, of which 11 were genuine worker substitutions and one verified only after the parent rewrote the source. Qwen3-Coder worker: 0\u002F45.",[61,5794,5795,5808],{},[64,5796,5797],{},[67,5798,5799,5802,5805],{},[70,5800,5801],{},"Study",[70,5803,5804],{},"Direct",[70,5806,5807],{},"Delegated",[80,5809,5810,5821,5832],{},[67,5811,5812,5815,5818],{},[85,5813,5814],{},"45-trial suite",[85,5816,5817],{},"15\u002F15",[85,5819,5820],{},"Qwen3-Coder 12\u002F15 (optimised), 14\u002F15 (batch-labelled)",[67,5822,5823,5826,5829],{},[85,5824,5825],{},"Second suite (15 tasks x 3)",[85,5827,5828],{},"45\u002F45",[85,5830,5831],{},"Nemotron 12\u002F45 (11 worker substitutions); Qwen 0\u002F45",[67,5833,5834,5837,5840],{},[85,5835,5836],{},"Same, counting interruptions as failures",[85,5838,5839],{},"37\u002F45",[85,5841,5842],{},"Nemotron 8\u002F45",[15,5844,5845],{},"That second suite also has a disclosure: 16 sessions were stopped by a subscription limit before any model call and were replaced. Eight were in the direct arm and eight in the Nemotron arm. Counting the original 16 as failures gives 37\u002F45 and 8\u002F45. Those are scheduling interruptions, not inference failures — but if you cite the headline numbers, cite both.",[15,5847,5848],{},"The 45\u002F45 deserves a caveat too. The tasks were built for this study, five categories with three runs each; there is no difficulty stratification.",[15,5850,5851],{},"One more honest detail: the task definitions and hidden tests were produced with AI assistance. The combined test file was created about 39 seconds before the first evaluated session and first committed after the last of the 45. What he can claim is narrower: task instructions and visible checks were identical across the 45 saved runs, and recorded score labels agree with the scorer as later saved. He cannot claim an independently snapshotted benchmark.",[15,5853,5854],{},"The first study's harness had limits: the parent's separate constraints field was not forwarded to the worker; the gateway dropped a duplicate system message and truncated messages over 1,800 characters; two of 41 reconstructed worker prompts lost about 220 characters, both in failed trials.",[15,5856,5857],{},"So \"the worker ignored instructions\" needs a discount. Some failures were capability; some were a worker that never received the full instruction.",[15,5859,5860],{},"What about 0\u002F45? It does not mean the model can do nothing. That arm had no direct control, parent acceptance and application were not replayed, and the parent's prompt-cache state was not reset between studies. The author scopes it to these tasks, this contract, this worker configuration.",[15,5862,5863],{},"Verification is level four. It is not a by-product of level three.",[39,5865,5867],{"id":5866},"the-bill-runs-the-other-way","The bill runs the other way",[15,5869,5870],{},"Frontier output tokens are expensive; a local worker is cheap. Delegate and save money — except delegation is not free. The parent writes the spec, reads what came back, decides whether to accept it, and cleans up failures. All of that is frontier tokens.",[15,5872,5873],{},"The cost model has five terms: parent context, specification, review, recovery, worker. The study measured the frontier terms. Electricity, hardware and engineering time were not measured — and unmeasured is not zero.",[15,5875,5876],{},"First suite, frontier cost per verified success: $0.136 direct; $0.214 optimised delegation; $0.199 batch-labelled. About 1.6x.",[15,5878,5879],{},"Second suite: $0.153 direct; $1.313 for Nemotron delegation. 8.6x.",[15,5881,5882],{},"Qwen's per-success cost is undefined, not zero — it had no successes.",[15,5884,5885],{},"One small case is worth keeping: a fee task where the worker's implementation passed and was applied, yet the frontier cost was $0.1856 versus $0.1293 direct, and wall time 126.4 seconds versus 27.4.",[15,5887,5888],{},"A successful substitution is not a saving.",[15,5890,5891,5892,5897],{},"This does not generalise to \"delegation never pays\". The author cites ",[19,5893,5896],{"href":5894,"rel":5895},"https:\u002F\u002Farxiv.org\u002Fabs\u002F2608.04804",[23],"Scrouting",": a small model scouts the repository, frontier models fix, and it solved 159 of 266 tasks against 158 for the strongest single model — one task apart — at roughly one fifth the cost. The direction is reversed: scout small, fix big. He limits his finding to his own contract, frontier-specifies \u002F local-implements.",[15,5899,5900],{},"The cost definition is narrow too: API list prices, no hardware, electricity or engineering time. \"8.6x\" is the frontier side of the bill, not your total.",[15,5902,5903],{},"Tokens are cheap. Coordination is not.",[39,5905,5907],{"id":5906},"the-acceptance-script-will-reject-good-code","The acceptance script will reject good code",[15,5909,5910],{},"The most practical part of the report is not about models. It is about acceptance.",[15,5912,5913,5914,5917,5918,5921],{},"Worker artifacts had to clear a static contract: imports, exports, and exported function parameter lists had to match. The stubs used underscored parameters for unused arguments (",[1067,5915,5916],{},"_entries","); workers naturally dropped the underscore (",[1067,5919,5920],{},"entries",").",[15,5923,5924],{},"For ordinary positional callers this changes nothing. But the contract compared parameter-name strings, so the artifact was rejected before parent review, before the hidden checks ran.",[15,5926,5927],{},"The post-hoc diagnostic: 36 valid-JSON artifacts rejected solely for dropping the underscore were applied unchanged to fresh task copies, with that rule bypassed, and re-run against the original sealed checks.",[15,5929,5930],{},"19 passed. 17 still failed.",[15,5932,5933],{},"So the rule was not simply wrong. It screened out both good and bad output. Rejecting bad output is its job; rejecting good output is its side effect — and if that gate is your only gate, the side effect is the final result.",[61,5935,5936,5952],{},[64,5937,5938],{},[67,5939,5940,5943,5946,5949],{},[70,5941,5942],{},"Worker",[70,5944,5945],{},"Rejected for underscores",[70,5947,5948],{},"Passed sealed checks unchanged",[70,5950,5951],{},"Still failed",[80,5953,5954,5968,5982],{},[67,5955,5956,5959,5962,5965],{},[85,5957,5958],{},"Qwen3-Coder",[85,5960,5961],{},"23",[85,5963,5964],{},"12",[85,5966,5967],{},"11",[67,5969,5970,5973,5976,5979],{},[85,5971,5972],{},"Nemotron",[85,5974,5975],{},"13",[85,5977,5978],{},"7",[85,5980,5981],{},"6",[67,5983,5984,5987,5990,5993],{},[85,5985,5986],{},"Total",[85,5988,5989],{},"36",[85,5991,5992],{},"19",[85,5994,5995],{},"17",[15,5997,5998],{},"He did not count the 19 as successes. They never went through parent review and application; the frozen scores stay Qwen 0\u002F45 and Nemotron 12\u002F45. He also did not conclude the contract should go. He proposed the next experiment: give the worker the specific validation error and allow one bounded repair.",[15,6000,6001],{},"That experiment has not been run.",[15,6003,6004],{},"On this contract, acceptance design failed more easily than model capability did.",[39,6006,6008],{"id":6007},"what-this-cannot-prove","What this cannot prove",[2438,6010,6011,6014,6017,6020],{},[2441,6012,6013],{},"A single-author retrospective, not a controlled experiment. The workload changed as the engineering changed.",[2441,6015,6016],{},"Small sample: five task categories, three runs per cell, one fixed parent, one worker configuration.",[2441,6018,6019],{},"Cost covers frontier tokens at list prices only.",[2441,6021,6022],{},"The raw records are private. You cannot recompute the numbers.",[15,6024,6025],{},"The studies are not pooled and do not rank models or runtimes.",[15,6027,6028],{},"So this gives you an acceptance framework, not a leaderboard. The most valuable part, in my reading, is not the numbers — it is that \"done\" becomes something you can check level by level.",[39,6030,6032],{"id":6031},"a-checklist-you-can-use-tomorrow","A checklist you can use tomorrow",[15,6034,6035],{},"The author's five rules, rewritten for a PoC document:",[5530,6037,6038,6044,6050,6056,6062],{},[2441,6039,6040,6043],{},[142,6041,6042],{},"Define success at the task boundary."," Accept on \"the output passed a check it does not own\", not on \"it responded\". Exercise the real client, not a harness you wrote.",[2441,6045,6046,6049],{},[142,6047,6048],{},"Treat deployment as a recipe, not a model name."," Checkpoint revision, quantisation, runtime, template, context settings, gateway, client, resident memory. Change one, re-verify.",[2441,6051,6052,6055],{},[142,6053,6054],{},"Make session continuity an application responsibility."," Stable prefix, no cosmetic re-serialisation, auxiliary requests must not evict the main conversation.",[2441,6057,6058,6061],{},[142,6059,6060],{},"Price delegation per verified success."," Not per token, not per call.",[2441,6063,6064,6067],{},[142,6065,6066],{},"Verify by behaviour where you can."," Static gates are cheap and worth keeping, but know what they reject.",[15,6069,6070],{},"If you are the buyer, three questions are enough:",[2438,6072,6073,6076,6079],{},[2441,6074,6075],{},"Which level does this demo stop at? \"It answers\" is level two.",[2441,6077,6078],{},"How do you verify level four, and who wrote the checks, and when were they frozen?",[2441,6080,6081],{},"Is the cost per call or per verified success?",[15,6083,6084,6085,6089],{},"Our own ",[19,6086,6088],{"href":6087},"\u002Fwork\u002Fvideo-transcript-pipeline","video transcription pipeline"," ran into the same class of problem: the model ran, the pipeline still stalled, and every failure looked like \"the model is a bit dumb\". We have not run measurements at this scale, so this essay borrows the paper's numbers.",[15,6091,6092,6093,6097],{},"The previous essay was about ",[19,6094,6096],{"href":6095},"\u002Fblog\u002Fbroken-on-arrival-quantized-models","files that look right and cannot be used","; this one is about models that look right and cannot finish the job.",[15,6099,6100,6101,6104],{},"If you want help filling in the table, our ",[19,6102,5439],{"href":6103},"\u002Fservices"," does exactly that: what can be handed to AI now, and what should wait.",[39,6106,6108],{"id":6107},"conclusion-which-level-counts-as-done","Conclusion: which level counts as done",[15,6110,6111],{},"The first three levels can be demonstrated in an afternoon. Load a model, send a few prompts, connect a client.",[15,6113,6114],{},"Level four takes weeks: design tasks, write hidden checks, run enough trials, classify failures, price the work. What you get is rarely \"this model is good\" — it is \"on these tasks, under this contract, it is good\".",[15,6116,6117],{},"The missing column on your acceptance sheet is where the price difference lives.",[15,6119,6120],{},"Tomorrow: write \"passes the tests\" into your PoC definition, and ask who wrote the tests and when they froze them.",{"title":2184,"searchDepth":2196,"depth":2196,"links":6122},[6123,6124,6125,6126,6127,6128,6129,6130,6131,6132],{"id":4850,"depth":2196,"text":4851},{"id":5527,"depth":2196,"text":5528},{"id":5653,"depth":2196,"text":5654},{"id":5699,"depth":2196,"text":5700},{"id":5767,"depth":2196,"text":5768},{"id":5866,"depth":2196,"text":5867},{"id":5906,"depth":2196,"text":5907},{"id":6007,"depth":2196,"text":6008},{"id":6031,"depth":2196,"text":6032},{"id":6107,"depth":2196,"text":6108},"\u002Fimages\u002Fblog\u002Fen\u002Frunning-is-not-done.svg","Cover for an essay on the four levels between a served model and finished work","2026-11-03","Same endpoint, different behaviour: tool calls, turn control, context, cache. A 4,165-turn solo report splits 'it runs' into four levels, finding local delegation costs 8.6x more per verified success.",{},"\u002Fimages\u002Fblog\u002Fen\u002Frunning-is-not-done.jpg","\u002Fblog\u002Fen\u002Frunning-is-not-done",{"title":4710,"description":6136},"running-is-not-done","blog\u002Fen\u002Frunning-is-not-done",[6144,6145,6146,6147],"local-llm","agents","serving","tool-calling","0DdevvjhzLCzG5RvWM1GlSfrg6_pwOm2oIXJq0yz_YU",1794643892912]