[{"data":1,"prerenderedAt":5008},["ShallowReactive",2],{"navigation":3,"blog-page":46,"blogs":56},[4],{"title":5,"path":6,"stem":7,"children":8,"page":45},"Blog","\u002Fblog","blog",[9,13,17,21,25,29,33,37,41],{"title":10,"path":11,"stem":12},"Audio Augmentation Techniques for Deep Learning","\u002Fblog\u002Faudio-augmentation","blog\u002Faudio-augmentation",{"title":14,"path":15,"stem":16},"Speed Up Whisper with CTranslate2","\u002Fblog\u002Fc-translate","blog\u002Fc-translate",{"title":18,"path":19,"stem":20},"Calling C++ and Go from Dart using FFI","\u002Fblog\u002Fdart-ffi-native-bridge","blog\u002Fdart-ffi-native-bridge",{"title":22,"path":23,"stem":24},"Russey Keo High School Memories","\u002Fblog\u002Fmemory","blog\u002Fmemory",{"title":26,"path":27,"stem":28},"My Dev Setup: Same Keys Everywhere","\u002Fblog\u002Fmy-dev-setup","blog\u002Fmy-dev-setup",{"title":30,"path":31,"stem":32},"Data Extraction with LangChain and Ollama Locally","\u002Fblog\u002Fpydantic-llm","blog\u002Fpydantic-llm",{"title":34,"path":35,"stem":36},"7-Segment LED Display with Proteus Simulation","\u002Fblog\u002Fseven-segment","blog\u002Fseven-segment",{"title":38,"path":39,"stem":40},"Understanding JWT Refresh Token Rotation & Revocation","\u002Fblog\u002Funderstand-refresh-token","blog\u002Funderstand-refresh-token",{"title":42,"path":43,"stem":44},"Khmer Speech Recognition","\u002Fblog\u002Fwhisper","blog\u002Fwhisper",false,{"id":47,"title":48,"body":49,"description":50,"extension":51,"links":49,"meta":52,"navigation":53,"path":6,"seo":54,"stem":7,"__hash__":55},"pages\u002Fblog.yml","Writing",null,"Notes on what I build and learn.","yml",{},true,{"title":48,"description":50},"MhtFC1wGI_Mak60730xcajRMtf6tgQ5IIK2tneEHIdk",[57,281,1940,2148,2760,3702,3840,4821,4972],{"id":58,"title":26,"author":59,"body":63,"date":271,"description":272,"extension":273,"image":274,"meta":275,"minRead":265,"navigation":53,"path":27,"seo":279,"stem":28,"__hash__":280},"blog\u002Fblog\u002Fmy-dev-setup.md",{"name":60,"avatar":61},"Phanith LIM",{"src":62,"alt":60},"\u002Fmyprofile.jpg",{"type":64,"value":65,"toc":260},"minimark",[66,74,77,82,89,100,104,110,113,141,148,152,158,165,169,180,184,191,235,238,242,251,254],[67,68,69,70],"p",{},"I switch between a lot of tools in a day: a terminal, Neovim, IntelliJ for Java, a browser. For a long time each one had its own shortcuts, and my hands kept pressing the wrong ones. So I rebuilt my setup around one rule: ",[71,72,73],"strong",{},"the same keys should do the same thing everywhere.",[67,75,76],{},"Here is what that looks like.",[78,79,81],"h3",{"id":80},"the-desktop-omarchy-and-hyprland","The desktop: Omarchy and Hyprland",[67,83,84],{},[85,86],"img",{"alt":87,"src":88},"Hyprland","\u002Fblog\u002Fmy-dev-setup\u002Fhyprland.svg",[67,90,91,92,99],{},"I run ",[93,94,98],"a",{"href":95,"rel":96},"https:\u002F\u002Fomarchy.org",[97],"nofollow","Omarchy",", an Arch Linux setup built on the Hyprland window manager. Windows tile themselves, and everything is a keyboard shortcut. Each monitor gets its own workspaces 1–10, and one key throws a window between my laptop and my external screen.",[78,101,103],{"id":102},"the-editor-neovim","The editor: Neovim",[67,105,106],{},[85,107],{"alt":108,"src":109},"Neovim","\u002Fblog\u002Fmy-dev-setup\u002Fneovim.png",[67,111,112],{},"Most of my writing and coding happens in Neovim with LazyVim. A few things I added on top:",[114,115,116,123,135],"ul",{},[117,118,119,122],"li",{},[71,120,121],{},"Harpoon"," for the four files I am working on right now, one key each.",[117,124,125,126,129,130,134],{},"A ",[71,127,128],{},"floating terminal"," on ",[131,132,133],"code",{},"Ctrl + \u002F",".",[117,136,137,140],{},[71,138,139],{},"Spring Boot support"," for Java, including a key that checks my Spring Modulith module boundaries.",[67,142,143,144,147],{},"When I want smooth animations, I open the same config in ",[71,145,146],{},"Neovide",", a GUI for Neovim.",[78,149,151],{"id":150},"sessions-tmux","Sessions: tmux",[67,153,154],{},[85,155],{"alt":156,"src":157},"tmux","\u002Fblog\u002Fmy-dev-setup\u002Ftmux.png",[67,159,160,161,164],{},"tmux keeps my terminals alive between sessions. I gave its bar rounded \"pills\" that recolour themselves when I change my desktop theme. In Neovide I copied tmux's ",[131,162,163],{},"Ctrl + b"," keys, so opening a tab or a split is the same in both.",[78,166,168],{"id":167},"java-intellij-with-vim-keys","Java: IntelliJ with Vim keys",[67,170,171,172,175,176,179],{},"For big Java projects I still use IntelliJ, with the IdeaVim plugin. I mapped it to match Neovim: the same leader key, the same ",[131,173,174],{},"gd",", the same ",[131,177,178],{},"Alt + 1–4"," to jump between my main files.",[78,181,183],{"id":182},"the-keyboard-cornix-lp","The keyboard: Cornix LP",[67,185,186,187,190],{},"The last piece is a split keyboard, the Cornix LP. Its layers turn the home row into shortcuts. My favourite part is that ",[71,188,189],{},"H \u002F J \u002F K \u002F L"," moves focus at every level:",[192,193,194,207],"table",{},[195,196,197],"thead",{},[198,199,200,204],"tr",{},[201,202,203],"th",{},"Hold",[201,205,206],{},"+ H \u002F J \u002F K \u002F L",[208,209,210,219,227],"tbody",{},[198,211,212,216],{},[213,214,215],"td",{},"Layer 1",[213,217,218],{},"Arrow keys",[198,220,221,224],{},[213,222,223],{},"Ctrl",[213,225,226],{},"Neovim splits and tmux panes",[198,228,229,232],{},[213,230,231],{},"Layer 2",[213,233,234],{},"Hyprland windows",[67,236,237],{},"One motion, three tools, no thinking.",[78,239,241],{"id":240},"try-it","Try it",[67,243,244,245,250],{},"Everything is in my ",[93,246,249],{"href":247,"rel":248},"https:\u002F\u002Fgithub.com\u002Fphanithlim\u002Fdotfiles",[97],"dotfiles on GitHub",", with a page of keybindings for each tool.",[67,252,253],{},"The best setup is not the one with the most features. It is the one where you stop noticing the tools.",[67,255,256],{},[257,258,259],"em",{},"Images: Neovim and tmux logos by Jason Long, Hyprland logo from the Hyprland project.",{"title":261,"searchDepth":262,"depth":262,"links":263},"",2,[264,266,267,268,269,270],{"id":80,"depth":265,"text":81},3,{"id":102,"depth":265,"text":103},{"id":150,"depth":265,"text":151},{"id":167,"depth":265,"text":168},{"id":182,"depth":265,"text":183},{"id":240,"depth":265,"text":241},"2026-10-05","Omarchy, Neovim, tmux, IntelliJ and a split keyboard, set up so one habit works in every tool.","md","\u002Fblog\u002Fmy-dev-setup\u002Fthumbnail.webp",{"tags":276},[277,108,278],"Linux","Productivity",{"title":26,"description":272},"I1CFIWCT88teEiYNfSjZB04U-TdAuJxa2AnLGdbHIPg",{"id":282,"title":18,"author":283,"body":285,"date":1931,"description":1932,"extension":273,"image":1933,"meta":1934,"minRead":375,"navigation":53,"path":19,"seo":1938,"stem":20,"__hash__":1939},"blog\u002Fblog\u002Fdart-ffi-native-bridge.md",{"name":60,"avatar":284},{"src":62,"alt":60},{"type":64,"value":286,"toc":1913},[287,301,304,307,312,319,323,334,455,458,496,500,515,789,792,833,837,843,1070,1073,1121,1124,1139,1142,1146,1149,1153,1279,1283,1359,1365,1367,1371,1375,1506,1510,1616,1619,1708,1711,1799,1802,1888,1890,1894,1902,1909],[67,288,289,292,293,296,297,300],{},[131,290,291],{},"dart:ffi"," lets Dart call into a shared library (",[131,294,295],{},".so"," or ",[131,298,299],{},".dylib",") directly. No platform channels, no plugin, no Flutter needed.",[67,302,303],{},"That is worth having when the code you want already exists in C or C++ and you would rather bind to it than port it, or when a hot loop is faster in a compiled language than in Dart. The cost is that you now have a build step per platform and a pointer or two to look after.",[67,305,306],{},"I wired up C++ and Go this way and worked out which other languages can join in. Both are below.",[308,309,311],"h2",{"id":310},"the-one-rule","The one rule",[67,313,314,315,318],{},"Dart can only call functions that expose a ",[71,316,317],{},"C ABI",": a plain C-compatible symbol with no name mangling. Every language reaches that differently, but the Dart side never changes: open the library, declare the types, look up the function.",[308,320,322],{"id":321},"c","C++",[67,324,325,326,329,330,333],{},"C++ mangles function names, so ",[131,327,328],{},"add"," becomes something unrecognisable. ",[131,331,332],{},"extern \"C\""," turns that off.",[335,336,340],"pre",{"className":337,"code":338,"language":339,"meta":261,"style":261},"language-cpp shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","\u002F\u002F native\u002Fcpp\u002Fmylib.cpp\n#include \u003Ccstdint>\n\nextern \"C\" {\n  int32_t add(int32_t a, int32_t b) {\n    return a + b;\n  }\n}\n","cpp",[131,341,342,351,368,373,392,426,443,449],{"__ignoreMap":261},[343,344,347],"span",{"class":345,"line":346},"line",1,[343,348,350],{"class":349},"sHwdD","\u002F\u002F native\u002Fcpp\u002Fmylib.cpp\n",[343,352,353,357,361,365],{"class":345,"line":262},[343,354,356],{"class":355},"s7zQu","#include",[343,358,360],{"class":359},"sMK4o"," \u003C",[343,362,364],{"class":363},"sfazB","cstdint",[343,366,367],{"class":359},">\n",[343,369,370],{"class":345,"line":265},[343,371,372],{"emptyLinePlaceholder":53},"\n",[343,374,376,380,383,386,389],{"class":345,"line":375},4,[343,377,379],{"class":378},"spNyl","extern",[343,381,382],{"class":359}," \"",[343,384,385],{"class":363},"C",[343,387,388],{"class":359},"\"",[343,390,391],{"class":359}," {\n",[343,393,395,398,402,405,408,412,415,418,421,424],{"class":345,"line":394},5,[343,396,397],{"class":378},"  int32_t",[343,399,401],{"class":400},"s2Zo4"," add",[343,403,404],{"class":359},"(",[343,406,407],{"class":378},"int32_t",[343,409,411],{"class":410},"sHdIc"," a",[343,413,414],{"class":359},",",[343,416,417],{"class":378}," int32_t",[343,419,420],{"class":410}," b",[343,422,423],{"class":359},")",[343,425,391],{"class":359},[343,427,429,432,435,438,440],{"class":345,"line":428},6,[343,430,431],{"class":355},"    return",[343,433,411],{"class":434},"sTEyZ",[343,436,437],{"class":359}," +",[343,439,420],{"class":434},[343,441,442],{"class":359},";\n",[343,444,446],{"class":345,"line":445},7,[343,447,448],{"class":359},"  }\n",[343,450,452],{"class":345,"line":451},8,[343,453,454],{"class":359},"}\n",[67,456,457],{},"Build it with CMake:",[335,459,463],{"className":460,"code":461,"language":462,"meta":261,"style":261},"language-bash shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","cmake -S native\u002Fcpp -B native\u002Fcpp\u002Fbuild -DCMAKE_BUILD_TYPE=Release\ncmake --build native\u002Fcpp\u002Fbuild\n","bash",[131,464,465,486],{"__ignoreMap":261},[343,466,467,471,474,477,480,483],{"class":345,"line":346},[343,468,470],{"class":469},"sBMFI","cmake",[343,472,473],{"class":363}," -S",[343,475,476],{"class":363}," native\u002Fcpp",[343,478,479],{"class":363}," -B",[343,481,482],{"class":363}," native\u002Fcpp\u002Fbuild",[343,484,485],{"class":363}," -DCMAKE_BUILD_TYPE=Release\n",[343,487,488,490,493],{"class":345,"line":262},[343,489,470],{"class":469},[343,491,492],{"class":363}," --build",[343,494,495],{"class":363}," native\u002Fcpp\u002Fbuild\n",[308,497,499],{"id":498},"go","Go",[67,501,502,503,506,507,510,511,514],{},"Go exports through CGo. Two details will bite you if you miss them: the file needs ",[131,504,505],{},"package main"," with an empty ",[131,508,509],{},"func main()",", and ",[131,512,513],{},"\u002F\u002Fexport"," must sit on the line directly above the function, with no blank line between.",[335,516,519],{"className":517,"code":518,"language":498,"meta":261,"style":261},"language-go shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","\u002F\u002F native\u002Fgo\u002Fmylib.go\npackage main\n\nimport \"C\"\n\n\u002F\u002Fexport multiply\nfunc multiply(a, b int32) int32 { return a * b }\n\n\u002F\u002Fexport fibonacci\nfunc fibonacci(n int32) int64 {\n    if n \u003C= 1 { return int64(n) }\n    a, b := int64(0), int64(1)\n    for i := int32(2); i \u003C= n; i++ { a, b = b, a+b }\n    return b\n}\n\nfunc main() {}\n",[131,520,521,526,534,538,550,554,559,599,603,609,631,662,695,757,765,770,775],{"__ignoreMap":261},[343,522,523],{"class":345,"line":346},[343,524,525],{"class":349},"\u002F\u002F native\u002Fgo\u002Fmylib.go\n",[343,527,528,531],{"class":345,"line":262},[343,529,530],{"class":359},"package",[343,532,533],{"class":469}," main\n",[343,535,536],{"class":345,"line":265},[343,537,372],{"emptyLinePlaceholder":53},[343,539,540,543,545,547],{"class":345,"line":375},[343,541,542],{"class":355},"import",[343,544,382],{"class":359},[343,546,385],{"class":469},[343,548,549],{"class":359},"\"\n",[343,551,552],{"class":345,"line":394},[343,553,372],{"emptyLinePlaceholder":53},[343,555,556],{"class":345,"line":428},[343,557,558],{"class":349},"\u002F\u002Fexport multiply\n",[343,560,561,564,567,569,571,573,575,578,580,582,585,588,591,594,597],{"class":345,"line":445},[343,562,563],{"class":359},"func",[343,565,566],{"class":400}," multiply",[343,568,404],{"class":359},[343,570,93],{"class":410},[343,572,414],{"class":359},[343,574,420],{"class":410},[343,576,577],{"class":378}," int32",[343,579,423],{"class":359},[343,581,577],{"class":378},[343,583,584],{"class":359}," {",[343,586,587],{"class":355}," return",[343,589,590],{"class":434}," a ",[343,592,593],{"class":359},"*",[343,595,596],{"class":434}," b ",[343,598,454],{"class":359},[343,600,601],{"class":345,"line":451},[343,602,372],{"emptyLinePlaceholder":53},[343,604,606],{"class":345,"line":605},9,[343,607,608],{"class":349},"\u002F\u002Fexport fibonacci\n",[343,610,612,614,617,619,622,624,626,629],{"class":345,"line":611},10,[343,613,563],{"class":359},[343,615,616],{"class":400}," fibonacci",[343,618,404],{"class":359},[343,620,621],{"class":410},"n",[343,623,577],{"class":378},[343,625,423],{"class":359},[343,627,628],{"class":378}," int64",[343,630,391],{"class":359},[343,632,634,637,640,643,647,649,651,653,655,657,659],{"class":345,"line":633},11,[343,635,636],{"class":355},"    if",[343,638,639],{"class":434}," n ",[343,641,642],{"class":359},"\u003C=",[343,644,646],{"class":645},"sbssI"," 1",[343,648,584],{"class":359},[343,650,587],{"class":355},[343,652,628],{"class":378},[343,654,404],{"class":359},[343,656,621],{"class":434},[343,658,423],{"class":359},[343,660,661],{"class":359}," }\n",[343,663,665,668,670,672,675,677,679,682,685,687,689,692],{"class":345,"line":664},12,[343,666,667],{"class":434},"    a",[343,669,414],{"class":359},[343,671,596],{"class":434},[343,673,674],{"class":359},":=",[343,676,628],{"class":378},[343,678,404],{"class":359},[343,680,681],{"class":645},"0",[343,683,684],{"class":359},"),",[343,686,628],{"class":378},[343,688,404],{"class":359},[343,690,691],{"class":645},"1",[343,693,694],{"class":359},")\n",[343,696,698,701,704,706,708,710,713,716,718,720,723,726,729,732,734,736,738,740,743,745,747,749,752,755],{"class":345,"line":697},13,[343,699,700],{"class":355},"    for",[343,702,703],{"class":434}," i ",[343,705,674],{"class":359},[343,707,577],{"class":378},[343,709,404],{"class":359},[343,711,712],{"class":645},"2",[343,714,715],{"class":359},");",[343,717,703],{"class":434},[343,719,642],{"class":359},[343,721,722],{"class":434}," n",[343,724,725],{"class":359},";",[343,727,728],{"class":434}," i",[343,730,731],{"class":359},"++",[343,733,584],{"class":359},[343,735,411],{"class":434},[343,737,414],{"class":359},[343,739,596],{"class":434},[343,741,742],{"class":359},"=",[343,744,420],{"class":434},[343,746,414],{"class":359},[343,748,411],{"class":434},[343,750,751],{"class":359},"+",[343,753,754],{"class":434},"b ",[343,756,454],{"class":359},[343,758,760,762],{"class":345,"line":759},14,[343,761,431],{"class":355},[343,763,764],{"class":434}," b\n",[343,766,768],{"class":345,"line":767},15,[343,769,454],{"class":359},[343,771,773],{"class":345,"line":772},16,[343,774,372],{"emptyLinePlaceholder":53},[343,776,778,780,783,786],{"class":345,"line":777},17,[343,779,563],{"class":359},[343,781,782],{"class":400}," main",[343,784,785],{"class":359},"()",[343,787,788],{"class":359}," {}\n",[67,790,791],{},"Build:",[335,793,795],{"className":460,"code":794,"language":462,"meta":261,"style":261},"cd native\u002Fgo && CGO_ENABLED=1 go build -o ..\u002Flibs\u002Flibgolib.so -buildmode=c-shared .\n",[131,796,797],{"__ignoreMap":261},[343,798,799,802,805,808,811,813,815,818,821,824,827,830],{"class":345,"line":346},[343,800,801],{"class":400},"cd",[343,803,804],{"class":363}," native\u002Fgo",[343,806,807],{"class":359}," &&",[343,809,810],{"class":434}," CGO_ENABLED",[343,812,742],{"class":359},[343,814,691],{"class":363},[343,816,817],{"class":469}," go",[343,819,820],{"class":363}," build",[343,822,823],{"class":363}," -o",[343,825,826],{"class":363}," ..\u002Flibs\u002Flibgolib.so",[343,828,829],{"class":363}," -buildmode=c-shared",[343,831,832],{"class":363}," .\n",[308,834,836],{"id":835},"the-dart-side","The Dart side",[67,838,839,840,134],{},"This part is the same no matter which language produced the library. Declare the native signature, declare the Dart signature, then ",[131,841,842],{},"lookupFunction",[335,844,848],{"className":845,"code":846,"language":847,"meta":261,"style":261},"language-dart shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","\u002F\u002F lib\u002Fsrc\u002Fgo_bindings.dart\nimport 'dart:ffi';\nimport 'package:path\u002Fpath.dart' as p;\n\ntypedef _FibonacciNative = Int64 Function(Int32 n);\ntypedef FibonacciDart = int Function(int n);\n\nclass GoLib {\n  final DynamicLibrary _lib;\n  late final FibonacciDart fibonacci;\n\n  GoLib(String libsDir)\n      : _lib = DynamicLibrary.open(p.join(libsDir, 'libgolib.so')) {\n    fibonacci = _lib.lookupFunction\u003C_FibonacciNative, FibonacciDart>('fibonacci');\n  }\n}\n","dart",[131,849,850,855,864,879,883,910,933,937,947,960,974,978,991,1027,1062,1066],{"__ignoreMap":261},[343,851,852],{"class":345,"line":346},[343,853,854],{"class":349},"\u002F\u002F lib\u002Fsrc\u002Fgo_bindings.dart\n",[343,856,857,859,862],{"class":345,"line":262},[343,858,542],{"class":645},[343,860,861],{"class":363}," 'dart:ffi'",[343,863,442],{"class":359},[343,865,866,868,871,874,877],{"class":345,"line":265},[343,867,542],{"class":645},[343,869,870],{"class":363}," 'package:path\u002Fpath.dart'",[343,872,873],{"class":645}," as",[343,875,876],{"class":434}," p",[343,878,442],{"class":359},[343,880,881],{"class":345,"line":375},[343,882,372],{"emptyLinePlaceholder":53},[343,884,885,888,891,894,897,900,902,905,908],{"class":345,"line":394},[343,886,887],{"class":359},"typedef",[343,889,890],{"class":469}," _FibonacciNative",[343,892,893],{"class":359}," =",[343,895,896],{"class":469}," Int64",[343,898,899],{"class":469}," Function",[343,901,404],{"class":434},[343,903,904],{"class":469},"Int32",[343,906,907],{"class":434}," n)",[343,909,442],{"class":359},[343,911,912,914,917,919,922,924,926,929,931],{"class":345,"line":428},[343,913,887],{"class":359},[343,915,916],{"class":469}," FibonacciDart",[343,918,893],{"class":359},[343,920,921],{"class":469}," int",[343,923,899],{"class":469},[343,925,404],{"class":434},[343,927,928],{"class":469},"int",[343,930,907],{"class":434},[343,932,442],{"class":359},[343,934,935],{"class":345,"line":445},[343,936,372],{"emptyLinePlaceholder":53},[343,938,939,942,945],{"class":345,"line":451},[343,940,941],{"class":359},"class",[343,943,944],{"class":469}," GoLib",[343,946,391],{"class":434},[343,948,949,952,955,958],{"class":345,"line":605},[343,950,951],{"class":378},"  final",[343,953,954],{"class":469}," DynamicLibrary",[343,956,957],{"class":434}," _lib",[343,959,442],{"class":359},[343,961,962,965,968,970,972],{"class":345,"line":611},[343,963,964],{"class":378},"  late",[343,966,967],{"class":378}," final",[343,969,916],{"class":469},[343,971,616],{"class":434},[343,973,442],{"class":359},[343,975,976],{"class":345,"line":633},[343,977,372],{"emptyLinePlaceholder":53},[343,979,980,983,985,988],{"class":345,"line":664},[343,981,982],{"class":469},"  GoLib",[343,984,404],{"class":434},[343,986,987],{"class":469},"String",[343,989,990],{"class":434}," libsDir)\n",[343,992,993,996,999,1001,1003,1005,1008,1011,1013,1016,1019,1021,1024],{"class":345,"line":697},[343,994,995],{"class":359},"      :",[343,997,998],{"class":434}," _lib ",[343,1000,742],{"class":359},[343,1002,954],{"class":469},[343,1004,134],{"class":359},[343,1006,1007],{"class":400},"open",[343,1009,1010],{"class":434},"(p",[343,1012,134],{"class":359},[343,1014,1015],{"class":400},"join",[343,1017,1018],{"class":434},"(libsDir",[343,1020,414],{"class":359},[343,1022,1023],{"class":363}," 'libgolib.so'",[343,1025,1026],{"class":434},")) {\n",[343,1028,1029,1032,1034,1036,1038,1040,1043,1046,1049,1052,1055,1058,1060],{"class":345,"line":759},[343,1030,1031],{"class":434},"    fibonacci ",[343,1033,742],{"class":359},[343,1035,957],{"class":434},[343,1037,134],{"class":359},[343,1039,842],{"class":400},[343,1041,1042],{"class":434},"\u003C",[343,1044,1045],{"class":469},"_FibonacciNative",[343,1047,1048],{"class":434},", ",[343,1050,1051],{"class":469},"FibonacciDart",[343,1053,1054],{"class":434},">(",[343,1056,1057],{"class":363},"'fibonacci'",[343,1059,423],{"class":434},[343,1061,442],{"class":359},[343,1063,1064],{"class":345,"line":767},[343,1065,448],{"class":434},[343,1067,1068],{"class":345,"line":772},[343,1069,454],{"class":434},[67,1071,1072],{},"Then call it like ordinary Dart:",[335,1074,1076],{"className":845,"code":1075,"language":847,"meta":261,"style":261},"final go = GoLib(libsDir);\nprint(go.fibonacci(10)); \u002F\u002F 55\n",[131,1077,1078,1095],{"__ignoreMap":261},[343,1079,1080,1083,1086,1088,1090,1093],{"class":345,"line":346},[343,1081,1082],{"class":378},"final",[343,1084,1085],{"class":434}," go ",[343,1087,742],{"class":359},[343,1089,944],{"class":469},[343,1091,1092],{"class":434},"(libsDir)",[343,1094,442],{"class":359},[343,1096,1097,1100,1103,1105,1108,1110,1113,1116,1118],{"class":345,"line":262},[343,1098,1099],{"class":400},"print",[343,1101,1102],{"class":434},"(go",[343,1104,134],{"class":359},[343,1106,1107],{"class":400},"fibonacci",[343,1109,404],{"class":434},[343,1111,1112],{"class":645},"10",[343,1114,1115],{"class":434},"))",[343,1117,725],{"class":359},[343,1119,1120],{"class":349}," \u002F\u002F 55\n",[67,1122,1123],{},"Numbers are the easy case. They copy across on their own, and nothing needs cleaning up afterwards.",[67,1125,1126,1127,1130,1131,1134,1135,1138],{},"Strings do not. A ",[131,1128,1129],{},"Pointer\u003CUtf8>"," is memory you asked for, so you convert with ",[131,1132,1133],{},"toNativeUtf8()",", pass the pointer, read the reply back with ",[131,1136,1137],{},"toDartString()",", and free what you allocated. Forget the free and you have a leak that no garbage collector will find for you.",[1140,1141],"hr",{},[308,1143,1145],{"id":1144},"which-languages-work","Which languages work",[67,1147,1148],{},"Anything that compiles to a shared library with C-compatible symbols.",[78,1150,1152],{"id":1151},"works","Works",[192,1154,1155,1165],{},[195,1156,1157],{},[198,1158,1159,1162],{},[201,1160,1161],{},"Language",[201,1163,1164],{},"Export mechanism",[208,1166,1167,1174,1184,1197,1210,1221,1232,1243,1254,1265],{},[198,1168,1169,1171],{},[213,1170,385],{},[213,1172,1173],{},"Native, no wrapper needed",[198,1175,1176,1178],{},[213,1177,322],{},[213,1179,1180,1183],{},[131,1181,1182],{},"extern \"C\" { ... }"," block",[198,1185,1186,1188],{},[213,1187,499],{},[213,1189,1190,1191,1193,1194],{},"CGo ",[131,1192,513],{}," + ",[131,1195,1196],{},"-buildmode=c-shared",[198,1198,1199,1202],{},[213,1200,1201],{},"Rust",[213,1203,1204,1193,1207],{},[131,1205,1206],{},"#[no_mangle] extern \"C\" fn",[131,1208,1209],{},"crate-type = [\"cdylib\"]",[198,1211,1212,1215],{},[213,1213,1214],{},"Zig",[213,1216,1217,1220],{},[131,1218,1219],{},"export fn"," + compile as shared library",[198,1222,1223,1226],{},[213,1224,1225],{},"Nim",[213,1227,1228,1231],{},[131,1229,1230],{},"{.exportc, dynlib.}"," pragma",[198,1233,1234,1237],{},[213,1235,1236],{},"D",[213,1238,1239,1242],{},[131,1240,1241],{},"extern(C)"," linkage",[198,1244,1245,1248],{},[213,1246,1247],{},"Kotlin\u002FNative",[213,1249,1250,1253],{},[131,1251,1252],{},"@CName"," annotation (not Kotlin\u002FJVM)",[198,1255,1256,1259],{},[213,1257,1258],{},"Swift",[213,1260,1261,1264],{},[131,1262,1263],{},"@_cdecl(\"name\")"," on a free function",[198,1266,1267,1270],{},[213,1268,1269],{},"Fortran",[213,1271,1272,1193,1275,1278],{},[131,1273,1274],{},"ISO_C_BINDING",[131,1276,1277],{},"bind(C)"," attribute",[78,1280,1282],{"id":1281},"does-not-work","Does not work",[192,1284,1285,1294],{},[195,1286,1287],{},[198,1288,1289,1291],{},[201,1290,1161],{},[201,1292,1293],{},"Reason",[208,1295,1296,1304,1312,1320,1328,1336,1344,1352],{},[198,1297,1298,1301],{},[213,1299,1300],{},"Python",[213,1302,1303],{},"No C ABI export, would need embedding CPython",[198,1305,1306,1309],{},[213,1307,1308],{},"Java",[213,1310,1311],{},"JVM bytecode only; JNI goes the other direction",[198,1313,1314,1317],{},[213,1315,1316],{},"Kotlin (JVM)",[213,1318,1319],{},"Runs on the JVM, no C ABI",[198,1321,1322,1325],{},[213,1323,1324],{},"Scala",[213,1326,1327],{},"JVM, no C ABI",[198,1329,1330,1333],{},[213,1331,1332],{},"C# \u002F .NET",[213,1334,1335],{},"CLR managed runtime, no native C ABI export",[198,1337,1338,1341],{},[213,1339,1340],{},"JavaScript \u002F TypeScript",[213,1342,1343],{},"Runs in a JS engine, no C ABI",[198,1345,1346,1349],{},[213,1347,1348],{},"Ruby",[213,1350,1351],{},"Interpreter-based, no C ABI export",[198,1353,1354,1357],{},[213,1355,1356],{},"PHP",[213,1358,1351],{},[67,1360,1361,1362,1364],{},"The pattern behind both tables: if the language can produce a ",[131,1363,295],{}," with plain C symbol names, Dart FFI can call it. If it needs its own runtime to be alive first, it cannot.",[1140,1366],{},[308,1368,1370],{"id":1369},"type-mappings","Type mappings",[78,1372,1374],{"id":1373},"c-and-c","C and C++",[192,1376,1377,1387],{},[195,1378,1379],{},[198,1380,1381,1384],{},[201,1382,1383],{},"Dart FFI type",[201,1385,1386],{},"C \u002F C++ type",[208,1388,1389,1401,1413,1423,1435,1447,1459,1471,1483,1494],{},[198,1390,1391,1396],{},[213,1392,1393],{},[131,1394,1395],{},"Int8",[213,1397,1398],{},[131,1399,1400],{},"int8_t",[198,1402,1403,1408],{},[213,1404,1405],{},[131,1406,1407],{},"Int16",[213,1409,1410],{},[131,1411,1412],{},"int16_t",[198,1414,1415,1419],{},[213,1416,1417],{},[131,1418,904],{},[213,1420,1421],{},[131,1422,407],{},[198,1424,1425,1430],{},[213,1426,1427],{},[131,1428,1429],{},"Int64",[213,1431,1432],{},[131,1433,1434],{},"int64_t",[198,1436,1437,1442],{},[213,1438,1439],{},[131,1440,1441],{},"Float",[213,1443,1444],{},[131,1445,1446],{},"float",[198,1448,1449,1454],{},[213,1450,1451],{},[131,1452,1453],{},"Double",[213,1455,1456],{},[131,1457,1458],{},"double",[198,1460,1461,1466],{},[213,1462,1463],{},[131,1464,1465],{},"Bool",[213,1467,1468],{},[131,1469,1470],{},"bool",[198,1472,1473,1478],{},[213,1474,1475],{},[131,1476,1477],{},"Void",[213,1479,1480],{},[131,1481,1482],{},"void",[198,1484,1485,1489],{},[213,1486,1487],{},[131,1488,1129],{},[213,1490,1491],{},[131,1492,1493],{},"char*",[198,1495,1496,1501],{},[213,1497,1498],{},[131,1499,1500],{},"Pointer\u003CT>",[213,1502,1503],{},[131,1504,1505],{},"T*",[78,1507,1509],{"id":1508},"go-via-cgo","Go (via CGo)",[192,1511,1512,1521],{},[195,1513,1514],{},[198,1515,1516,1518],{},[201,1517,1383],{},[201,1519,1520],{},"Go \u002F CGo type",[208,1522,1523,1538,1552,1566,1580,1593,1604],{},[198,1524,1525,1529],{},[213,1526,1527],{},[131,1528,904],{},[213,1530,1531,1534,1535],{},[131,1532,1533],{},"int32"," \u002F ",[131,1536,1537],{},"C.int",[198,1539,1540,1544],{},[213,1541,1542],{},[131,1543,1429],{},[213,1545,1546,1534,1549],{},[131,1547,1548],{},"int64",[131,1550,1551],{},"C.longlong",[198,1553,1554,1558],{},[213,1555,1556],{},[131,1557,1441],{},[213,1559,1560,1534,1563],{},[131,1561,1562],{},"float32",[131,1564,1565],{},"C.float",[198,1567,1568,1572],{},[213,1569,1570],{},[131,1571,1453],{},[213,1573,1574,1534,1577],{},[131,1575,1576],{},"float64",[131,1578,1579],{},"C.double",[198,1581,1582,1586],{},[213,1583,1584],{},[131,1585,1465],{},[213,1587,1588,1534,1590,1592],{},[131,1589,1470],{},[131,1591,1537],{}," (0\u002F1)",[198,1594,1595,1599],{},[213,1596,1597],{},[131,1598,1129],{},[213,1600,1601],{},[131,1602,1603],{},"*C.char",[198,1605,1606,1611],{},[213,1607,1608],{},[131,1609,1610],{},"Pointer\u003CInt32>",[213,1612,1613],{},[131,1614,1615],{},"*C.int",[78,1617,1201],{"id":1618},"rust",[192,1620,1621,1630],{},[195,1622,1623],{},[198,1624,1625,1627],{},[201,1626,1383],{},[201,1628,1629],{},"Rust type",[208,1631,1632,1643,1654,1665,1676,1686,1697],{},[198,1633,1634,1638],{},[213,1635,1636],{},[131,1637,904],{},[213,1639,1640],{},[131,1641,1642],{},"i32",[198,1644,1645,1649],{},[213,1646,1647],{},[131,1648,1429],{},[213,1650,1651],{},[131,1652,1653],{},"i64",[198,1655,1656,1660],{},[213,1657,1658],{},[131,1659,1441],{},[213,1661,1662],{},[131,1663,1664],{},"f32",[198,1666,1667,1671],{},[213,1668,1669],{},[131,1670,1453],{},[213,1672,1673],{},[131,1674,1675],{},"f64",[198,1677,1678,1682],{},[213,1679,1680],{},[131,1681,1465],{},[213,1683,1684],{},[131,1685,1470],{},[198,1687,1688,1692],{},[213,1689,1690],{},[131,1691,1129],{},[213,1693,1694],{},[131,1695,1696],{},"*const c_char",[198,1698,1699,1703],{},[213,1700,1701],{},[131,1702,1500],{},[213,1704,1705],{},[131,1706,1707],{},"*mut T",[78,1709,1247],{"id":1710},"kotlinnative",[192,1712,1713,1722],{},[195,1714,1715],{},[198,1716,1717,1719],{},[201,1718,1383],{},[201,1720,1721],{},"Kotlin type",[208,1723,1724,1735,1746,1756,1766,1777,1788],{},[198,1725,1726,1730],{},[213,1727,1728],{},[131,1729,904],{},[213,1731,1732],{},[131,1733,1734],{},"Int",[198,1736,1737,1741],{},[213,1738,1739],{},[131,1740,1429],{},[213,1742,1743],{},[131,1744,1745],{},"Long",[198,1747,1748,1752],{},[213,1749,1750],{},[131,1751,1441],{},[213,1753,1754],{},[131,1755,1441],{},[198,1757,1758,1762],{},[213,1759,1760],{},[131,1761,1453],{},[213,1763,1764],{},[131,1765,1453],{},[198,1767,1768,1772],{},[213,1769,1770],{},[131,1771,1465],{},[213,1773,1774],{},[131,1775,1776],{},"Boolean",[198,1778,1779,1783],{},[213,1780,1781],{},[131,1782,1129],{},[213,1784,1785],{},[131,1786,1787],{},"CPointer\u003CByteVar>?",[198,1789,1790,1794],{},[213,1791,1792],{},[131,1793,1610],{},[213,1795,1796],{},[131,1797,1798],{},"CPointer\u003CIntVar>?",[78,1800,1258],{"id":1801},"swift",[192,1803,1804,1813],{},[195,1805,1806],{},[198,1807,1808,1810],{},[201,1809,1383],{},[201,1811,1812],{},"Swift type",[208,1814,1815,1825,1835,1845,1855,1865,1876],{},[198,1816,1817,1821],{},[213,1818,1819],{},[131,1820,904],{},[213,1822,1823],{},[131,1824,904],{},[198,1826,1827,1831],{},[213,1828,1829],{},[131,1830,1429],{},[213,1832,1833],{},[131,1834,1429],{},[198,1836,1837,1841],{},[213,1838,1839],{},[131,1840,1441],{},[213,1842,1843],{},[131,1844,1441],{},[198,1846,1847,1851],{},[213,1848,1849],{},[131,1850,1453],{},[213,1852,1853],{},[131,1854,1453],{},[198,1856,1857,1861],{},[213,1858,1859],{},[131,1860,1465],{},[213,1862,1863],{},[131,1864,1465],{},[198,1866,1867,1871],{},[213,1868,1869],{},[131,1870,1129],{},[213,1872,1873],{},[131,1874,1875],{},"UnsafePointer\u003CCChar>?",[198,1877,1878,1882],{},[213,1879,1880],{},[131,1881,1129],{},[213,1883,1884,1887],{},[131,1885,1886],{},"UnsafeMutablePointer\u003CCChar>?"," (out)",[1140,1889],{},[308,1891,1893],{"id":1892},"source","Source",[67,1895,1896,1897],{},"Source code: ",[93,1898,1901],{"href":1899,"rel":1900},"https:\u002F\u002Fgithub.com\u002Fphanithlim\u002Fdart-native",[97],"phanithlim\u002Fdart-native",[67,1903,1904,1905,1908],{},"Everything lives under ",[131,1906,1907],{},"native\u002F",", one directory per language, each with its own README and build steps. Clone it, build one of the libraries, and the Dart side will call straight into it.",[1910,1911,1912],"style",{},"html pre.shiki code .sBMFI, html code.shiki .sBMFI{--shiki-light:#E2931D;--shiki-default:#FFCB6B;--shiki-dark:#FFCB6B}html pre.shiki code .sfazB, html code.shiki .sfazB{--shiki-light:#91B859;--shiki-default:#C3E88D;--shiki-dark:#C3E88D}html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html pre.shiki code .s2Zo4, html code.shiki .s2Zo4{--shiki-light:#6182B8;--shiki-default:#82AAFF;--shiki-dark:#82AAFF}html pre.shiki code .sMK4o, html code.shiki .sMK4o{--shiki-light:#39ADB5;--shiki-default:#89DDFF;--shiki-dark:#89DDFF}html pre.shiki code .sTEyZ, html code.shiki .sTEyZ{--shiki-light:#90A4AE;--shiki-default:#EEFFFF;--shiki-dark:#BABED8}html pre.shiki code .sHwdD, html code.shiki .sHwdD{--shiki-light:#90A4AE;--shiki-light-font-style:italic;--shiki-default:#546E7A;--shiki-default-font-style:italic;--shiki-dark:#676E95;--shiki-dark-font-style:italic}html pre.shiki code .s7zQu, html code.shiki .s7zQu{--shiki-light:#39ADB5;--shiki-light-font-style:italic;--shiki-default:#89DDFF;--shiki-default-font-style:italic;--shiki-dark:#89DDFF;--shiki-dark-font-style:italic}html pre.shiki code .spNyl, html code.shiki .spNyl{--shiki-light:#9C3EDA;--shiki-default:#C792EA;--shiki-dark:#C792EA}html pre.shiki code .sHdIc, html code.shiki .sHdIc{--shiki-light:#90A4AE;--shiki-light-font-style:italic;--shiki-default:#EEFFFF;--shiki-default-font-style:italic;--shiki-dark:#BABED8;--shiki-dark-font-style:italic}html pre.shiki code .sbssI, html code.shiki .sbssI{--shiki-light:#F76D47;--shiki-default:#F78C6C;--shiki-dark:#F78C6C}",{"title":261,"searchDepth":262,"depth":262,"links":1914},[1915,1916,1917,1918,1919,1923,1930],{"id":310,"depth":262,"text":311},{"id":321,"depth":262,"text":322},{"id":498,"depth":262,"text":499},{"id":835,"depth":262,"text":836},{"id":1144,"depth":262,"text":1145,"children":1920},[1921,1922],{"id":1151,"depth":265,"text":1152},{"id":1281,"depth":265,"text":1282},{"id":1369,"depth":262,"text":1370,"children":1924},[1925,1926,1927,1928,1929],{"id":1373,"depth":265,"text":1374},{"id":1508,"depth":265,"text":1509},{"id":1618,"depth":265,"text":1201},{"id":1710,"depth":265,"text":1247},{"id":1801,"depth":265,"text":1258},{"id":1892,"depth":262,"text":1893},"2026-06-02","Calling native libraries from Dart with dart:ffi, and which languages work.","\u002Fblog\u002Fdart-ffi-native-bridge\u002Fcover.jpg",{"tags":1935},[1936,1937],"Dart","FFI",{"title":18,"description":1932},"ZhFB4P4xj5eoA7PeGlInQmuG8DbLq9e34CcYib-16dc",{"id":1941,"title":38,"author":1942,"body":1944,"date":2139,"description":2140,"extension":273,"image":2141,"meta":2142,"minRead":262,"navigation":53,"path":39,"seo":2146,"stem":40,"__hash__":2147},"blog\u002Fblog\u002Funderstand-refresh-token.md",{"name":60,"avatar":1943},{"src":62,"alt":60},{"type":64,"value":1945,"toc":2132},[1946,1953,1956,1960,1963,1984,1987,1991,1994,2008,2011,2015,2018,2081,2087,2107,2111,2122,2125,2129],[67,1947,1948,1949,1952],{},"If you rotate refresh tokens, every refresh hands back a new access token ",[257,1950,1951],{},"and"," a new refresh token. So why set the refresh token to 7 days? If it gets replaced anyway, why not 30 minutes?",[67,1954,1955],{},"Because the expiry is not about how long the token lives. It is about how long a user can be away.",[78,1957,1959],{"id":1958},"a-30-minute-token-breaks-lunch","A 30-minute token breaks lunch",[67,1961,1962],{},"Say you set the refresh token to 30 minutes, for extra security.",[1964,1965,1966,1972,1978],"ol",{},[117,1967,1968,1971],{},[71,1969,1970],{},"12:00",": the user logs in. Access token: 15 minutes. Refresh token: 30 minutes.",[117,1973,1974,1977],{},[71,1975,1976],{},"12:15",": they finish what they were doing and close the laptop to get lunch.",[117,1979,1980,1983],{},[71,1981,1982],{},"12:50",": they come back and open the app.",[67,1985,1986],{},"Both tokens are dead. The access token expired at 12:15, the refresh token at 12:30. The user has to type their password again because they stepped away for 35 minutes, and they blame your app for it.",[78,1988,1990],{"id":1989},"_7-days-is-a-sliding-window","7 days is a sliding window",[67,1992,1993],{},"With a 7-day refresh token you get a window that keeps moving:",[114,1995,1996,2002],{},[117,1997,1998,2001],{},[71,1999,2000],{},"Active user:"," every refresh resets the 7-day timer. As long as they keep using the app, they stay logged in.",[117,2003,2004,2007],{},[71,2005,2006],{},"User on holiday:"," no app for 10 days, the timer runs out, and they log in again when they return.",[67,2009,2010],{},"That is the real setting you are choosing: how long inactivity is allowed before a password is required.",[78,2012,2014],{"id":2013},"expiry-is-the-clock-revocation-is-the-kill-switch","Expiry is the clock, revocation is the kill switch",[67,2016,2017],{},"Expiry is a token dying of old age. Revocation is you killing it on purpose.",[192,2019,2020,2034],{},[195,2021,2022],{},[198,2023,2024,2028,2031],{},[201,2025,2027],{"align":2026},"left","Feature",[201,2029,2030],{"align":2026},"Expiration",[201,2032,2033],{"align":2026},"Revocation",[208,2035,2036,2049,2062],{},[198,2037,2038,2043,2046],{},[213,2039,2040],{"align":2026},[71,2041,2042],{},"Controlled by",[213,2044,2045],{"align":2026},"The clock",[213,2047,2048],{"align":2026},"The admin \u002F server",[198,2050,2051,2056,2059],{},[213,2052,2053],{"align":2026},[71,2054,2055],{},"Question",[213,2057,2058],{"align":2026},"\"Is this token too old?\"",[213,2060,2061],{"align":2026},"\"Is this token still allowed?\"",[198,2063,2064,2069,2075],{},[213,2065,2066],{"align":2026},[71,2067,2068],{},"Database",[213,2070,2071,2072],{"align":2026},"Checked via ",[131,2073,2074],{},"expiry_date",[213,2076,2071,2077,2080],{"align":2026},[131,2078,2079],{},"is_revoked"," column",[67,2082,2083,2084,2086],{},"That ",[131,2085,2079],{}," column earns its place three times over:",[114,2088,2089,2095,2101],{},[117,2090,2091,2094],{},[71,2092,2093],{},"Logout."," The server marks the token revoked, so it can never be used again.",[117,2096,2097,2100],{},[71,2098,2099],{},"Password reset."," Revoke every token for that user and all their sessions drop at once.",[117,2102,2103,2106],{},[71,2104,2105],{},"Stolen token."," If someone replays an old rotated token, the server sees it is already revoked and can lock the account.",[78,2108,2110],{"id":2109},"rotation-step-by-step","Rotation, step by step",[1964,2112,2113,2116,2119],{},[117,2114,2115],{},"The client logs in with a username and password. The server stores refresh token 1 with a 7-day expiry and returns access token 1 and refresh token 1.",[117,2117,2118],{},"Access token 1 expires. The client asks for a refresh and sends refresh token 1.",[117,2120,2121],{},"The server checks that refresh token 1 is valid and not revoked, marks it revoked, stores a new refresh token 2 with a fresh 7-day window, and returns access token 2 and refresh token 2.",[67,2123,2124],{},"Step 3 is the whole trick. The old token is revoked the moment it is used, so any later attempt to reuse it is both rejected and a signal that something is wrong.",[78,2126,2128],{"id":2127},"in-short","In short",[67,2130,2131],{},"The 7-day expiry is your inactivity limit: it keeps active users logged in. Rotation and revocation are what make a long-lived session safe to hand out.",{"title":261,"searchDepth":262,"depth":262,"links":2133},[2134,2135,2136,2137,2138],{"id":1958,"depth":265,"text":1959},{"id":1989,"depth":265,"text":1990},{"id":2013,"depth":265,"text":2014},{"id":2109,"depth":265,"text":2110},{"id":2127,"depth":265,"text":2128},"2026-01-21","Why a refresh token lives for 7 days, and what the revoke column is for.","\u002Fblog\u002Funderstand-refresh-token\u002Fcover.jpg",{"tags":2143},[2144,2145],"Security","Authentication",{"title":38,"description":2140},"F_pD6V8k8F6n66V8xTQzFATiIVrA5DSsQTzSej3E8pQ",{"id":2149,"title":14,"author":2150,"body":2152,"date":2753,"description":2754,"extension":273,"image":2340,"meta":2755,"minRead":262,"navigation":53,"path":15,"seo":2758,"stem":16,"__hash__":2759},"blog\u002Fblog\u002Fc-translate.md",{"name":60,"avatar":2151},{"src":62,"alt":60},{"type":64,"value":2153,"toc":2746},[2154,2157,2160,2164,2184,2188,2213,2222,2226,2282,2293,2304,2332,2335,2341,2345,2349,2359,2373,2376,2461,2485,2658,2661,2664,2669,2716,2720,2723,2743],[67,2155,2156],{},"Whisper transcribes well, but the Python implementation is slow and eats memory. That is a problem when the goal is Khmer transcription running on a cheap device, close to real time.",[67,2158,2159],{},"CTranslate2 fixes most of it. Here is the conversion I use on my own fine-tuned model.",[78,2161,2163],{"id":2162},"what-faster-whisper-is","What faster-whisper is",[67,2165,2166,2171,2172,2177,2178,2183],{},[93,2167,2170],{"href":2168,"rel":2169},"https:\u002F\u002Fgithub.com\u002FSYSTRAN\u002Ffaster-whisper",[97],"faster-whisper"," reimplements Whisper on top of ",[93,2173,2176],{"href":2174,"rel":2175},"https:\u002F\u002Fopennmt.net\u002FCTranslate2\u002Findex.html#",[97],"CTranslate2",", an inference engine for Transformer models. It runs up to 4x faster than ",[93,2179,2182],{"href":2180,"rel":2181},"https:\u002F\u002Fhuggingface.co\u002Fopenai\u002Fwhisper-tiny",[97],"openai\u002Fwhisper"," on less memory, and 8-bit quantization buys you more on both CPU and GPU.",[78,2185,2187],{"id":2186},"step-1-install-the-converter","Step 1: Install the converter",[335,2189,2191],{"className":460,"code":2190,"language":462,"meta":261,"style":261},"pip install transformers\npip install ctranslate2\n",[131,2192,2193,2204],{"__ignoreMap":261},[343,2194,2195,2198,2201],{"class":345,"line":346},[343,2196,2197],{"class":469},"pip",[343,2199,2200],{"class":363}," install",[343,2202,2203],{"class":363}," transformers\n",[343,2205,2206,2208,2210],{"class":345,"line":262},[343,2207,2197],{"class":469},[343,2209,2200],{"class":363},[343,2211,2212],{"class":363}," ctranslate2\n",[67,2214,2215,2216,2221],{},"CTranslate2 only supports a subset of Hugging Face Transformers models, so check that yours is ",[93,2217,2220],{"href":2218,"rel":2219},"https:\u002F\u002Fopennmt.net\u002FCTranslate2\u002Fguides\u002Ftransformers.html",[97],"on the list"," before you go further.",[78,2223,2225],{"id":2224},"step-2-convert-the-model","Step 2: Convert the model",[335,2227,2229],{"className":460,"code":2228,"language":462,"meta":261,"style":261},"ct2-transformers-converter --model \u003Cmodel_id> --output_dir \u003Coutput_path> --copy_files tokenizer_config.json --quantization \u003Ctypes>\n",[131,2230,2231],{"__ignoreMap":261},[343,2232,2233,2236,2239,2241,2244,2247,2250,2253,2255,2258,2261,2263,2266,2269,2272,2274,2277,2280],{"class":345,"line":346},[343,2234,2235],{"class":469},"ct2-transformers-converter",[343,2237,2238],{"class":363}," --model",[343,2240,360],{"class":359},[343,2242,2243],{"class":363},"model_i",[343,2245,2246],{"class":434},"d",[343,2248,2249],{"class":359},">",[343,2251,2252],{"class":363}," --output_dir",[343,2254,360],{"class":359},[343,2256,2257],{"class":363},"output_pat",[343,2259,2260],{"class":434},"h",[343,2262,2249],{"class":359},[343,2264,2265],{"class":363}," --copy_files",[343,2267,2268],{"class":363}," tokenizer_config.json",[343,2270,2271],{"class":363}," --quantization",[343,2273,360],{"class":359},[343,2275,2276],{"class":363},"type",[343,2278,2279],{"class":434},"s",[343,2281,367],{"class":359},[67,2283,2284,2287,2288,134],{},[131,2285,2286],{},"--quantization int8"," is where the size and speed win comes from, and ",[93,2289,2292],{"href":2290,"rel":2291},"https:\u002F\u002Fopennmt.net\u002FCTranslate2\u002Fquantization.html",[97],"the docs explain the trade-off",[67,2294,2295,2296,2303],{},"For my fine-tuned model, ",[93,2297,2300],{"href":2298,"rel":2299},"https:\u002F\u002Fhuggingface.co\u002FPhanithLIM\u002Fwhisper-tiny-aug-29dec",[97],[131,2301,2302],{},"PhanithLIM\u002Fwhisper-tiny-aug-29dec",":",[335,2305,2307],{"className":460,"code":2306,"language":462,"meta":261,"style":261},"ct2-transformers-converter --model PhanithLIM\u002Fwhisper-tiny-aug-29dec --output_dir whisper-tiny-int8 --copy_files tokenizer_config.json --quantization int8\n",[131,2308,2309],{"__ignoreMap":261},[343,2310,2311,2313,2315,2318,2320,2323,2325,2327,2329],{"class":345,"line":346},[343,2312,2235],{"class":469},[343,2314,2238],{"class":363},[343,2316,2317],{"class":363}," PhanithLIM\u002Fwhisper-tiny-aug-29dec",[343,2319,2252],{"class":363},[343,2321,2322],{"class":363}," whisper-tiny-int8",[343,2324,2265],{"class":363},[343,2326,2268],{"class":363},[343,2328,2271],{"class":363},[343,2330,2331],{"class":363}," int8\n",[67,2333,2334],{},"The converted model lands in the output directory:",[67,2336,2337],{},[85,2338],{"alt":2339,"src":2340},"Converted model files stored in the output directory","\u002Fblog\u002Fc-translate\u002Fimage1.png",[67,2342,2343],{},[257,2344,2339],{},[78,2346,2348],{"id":2347},"step-3-transcribe-something","Step 3: Transcribe something",[67,2350,2351,2352,2358],{},"Install ",[93,2353,2356],{"href":2354,"rel":2355},"https:\u002F\u002Fpypi.org\u002Fproject\u002Ffaster-whisper\u002F",[97],[131,2357,2170],{}," to run the converted model:",[335,2360,2362],{"className":460,"code":2361,"language":462,"meta":261,"style":261},"pip install faster-whisper\n",[131,2363,2364],{"__ignoreMap":261},[343,2365,2366,2368,2370],{"class":345,"line":346},[343,2367,2197],{"class":469},[343,2369,2200],{"class":363},[343,2371,2372],{"class":363}," faster-whisper\n",[67,2374,2375],{},"Then point it at the local directory:",[335,2377,2381],{"className":2378,"code":2379,"language":2380,"meta":261,"style":261},"language-python shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","from faster_whisper import WhisperModel\nmodel_size = \"\u002Fcontent\u002Fwhisper-tiny-int8\"\nmodel = WhisperModel(model_size, device=\"cpu\", compute_type=\"int8\", local_files_only=True)\n","python",[131,2382,2383,2396,2410],{"__ignoreMap":261},[343,2384,2385,2388,2391,2393],{"class":345,"line":346},[343,2386,2387],{"class":355},"from",[343,2389,2390],{"class":434}," faster_whisper ",[343,2392,542],{"class":355},[343,2394,2395],{"class":434}," WhisperModel\n",[343,2397,2398,2401,2403,2405,2408],{"class":345,"line":262},[343,2399,2400],{"class":434},"model_size ",[343,2402,742],{"class":359},[343,2404,382],{"class":359},[343,2406,2407],{"class":363},"\u002Fcontent\u002Fwhisper-tiny-int8",[343,2409,549],{"class":359},[343,2411,2412,2415,2417,2420,2422,2425,2427,2430,2432,2434,2437,2439,2441,2444,2446,2448,2451,2453,2455,2458],{"class":345,"line":265},[343,2413,2414],{"class":434},"model ",[343,2416,742],{"class":359},[343,2418,2419],{"class":400}," WhisperModel",[343,2421,404],{"class":359},[343,2423,2424],{"class":400},"model_size",[343,2426,414],{"class":359},[343,2428,2429],{"class":410}," device",[343,2431,742],{"class":359},[343,2433,388],{"class":359},[343,2435,2436],{"class":363},"cpu",[343,2438,388],{"class":359},[343,2440,414],{"class":359},[343,2442,2443],{"class":410}," compute_type",[343,2445,742],{"class":359},[343,2447,388],{"class":359},[343,2449,2450],{"class":363},"int8",[343,2452,388],{"class":359},[343,2454,414],{"class":359},[343,2456,2457],{"class":410}," local_files_only",[343,2459,2460],{"class":359},"=True)\n",[2462,2463,2464,2469,2478],"blockquote",{},[67,2465,2466],{},[71,2467,2468],{},"Local or Hugging Face",[67,2470,2471,2474,2475,134],{},[131,2472,2473],{},"local_files_only=True"," loads from the directory on disk. If you pushed the converted model to Hugging Face instead, set it to ",[131,2476,2477],{},"False",[67,2479,2480,2481,2484],{},"For GPU, change ",[131,2482,2483],{},"device=\"cuda\"",". The difference is large.",[335,2486,2488],{"className":2378,"code":2487,"language":2380,"meta":261,"style":261},"segments, info = model.transcribe(\"\u002Fcontent\u002F1_0.wav\", beam_size=8)\nprint(\"Detected language '%s' with probability %f\" % (info.language, info.language_probability))\nfor segment in segments:\n    print(\"[%.2fs -> %.2fs] %s\" % (segment.start, segment.end, segment.text))\n",[131,2489,2490,2531,2581,2598],{"__ignoreMap":261},[343,2491,2492,2495,2497,2500,2502,2505,2507,2510,2512,2514,2517,2519,2521,2524,2526,2529],{"class":345,"line":346},[343,2493,2494],{"class":434},"segments",[343,2496,414],{"class":359},[343,2498,2499],{"class":434}," info ",[343,2501,742],{"class":359},[343,2503,2504],{"class":434}," model",[343,2506,134],{"class":359},[343,2508,2509],{"class":400},"transcribe",[343,2511,404],{"class":359},[343,2513,388],{"class":359},[343,2515,2516],{"class":363},"\u002Fcontent\u002F1_0.wav",[343,2518,388],{"class":359},[343,2520,414],{"class":359},[343,2522,2523],{"class":410}," beam_size",[343,2525,742],{"class":359},[343,2527,2528],{"class":645},"8",[343,2530,694],{"class":359},[343,2532,2533,2535,2537,2539,2542,2545,2548,2551,2553,2556,2559,2562,2564,2568,2570,2573,2575,2578],{"class":345,"line":262},[343,2534,1099],{"class":400},[343,2536,404],{"class":359},[343,2538,388],{"class":359},[343,2540,2541],{"class":363},"Detected language '",[343,2543,2544],{"class":645},"%s",[343,2546,2547],{"class":363},"' with probability ",[343,2549,2550],{"class":645},"%f",[343,2552,388],{"class":359},[343,2554,2555],{"class":359}," %",[343,2557,2558],{"class":359}," (",[343,2560,2561],{"class":400},"info",[343,2563,134],{"class":359},[343,2565,2567],{"class":2566},"swJcz","language",[343,2569,414],{"class":359},[343,2571,2572],{"class":400}," info",[343,2574,134],{"class":359},[343,2576,2577],{"class":2566},"language_probability",[343,2579,2580],{"class":359},"))\n",[343,2582,2583,2586,2589,2592,2595],{"class":345,"line":265},[343,2584,2585],{"class":355},"for",[343,2587,2588],{"class":434}," segment ",[343,2590,2591],{"class":355},"in",[343,2593,2594],{"class":434}," segments",[343,2596,2597],{"class":359},":\n",[343,2599,2600,2603,2605,2607,2610,2613,2616,2618,2621,2623,2625,2627,2629,2632,2634,2637,2639,2642,2644,2647,2649,2651,2653,2656],{"class":345,"line":375},[343,2601,2602],{"class":400},"    print",[343,2604,404],{"class":359},[343,2606,388],{"class":359},[343,2608,2609],{"class":363},"[",[343,2611,2612],{"class":645},"%.2f",[343,2614,2615],{"class":363},"s -> ",[343,2617,2612],{"class":645},[343,2619,2620],{"class":363},"s] ",[343,2622,2544],{"class":645},[343,2624,388],{"class":359},[343,2626,2555],{"class":359},[343,2628,2558],{"class":359},[343,2630,2631],{"class":400},"segment",[343,2633,134],{"class":359},[343,2635,2636],{"class":2566},"start",[343,2638,414],{"class":359},[343,2640,2641],{"class":400}," segment",[343,2643,134],{"class":359},[343,2645,2646],{"class":2566},"end",[343,2648,414],{"class":359},[343,2650,2641],{"class":400},[343,2652,134],{"class":359},[343,2654,2655],{"class":2566},"text",[343,2657,2580],{"class":359},[67,2659,2660],{},"On a 3 minute 2 second file this took 1 minute on CPU and 5 seconds on GPU.",[67,2662,2663],{},"The Khmer output itself is rough. That is not CTranslate2's fault: this is the tiny model trained on a small dataset, and the conversion keeps whatever accuracy you started with.",[67,2665,2666],{},[71,2667,2668],{},"Transcription output",[335,2670,2674],{"className":2671,"code":2672,"language":2673,"meta":261,"style":261},"language-txt shiki shiki-themes material-theme-lighter material-theme material-theme-palenight","Detected language 'km' with probability 0.998886\n[0.00s -> 30.00s]ក្រសួងសុខាភិបាលកាលពីថ្ងៃចិន្នបានធ្វើការអំពីនាលដល់បង់ប្អូនប្រជាពលរត់ទាំងឱ្យមានការប្រុមប្រយៈការពីសុខភាពពី\n[30.00s -> 60.00s] ក្យមដូចជាក្រុមប្ដាសាយ ហូរសម្បរ ប្រលាក់បំពង្ករកកកន្តាស៍ ជួមមេនមេតភ្នោះមីតភ្នំពី ៦៦ឆ្នាំ ឬអ្នកមានពីរកម្ម ក\n[60.00s -> 90.00s] ាពី ភាគច្រើន បានជាសាស្ប្បីឡើងវិញ គាត់ វាជាសាស្ត្របន្នស្មានថា ប្រហែល ១១២០ ទៅ ១២០ នៃភាគច្រើន គឺស្រាល ប៉ុន្ត\n[90.00s -> 120.00s] ទីដែលមានមានរក បន្ទាប់មក ប៉ុន្មត្មម៉ុន ឬភ្នេះ ដ៏ល្លែងសញ្ញា នៃជំងឺនេះ រួមមាន ក៏ ក្រុងក្ដៅ ទឹងច្រម៉ុស ហូរសម្ប\n[120.00s -> 150.00s] កានអាណាម៉ៃ លានសម្អាតដៃឱ្យ បានមានញាប់ជាមួយសប្បូរ ឬទឹកឱ្យកដែលប៉ះពាល់ភ្នេកច្រុមមក និងមាត់ ប្រសិម្បល់មិនពាន\n[150.00s -> 180.00s] ទឹងបរិខា បុរិភោគជាមួយអ្នកដទ្រី នឹងពាក្យសុំស្លៀប ពាក់សម្លៀបសម្លៀបបំពាក្ក្នសំស្រប ដើម្បីបង្ការភាពប្រជាកព\n[180.00s -> 181.82s] សប្បនេះមិត្តសេរីការអាយ។\n","txt",[131,2675,2676,2681,2686,2691,2696,2701,2706,2711],{"__ignoreMap":261},[343,2677,2678],{"class":345,"line":346},[343,2679,2680],{},"Detected language 'km' with probability 0.998886\n",[343,2682,2683],{"class":345,"line":262},[343,2684,2685],{},"[0.00s -> 30.00s]ក្រសួងសុខាភិបាលកាលពីថ្ងៃចិន្នបានធ្វើការអំពីនាលដល់បង់ប្អូនប្រជាពលរត់ទាំងឱ្យមានការប្រុមប្រយៈការពីសុខភាពពី\n",[343,2687,2688],{"class":345,"line":265},[343,2689,2690],{},"[30.00s -> 60.00s] ក្យមដូចជាក្រុមប្ដាសាយ ហូរសម្បរ ប្រលាក់បំពង្ករកកកន្តាស៍ ជួមមេនមេតភ្នោះមីតភ្នំពី ៦៦ឆ្នាំ ឬអ្នកមានពីរកម្ម ក\n",[343,2692,2693],{"class":345,"line":375},[343,2694,2695],{},"[60.00s -> 90.00s] ាពី ភាគច្រើន បានជាសាស្ប្បីឡើងវិញ គាត់ វាជាសាស្ត្របន្នស្មានថា ប្រហែល ១១២០ ទៅ ១២០ នៃភាគច្រើន គឺស្រាល ប៉ុន្ត\n",[343,2697,2698],{"class":345,"line":394},[343,2699,2700],{},"[90.00s -> 120.00s] ទីដែលមានមានរក បន្ទាប់មក ប៉ុន្មត្មម៉ុន ឬភ្នេះ ដ៏ល្លែងសញ្ញា នៃជំងឺនេះ រួមមាន ក៏ ក្រុងក្ដៅ ទឹងច្រម៉ុស ហូរសម្ប\n",[343,2702,2703],{"class":345,"line":428},[343,2704,2705],{},"[120.00s -> 150.00s] កានអាណាម៉ៃ លានសម្អាតដៃឱ្យ បានមានញាប់ជាមួយសប្បូរ ឬទឹកឱ្យកដែលប៉ះពាល់ភ្នេកច្រុមមក និងមាត់ ប្រសិម្បល់មិនពាន\n",[343,2707,2708],{"class":345,"line":445},[343,2709,2710],{},"[150.00s -> 180.00s] ទឹងបរិខា បុរិភោគជាមួយអ្នកដទ្រី នឹងពាក្យសុំស្លៀប ពាក់សម្លៀបសម្លៀបបំពាក្ក្នសំស្រប ដើម្បីបង្ការភាពប្រជាកព\n",[343,2712,2713],{"class":345,"line":451},[343,2714,2715],{},"[180.00s -> 181.82s] សប្បនេះមិត្តសេរីការអាយ។\n",[78,2717,2719],{"id":2718},"worth-it","Worth it?",[67,2721,2722],{},"Yes, if you need speech recognition on a machine without a big GPU. The conversion takes a minute and the model gets smaller and faster for it. Accuracy still comes from your training data.",[335,2724,2726],{"className":2378,"code":2725,"language":2380,"meta":261,"style":261},"print(\"Thank you for reading!\")\n",[131,2727,2728],{"__ignoreMap":261},[343,2729,2730,2732,2734,2736,2739,2741],{"class":345,"line":346},[343,2731,1099],{"class":400},[343,2733,404],{"class":359},[343,2735,388],{"class":359},[343,2737,2738],{"class":363},"Thank you for reading!",[343,2740,388],{"class":359},[343,2742,694],{"class":359},[1910,2744,2745],{},"html pre.shiki code .s7zQu, html code.shiki .s7zQu{--shiki-light:#39ADB5;--shiki-light-font-style:italic;--shiki-default:#89DDFF;--shiki-default-font-style:italic;--shiki-dark:#89DDFF;--shiki-dark-font-style:italic}html pre.shiki code .sTEyZ, html code.shiki .sTEyZ{--shiki-light:#90A4AE;--shiki-default:#EEFFFF;--shiki-dark:#BABED8}html pre.shiki code .sMK4o, html code.shiki .sMK4o{--shiki-light:#39ADB5;--shiki-default:#89DDFF;--shiki-dark:#89DDFF}html pre.shiki code .sfazB, html code.shiki .sfazB{--shiki-light:#91B859;--shiki-default:#C3E88D;--shiki-dark:#C3E88D}html pre.shiki code .s2Zo4, html code.shiki .s2Zo4{--shiki-light:#6182B8;--shiki-default:#82AAFF;--shiki-dark:#82AAFF}html pre.shiki code .sHdIc, html code.shiki .sHdIc{--shiki-light:#90A4AE;--shiki-light-font-style:italic;--shiki-default:#EEFFFF;--shiki-default-font-style:italic;--shiki-dark:#BABED8;--shiki-dark-font-style:italic}html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html pre.shiki code .sbssI, html code.shiki .sbssI{--shiki-light:#F76D47;--shiki-default:#F78C6C;--shiki-dark:#F78C6C}html pre.shiki code .swJcz, html code.shiki .swJcz{--shiki-light:#E53935;--shiki-default:#F07178;--shiki-dark:#F07178}html pre.shiki code .sBMFI, html code.shiki .sBMFI{--shiki-light:#E2931D;--shiki-default:#FFCB6B;--shiki-dark:#FFCB6B}",{"title":261,"searchDepth":262,"depth":262,"links":2747},[2748,2749,2750,2751,2752],{"id":2162,"depth":265,"text":2163},{"id":2186,"depth":265,"text":2187},{"id":2224,"depth":265,"text":2225},{"id":2347,"depth":265,"text":2348},{"id":2718,"depth":265,"text":2719},"2025-03-08","Converting Whisper with CTranslate2 so it runs faster and lighter.",{"tags":2756},[2757,1300],"Machine Learning",{"title":14,"description":2754},"cu1Mi_XASSFchhQZamQlpT3UFnugSsCOJaltpeLeSJA",{"id":2761,"title":30,"author":2762,"body":2764,"date":3695,"description":3696,"extension":273,"image":3697,"meta":3698,"minRead":265,"navigation":53,"path":31,"seo":3700,"stem":32,"__hash__":3701},"blog\u002Fblog\u002Fpydantic-llm.md",{"name":60,"avatar":2763},{"src":62,"alt":60},{"type":64,"value":2765,"toc":3686},[2766,2769,2772,2776,2817,2838,2842,2932,2936,2943,3132,3136,3142,3197,3200,3204,3211,3260,3264,3267,3411,3421,3425,3480,3658,3665,3683],[67,2767,2768],{},"An LLM will happily answer \"give me the top 10 phones\" in prose. What I want is rows I can put in a DataFrame.",[67,2770,2771],{},"Here is how I do that on my own machine: Pydantic describes the shape I expect, LangChain's output parser forces the model to answer in that shape, and pandas takes it from there. Nothing leaves the laptop.",[78,2773,2775],{"id":2774},"_1-install","1. Install",[335,2777,2779],{"className":460,"code":2778,"language":462,"meta":261,"style":261},"pip install langchain\npip install ollama\npip install pydantic\npip install pandas\n",[131,2780,2781,2790,2799,2808],{"__ignoreMap":261},[343,2782,2783,2785,2787],{"class":345,"line":346},[343,2784,2197],{"class":469},[343,2786,2200],{"class":363},[343,2788,2789],{"class":363}," langchain\n",[343,2791,2792,2794,2796],{"class":345,"line":262},[343,2793,2197],{"class":469},[343,2795,2200],{"class":363},[343,2797,2798],{"class":363}," ollama\n",[343,2800,2801,2803,2805],{"class":345,"line":265},[343,2802,2197],{"class":469},[343,2804,2200],{"class":363},[343,2806,2807],{"class":363}," pydantic\n",[343,2809,2810,2812,2814],{"class":345,"line":375},[343,2811,2197],{"class":469},[343,2813,2200],{"class":363},[343,2815,2816],{"class":363}," pandas\n",[2462,2818,2819,2824],{},[67,2820,2821],{},[71,2822,2823],{},"You need Ollama running",[67,2825,2351,2826,2831,2832,2837],{},[93,2827,2830],{"href":2828,"rel":2829},"https:\u002F\u002Follama.com\u002F",[97],"Ollama",", open it, and let the server start. If it is new to you, these ",[93,2833,2836],{"href":2834,"rel":2835},"https:\u002F\u002Fwww.hostinger.com\u002Ftutorials\u002Follama-cli-tutorial",[97],"command-line basics"," cover enough to get going.",[78,2839,2841],{"id":2840},"_2-imports","2. Imports",[335,2843,2845],{"className":2378,"code":2844,"language":2380,"meta":261,"style":261},"from pydantic import BaseModel, Field\nfrom langchain_ollama.llms import OllamaLLM\nfrom langchain.output_parsers import PydanticOutputParser\nfrom langchain.prompts import HumanMessagePromptTemplate, ChatPromptTemplate\nimport pandas as pd\n",[131,2846,2847,2864,2881,2898,2919],{"__ignoreMap":261},[343,2848,2849,2851,2854,2856,2859,2861],{"class":345,"line":346},[343,2850,2387],{"class":355},[343,2852,2853],{"class":434}," pydantic ",[343,2855,542],{"class":355},[343,2857,2858],{"class":434}," BaseModel",[343,2860,414],{"class":359},[343,2862,2863],{"class":434}," Field\n",[343,2865,2866,2868,2871,2873,2876,2878],{"class":345,"line":262},[343,2867,2387],{"class":355},[343,2869,2870],{"class":434}," langchain_ollama",[343,2872,134],{"class":359},[343,2874,2875],{"class":434},"llms ",[343,2877,542],{"class":355},[343,2879,2880],{"class":434}," OllamaLLM\n",[343,2882,2883,2885,2888,2890,2893,2895],{"class":345,"line":265},[343,2884,2387],{"class":355},[343,2886,2887],{"class":434}," langchain",[343,2889,134],{"class":359},[343,2891,2892],{"class":434},"output_parsers ",[343,2894,542],{"class":355},[343,2896,2897],{"class":434}," PydanticOutputParser\n",[343,2899,2900,2902,2904,2906,2909,2911,2914,2916],{"class":345,"line":375},[343,2901,2387],{"class":355},[343,2903,2887],{"class":434},[343,2905,134],{"class":359},[343,2907,2908],{"class":434},"prompts ",[343,2910,542],{"class":355},[343,2912,2913],{"class":434}," HumanMessagePromptTemplate",[343,2915,414],{"class":359},[343,2917,2918],{"class":434}," ChatPromptTemplate\n",[343,2920,2921,2923,2926,2929],{"class":345,"line":394},[343,2922,542],{"class":355},[343,2924,2925],{"class":434}," pandas ",[343,2927,2928],{"class":355},"as",[343,2930,2931],{"class":434}," pd\n",[78,2933,2935],{"id":2934},"_3-describe-what-you-want-back","3. Describe what you want back",[67,2937,2938,2939,2942],{},"The Pydantic model is the contract. Each ",[131,2940,2941],{},"description"," is a hint the model actually reads, so it is worth writing them properly.",[335,2944,2946],{"className":2378,"code":2945,"language":2380,"meta":261,"style":261},"class PhoneModel(BaseModel):\n    brand: str = Field(description=\"Brand of the phone\")\n    model: str = Field(description=\"Model of the phone\")\n    year: int = Field(description=\"Year of the phone\")\n    price: float = Field(description=\"Price of the phone\")\n\nclass PhonesModel(BaseModel):\n    phones: list[PhoneModel] = Field(description=\"List of phones\")\n",[131,2947,2948,2963,2993,3021,3049,3078,3082,3095],{"__ignoreMap":261},[343,2949,2950,2952,2955,2957,2960],{"class":345,"line":346},[343,2951,941],{"class":378},[343,2953,2954],{"class":469}," PhoneModel",[343,2956,404],{"class":359},[343,2958,2959],{"class":469},"BaseModel",[343,2961,2962],{"class":359},"):\n",[343,2964,2965,2968,2970,2973,2975,2978,2980,2982,2984,2986,2989,2991],{"class":345,"line":262},[343,2966,2967],{"class":434},"    brand",[343,2969,2303],{"class":359},[343,2971,2972],{"class":469}," str",[343,2974,893],{"class":359},[343,2976,2977],{"class":400}," Field",[343,2979,404],{"class":359},[343,2981,2941],{"class":410},[343,2983,742],{"class":359},[343,2985,388],{"class":359},[343,2987,2988],{"class":363},"Brand of the phone",[343,2990,388],{"class":359},[343,2992,694],{"class":359},[343,2994,2995,2998,3000,3002,3004,3006,3008,3010,3012,3014,3017,3019],{"class":345,"line":265},[343,2996,2997],{"class":434},"    model",[343,2999,2303],{"class":359},[343,3001,2972],{"class":469},[343,3003,893],{"class":359},[343,3005,2977],{"class":400},[343,3007,404],{"class":359},[343,3009,2941],{"class":410},[343,3011,742],{"class":359},[343,3013,388],{"class":359},[343,3015,3016],{"class":363},"Model of the phone",[343,3018,388],{"class":359},[343,3020,694],{"class":359},[343,3022,3023,3026,3028,3030,3032,3034,3036,3038,3040,3042,3045,3047],{"class":345,"line":375},[343,3024,3025],{"class":434},"    year",[343,3027,2303],{"class":359},[343,3029,921],{"class":469},[343,3031,893],{"class":359},[343,3033,2977],{"class":400},[343,3035,404],{"class":359},[343,3037,2941],{"class":410},[343,3039,742],{"class":359},[343,3041,388],{"class":359},[343,3043,3044],{"class":363},"Year of the phone",[343,3046,388],{"class":359},[343,3048,694],{"class":359},[343,3050,3051,3054,3056,3059,3061,3063,3065,3067,3069,3071,3074,3076],{"class":345,"line":394},[343,3052,3053],{"class":434},"    price",[343,3055,2303],{"class":359},[343,3057,3058],{"class":469}," float",[343,3060,893],{"class":359},[343,3062,2977],{"class":400},[343,3064,404],{"class":359},[343,3066,2941],{"class":410},[343,3068,742],{"class":359},[343,3070,388],{"class":359},[343,3072,3073],{"class":363},"Price of the phone",[343,3075,388],{"class":359},[343,3077,694],{"class":359},[343,3079,3080],{"class":345,"line":428},[343,3081,372],{"emptyLinePlaceholder":53},[343,3083,3084,3086,3089,3091,3093],{"class":345,"line":445},[343,3085,941],{"class":378},[343,3087,3088],{"class":469}," PhonesModel",[343,3090,404],{"class":359},[343,3092,2959],{"class":469},[343,3094,2962],{"class":359},[343,3096,3097,3100,3102,3105,3107,3110,3113,3115,3117,3119,3121,3123,3125,3128,3130],{"class":345,"line":451},[343,3098,3099],{"class":434},"    phones",[343,3101,2303],{"class":359},[343,3103,3104],{"class":434}," list",[343,3106,2609],{"class":359},[343,3108,3109],{"class":434},"PhoneModel",[343,3111,3112],{"class":359},"]",[343,3114,893],{"class":359},[343,3116,2977],{"class":400},[343,3118,404],{"class":359},[343,3120,2941],{"class":410},[343,3122,742],{"class":359},[343,3124,388],{"class":359},[343,3126,3127],{"class":363},"List of phones",[343,3129,388],{"class":359},[343,3131,694],{"class":359},[78,3133,3135],{"id":3134},"_4-turn-the-schema-into-instructions","4. Turn the schema into instructions",[67,3137,3138,3141],{},[131,3139,3140],{},"PydanticOutputParser"," writes the formatting instructions for you and parses the reply back into your objects.",[335,3143,3145],{"className":2378,"code":3144,"language":2380,"meta":261,"style":261},"output_parser = PydanticOutputParser(pydantic_object = PhonesModel)\nformat_instructions = output_parser.get_format_instructions()\nprint(format_instructions)\n",[131,3146,3147,3168,3186],{"__ignoreMap":261},[343,3148,3149,3152,3154,3157,3159,3162,3164,3166],{"class":345,"line":346},[343,3150,3151],{"class":434},"output_parser ",[343,3153,742],{"class":359},[343,3155,3156],{"class":400}," PydanticOutputParser",[343,3158,404],{"class":359},[343,3160,3161],{"class":410},"pydantic_object",[343,3163,893],{"class":359},[343,3165,3088],{"class":400},[343,3167,694],{"class":359},[343,3169,3170,3173,3175,3178,3180,3183],{"class":345,"line":262},[343,3171,3172],{"class":434},"format_instructions ",[343,3174,742],{"class":359},[343,3176,3177],{"class":434}," output_parser",[343,3179,134],{"class":359},[343,3181,3182],{"class":400},"get_format_instructions",[343,3184,3185],{"class":359},"()\n",[343,3187,3188,3190,3192,3195],{"class":345,"line":265},[343,3189,1099],{"class":400},[343,3191,404],{"class":359},[343,3193,3194],{"class":400},"format_instructions",[343,3196,694],{"class":359},[67,3198,3199],{},"Print them once and read them. That text is what the model is being told.",[78,3201,3203],{"id":3202},"_5-pick-a-local-model","5. Pick a local model",[67,3205,3206,3207,3210],{},"Pull whichever model you want from Ollama. I used ",[131,3208,3209],{},"llama3.2:3b"," here, with temperature at 0 so the output stays repeatable.",[335,3212,3214],{"className":2378,"code":3213,"language":2380,"meta":261,"style":261},"model = OllamaLLM(\n    model='llama3.2:3b',\n    temperature=0.0,\n)\n",[131,3215,3216,3228,3244,3256],{"__ignoreMap":261},[343,3217,3218,3220,3222,3225],{"class":345,"line":346},[343,3219,2414],{"class":434},[343,3221,742],{"class":359},[343,3223,3224],{"class":400}," OllamaLLM",[343,3226,3227],{"class":359},"(\n",[343,3229,3230,3232,3234,3237,3239,3241],{"class":345,"line":262},[343,3231,2997],{"class":410},[343,3233,742],{"class":359},[343,3235,3236],{"class":359},"'",[343,3238,3209],{"class":363},[343,3240,3236],{"class":359},[343,3242,3243],{"class":359},",\n",[343,3245,3246,3249,3251,3254],{"class":345,"line":265},[343,3247,3248],{"class":410},"    temperature",[343,3250,742],{"class":359},[343,3252,3253],{"class":645},"0.0",[343,3255,3243],{"class":359},[343,3257,3258],{"class":345,"line":375},[343,3259,694],{"class":359},[78,3261,3263],{"id":3262},"_6-chain-prompt-model-and-parser","6. Chain prompt, model, and parser",[67,3265,3266],{},"The prompt carries my instruction plus the format instructions. The pipe operator puts the three pieces in a row, which is where the name LangChain comes from.",[335,3268,3270],{"className":2378,"code":3269,"language":2380,"meta":261,"style":261},"human_text = \"{instruction}\\n{format_instructions}\"\nmessage = HumanMessagePromptTemplate.from_template(human_text)\nprompt = ChatPromptTemplate.from_messages([message])\n\nchain = prompt | model | output_parser\nproducts = chain.invoke({\"instruction\":\"Give top 10 popular phone now\", \"format_instructions\":format_instructions})\n",[131,3271,3272,3292,3313,3337,3341,3362],{"__ignoreMap":261},[343,3273,3274,3277,3279,3281,3284,3287,3290],{"class":345,"line":346},[343,3275,3276],{"class":434},"human_text ",[343,3278,742],{"class":359},[343,3280,382],{"class":359},[343,3282,3283],{"class":645},"{instruction}",[343,3285,3286],{"class":434},"\\n",[343,3288,3289],{"class":645},"{format_instructions}",[343,3291,549],{"class":359},[343,3293,3294,3297,3299,3301,3303,3306,3308,3311],{"class":345,"line":262},[343,3295,3296],{"class":434},"message ",[343,3298,742],{"class":359},[343,3300,2913],{"class":434},[343,3302,134],{"class":359},[343,3304,3305],{"class":400},"from_template",[343,3307,404],{"class":359},[343,3309,3310],{"class":400},"human_text",[343,3312,694],{"class":359},[343,3314,3315,3318,3320,3323,3325,3328,3331,3334],{"class":345,"line":265},[343,3316,3317],{"class":434},"prompt ",[343,3319,742],{"class":359},[343,3321,3322],{"class":434}," ChatPromptTemplate",[343,3324,134],{"class":359},[343,3326,3327],{"class":400},"from_messages",[343,3329,3330],{"class":359},"([",[343,3332,3333],{"class":400},"message",[343,3335,3336],{"class":359},"])\n",[343,3338,3339],{"class":345,"line":375},[343,3340,372],{"emptyLinePlaceholder":53},[343,3342,3343,3346,3348,3351,3354,3357,3359],{"class":345,"line":394},[343,3344,3345],{"class":434},"chain ",[343,3347,742],{"class":359},[343,3349,3350],{"class":434}," prompt ",[343,3352,3353],{"class":359},"|",[343,3355,3356],{"class":434}," model ",[343,3358,3353],{"class":359},[343,3360,3361],{"class":434}," output_parser\n",[343,3363,3364,3367,3369,3372,3374,3377,3380,3382,3385,3387,3389,3391,3394,3396,3398,3400,3402,3404,3406,3408],{"class":345,"line":428},[343,3365,3366],{"class":434},"products ",[343,3368,742],{"class":359},[343,3370,3371],{"class":434}," chain",[343,3373,134],{"class":359},[343,3375,3376],{"class":400},"invoke",[343,3378,3379],{"class":359},"({",[343,3381,388],{"class":359},[343,3383,3384],{"class":363},"instruction",[343,3386,388],{"class":359},[343,3388,2303],{"class":359},[343,3390,388],{"class":359},[343,3392,3393],{"class":363},"Give top 10 popular phone now",[343,3395,388],{"class":359},[343,3397,414],{"class":359},[343,3399,382],{"class":359},[343,3401,3194],{"class":363},[343,3403,388],{"class":359},[343,3405,2303],{"class":359},[343,3407,3194],{"class":400},[343,3409,3410],{"class":359},"})\n",[67,3412,3413,3416,3417,3420],{},[131,3414,3415],{},"products"," is a ",[131,3418,3419],{},"PhonesModel",", not a string. If the model answers in the wrong shape, the parser raises instead of quietly handing me junk.",[78,3422,3424],{"id":3423},"_7-into-a-dataframe","7. Into a DataFrame",[335,3426,3428],{"className":2378,"code":3427,"language":2380,"meta":261,"style":261},"df = pd.DataFrame([product.model_dump() for product in products.phones])\ndf\n",[131,3429,3430,3475],{"__ignoreMap":261},[343,3431,3432,3435,3437,3440,3442,3445,3447,3450,3452,3455,3457,3460,3463,3465,3468,3470,3473],{"class":345,"line":346},[343,3433,3434],{"class":434},"df ",[343,3436,742],{"class":359},[343,3438,3439],{"class":434}," pd",[343,3441,134],{"class":359},[343,3443,3444],{"class":400},"DataFrame",[343,3446,3330],{"class":359},[343,3448,3449],{"class":400},"product",[343,3451,134],{"class":359},[343,3453,3454],{"class":400},"model_dump",[343,3456,785],{"class":359},[343,3458,3459],{"class":355}," for",[343,3461,3462],{"class":400}," product ",[343,3464,2591],{"class":355},[343,3466,3467],{"class":400}," products",[343,3469,134],{"class":359},[343,3471,3472],{"class":2566},"phones",[343,3474,3336],{"class":359},[343,3476,3477],{"class":345,"line":262},[343,3478,3479],{"class":434},"df\n",[192,3481,3482,3500],{},[195,3483,3484],{},[198,3485,3486,3488,3491,3494,3497],{},[201,3487],{},[201,3489,3490],{},"brand",[201,3492,3493],{},"model",[201,3495,3496],{},"year",[201,3498,3499],{},"price",[208,3501,3502,3518,3533,3548,3564,3580,3596,3612,3628,3643],{},[198,3503,3504,3506,3509,3512,3515],{},[213,3505,681],{},[213,3507,3508],{},"Apple",[213,3510,3511],{},"iPhone 13",[213,3513,3514],{},"2021",[213,3516,3517],{},"699.0",[198,3519,3520,3522,3525,3528,3530],{},[213,3521,691],{},[213,3523,3524],{},"Samsung",[213,3526,3527],{},"Galaxy S21",[213,3529,3514],{},[213,3531,3532],{},"799.0",[198,3534,3535,3537,3540,3543,3545],{},[213,3536,712],{},[213,3538,3539],{},"Google",[213,3541,3542],{},"Pixel 6",[213,3544,3514],{},[213,3546,3547],{},"599.0",[198,3549,3550,3553,3556,3559,3561],{},[213,3551,3552],{},"3",[213,3554,3555],{},"OnePlus",[213,3557,3558],{},"9 Pro",[213,3560,3514],{},[213,3562,3563],{},"969.0",[198,3565,3566,3569,3572,3575,3577],{},[213,3567,3568],{},"4",[213,3570,3571],{},"Xiaomi",[213,3573,3574],{},"Mi 11",[213,3576,3514],{},[213,3578,3579],{},"749.0",[198,3581,3582,3585,3588,3591,3593],{},[213,3583,3584],{},"5",[213,3586,3587],{},"Oppo",[213,3589,3590],{},"Find X3 Pro",[213,3592,3514],{},[213,3594,3595],{},"1149.0",[198,3597,3598,3601,3604,3607,3610],{},[213,3599,3600],{},"6",[213,3602,3603],{},"Realme",[213,3605,3606],{},"GT 2 Pro",[213,3608,3609],{},"2022",[213,3611,3547],{},[198,3613,3614,3617,3620,3623,3625],{},[213,3615,3616],{},"7",[213,3618,3619],{},"Vivo",[213,3621,3622],{},"X70 Pro",[213,3624,3514],{},[213,3626,3627],{},"899.0",[198,3629,3630,3632,3635,3638,3640],{},[213,3631,2528],{},[213,3633,3634],{},"Sony",[213,3636,3637],{},"Xperia 1 III",[213,3639,3514],{},[213,3641,3642],{},"1299.0",[198,3644,3645,3648,3651,3654,3656],{},[213,3646,3647],{},"9",[213,3649,3650],{},"Motorola",[213,3652,3653],{},"Edge 30 Pro",[213,3655,3609],{},[213,3657,3532],{},[67,3659,3660,3661,3664],{},"One thing worth saying out loud: the model made these numbers up from what it remembers. The schema guarantees the ",[257,3662,3663],{},"shape"," of the answer, never the truth of it. For anything that matters, check the values against a real source.",[335,3666,3667],{"className":2378,"code":2725,"language":2380,"meta":261,"style":261},[131,3668,3669],{"__ignoreMap":261},[343,3670,3671,3673,3675,3677,3679,3681],{"class":345,"line":346},[343,3672,1099],{"class":400},[343,3674,404],{"class":359},[343,3676,388],{"class":359},[343,3678,2738],{"class":363},[343,3680,388],{"class":359},[343,3682,694],{"class":359},[1910,3684,3685],{},"html pre.shiki code .sBMFI, html code.shiki .sBMFI{--shiki-light:#E2931D;--shiki-default:#FFCB6B;--shiki-dark:#FFCB6B}html pre.shiki code .sfazB, html code.shiki .sfazB{--shiki-light:#91B859;--shiki-default:#C3E88D;--shiki-dark:#C3E88D}html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html pre.shiki code .s7zQu, html code.shiki .s7zQu{--shiki-light:#39ADB5;--shiki-light-font-style:italic;--shiki-default:#89DDFF;--shiki-default-font-style:italic;--shiki-dark:#89DDFF;--shiki-dark-font-style:italic}html pre.shiki code .sTEyZ, html code.shiki .sTEyZ{--shiki-light:#90A4AE;--shiki-default:#EEFFFF;--shiki-dark:#BABED8}html pre.shiki code .sMK4o, html code.shiki .sMK4o{--shiki-light:#39ADB5;--shiki-default:#89DDFF;--shiki-dark:#89DDFF}html pre.shiki code .spNyl, html code.shiki .spNyl{--shiki-light:#9C3EDA;--shiki-default:#C792EA;--shiki-dark:#C792EA}html pre.shiki code .s2Zo4, html code.shiki .s2Zo4{--shiki-light:#6182B8;--shiki-default:#82AAFF;--shiki-dark:#82AAFF}html pre.shiki code .sHdIc, html code.shiki .sHdIc{--shiki-light:#90A4AE;--shiki-light-font-style:italic;--shiki-default:#EEFFFF;--shiki-default-font-style:italic;--shiki-dark:#BABED8;--shiki-dark-font-style:italic}html pre.shiki code .sbssI, html code.shiki .sbssI{--shiki-light:#F76D47;--shiki-default:#F78C6C;--shiki-dark:#F78C6C}html pre.shiki code .swJcz, html code.shiki .swJcz{--shiki-light:#E53935;--shiki-default:#F07178;--shiki-dark:#F07178}",{"title":261,"searchDepth":262,"depth":262,"links":3687},[3688,3689,3690,3691,3692,3693,3694],{"id":2774,"depth":265,"text":2775},{"id":2840,"depth":265,"text":2841},{"id":2934,"depth":265,"text":2935},{"id":3134,"depth":265,"text":3135},{"id":3202,"depth":265,"text":3203},{"id":3262,"depth":265,"text":3263},{"id":3423,"depth":265,"text":3424},"2025-03-01","Getting typed rows out of a local LLM with Pydantic and LangChain.","\u002Fblog\u002Fpydantic-llm\u002Fcover.jpg",{"tags":3699},[2757,1300],{"title":30,"description":3696},"5cjs_kXCP4cAoBKbVN2gF4q1hmoVlkH19jgBkD_Nk88",{"id":3703,"title":42,"author":3704,"body":3706,"date":3832,"description":3833,"extension":273,"image":3834,"meta":3835,"minRead":262,"navigation":53,"path":43,"seo":3838,"stem":44,"__hash__":3839},"blog\u002Fblog\u002Fwhisper.md",{"name":60,"avatar":3705},{"src":62,"alt":60},{"type":64,"value":3707,"toc":3827},[3708,3711,3714,3718,3721,3724,3728,3731,3734,3739,3756,3761,3772,3775,3778,3782,3785,3788,3795,3798,3803],[67,3709,3710],{},"Whisper transcribes English well. On Khmer it mostly guesses.",[67,3712,3713],{},"So I fine-tuned it: custom scripts, a dataset I put together, and a training pipeline that runs end to end. Almost all of the effort went into the data, and almost none into the model.",[78,3715,3717],{"id":3716},"why-khmer-is-awkward","Why Khmer is awkward",[67,3719,3720],{},"Khmer runs words together. No spaces between them, and almost no punctuation. A sentence arrives as one unbroken string of characters.",[67,3722,3723],{},"That breaks an assumption most speech pipelines quietly make, which is that you can line words up with sounds because you can see where the words are. Here you cannot, so the normalization step below carries weight that it would not carry in English.",[78,3725,3727],{"id":3726},"cutting-audio-into-training-pairs","Cutting audio into training pairs",[67,3729,3730],{},"Whisper learns from short clips paired with the exact text spoken in them. My recordings were long, with a single transcript covering the whole file, so something has to decide where each word falls in time. That is forced alignment.",[67,3732,3733],{},"Two paths run in parallel and meet at the aligner.",[67,3735,3736],{},[71,3737,3738],{},"The transcript side",[1964,3740,3741,3744,3747,3750,3753],{},[117,3742,3743],{},"Start with the raw transcript.",[117,3745,3746],{},"Normalize it, fixing punctuation, casing and formatting.",[117,3748,3749],{},"Romanize the Khmer script into Latin characters.",[117,3751,3752],{},"Run uroman normalization so the text matches what the model expects.",[117,3754,3755],{},"Tokenize it into a token sequence.",[67,3757,3758],{},[71,3759,3760],{},"The audio side",[1964,3762,3763,3766,3769],{},[117,3764,3765],{},"Start with the waveform.",[117,3767,3768],{},"Run it through the acoustic model.",[117,3770,3771],{},"Read off the emission probabilities, one distribution over tokens per frame.",[67,3773,3774],{},"The aligner takes the token sequence and those probabilities and works out which token was being spoken in which frame. What comes back is every token with a timestamp and a confidence score.",[67,3776,3777],{},"The timestamps give the cut points. The confidence scores are the useful part: a clip the aligner is unsure about is usually a clip where the transcript and the audio disagree, and those are worth dropping before they teach the model something wrong.",[78,3779,3781],{"id":3780},"trying-it-out","Trying it out",[67,3783,3784],{},"The fine-tuned model runs in a small demo that takes a microphone or an uploaded file and returns the transcription. It is the screenshot at the top of this post.",[67,3786,3787],{},"The video walks through the results:",[67,3789,3790],{},[93,3791,3794],{"href":3792,"rel":3793},"https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=kjtTuh6uOUA",[97],"Watch the video on YouTube",[67,3796,3797],{},"Khmer speech recognition is still far behind English, and a small model trained on a small dataset is not going to close that. What this does show is that the gap is mostly a data problem, and data problems are the kind you can work at.",[3799,3800,3802],"h4",{"id":3801},"resources","Resources",[114,3804,3805,3813,3820],{},[117,3806,3807,3812],{},[93,3808,3811],{"href":3809,"rel":3810},"https:\u002F\u002Fgithub.com\u002Fphanithlim\u002Ffine-tuning-whisper",[97],"phanithlim\u002Ffine-tuning-whisper",", the scripts and the pipeline",[117,3814,3815],{},[93,3816,3819],{"href":3817,"rel":3818},"https:\u002F\u002Fhuggingface.co\u002Fcollections\u002FPhanithLIM\u002Fkhmer-automation-speech-recognition-67d7f77f52ba357dc72abd4e",[97],"Models and datasets on Hugging Face",[117,3821,3822],{},[93,3823,3826],{"href":3824,"rel":3825},"https:\u002F\u002Fdrive.google.com\u002Fdrive\u002Ffolders\u002F1bApbUyeVRhIDEy0SzwvSQR8lka-mV_LM?usp=sharing",[97],"Report & Slide",{"title":261,"searchDepth":262,"depth":262,"links":3828},[3829,3830,3831],{"id":3716,"depth":265,"text":3717},{"id":3726,"depth":265,"text":3727},{"id":3780,"depth":265,"text":3781},"2025-02-27","Fine-tuning Whisper for Khmer, and why the data was the hard part.","\u002Fblog\u002Fwhisper\u002Fimage1.png",{"tags":3836},[2757,3837],"Audio",{"title":42,"description":3833},"SMfWWO50jkoF4h7XkLR5XjDg7zT3HAFLJZtu6Hho_vU",{"id":3841,"title":10,"author":3842,"body":3844,"date":4814,"description":4815,"extension":273,"image":4816,"meta":4817,"minRead":394,"navigation":53,"path":11,"seo":4819,"stem":12,"__hash__":4820},"blog\u002Fblog\u002Faudio-augmentation.md",{"name":60,"avatar":3843},{"src":62,"alt":60},{"type":64,"value":3845,"toc":4806},[3846,3849,3852,3855,3859,3862,3865,3919,3925,3929,3936,3942,3946,3952,3958,3962,3965,3972,3975,4205,4208,4317,4326,4329,4335,4345,4349,4352,4430,4437,4443,4447,4453,4459,4463,4467,4470,4552,4584,4588,4591,4674,4704,4708,4715,4721,4725,4758,4761,4778,4782,4785,4803],[67,3847,3848],{},"When you do not have enough audio to train on, you make more out of what you have. Add some noise, speed a clip up, shift its pitch: the words stay the same, so the label stays the same, but the model sees a new example.",[67,3850,3851],{},"That is audio augmentation. It is cheaper than collecting more recordings, and it stops a model from memorising the quirks of the few recordings it has.",[67,3853,3854],{},"Here are the six transformations I keep coming back to.",[78,3856,3858],{"id":3857},"_1-noise-injection","1. Noise injection",[67,3860,3861],{},"Real recordings are never clean. Adding random noise to a clip teaches the model to listen past a fan, a street, a bad microphone.",[67,3863,3864],{},"The noise comes from a distribution, and the two I use are the normal and the uniform distribution:",[192,3866,3867,3877],{},[195,3868,3869],{},[198,3870,3871,3874],{},[201,3872,3873],{},"Normal Distribution",[201,3875,3876],{},"Uniform Distribution",[208,3878,3879,3887,3895,3903,3911],{},[198,3880,3881,3884],{},[213,3882,3883],{},"Depicted as a bell curve.",[213,3885,3886],{},"Depicted as a flat, horizontal line.",[198,3888,3889,3892],{},[213,3890,3891],{},"Probability is concentrated around the mean.",[213,3893,3894],{},"Probability is spread evenly across a range.",[198,3896,3897,3900],{},[213,3898,3899],{},"Values closer to the mean are more likely.",[213,3901,3902],{},"All values within the range are equally likely.",[198,3904,3905,3908],{},[213,3906,3907],{},"Represented by a peak at the mean, with tails.",[213,3909,3910],{},"Represented by a flat line indicating constant probability.",[198,3912,3913,3916],{},[213,3914,3915],{},"Gaussian noise is generated using the normal distribution.",[213,3917,3918],{},"Uniform noise is generated using the uniform distribution.",[67,3920,3921],{},[85,3922],{"alt":3923,"src":3924},"Comparison of Normal and Uniform Distributions","\u002Fblog\u002Faudio-augmentation\u002Fimage1.png",[67,3926,3927],{},[257,3928,3923],{},[67,3930,3931,3932,3935],{},"Think of a classroom. ",[71,3933,3934],{},"Gaussian noise"," is the air conditioner humming in the background: constant, and easy enough to listen past.",[67,3937,3938],{},[85,3939],{"alt":3940,"src":3941},"Gaussian noise added to the original signal","\u002Fblog\u002Faudio-augmentation\u002Fimage2.png",[67,3943,3944],{},[257,3945,3940],{},[67,3947,3948,3951],{},[71,3949,3950],{},"Uniform noise"," is a sudden clap. It cuts across the teacher's voice, and the student (the model) loses the word.",[67,3953,3954],{},[85,3955],{"alt":3956,"src":3957},"Uniform noise added to the original signal","\u002Fblog\u002Faudio-augmentation\u002Fimage3.png",[67,3959,3960],{},[257,3961,3956],{},[67,3963,3964],{},"The formula is just an addition:",[335,3966,3970],{"className":3967,"code":3969,"language":2655,"meta":261},[3968],"language-text","Noisy Signal = Original Signal + (Noise Amplitude * Random Noise)\n",[131,3971,3969],{"__ignoreMap":261},[67,3973,3974],{},"To see what any augmentation did, plot the two signals side by side:",[335,3976,3978],{"className":2378,"code":3977,"language":2380,"meta":261,"style":261},"def _plot_signal_and_augmented_signal(signal, augmented_signal, sr):\n    fig, ax = plt.subplots(ncols=2, figsize=(15, 4))\n    librosa.display.waveshow(signal, sr=sr, ax=ax[0])\n    ax[0].set(title=\"Original signal\")\n    librosa.display.waveshow(augmented_signal, sr=sr, ax=ax[1])\n    ax[1].set(title=\"Augmentation signal\")\n    plt.show()\n",[131,3979,3980,4005,4052,4096,4127,4166,4193],{"__ignoreMap":261},[343,3981,3982,3985,3988,3990,3993,3995,3998,4000,4003],{"class":345,"line":346},[343,3983,3984],{"class":378},"def",[343,3986,3987],{"class":400}," _plot_signal_and_augmented_signal",[343,3989,404],{"class":359},[343,3991,3992],{"class":410},"signal",[343,3994,414],{"class":359},[343,3996,3997],{"class":410}," augmented_signal",[343,3999,414],{"class":359},[343,4001,4002],{"class":410}," sr",[343,4004,2962],{"class":359},[343,4006,4007,4010,4012,4015,4017,4020,4022,4025,4027,4030,4032,4034,4036,4039,4042,4045,4047,4050],{"class":345,"line":262},[343,4008,4009],{"class":434},"    fig",[343,4011,414],{"class":359},[343,4013,4014],{"class":434}," ax ",[343,4016,742],{"class":359},[343,4018,4019],{"class":434}," plt",[343,4021,134],{"class":359},[343,4023,4024],{"class":400},"subplots",[343,4026,404],{"class":359},[343,4028,4029],{"class":410},"ncols",[343,4031,742],{"class":359},[343,4033,712],{"class":645},[343,4035,414],{"class":359},[343,4037,4038],{"class":410}," figsize",[343,4040,4041],{"class":359},"=(",[343,4043,4044],{"class":645},"15",[343,4046,414],{"class":359},[343,4048,4049],{"class":645}," 4",[343,4051,2580],{"class":359},[343,4053,4054,4057,4059,4062,4064,4067,4069,4071,4073,4075,4077,4080,4082,4085,4087,4090,4092,4094],{"class":345,"line":265},[343,4055,4056],{"class":434},"    librosa",[343,4058,134],{"class":359},[343,4060,4061],{"class":2566},"display",[343,4063,134],{"class":359},[343,4065,4066],{"class":400},"waveshow",[343,4068,404],{"class":359},[343,4070,3992],{"class":400},[343,4072,414],{"class":359},[343,4074,4002],{"class":410},[343,4076,742],{"class":359},[343,4078,4079],{"class":400},"sr",[343,4081,414],{"class":359},[343,4083,4084],{"class":410}," ax",[343,4086,742],{"class":359},[343,4088,4089],{"class":400},"ax",[343,4091,2609],{"class":359},[343,4093,681],{"class":645},[343,4095,3336],{"class":359},[343,4097,4098,4101,4103,4105,4108,4111,4113,4116,4118,4120,4123,4125],{"class":345,"line":375},[343,4099,4100],{"class":434},"    ax",[343,4102,2609],{"class":359},[343,4104,681],{"class":645},[343,4106,4107],{"class":359},"].",[343,4109,4110],{"class":400},"set",[343,4112,404],{"class":359},[343,4114,4115],{"class":410},"title",[343,4117,742],{"class":359},[343,4119,388],{"class":359},[343,4121,4122],{"class":363},"Original signal",[343,4124,388],{"class":359},[343,4126,694],{"class":359},[343,4128,4129,4131,4133,4135,4137,4139,4141,4144,4146,4148,4150,4152,4154,4156,4158,4160,4162,4164],{"class":345,"line":394},[343,4130,4056],{"class":434},[343,4132,134],{"class":359},[343,4134,4061],{"class":2566},[343,4136,134],{"class":359},[343,4138,4066],{"class":400},[343,4140,404],{"class":359},[343,4142,4143],{"class":400},"augmented_signal",[343,4145,414],{"class":359},[343,4147,4002],{"class":410},[343,4149,742],{"class":359},[343,4151,4079],{"class":400},[343,4153,414],{"class":359},[343,4155,4084],{"class":410},[343,4157,742],{"class":359},[343,4159,4089],{"class":400},[343,4161,2609],{"class":359},[343,4163,691],{"class":645},[343,4165,3336],{"class":359},[343,4167,4168,4170,4172,4174,4176,4178,4180,4182,4184,4186,4189,4191],{"class":345,"line":428},[343,4169,4100],{"class":434},[343,4171,2609],{"class":359},[343,4173,691],{"class":645},[343,4175,4107],{"class":359},[343,4177,4110],{"class":400},[343,4179,404],{"class":359},[343,4181,4115],{"class":410},[343,4183,742],{"class":359},[343,4185,388],{"class":359},[343,4187,4188],{"class":363},"Augmentation signal",[343,4190,388],{"class":359},[343,4192,694],{"class":359},[343,4194,4195,4198,4200,4203],{"class":345,"line":445},[343,4196,4197],{"class":434},"    plt",[343,4199,134],{"class":359},[343,4201,4202],{"class":400},"show",[343,4204,3185],{"class":359},[67,4206,4207],{},"And the noise itself:",[335,4209,4211],{"className":2378,"code":4210,"language":2380,"meta":261,"style":261},"def add_white_noise(signal, noise_percentage_factor):\n    noise = np.random.normal(loc=0, scale=signal.std(), size=signal.shape)\n    augmented_signal = signal + noise * noise_percentage_factor\n    return augmented_signal\n",[131,4212,4213,4231,4290,4310],{"__ignoreMap":261},[343,4214,4215,4217,4220,4222,4224,4226,4229],{"class":345,"line":346},[343,4216,3984],{"class":378},[343,4218,4219],{"class":400}," add_white_noise",[343,4221,404],{"class":359},[343,4223,3992],{"class":410},[343,4225,414],{"class":359},[343,4227,4228],{"class":410}," noise_percentage_factor",[343,4230,2962],{"class":359},[343,4232,4233,4236,4238,4241,4243,4246,4248,4251,4253,4256,4258,4260,4262,4265,4267,4269,4271,4274,4277,4280,4282,4284,4286,4288],{"class":345,"line":262},[343,4234,4235],{"class":434},"    noise ",[343,4237,742],{"class":359},[343,4239,4240],{"class":434}," np",[343,4242,134],{"class":359},[343,4244,4245],{"class":2566},"random",[343,4247,134],{"class":359},[343,4249,4250],{"class":400},"normal",[343,4252,404],{"class":359},[343,4254,4255],{"class":410},"loc",[343,4257,742],{"class":359},[343,4259,681],{"class":645},[343,4261,414],{"class":359},[343,4263,4264],{"class":410}," scale",[343,4266,742],{"class":359},[343,4268,3992],{"class":400},[343,4270,134],{"class":359},[343,4272,4273],{"class":400},"std",[343,4275,4276],{"class":359},"(),",[343,4278,4279],{"class":410}," size",[343,4281,742],{"class":359},[343,4283,3992],{"class":400},[343,4285,134],{"class":359},[343,4287,3663],{"class":2566},[343,4289,694],{"class":359},[343,4291,4292,4295,4297,4300,4302,4305,4307],{"class":345,"line":265},[343,4293,4294],{"class":434},"    augmented_signal ",[343,4296,742],{"class":359},[343,4298,4299],{"class":434}," signal ",[343,4301,751],{"class":359},[343,4303,4304],{"class":434}," noise ",[343,4306,593],{"class":359},[343,4308,4309],{"class":434}," noise_percentage_factor\n",[343,4311,4312,4314],{"class":345,"line":375},[343,4313,431],{"class":355},[343,4315,4316],{"class":434}," augmented_signal\n",[67,4318,4319,4321,4322,4325],{},[131,4320,3992],{}," is the audio as a NumPy array. ",[131,4323,4324],{},"noise_percentage_factor"," is how loud the noise should be.",[67,4327,4328],{},"Two choices in that function are worth explaining.",[67,4330,4331,4334],{},[71,4332,4333],{},"Why scale the noise by the signal's standard deviation?"," Because the noise has to be loud enough to matter and quiet enough to leave the speech intact. The standard deviation ties it to the volume of this particular clip, so the same code works on a whisper and on a shout.",[67,4336,4337,4344],{},[71,4338,4339,4340,4343],{},"Why ",[131,4341,4342],{},"loc=0","?"," A mean of zero means the noise pushes the waveform up as often as it pushes it down. Any other mean would shift the whole signal in one direction and change its amplitude, which is not what you want.",[78,4346,4348],{"id":4347},"_2-time-stretching","2. Time stretching",[67,4350,4351],{},"Time stretching changes how fast a clip plays without changing its pitch. People speak at different speeds, and this is how you show the model that.",[335,4353,4355],{"className":2378,"code":4354,"language":2380,"meta":261,"style":261},"def time_stretch(signal, time_stretch_rate):\n    \"\"\"Time stretching using librosa\"\"\"\n    augmented_signal = librosa.effects.time_stretch(signal, rate=time_stretch_rate)  # Specify rate\n    return augmented_signal\n",[131,4356,4357,4375,4386,4424],{"__ignoreMap":261},[343,4358,4359,4361,4364,4366,4368,4370,4373],{"class":345,"line":346},[343,4360,3984],{"class":378},[343,4362,4363],{"class":400}," time_stretch",[343,4365,404],{"class":359},[343,4367,3992],{"class":410},[343,4369,414],{"class":359},[343,4371,4372],{"class":410}," time_stretch_rate",[343,4374,2962],{"class":359},[343,4376,4377,4380,4383],{"class":345,"line":262},[343,4378,4379],{"class":355},"    \"\"\"",[343,4381,4382],{"class":349},"Time stretching using librosa",[343,4384,4385],{"class":355},"\"\"\"\n",[343,4387,4388,4390,4392,4395,4397,4400,4402,4405,4407,4409,4411,4414,4416,4419,4421],{"class":345,"line":265},[343,4389,4294],{"class":434},[343,4391,742],{"class":359},[343,4393,4394],{"class":434}," librosa",[343,4396,134],{"class":359},[343,4398,4399],{"class":2566},"effects",[343,4401,134],{"class":359},[343,4403,4404],{"class":400},"time_stretch",[343,4406,404],{"class":359},[343,4408,3992],{"class":400},[343,4410,414],{"class":359},[343,4412,4413],{"class":410}," rate",[343,4415,742],{"class":359},[343,4417,4418],{"class":400},"time_stretch_rate",[343,4420,423],{"class":359},[343,4422,4423],{"class":349},"  # Specify rate\n",[343,4425,4426,4428],{"class":345,"line":375},[343,4427,431],{"class":355},[343,4429,4316],{"class":434},[67,4431,4432,4433,4436],{},"With ",[131,4434,4435],{},"time_stretch_rate > 1"," the clip gets shorter and faster:",[67,4438,4439],{},[85,4440],{"alt":4441,"src":4442},"Compressed (faster) signal with time_stretch_rate > 1","\u002Fblog\u002Faudio-augmentation\u002Fimage4.png",[67,4444,4445],{},[257,4446,4441],{},[67,4448,4432,4449,4452],{},[131,4450,4451],{},"time_stretch_rate \u003C 1"," it gets longer and slower:",[67,4454,4455],{},[85,4456],{"alt":4457,"src":4458},"Expanded (slower) signal with time_stretch_rate \u003C 1","\u002Fblog\u002Faudio-augmentation\u002Fimage5.png",[67,4460,4461],{},[257,4462,4457],{},[78,4464,4466],{"id":4465},"_3-pitch-scaling","3. Pitch scaling",[67,4468,4469],{},"Pitch scaling moves the pitch and leaves the speed alone, so a high voice becomes a lower one, or the other way around.",[335,4471,4473],{"className":2378,"code":4472,"language":2380,"meta":261,"style":261},"def pitch_scale(signal, sr, num_semitones):\n    \"\"\"Pitch scaling using librosa\"\"\"\n    augmented_signal = librosa.effects.pitch_shift(signal, sr=sr, n_steps= num_semitones)\n    return augmented_signal\n",[131,4474,4475,4497,4506,4546],{"__ignoreMap":261},[343,4476,4477,4479,4482,4484,4486,4488,4490,4492,4495],{"class":345,"line":346},[343,4478,3984],{"class":378},[343,4480,4481],{"class":400}," pitch_scale",[343,4483,404],{"class":359},[343,4485,3992],{"class":410},[343,4487,414],{"class":359},[343,4489,4002],{"class":410},[343,4491,414],{"class":359},[343,4493,4494],{"class":410}," num_semitones",[343,4496,2962],{"class":359},[343,4498,4499,4501,4504],{"class":345,"line":262},[343,4500,4379],{"class":355},[343,4502,4503],{"class":349},"Pitch scaling using librosa",[343,4505,4385],{"class":355},[343,4507,4508,4510,4512,4514,4516,4518,4520,4523,4525,4527,4529,4531,4533,4535,4537,4540,4542,4544],{"class":345,"line":265},[343,4509,4294],{"class":434},[343,4511,742],{"class":359},[343,4513,4394],{"class":434},[343,4515,134],{"class":359},[343,4517,4399],{"class":2566},[343,4519,134],{"class":359},[343,4521,4522],{"class":400},"pitch_shift",[343,4524,404],{"class":359},[343,4526,3992],{"class":400},[343,4528,414],{"class":359},[343,4530,4002],{"class":410},[343,4532,742],{"class":359},[343,4534,4079],{"class":400},[343,4536,414],{"class":359},[343,4538,4539],{"class":410}," n_steps",[343,4541,742],{"class":359},[343,4543,4494],{"class":400},[343,4545,694],{"class":359},[343,4547,4548,4550],{"class":345,"line":375},[343,4549,431],{"class":355},[343,4551,4316],{"class":434},[114,4553,4554,4561,4568],{},[117,4555,4556,4560],{},[71,4557,4558],{},[131,4559,3992],{},": audio as a NumPy array.",[117,4562,4563,4567],{},[71,4564,4565],{},[131,4566,4079],{},": sampling rate, for example 44,100 Hz.",[117,4569,4570,4575,4576,4579,4580,4583],{},[71,4571,4572],{},[131,4573,4574],{},"num_semitones",": how far to shift: ",[131,4577,4578],{},"+4"," raises the pitch by four semitones, ",[131,4581,4582],{},"-3"," lowers it by three.",[78,4585,4587],{"id":4586},"_4-random-gain","4. Random gain",[67,4589,4590],{},"Gain is volume. Multiplying a clip by a random factor gives the model quiet recordings and loud ones from the same source.",[335,4592,4594],{"className":2378,"code":4593,"language":2380,"meta":261,"style":261},"def random_gain(signal, min_factor=0.1, max_factor=0.12):\n    gain_rate = random.uniform(min_factor, max_factor)\n    augmented_signal = signal * gain_rate\n    return augmented_signal\n",[131,4595,4596,4629,4655,4668],{"__ignoreMap":261},[343,4597,4598,4600,4603,4605,4607,4609,4612,4614,4617,4619,4622,4624,4627],{"class":345,"line":346},[343,4599,3984],{"class":378},[343,4601,4602],{"class":400}," random_gain",[343,4604,404],{"class":359},[343,4606,3992],{"class":410},[343,4608,414],{"class":359},[343,4610,4611],{"class":410}," min_factor",[343,4613,742],{"class":359},[343,4615,4616],{"class":645},"0.1",[343,4618,414],{"class":359},[343,4620,4621],{"class":410}," max_factor",[343,4623,742],{"class":359},[343,4625,4626],{"class":645},"0.12",[343,4628,2962],{"class":359},[343,4630,4631,4634,4636,4639,4641,4644,4646,4649,4651,4653],{"class":345,"line":262},[343,4632,4633],{"class":434},"    gain_rate ",[343,4635,742],{"class":359},[343,4637,4638],{"class":434}," random",[343,4640,134],{"class":359},[343,4642,4643],{"class":400},"uniform",[343,4645,404],{"class":359},[343,4647,4648],{"class":400},"min_factor",[343,4650,414],{"class":359},[343,4652,4621],{"class":400},[343,4654,694],{"class":359},[343,4656,4657,4659,4661,4663,4665],{"class":345,"line":265},[343,4658,4294],{"class":434},[343,4660,742],{"class":359},[343,4662,4299],{"class":434},[343,4664,593],{"class":359},[343,4666,4667],{"class":434}," gain_rate\n",[343,4669,4670,4672],{"class":345,"line":375},[343,4671,431],{"class":355},[343,4673,4316],{"class":434},[114,4675,4676,4682,4693],{},[117,4677,4678,4560],{},[71,4679,4680],{},[131,4681,3992],{},[117,4683,4684,4688,4689,4692],{},[71,4685,4686],{},[131,4687,4648],{},": quietest allowed, for example ",[131,4690,4691],{},"0.5"," for half volume.",[117,4694,4695,4700,4701,134],{},[71,4696,4697],{},[131,4698,4699],{},"max_factor",": loudest allowed, for example ",[131,4702,4703],{},"1.5",[78,4705,4707],{"id":4706},"_5-inverting-polarity","5. Inverting polarity",[67,4709,4710,4711,4714],{},"Multiply the signal by ",[131,4712,4713],{},"-1"," and the waveform flips: peaks become troughs, troughs become peaks. It sounds identical on its own, and that is the point. What changes is how it behaves when mixed with the original.",[67,4716,4717],{},[85,4718],{"alt":4719,"src":4720},"Inverted polarity: peaks become troughs and vice versa","\u002Fblog\u002Faudio-augmentation\u002Fimage6.png",[67,4722,4723],{},[257,4724,4719],{},[335,4726,4728],{"className":2378,"code":4727,"language":2380,"meta":261,"style":261},"def invert_polarity(signal):\n    return signal * -1,\n",[131,4729,4730,4743],{"__ignoreMap":261},[343,4731,4732,4734,4737,4739,4741],{"class":345,"line":346},[343,4733,3984],{"class":378},[343,4735,4736],{"class":400}," invert_polarity",[343,4738,404],{"class":359},[343,4740,3992],{"class":410},[343,4742,2962],{"class":359},[343,4744,4745,4747,4749,4751,4754,4756],{"class":345,"line":262},[343,4746,431],{"class":355},[343,4748,4299],{"class":434},[343,4750,593],{"class":359},[343,4752,4753],{"class":359}," -",[343,4755,691],{"class":645},[343,4757,3243],{"class":359},[67,4759,4760],{},"Where polarity inversion earns its keep:",[114,4762,4763,4766,4769,4772,4775],{},[117,4764,4765],{},"Cancelling noise by phase inversion.",[117,4767,4768],{},"Fixing phase problems in stereo or multi-channel audio.",[117,4770,4771],{},"Pulling vocals out for a karaoke or instrumental track.",[117,4773,4774],{},"Building sound effects.",[117,4776,4777],{},"Cutting feedback in live performances.",[78,4779,4781],{"id":4780},"_6-stacking-them","6. Stacking them",[67,4783,4784],{},"The transformations compose. Noise plus a time stretch plus a pitch shift gives you a clip that is further from the original than any single one of them, which is usually what you want from a small dataset. Just keep it plausible: if the result sounds like nothing a real microphone would ever record, the model learns from something it will never meet.",[335,4786,4787],{"className":2378,"code":2725,"language":2380,"meta":261,"style":261},[131,4788,4789],{"__ignoreMap":261},[343,4790,4791,4793,4795,4797,4799,4801],{"class":345,"line":346},[343,4792,1099],{"class":400},[343,4794,404],{"class":359},[343,4796,388],{"class":359},[343,4798,2738],{"class":363},[343,4800,388],{"class":359},[343,4802,694],{"class":359},[1910,4804,4805],{},"html pre.shiki code .spNyl, html code.shiki .spNyl{--shiki-light:#9C3EDA;--shiki-default:#C792EA;--shiki-dark:#C792EA}html pre.shiki code .s2Zo4, html code.shiki .s2Zo4{--shiki-light:#6182B8;--shiki-default:#82AAFF;--shiki-dark:#82AAFF}html pre.shiki code .sMK4o, html code.shiki .sMK4o{--shiki-light:#39ADB5;--shiki-default:#89DDFF;--shiki-dark:#89DDFF}html pre.shiki code .sHdIc, html code.shiki .sHdIc{--shiki-light:#90A4AE;--shiki-light-font-style:italic;--shiki-default:#EEFFFF;--shiki-default-font-style:italic;--shiki-dark:#BABED8;--shiki-dark-font-style:italic}html pre.shiki code .sTEyZ, html code.shiki .sTEyZ{--shiki-light:#90A4AE;--shiki-default:#EEFFFF;--shiki-dark:#BABED8}html pre.shiki code .sbssI, html code.shiki .sbssI{--shiki-light:#F76D47;--shiki-default:#F78C6C;--shiki-dark:#F78C6C}html pre.shiki code .swJcz, html code.shiki .swJcz{--shiki-light:#E53935;--shiki-default:#F07178;--shiki-dark:#F07178}html pre.shiki code .sfazB, html code.shiki .sfazB{--shiki-light:#91B859;--shiki-default:#C3E88D;--shiki-dark:#C3E88D}html .light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html.light .shiki span {color: var(--shiki-light);background: var(--shiki-light-bg);font-style: var(--shiki-light-font-style);font-weight: var(--shiki-light-font-weight);text-decoration: var(--shiki-light-text-decoration);}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html pre.shiki code .s7zQu, html code.shiki .s7zQu{--shiki-light:#39ADB5;--shiki-light-font-style:italic;--shiki-default:#89DDFF;--shiki-default-font-style:italic;--shiki-dark:#89DDFF;--shiki-dark-font-style:italic}html pre.shiki code .sHwdD, html code.shiki .sHwdD{--shiki-light:#90A4AE;--shiki-light-font-style:italic;--shiki-default:#546E7A;--shiki-default-font-style:italic;--shiki-dark:#676E95;--shiki-dark-font-style:italic}",{"title":261,"searchDepth":262,"depth":262,"links":4807},[4808,4809,4810,4811,4812,4813],{"id":3857,"depth":265,"text":3858},{"id":4347,"depth":265,"text":4348},{"id":4465,"depth":265,"text":4466},{"id":4586,"depth":265,"text":4587},{"id":4706,"depth":265,"text":4707},{"id":4780,"depth":265,"text":4781},"2025-02-20","Six ways to stretch a small audio dataset, with the code I use.","\u002Fblog\u002Faudio-augmentation\u002Fthumbnail.png",{"tags":4818},[2757,3837,1300],{"title":10,"description":4815},"21TaIrmtFsgDoy_h2nB7ilHQ_AZ289VCG3jIH7hw9Rc",{"id":4822,"title":34,"author":4823,"body":4825,"date":4965,"description":4966,"extension":273,"image":4892,"meta":4967,"minRead":265,"navigation":53,"path":35,"seo":4970,"stem":36,"__hash__":4971},"blog\u002Fblog\u002Fseven-segment.md",{"name":60,"avatar":4824},{"src":62,"alt":60},{"type":64,"value":4826,"toc":4959},[4827,4830,4833,4837,4850,4853,4870,4873,4877,4887,4893,4898,4901,4907,4910,4914,4917,4920,4926,4930,4933,4941,4943],[67,4828,4829],{},"A 7-segment display shows a digit by lighting some of its seven bars. The question this project answers is a small one: given a number in binary, which bars should be on?",[67,4831,4832],{},"I built the answer as a logic circuit and simulated it in Proteus. This was for my mathematics class in my first year at the ITE Department, Faculty of Engineering, and it is still one of my favourite things I made that year. 😁",[78,4834,4836],{"id":4835},"seven-questions-not-one","Seven questions, not one",[67,4838,4839,4840,4843,4844,4846,4847,134],{},"The input is four bits, ",[71,4841,4842],{},"A B C D",", enough to count from 0 to 15. The output is seven wires, one per segment, named ",[71,4845,93],{}," through ",[71,4848,4849],{},"g",[67,4851,4852],{},"That means there is no single circuit to design. Each segment is its own yes-or-no question, asked of the same four inputs:",[114,4854,4855,4861,4867],{},[117,4856,4857,4858,4860],{},"Segment ",[71,4859,93],{}," is on for 0, 2, 3, 5, 6, 7, 8 and 9.",[117,4862,4857,4863,4866],{},[71,4864,4865],{},"e"," is on for 0, 2, 6 and 8.",[117,4868,4869],{},"And so on for the other five.",[67,4871,4872],{},"Seven segments, seven boolean functions, all sharing the same four input bits.",[78,4874,4876],{"id":4875},"where-the-dont-cares-help","Where the don't-cares help",[67,4878,4879,4880,4883,4884,4886],{},"Four bits give sixteen combinations, but a digit only needs ten of them. The inputs for 10 to 15 never happen, so the output for those rows does not matter. On a Karnaugh map they are written as ",[71,4881,4882],{},"x",", and that is where the simplification comes from: an ",[71,4885,4882],{}," can be read as 1 when it helps a group grow, or ignored when it does not.",[67,4888,4889],{},[85,4890],{"alt":4891,"src":4892},"K-maps for segments e, f and g, with the don't-care cells marked x","\u002Fblog\u002Fseven-segment\u002Fseven.png",[67,4894,4895],{},[257,4896,4897],{},"K-maps for segments e, f and g. The x cells are the inputs a digit never uses.",[67,4899,4900],{},"Grouping the cells gives a short expression for each segment:",[335,4902,4905],{"className":4903,"code":4904,"language":2655,"meta":261},[3968],"e = CD' + B'D'\nf = A + BC' + C'D' + BD'\n",[131,4906,4904],{"__ignoreMap":261},[67,4908,4909],{},"Without the don't-cares these expressions would be noticeably longer, which means more gates to wire up. The same method gives a, b, c, d and g. All seven are worked out in the PDFs at the bottom.",[78,4911,4913],{"id":4912},"building-it-in-proteus","Building it in Proteus",[67,4915,4916],{},"From there it is just construction. Each expression becomes a small arrangement of AND, OR and NOT gates, the seven outputs go to the seven segment pins, and four switches drive the input bits.",[67,4918,4919],{},"Proteus runs the whole thing without any hardware, so you can flip the input switches and watch the digit change. When a segment stays dark that should be lit, the mistake is either in the grouping on the K-map or in the wiring, and both are quick to check.",[67,4921,4922],{},[93,4923,3794],{"href":4924,"rel":4925},"https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=zUS0YkRH8gE",[97],[78,4927,4929],{"id":4928},"looking-back","Looking back",[67,4931,4932],{},"What stuck with me is that the interesting part was not the circuit. It was noticing that six impossible inputs are a gift rather than a gap, and that a table of ones and zeros can be squeezed into something much smaller before you build anything.",[67,4934,4935,4936],{},"I wrote this up on my first blog back then, which is still online: ",[93,4937,4940],{"href":4938,"rel":4939},"https:\u002F\u002Fphanithlim.blogspot.com\u002F",[97],"phanithlim.blogspot.com",[3799,4942,3802],{"id":3801},[114,4944,4945,4952],{},[117,4946,4947],{},[93,4948,4951],{"href":4949,"rel":4950},"https:\u002F\u002Fdrive.google.com\u002Ffile\u002Fd\u002F1PK1P5Y7ayJfk7N-3tk3EFLCK21_HG7-8\u002Fpreview",[97],"Report of 7-segment (PDF)",[117,4953,4954],{},[93,4955,4958],{"href":4956,"rel":4957},"https:\u002F\u002Fdrive.google.com\u002Ffile\u002Fd\u002F1PUhr_0uNY2m1WRCaUDKBKKK_zfCny8RL\u002Fpreview",[97],"K-map (PDF)",{"title":261,"searchDepth":262,"depth":262,"links":4960},[4961,4962,4963,4964],{"id":4835,"depth":265,"text":4836},{"id":4875,"depth":265,"text":4876},{"id":4912,"depth":265,"text":4913},{"id":4928,"depth":265,"text":4929},"2022-04-25","Turning four input bits into ten digits with K-maps, then simulating the circuit in Proteus.",{"tags":4968},[4969],"Electronics",{"title":34,"description":4966},"I7cJrXPbzOMu4c5ovxzPSBIfrty01S6STZpy0yfh6cQ",{"id":4973,"title":22,"author":4974,"body":4976,"date":5000,"description":5001,"extension":273,"image":5002,"meta":5003,"minRead":346,"navigation":53,"path":23,"seo":5006,"stem":24,"__hash__":5007},"blog\u002Fblog\u002Fmemory.md",{"name":60,"avatar":4975},{"src":62,"alt":60},{"type":64,"value":4977,"toc":4998},[4978,4989,4992],[67,4979,4980,4981,4984,4985,4988],{},"This video is my high school years in one place, from ",[71,4982,4983],{},"Grade 10"," to ",[71,4986,4987],{},"Grade 12",". Friends, class activities, school trips, and the small everyday moments in between.",[67,4990,4991],{},"Two days after finishing the BACII exam I gathered the photos and put them together in After Effects. Going through them all again was the best part of making it.",[67,4993,4994],{},[93,4995,3794],{"href":4996,"rel":4997},"https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=gXlXc_baHvI",[97],{"title":261,"searchDepth":262,"depth":262,"links":4999},[],"2021-10-28","A video of my high school years, made after the BACII exam.","\u002Fblog\u002Fmemory\u002Fmemory.jpg",{"tags":5004},[5005],"Personal",{"title":22,"description":5001},"qzvbeHOt-LKL9gKbuUSkwx35Zuwe76tmCrJaCpRW2ds",1791981008542]