Microsoft's Secret Binary Optimization: BBT (Binary Basic Block Tools) and Its Impact on Product Performance

Dave Plummer, retired Microsoft operating systems engineer, reveals the existence and inner workings of an obscure internal tool called BBT (Binary Basic Block Tools), nicknamed "Lego," used during the Windows 95 and Windows NT4 era. BBT operated post-link, manipulating finished executable binaries (EXEs and DLLs) to optimize memory usage and performance, specifically targeting the working set size to ensure products like Windows and Office ran well on extremely constrained hardware (e.g., 12MB–64MB RAM). Unlike traditional compiler and linker optimizations focused on source code, BBT rearranged basic blocks—the smallest, linear chunks of machine instructions—so that frequently executed code paths were grouped together in physical memory pages. Rarely used code, like error handlers, was pushed further away, reducing unnecessary page access, instruction cache misses, and ITLB misses.

Microsoft's need was driven by a world where "memory was the weather," and every additional page fault had tangible performance costs ("grinding the disks like a raccoon trapped in a coffee can"). The aim was to make startup and common application paths responsive while minimizing resource use. BBT made use of execution profiles—collected from millions of machines and extensive test farms—to identify which code sections ran most frequently and restructure the binaries accordingly. This approach resembles modern profile-guided optimizations, such as LLVM's Bolt, emphasizing that locality and code layout still matter even as hardware improved, and complexity increased.

Rewriting binaries safely post-link is dangerous and challenging due to intricate dependencies, jumps, entry points, and metadata. Microsoft was uniquely positioned to undertake this risk, leveraging extensive testing and incentives for large-scale performance improvements. Such invisible optimizations, uncelebrated yet vital, contributed to the professional polish and user experience: faster startup, better multitasking, and smoother application launches. The speaker underscores a key philosophical insight: “The binary layout is the user interface.” The lessons from BBT persist, and Plummer recommends tools like Bolt and Propeller for similar modern optimizations. He emphasizes measurement and representative workloads are essential for meaningful performance gains, advocating for enduring attention to locality and hot path layout in the face of modern hardware abundance.