Propagation Cost: The Number That Measured Mozilla's Rewrite

In 1998 Netscape released the source code of its browser, and the volunteers who showed up found it so entangled that the project rewrote a large part of it. Years later, three researchers published a single number that says how much the rewrite helped.

The rewrite that got measured

After my posts on the Modularity Maturity Index and fitness functions, I went looking for older, harder evidence. I wanted proof that architecture can be measured rather than argued about. The trail leads to a 2006 paper in Management Science: MacCormack, Rusnak and Baldwin, "Exploring the Structure of Complex Software Designs". They picked two codebases the whole industry knew, Mozilla and the Linux kernel. Their question was blunt. When you change one source file, how many other files can that change reach?

Their method was direct. Treat every source file as a row and a column in a matrix. Extract every function call between files with a static analyzer (they used Understand C++) and mark the cell. Then compute the transitive closure. If A calls B and B calls C, a change in C can reach A through the chain. So that cell gets marked too. The share of marked cells in the whole matrix is what they named propagation cost. It is the proportion of the system that could be affected, on average, when one file changes.

The numbers told the story. In April 1998, Mozilla stood at 1,684 source files with a propagation cost of 17.35%. The size-matched Linux kernel (version 2.1.105, 1,678 files) sat at 5.82%. A change in Mozilla could ripple three times as far as a change in Linux. That fit the complaints of the volunteers who had tried to contribute. Then Mozilla ran its redesign in the fall of 1998, and the December build measured 2.78%. The paper puts it plainly:

Changes to a source file have the potential to impact 80% fewer source files, on average, after the re-design.

MacCormack, Rusnak & Baldwin, Management Science, 2006

One erratum worth flagging: the paper's body text says 5.16% for Linux where its Table 1 prints 5.82%. I use the table value here. Either way the claim holds. The redesigned Mozilla came out measurably more modular than the kernel it was being shamed against.

What the number actually is

I find the metric easiest to hold onto as three pictures. There is the import graph you already know, then the matrix version of that graph, then the matrix after you follow every chain to its end.

Read the middle matrix row by row: file A imports B, so row A, column B is marked. The right-hand matrix is what the literature calls the visibility matrix. It follows every chain. A imports B, B imports C, C imports D. A change anywhere down that chain can surface in A. Three orange cells appear that no single import explains. By convention a file is always visible to itself, which is why the diagonal counts. Thirteen of thirty-six cells are filled, so this little system's propagation cost is 36%. That is the entire metric. There are no weights or tuning constants hiding anywhere.

Computing it on codebases you know

The 2006 paper used function calls between C files. For JavaScript and TypeScript, the import graph is the practical stand-in. madge already extracts it (the same tool I used for the MMI post). What madge gives back is a plain object mapping each file to the files it imports. That is the middle matrix of the figure above. From there, propagation cost is a breadth-first search from each file and one division:

// visibility: every file reachable from `file`, including itself
const reach = new Set([file]);
const queue = [file];
while (queue.length) {
  for (const dep of graph[queue.shift()] ?? []) {
    if (!reach.has(dep)) { reach.add(dep); queue.push(dep); }
  }
}
// propagation cost = sum of |reach| over all files / n²

I pinned eleven codebases at specific releases. I ran madge on each one's main source directory and computed the number. The full harness, the raw dependency graphs, and the results live in kmaxat/propagation-cost-demo. Rerun it there, or add your own repos:

$ node analyze.mjs
madge eslint (lib)
  files=378 edges=630 pc=4.75%
madge webpack (lib)
  files=620 edges=2278 pc=66.11%
madge vue-runtime-core (packages/runtime-core/src)
  files=65 edges=436 pc=95.46%
...

The pair worth staring at is eslint against webpack. They are the two largest codebases in the set, the same order of magnitude in file count. They land a factor of fourteen apart. The reason shows up if you ask madge one more question. madge --circular finds zero circular dependencies among eslint's 378 files, and 187 circular chains in webpack's 620. eslint's architecture makes most files leaves. Each rule is a self-contained module that nothing else imports, so a change to it can reach almost nothing. In webpack, 412 of the 620 files form a single strongly connected core. Every file in that core can reach every other one. The file with the widest reach touches 614. This is the same shape of contrast the paper found between Linux and pre-redesign Mozilla. I reproduced it on a laptop in a few minutes.

vue's runtime-core at 95.5% looks alarming until you notice what I measured. It is one package inside a monorepo, deliberately cohesive, with 150 circular import chains that the Vue team lives with internally. Propagation cost is a property of the boundary you draw. Inside a module that is supposed to be one unit, high visibility is close to the definition of cohesion. Across a whole system, the same number would mean almost any change can reach almost any file. The metric has no way to tell which of the two situations it was handed.

The two mmi-demo rows are the toy library from my MMI post. I wrote it once with tangled dependencies and once cleanly, with the same eight modules. Propagation cost sees the repair: removing the cycles and the junk-drawer module drops it from 57.8% to 46.9%. It also shows the metric's floor for tiny codebases. With eight files, the diagonal and the entry point together fill nearly a quarter of the matrix before any real coupling is counted. The same caveat applies to express. express v5 keeps only six files in lib/ and delegates the rest to packages, so its 38.9% says more about measuring six files than about express.

What the number can't tell you

Robert Nord, Ipek Ozkaya and their colleagues at the Software Engineering Institute took a hard look at the metric in an ICSM 2013 paper. They found it gives inconsistent signals across some perfectly legitimate transformations. The clearest failure is the interface layer, the standard move for breaking a dependency. Adding one can raise propagation cost, because the new indirection adds visible pairs even as it makes the system easier to change. They propose weighting the matrix with pattern knowledge before trusting comparisons across refactorings.

My own runs add two more warnings. The number is only comparable at similar scale. The original authors respected that by size-matching their builds (1,684 Mozilla files against 1,678 Linux files). My six-file express row cheerfully violates it. And the dependency type matters. madge sees static imports, not function calls, so dependency injection, dynamic require, and event buses are invisible. The absolute values here are not comparable to the paper's. The comparisons between repos measured the same way are the meaningful part.

Running it in CI

I think propagation cost is most useful as a trend line on a single codebase, which makes it a natural fitness function. The whole computation is madge plus fifty lines of JavaScript with no dependencies beyond it. That is cheap enough to run on every merge. The harness in the demo repo recomputed every number in this post in a few minutes on my laptop. Adding your own repo to its repos.json is one line.

A CI step that fails when propagation cost jumps past its baseline turns "I think this PR couples things" into a diff you can point at.

References

• Alan MacCormack, John Rusnak, Carliss Baldwin, "Exploring the Structure of Complex Software Designs: An Empirical Study of Open Source and Proprietary Code" — Management Science 52(7), 2006. The working-paper PDF is free; all quotes and numbers above come from it.
• Carliss Baldwin, Alan MacCormack, John Rusnak, "Hidden Structure: Using Network Methods to Map System Architecture" — Research Policy 43(8), 2014. The follow-up that uses the same machinery to sort architectures into core-periphery types.
• Robert Nord, Ipek Ozkaya, Raghvinder Sangwan, Julien Delange, Marco González, Philippe Kruchten, "Variations on Using Propagation Cost to Measure Architecture Modifiability Properties" — ICSM 2013. The caveats section above draws on it.
• madge — the import-graph extractor used for every measurement here.
• kmaxat/propagation-cost-demo — the harness, pinned repo list, raw graphs, and results.