Conway's Law, Measured: Git Can Only See Half

In short: Git can test half of Conway's law. I built a contributor-by-module commit matrix for webpack, ESLint, Prettier, Rollup and Vite. Then I compared each one's normalized mutual information with 1,000 shuffles that keep every activity total fixed. Three repositories showed more contributor-module structure than their activity predicts. Two did not. Nothing in a repository says who reports to whom or who talks to whom, so none of this grades a team.

A Windows Vista model reached 86.2 percent precision and 84.0 percent recall when it used organizational metrics to predict failure-prone binaries. It is a striking result. It is also not a direct test of Conway's law.

The famous result predicts failures

Nagappan, Murphy and Basili built eight measures from Microsoft's organizational data. They counted developers, ownership depth, reporting distance, edits and other process signals for each Vista binary. Their model then predicted which binaries would fail after release. The Vista model reached 86.2 percent precision and 84.0 percent recall.

That is good defect prediction. It does not compare an org chart with the system's module map. The dependent variable is failure proneness, not architectural similarity. The paper offers strong evidence that organizational context carries useful information about software quality. Reading it as a clean confirmation of Conway's law asks the experiment to answer a different question.

I wanted to test a smaller question with public data. Do contributors tend to work inside declared module boundaries more than their activity levels alone would predict? Git can help with that question. It cannot tell me whether those contributors sit on the same team or talk every morning.

I used the same pinned versions of webpack, ESLint, Prettier, Rollup and Vite as the earlier posts in this series. Each project has checked-in module rules. Workspaces define modules where they make sense. Reviewed source directories cover the rest.

The matrix is simple. A row is a contributor and a column is a module. A cell counts distinct commits where that contributor touched source in that module. One large commit still counts once for each module it touches.

I summarize the matrix with normalized mutual information, or NMI. In plain terms, it asks how much knowing the contributor reduces uncertainty about the module, and how much the module tells us about the contributor. The scale runs from zero to one. Higher values mean a tighter association. They do not mean a better architecture.

A prolific contributor or a busy module can create apparent structure on its own. So I shuffle the module labels while preserving every contributor total and every module total. The middle ninety-five percent of those shuffled scores is the baseline interval. An observed score above that interval is more structured than the activity totals would normally produce.

Three clear signals, two ordinary ones

webpack, ESLint and Prettier finished above their shuffled intervals. Rollup and Vite stayed inside. Rollup went a little further in the other direction: its observed score was below the shuffled median. The five repositories did not give me a neat confirmation story.

The table also reports two supporting measures. Effective owners rises when a module's work is spread across contributors. Breadth rises when a contributor works across modules. Both discount a long tail of one-off touches, which makes them more useful than a raw headcount.

webpack has the strongest separation from its baseline, yet its NMI is the lowest observed score in the table. That is not a contradiction. Its many modules and uneven activity create a low null expectation. The comparison with the shuffle carries the meaning, not the raw score by itself.

Rollup is the useful disappointment. Its contributors and modules form no stronger association than the shuffled activity pattern. Vite's observed score is higher, but its wide interval still contains it. Calling either repository a violation of Conway's law would invent organization data I do not have.

ESLint had the highest observed NMI among the three repositories above baseline, so it makes the clearest readable matrix. Dark cells show repeated work by one contributor in one module. Pale rows belong to occasional contributors. The dense rules column reflects a real center of activity.

The median ESLint contributor touched one effective module, but the median module had more than five effective owners. Those facts fit together. Many people can each make a narrow contribution to a shared module. A single NMI value hides that ownership shape.

I would inspect the wide rows and crowded columns first. A wide row may belong to a maintainer, a release task or a change that cuts across boundaries. A crowded column may be a healthy shared service. It may also have unclear ownership. The matrix chooses where to ask. It does not supply the answer.

The window check changed the picture most for ESLint and Vite. Short windows amplify a burst of focused work. Longer windows mix releases, maintenance periods and contributor turnover. That is why I do not treat the headline as a permanent project property.

The identity result surprised me. The analyzer used each repository's .mailmap and then exact normalized email. It did not guess that similar names were the same person. In these pinned windows, the mailmaps resolved no aliases. Zhu and Wei found that multiple names and addresses are common enough to threaten repository studies. My zero change should stay local to this dataset.

Research did not settle into one result

The Vista finding was never the last word. Caglayan and colleagues repeated the organizational-metric study in another commercial system. Their organizational model beat churn, but a model based on pre-release defects beat it. They concluded that performance depends on the project and prediction level.

Socio-technical congruence research asks a closer coordination question. Cataldo, Herbsleb and Carley compared coordination needs with observed coordination. Better congruence was associated with faster change-request resolution. The result depended on which technical dependencies created the supposed need to coordinate. Logical dependencies mattered more than calls or shared data.

Kwan, Schroter and Damian then studied build success. Congruence mattered for some build types, not all. In some settings, higher congruence went with lower build success. Even zero-congruence builds often succeeded. Context did more work than the slogan.

Open-source studies add another caution. Kamola inferred both developer groups and module groups from repository networks, then found that the clusters matched poorly. Mauerer and colleagues tried several socio-technical constructions across twenty-five projects. They found no relation to bugs or churn in any time setting they tested.

These papers do not cancel one another. They measure different social traces, technical dependencies and outcomes. Together they make a compliance score look hard to defend. The result changes when the definition of required coordination changes.

My own definition is narrow on purpose. The analyzer reads non-merge commits inside checked source boundaries and follows renames where Git exposes them. It removes tests, fixtures, generated and vendored files, lockfiles, minified files, binaries, bots and configured bulk commits. A source file must match one module rule. Ambiguous and unmatched files stop the run.

Each contributor-module observation is one distinct commit touch. NMI is normalized by the smaller of contributor entropy and module entropy. The null model permutes module labels but retains contributor totals and module totals. It measures unusual association after activity is held fixed. It does not test communication or organizational cause.

Some declared modules had no eligible touches in shorter windows. I kept those boundaries in configuration instead of changing the module map after seeing the data. The reported module count is the set that had observed work in that window. At the headline cutoff, ESLint had no touches in mcp or types. Prettier had none in experimental-cli, Rollup had none in watch, and Vite had none in runtime, types or root. Every webpack module had work.

Five JavaScript and TypeScript tools are a narrow sample. Module boundaries are reviewed choices, not facts found in nature. Commit touches are not effort. An email address is not always one person, and one person can still have several addresses. Private messages, meetings, issue discussions and reporting lines remain missing. Nothing here links alignment to defects, speed or quality.

Use it as a review queue

I would run the matrix before changing team ownership or splitting a package. Start with modules that have diffuse ownership and contributors who cross many boundaries. Open their recent changes. Then ask the people involved how coordination works and whether the module map still fits the work.

A concentrated module may be well owned or guarded by one person. A wide contributor may be a system expert or a sign that the boundaries leak. Those interpretations require facts Git does not contain. That missing half is not noise to clean away. It is the part needed before making an organizational claim.

Three repositories showed contributor-module association beyond their shuffled activity patterns. Two did not. I would use the matrix to choose a coordination question, never to grade a team or declare that a repository obeys Conway's law.

I'm Maxat, a software engineer testing architecture research against real repositories. The dependency-free analyzer, pinned commits, raw results and generated figures behind this post live in this site's repository.

References

• Nachiappan Nagappan, Brendan Murphy and Victor Basili, "The Influence of Organizational Structure on Software Quality: An Empirical Case Study", ICSE 2008. Source of the eight organizational measures and the Vista precision and recall result.
• Bora Caglayan, Burak Turhan, Ayse Bener, Mayy Habayeb, Andriy Miransky and Enzo Cialini, "Merits of Organizational Metrics in Defect Prediction: An Industrial Replication", ICSE 2015. The commercial replication with a different ranking of metric families.
• Marcelo Cataldo, James Herbsleb and Kathleen Carley, "Socio-Technical Congruence: A Framework for Assessing the Impact of Technical and Work Dependencies on Software Development Productivity", ESEM 2008. Source of the congruence framework and dependency definition caution.
• Irwin Kwan, Adrian Schroter and Daniela Damian, "Does Socio-Technical Congruence Have an Effect on Software Build Success?", IEEE Transactions on Software Engineering, 2011. Source of the contextual and inverted build results.
• Mariusz Kamola, "How to Verify Conway's Law for Open Source Projects", IEEE Access, 2019. An open-source clustering study with poor developer-to-module matches.
• Wolfgang Mauerer, Mitchell Joblin, Damian Tamburri, Carlos Paradis, Rick Kazman and Sven Apel, "In Search of Socio-Technical Congruence: A Large-Scale Longitudinal Study", IEEE Transactions on Software Engineering, 2021. The twenty-five-project null result for bugs and churn.
• Jiaxin Zhu and Jun Wei, "An Empirical Study of Multiple Names and Email Addresses in OSS Version Control Repositories", MSR 2019. Source for author aliasing as a validity threat.