Why exceptional people cluster together and how small differences in talent create massive differences in productivity
Ajay Shah is an economist who has held positions at various government and academic institutions, known for his work on public policy and institutional reform. Amit Varma is a writer, podcaster, and the creator of "The Seen and the Unseen," one of India's most respected long-form conversation shows. Together, they host "Everything is Everything," where they explore big ideas through the lens of first principles, books, history, and lived experience.
Small differences in talent create exponential differences in output. This fundamental insight from economist Michael Kremer's O-ring theory explains why brilliant minds congregate in specific places - from Silicon Valley's programmers to Harvard's academics. Just as the $10 O-ring that failed brought down the $3.2 billion Challenger shuttle, teams with slightly less skilled members produce dramatically worse results than teams of exceptional performers.
Amit and Ajay explore how this clustering happens inevitably through market incentives, why creative people migrate to centers of excellence, and how technology is changing these patterns. They discuss brain drain from India, the Soviet chess system that dominated the world, and why someone like Viswanathan Anand achieving chess mastery without systematic training was as unlikely as winning Formula 1 in a Maruti 800. The conversation extends to institution building, examining why new organizations with coercive power tend to fail and what successful transitions like East Germany can teach us about building state capability.
This leads to broader questions about development, the role of authenticity in humanities versus universality in STEM fields, and whether India can create the conditions for talent to flourish at home rather than migrate abroad.
Shah, Ajay, and Amit Varma. "Why Talent Comes in Clusters." Episode 8 of Everything is Everything. XKDR Forum, August 18, 2023. Podcast, video, 1:09:03. https://www.xkdr.org/viewpoints/why-talent-comes-in-clusters-episode-8-everything-is-everything
The O-ring theory, developed by economist Michael Kremer, takes its name from the $10 component that caused the $3.2 billion Challenger space shuttle to explode in 1986. This wasn't just about weak links in a chain - it revealed something profound about how talent and productivity interact.
Consider ten people working together, each performing at 0.99 level - incredibly talented. Their combined output reaches 9.05. Drop their individual performance slightly to 0.95 - still exceptionally talented - and total output plummets to 5.99. At 0.9 each, which is still better than 90% of humanity, total output becomes just 3.49.
Amit explains the implications:
"A small difference in talent leads to a massive difference in output and productivity. Therefore the incentives are such that if you are a staggeringly competent or qualified or talented person, you need to be around other staggeringly talented and qualified people to get a full expression of your abilities."
This effect becomes most pronounced in two scenarios: when quality matters more than quantity (you want one great novelist, not three mediocre ones), and in complex tasks with thousands of interdependent steps where every component must perform flawlessly.
The O-ring effect creates powerful incentives for exceptional people to seek each other out. This isn't just preference - it's economic necessity. Someone performing at 0.99 level can justifiably earn twice or thrice what someone at 0.95 level makes, despite the seemingly small difference.
Historical examples illuminate this pattern. Oppenheimer struggled in school and college, even attempting to poison a physics teacher. But at the University of Göttingen, taught by Max Born (who coined the term "quantum mechanics") alongside classmates like Dirac and John von Neumann, he reached an entirely different level.
This explains why software professionals migrate to Silicon Valley and filmmakers to Bombay. Network effects amplify the clustering - entrepreneurs know the best coders are in Silicon Valley, and coders know that's where the best startups launch.
Richard Florida's concept of the "creative class" adds another dimension. His research showed that economic growth follows creative congregations - cities full of entrepreneurs, artists, and diverse populations including LGBT communities foster greater openness and talent flourishing. Capital then chases this talent, creating virtuous cycles.
Some individuals succeed despite lacking access to talent clusters, but these cases prove the rule rather than contradict it. The Soviet chess system exemplifies systematic talent development. As one functionary declared in the 1930s:
"We must finish once and for all with the neutrality of chess. We must condemn once and for all the formula chess for the sake of chess. We must organize shock brigades of chess players and begin immediate realization of a five-year plan for chess."
Chess was taught in Soviet schools like mathematics. This massive sample size made talent identification easy, with systematic training from beginner to grandmaster level. The "Soviet School of Chess" developed sophisticated heuristics and strategic thinking methods.
Devangshu Datta, who played serious chess for India in the 1980s, described the cultural gap:
"When I started playing East Europeans in the 1980s, the difference in chess culture was stark. We knew so much less, it wasn't funny. To take an analogy, it was like putting a bunch of talented kids with a basic knowledge of self-taught HSC level maths into direct competition with people who had postgrad math degrees."
Against this backdrop, Viswanathan Anand's rise was extraordinary. Coming from India without systematic training, competing against the Soviet machine was like "taking a Maruti 800 into a Formula 1 race and winning the damn thing" or "winning Wimbledon with a wooden racket."
Remote collaboration has evolved dramatically. Ajay recounts writing a paper with statistician Achim Zeileis in Vienna and economist Ila Patnaik without ever meeting:
"We just struck up a conversation over mailing lists and the internet, and we realized we were chasing different pieces of a correlated puzzle and we came together to solve it and we got a beautiful paper done. Only later we met and we started talking and it was great. But we got all the way to our first paper without even video calls, just on email and SVN."
However, human rapport remains essential. Successful remote teams combine periodic in-person meetings with distributed work. The contracting and trust-building phases require physical presence - understanding someone's humor, building relationships, negotiating resource allocation. Once these foundations exist, actual work can happen globally.
This creates hybrid models where key people establish relationships and agreements in person, then implement across distances. Many breakthrough collaborations in London, New York, and Cambridge, Massachusetts happen because people can form coalitions face-to-face before distributing the work.
Different disciplines have varying requirements for local presence. STEM subjects are increasingly universal - people worldwide pursue similar problems with growing cross-border collaboration capabilities. Standardization and APIs in software allow global teams to work together effectively.
But humanities and social sciences require authenticity and local knowledge. As Ajay notes:
"Being inside your subject and being closer to your subject matters greatly. Being in India, you are constantly inside your subject matter and you're invisibly picking up knowledge every day."
This applies to filmmaking too. Bollywood exists in Bombay because that's where the cultural conversation happens, where innovation emerges, where you understand what's next. You can't tap into that state-of-the-art knowledge from outside the conversation.
The University of Chicago's early greatness partly stemmed from being away from East Coast fashion, fostering independent thinking. But for understanding human society, proximity to your subject provides daily invisible education that distance cannot replace.
Most attempts to create new government organizations in India fail because they're thrust into complexity without adequate preparation. When organizations suddenly receive coercive power, they enter firefighting mode immediately and never recover.
Ajay explains the typical failure pattern:
"That organization goes into firefighting from day three and it never recovers from the firefighting. It is just one big mess because it fails to have rule of law in the beginning. Then you get all the wrong incentives of the people lobbying and pushing to do special favors at a transaction level and that becomes the institutional DNA."
The solution requires treating institution building like bridge construction - recognizing it takes time and systematic preparation. Organizations need organizational design, training, IT systems, and careful checks and balances before receiving full power.
The closest India came to this approach was in planning the Insolvency and Bankruptcy Board (IBBI), where a detailed project plan outlined a two-year process to create a high-quality regulator. Unfortunately, the law was notified and five people were expected to perform immediately, abandoning the systematic approach.
East Germany succeeded because it inherited West Germany's complete institutional package - liberal democracy, constitutionalism, rule of law, and checks and balances. This wasn't just elections every five years, but an entire system that attracted decent people to leadership positions.
Eastern Europe and Baltic republics also succeeded, helped by three factors: experience with communism's failures, making them eager to escape; EU membership providing 20,000 pages of laws and restrictions that prevented certain mistakes; and the dramatic symbolism of transformation - like Warsaw converting the Communist Party headquarters into a stock exchange.
Russia's failure stemmed from focusing only on economic liberalization while neglecting state institutions. As Ajay observes:
"There's no running away from that. You need the state. You need liberal democracy, you need freedom. And freedom is coded into laws and state institutions. If you have unchecked state violence in those state institutions, then it's never going to work."
For China's future, Taiwan's institutional DNA and capability offer more promise than exile structures like the Central Tibetan Administration, which operate as philanthropic organizations without the harsh realities of wielding coercive power.
George Orwell's essays demonstrate how writing and thinking interconnect. Reading Orwell can literally improve your own thinking and expression - not through the specific content, but through exposure to clarity of language and rhythm.
The relationship works both ways: clear thinkers tend to write clearly, but forcing yourself to write clearly also compels better thinking. You cannot hide behind abstract words or vague jargon when pressed to explain something simply.
As Amit notes, echoing Joan Didion: "I don't know what I think until I write it down." Writing becomes research, self-discovery, and intellectual development. This principle applies beyond non-fiction - to personal essays, fiction, and any creative process where clarity of expression reveals clarity of thought.