The Role of Chance Encounters in Silicon Valley Innovation

Cellphone signals and patent citations approximate a theory’s long-sought paper trail

As Silicon Valley legend has it, back in the day semiconductor hotshots gathered at Walker’s Wagon Wheel to drink and gossip and, by the third beer, a new technology breakthrough had often been devised. Repeated millions of times over, at any place but the office, chance encounters have kept Silicon Valley cranking out patents — and technology startups — faster and more profitably than any other region of the world.    

Proving this phenomenon, however, has eluded economists, even as cities around the world sought to recreate the Silicon Valley effect. Nobel Laureate Paul Krugman pointed out in 1991: “Knowledge flows are invisible; they leave no paper trail by which they may be measured and tracked, and there is nothing to prevent the theorist from assuming anything about them.”  

Enter the smartphone. Using access to vast data on cellphones, UCLA Anderson’s Keith Chen has in recent years helped explain the spread of COVID-19 at nursing homes, voter suppression and even the disinclination of politically opposed relatives to linger over a holiday meal.      

In a working paper, Chen and MIT’s David Atkin and Anton Popov track 425,000 cellphones around Silicon Valley, through 218 million episodes in which two seemingly unrelated workers cross paths in coffee shops, transport depots, bars, apartment complexes, schools, restaurants, doctors’ waiting rooms and other locations away from their own offices between September 2016 and November 2017.      

Focusing on encounters unlikely to represent planned meetings, the researchers estimate how much of the region’s new patent activity is due to chance encounters between workers of different firms.

Fancy Meeting You Here

Without such encounters, patent citations between Silicon Valley inventors would decline by about 8%, according to the study. The innovation boost calculated from these encounters, which for study purposes occur miles away from either worker’s office, is about twice as large as a similar effect found when two firms’ offices are simply located near each other.

The study can’t draw conclusions about exactly what follows when a programmer and an actuary walk into a bar. “But what we did ask is, do tech firms who happen to be located in a way that their workers serendipitously bump into each other a lot… do they cite each other more [in patent filings]?” Chen says.

“And the answer is, quite a bit.”

The above calculation isn’t end-all proof that conveniently located bars and parks and golf clubs help inspire new technology. The paper’s findings are the result of multiple statistical regressions and design workarounds necessary when one seeks evidence that a Nobel Prize winner has deemed invisible. No one has found a straightforward way to measure the effects of informal knowledge flows yet.

But detailed cellphone data is a better place to start, Chen contends.

Research in this field doesn’t usually track actual encounters between workers. The studies typically focus on changes that make face-to-face meetings easier or harder to achieve, like more or less physical distance between firms, a low-cost airline entering the market, or bar closures during Prohibition. They essentially assume meetings took place when there’s evidence of increased innovation involving both parties, such as more academic collaborations or mutual citations in patent applications.

With cellphone location data, Chen can see that a lot of workers are crossing paths at Shoreline Golf Links, John F. Kennedy Middle School, Stein’s Beer Garden and other popular spots across Silicon Valley. “I can see that two people go into the same coffee house at the same time,” Chen says. And he knows where all of them work and sleep. All that makes a stronger setup for proxying actual meetings, he says, even though he can’t prove those two coffee drinkers ever spoke to each other.

The researchers suspect that the most fruitful chance meetings are between acquaintances; workers from Apple and Cisco who once met at a party find themselves waiting for service at the same car dealership, for example, rather than two strangers starting a conversation at a bar. Such serendipitous meetings, they hypothesize, rekindle relationships that can develop over time into collaboration.

What Phones Reveal      

There are nearly 300 million smartphones in the U.S., and the vast majority ping their location to cellphone towers every 5 to 10 minutes; more often when they’re moving fast, like in a car. In the anonymized dataset Chen collected, each ping records a device identifier, timestamp, latitude and longitude, and estimates about the location accuracy. (It averages about 20 meters.) The phone has to have reception and a location-enabled app running in the background to ping, so usually the total number of times devices ID location is a lot less than the 250-ish times a day they potentially could.

Companies innovate by way of planned meetings all the time, of course, and the researchers had to account for that. Maybe all the invention is due to Apple engineers scheduling meetings with Cisco workers because they know the two teams are working on related issues.

Atkin, Chen and Popov have spent about seven years on this work, including developing a multitude of measures to address a host of such confounds. Is the geography really facilitating these meetings, or is it just the result of proximity between offices? Does that phone owner work at small company A or for small company B in the same building? Do differences in cellphone reception skew the results?

None of those questions would be relevant without clearing up the planned versus unplanned meeting issue. The results don’t tell us much if Apple contacted Cisco for that coffee.

Atkin, Chen and Popov mapped out the office locations of 18,360 patenting companies in Silicon Valley and used the phone location data to match them with some 51,580 workers at patenting companies. If a phone stayed in Oracle’s offices more than 20 hours a week, for example, its owner was deemed an Oracle employee. Wherever the phone stayed overnight was considered the worker’s home.

Patent Citations Linking Firms and People

Meetings, for study purposes, occurred every time two phones from workers at different employers ended up under the same roof for at least 10 minutes in a 30-minute segment.

The researchers scraped citation data in about 10 years of patent applications to find incidents of one Silicon Valley inventor citing the work of another Silicon Valley inventor.

To tease out how much of this mutual citation activity flowed from serendipitous encounters, rather than commonplace collaboration, the researchers built a sample of meetings between workers from companies that don’t ever file for patents. They excluded meetings between workers in the same industries or with supply relationships. They also culled pairings at popular places for planned meetings, like Stanford and the convention center. And they excluded encounters at places within 5 kilometers of either worker’s office, which would be likely places for planned meetups.

The resulting sample includes more than 200 million observations of crossed paths between people highly unlikely to ever arrange for a work meeting. To estimate how often workers from, say, Google and Intel accidentally cross paths, the researchers looked at meetings of unrelated workers from offices near each patenting firm; a meeting between a social media programmer with offices near Google and a CAD engineer working at a company near Intel, for example. Serving as proxies, these nonpatenting pairings allowed researchers to statistically estimate the frequency of unplanned meetings between Google and Intel.

Innovation’s Work-From-Home Penalty    

Their findings suggest that by simply crossing paths outside of work frequently, Silicon Valley workers are generating new technology that would not happen, or at least would not happen as quickly, if workers drove straight home to suburbs. So what if something makes these unplanned run-ins less likely to occur?

Of course, COVID-19 did just that. Starting in 2020, millions of workers were sent home to do their jobs, and many millions of former office workers are still working remotely. Economists have been scrambling ever since to learn how that change affects productivity. Of keen interest: Can the U.S. keep its innovation edge if top producers like Apple, Cisco and IBM permanently free workers from traditional offices?

Chen and his co-authors, whose data pre-dates COVID-19, nevertheless believe they found clues in the phone data. There, they witnessed some phones that normally go into offices part of the week spend predictable work days at home. On at-home days, the workers crossed paths with other workers much less frequently than their office-based colleagues. Using these variations in the likelihood workers would serendipitously meet, the researchers estimated how much patent citation would fall if some remote work becomes permanent.

Serendipitous face-to-face meetings would fall by 17% and relevant citations by 5.2% if 25% of Silicon Valley’s office workers worked from home, according to the researcher’s cursory estimates. If half worked from home, those losses rise to 35.1% and 11.8% respectively. If those former office workers move away — they could go almost anywhere else for cheaper housing — the region’s patent activity would fall further, the research suggests.

To mimic the exodus of a major patenting company — if, say, Microsoft’s team fled Silicon Valley offices permanently to work remotely — the researchers ran regressions assuming the likelihood of serendipitous meetings would fall from whatever level previously calculated (using proxy meetings) to almost nothing. They repeated the exercise for each of Silicon Valley’s Top 20 patenting companies.

The results are counterintuitive. An exit of Oracle or Genentech office workers would drop overall citations between Silicon Valley inventors by 19% or 13% respectively, the study estimates. Losing Google or Apple office workers, which show up in about three times as many citations, would barely be noticed by that measure. Perhaps their employees, working on huge campuses with amenities ranging from yoga to beer to laundry facilities, draw less inspiration generally from the outside world.