Kaggle Challenges and the Value of Data Science

The impact of data on business outcomes is covered with buzzwords. The people in the loop say real things sometimes (examples here), but there’s a twist. Vendors picks only the best cases that sell their stuff, and their clients conceal successes to leave competitors guessing.

Let’s turn to Kaggle for balanced statistics. The Kaggle competitions put participants in the same conditions, which allow for easy comparison. The website maintains public and private leaderboards for each competition, based on test data. I use the set of public leaderboards available here.

Businesses hire many data scientists now. And the first interesting question to the data is: should I select talents carefully or hire people fast? Here’s a test: let’s look at the winning margins on the top of leaderboards. If they’re large, then the skill premium may be large as well, so it’s worth looking for better candidates and pay them more. This is the answer in one chart:

kaggle_winners_handicap

Each line represents a competition. The y-scale shows the final score of a participant as a fraction of the winner’s score. The score is a statistical metrics reflecting the quality of a (typically) prediction of interest, such as revenues, votes, or purchases. In some cases, the higher score is better, in the others, it’s the opposite. Lines are moving in the respective directions.

A single leaderboard from that chart may look like this (insurance-related competition):

kaggle_ranking

This case is slightly unusual because it has distinctive leaders with large handicaps. Still, those who try—the red dots—eventually succeed. The problem is, very few do try:

kaggle_submissions_per_team

In 4,000 cases, a team submitted only a single solution in a competition. Really serious attacks on the problem start with 10+ submissions, which few teams make.

Despite this, many participants end close to the winner:

kaggle_scores_kde

Looking from a different perspective on individual performance, I compare how the same users completed different competitions:

kaggle_place_matrix

These five races involved 500+ users each, and some users overlap. The overlapping shows the Kaggle core: the people who compete regularly and finish high (left-bottom corners of each subplot). Elsewhere, the relationships are weak.

These modest evidences suggest that people matter less and commitment more.

Does time matter? I take the means by the days remaining until the last submission:

kaggle_progress_5

This data belongs to the attempts to predict lemons at car auctions. The higher score is better here, and you see that additional submissions don’t improve the quality of an average submission. The leaders do improve slowly, however. Data scientists find low-hanging fruits in available data quickly and then fight for small improvements with much time investments. For one example, read this detailed journal by Kevin Markham.

A typical disclaimer would mention various limitations of these plots for decision making or of Kaggle competitions for real cases. Yes, while hiring, you need to know more than this. I would emphasize a different thing. Managers like intuitive decisions and confirm them with favorable evidences, including statistical insights. But having numbers this way isn’t the same as thinking that starts from numbers. Most businesses can get almost nothing from data scientists before their managers start thinking from numbers, not to numbers. And this transition from intuition to balanced evidences yields more than improving a single prediction by a few percentage points mentioned here.

Data and replication files on GitHub

Reddit’s Big Brother Is Watching Your Menus

For those who are interested in the impact of design on substantial outcomes, here’s the number of subscribers in the 54 most popular subreddits on reddit.com:

reddit_top54

It’s a nice, smooth transition reflecting the popularity of each subreddit.

The same goes for subreddits after the 54th one:

reddit_top55-100

But let’s put them together:

reddit_sub50-60

A huge discontinuity appears between “TwoXChromosomes” and “woahdude”. Like 3.5 times. Normally, the data doesn’t behave this way.

But it does here. The reason is, the designers put exactly 53 elements in the two main menus (on the left and on the top):

reddit_menus

(Check the numbers)

These elements, as adeadhead reminds in the comments, reflect the default choice of subscriptions added to a new user’s list. I can think of additional channels, like user attention and search engines, that create this big difference between included and excluded elements. However, the connection is causal and at the disposal of designers.

PS: You could notice the “atheism” subreddit that slightly spoils the discontinuity. The subreddit had been in the menus before (snapshot) but was removed after 2012. The subscribers remained, though few new ones signed in, because the subreddit wasn’t in the menus.

See also

Best Time to Post? It’s Irrelevant

While social media invent various algorithms to show relevant information to users, companies like Buffer are trying to understand how to circumvent these algorithms to promote their clients’ content. This is not necessarily a zero-sum game, as it may seem. Optimizers add more structure to the content, pick relevant addressees, and distribute content to the media where information overload is less extreme.

The simplest problem around is to pick the best time for posting when you already have certain content. I looked into this once for StackExchange.com, and the optimal timing happened to depend a lot on the subsite in question. StackExchange is a network of Q&A websites built on a common technology but with somewhat segregated users and different rules. The subsites look alike, integrated, and you normally expect the common features to prevail over everything else. But according to the data, the patterns of performance, such as time-to-answer, vary across the subsites. The soft rules—those that are not engraved in the common software code—and people make them vary.

Here’s another example: Y-Combinator’s Hacker News, which has a solid community and transparent ranking algorithm. The rules are simple: a user submits a link and title, the community upvote this submission. Good submissions make the front page, bad submissions are unread and forgotten. The service receives more than 300,000 submissions annually. The question is the same: given a submission, what’s the best time to post it? I took the number of expected upvotes as the criterion.

Many studied the Hacker News dataset before. A good example is this one. There’s even a special app for picking the time (I didn’t get what it does exactly). They answered different questions, though.

Here’s my version of events. In this post, however, I’d make another point based on this data.

First, just looking at upvotes shows that weekends are the better days for posting (0 is Monday, 6 is Sunday):

Source
Source

In particular:

Source
Source

However, this approach can’t say much. Time affects not only users who read links submitted to Hacker News (demand), but also those who submit the links (supply). You have causation suspects right away. Like, maybe users submit better links on weekends because they have more time to pick the good ones. Then scheduling the same submission of yours to weekends would not increase the upvotes it gets.

For a bunch of typical reasons (few variables available, unstructured data, and no suitable natural experiments), the impact of time on upvotes is hard to separate from other factors. You have only indirect evidences. For example, less competition on weekends may increase expected upvotes:

Source
Source

It remains unclear how to sum up indirect evidences into conclusions. Statistical models would disappoint. Time-related variables explain less than 1% of variation—meaning, unsurprisingly, that the other 99% depend on something else. This something includes the page you link to, the readers, and nuances of Hacker News’ architecture.

My point is, even a simple algorithm can be efficient, meaning, its outcome is independent of irrelevant factors, like time. A complex algorithm may perform worse, in fact. If content promotion depends on the author’s social capital (followers, friends, subscribers), ranking relies on the author’s past submissions rather than the current one. So, Facebook’s or Quora’s algorithms for sorting things for users are not only harder to pass through; they also may distort important outcomes.

See also: Python notebook with Hacker News data analysis

Shakespeare Or Death

Leo Tolstoy had a life-long feud with Shakespeare:

I remember the astonishment I felt when I first read Shakespeare. I expected to receive a powerful esthetic pleasure, but having read, one after the other, works regarded as his best: “King Lear,” “Romeo and Juliet,” “Hamlet” and “Macbeth,” not only did I feel no delight, but I felt an irresistible repulsion and tedium, and doubted as to whether I was senseless in feeling works regarded as the summit of perfection by the whole of the civilized world to be trivial and positively bad, or whether the significance which this civilized world attributes to the works of Shakespeare was itself senseless.

Naturally, after all, Tolstoy discovered that “the significance which this civilized world attributes to the works of Shakespeare was itself senseless.” The reason was:

Until the end of the eighteenth century Shakespeare not only failed to gain any special fame in England, but was valued less than his contemporary dramatists: Ben Jonson, Fletcher, Beaumont, and others. His fame originated in Germany, and thence was transferred to England. […] These men, German esthetic critics, for the most part utterly devoid of esthetic feeling, without that simple, direct artistic sensibility which, for people with a feeling for art, clearly distinguishes esthetic impressions from all others, but believing the authority which had recognized Shakespeare as a great poet, began to praise the whole of Shakespeare indiscriminately, especially distinguishing such passages as struck them by their effects, or which expressed thoughts corresponding to their views of life, imagining that these effects and these thoughts constitute the essence of what is called art.

On Shakespeare, [103–114]

The hypothesis in bold is easy to check. The mentions of Shakespeare in the 19th century books:

Google N-grams
Google N-grams

(Bacon and Chaucer included for controls.)

Shakespeare’s rating is flat since 1564, though the early data is noisier. His mentions in German books increase alongside with those in English literature.

So, Shakespeare indeed was rediscovered 300 years after his birth. Is Tolstoy correct about the quality? In a typical rebuttal, George Orwell says it’s not up to Tolstoy to judge because “there is no test of literary merit except survival.”

But I think Orwell is missing the point. By the end of his life, Tolstoy became an overwhelmingly social—not literary—critic, and so his piece is about “the significance which this civilized world attributes to the works of Shakespeare.” The poet turned out to be a useful illustration that the civilized world saw nothing in Shakespeare for three centuries, then suddenly woke up and made him the number-one celebrity in the nomination (like here). Tolstoy attacks not the poetry but the idolatry, promoted by the likes of Harold Bloom, himself an authority, who put Shakespeare in the “Center of the Western Canon” in 1994.

How can we separate quality from reputation in this case? Well, schools and universities remain the major marketing channel for Shakespeare. For a good writer, getting into a mandatory reading list shouldn’t be a great deal since he’s always in demand. But for Shakespeare, popularity depends on the schooling cycle:

Google Trends
Google Trends, US only

Red peaks show how students buy textbooks in January and August. Shakespeare’s popularity peaks in April and vanishes by the summer break. Is summer a bad time for reading? On the contrary, kids have more free time. For example, publishers released all Harry Potter books in either June or July—just to get into the reading season. The graph says Shakespeare loses when people read books they like. And since it’s a blasphemy to compare the poet to popular authors, can we understand what attracts free people to Shakespeare?

Instead of getting into value judgements of his readers, I’d make a couple of economic guesses about his popularity. First, the McDonald’s hypothesis. Despite its bad reputation, McDonald’s provides certain quality and menu everywhere, and the client gets exactly what he expected (which may be better than having seafoods in tropics and ending up with intoxication). Similarly, wherever you happen to be, Shakespeare is on stage of the nearest theater. They had Shakespeare in London and New York, in Soviet Moscow, and on screens adapted by Japanese director Akira Kurosawa. Hitler personally excluded Shakespeare from the list of authors banned by the Nazi. Maybe the only big historical case of Shakespeare deprivation was China during the Cultural Revolution.

The second version is industrial. As performing arts got more complex, theater professionals needed a yardstick that separates performers from authors. If you saw something good on stage, you might be puzzled whether actors were good or the stuff was well written. And you could easily separate performance out if you keep the writings constant. Shakespeare is a perfect candidate for his cartoonish characters and stories. If you saw ten Hamlets in action, you can pick the favorite one. It’s like running 100 meters: everyone runs in the same conditions. Here, Shakespeare survived because he became the industry standard.

These versions are difficult to check, of course. And here I like Tolstoy’s approach more because his appraisal is not about proofs. As an Enlightenment guy, Tolstoy only encourages to question the quality of Shakespeare’s writings, instead of rationalizing his popularity. That’s a good exercise to try on contemporary authors and their works.

Alibaba, The State

Alibaba is sort of doing fine after the IPO. But what does it do? It replaces the state.

Roughly, if a firm picks a supplier, it wants supplies to be fine and to arrive in time. The supplier, in turn, want to make sure that the client pays as agreed.

Now, there are two ways to provide it for sure. Option A is the threat of legal actions if things went terribly wrong. Option B is to avoid bad partners at all. The state offers both options. It has licensing and regulators to prevent very bad companies from operating in the market. And the state also has a more traditional function of bashing bad businesses for violating the law.

Obviously, Alibaba is not the British East India Company—it cannot apply violence freely. But it does offer an alternative to government regulations, especially in countries where governments are not trusted. The website routinely offers inspections and secure payments. It encourages buyers to leave feedbacks. As a matter of punishment, it can ban businesses from the marketplace.

Alibaba reduces the risk, which would otherwise require more resources to meet. Though private inspections, insurance, and feedbacks have been there for centuries, IT technologies made them extremely centralized and embedded in a single company. The state also implies a monopoly—and online marketplaces have it! Amazon, eBay, and Alibaba have no strong competitors in their respective markets.

Does this replacement for weak governance affect economic development? Possibly. Alibaba is an international trade hub. Normally, small and medium enterprises are reluctant to deal with international partners due to uncertainty. For example, the World Bank points at political risks:

Source
Source

Those investors who actually work in emerging markets estimate the risk as being three times lower than that by investors who don’t consider investing in emerging markets at all. Uninformed investors overstate risks and stay away from what can be a perfectly normal market.

Many of the B2B transactions mediated by Alibaba might not have happened at all without the relevant information. For one reason, the baseline risk is high as governments in emerging markets are reluctant to prosecute local crooks. For another, western mass media cover these markets biasedly. Someone who read the Financial Times throughout 2014 might have an impression that China is nothing more but corruption, political trials, empty infrastructure, ghost cities, and permanently slowing down economic growth. Even if these materials are not necessarily biased against one country (the media look for a drama everywhere, right?), the readers can’t simply go out in the streets and check how things really are, as for domestic coverage. Therefore, businesses need a middleman who is more motivated than the state and more systematic than the media in helping shoppers in emerging markets.

Twitter, Brevity, Innovation

Singapore’s Minister for Education [sic] recollects his lessons from Lee Kuan Yew:

I learned [from Lee] this [economy of effort] the hard way. Once, in response to a question, I wrote him three paragraphs. I thought I was comprehensive. Instead, he said, “I only need a one sentence answer, why did you give me three paragraphs?” I reflected long and hard on this, and realised that that was how he cut through clutter. When he was the Prime Minister, it was critical to distinguish between the strategic and the peripheral issues.

And that’s what Twitter does. It teaches brevity to millions. Academics and other professionals who face tons of information daily must love it. First, because it saves their time. Second, it prioritizes small pieces of important information.

Emails and traditional media do this badly because people can’t resist the temptation to get into “important details.” But my details are important only after you asked for them. And Twitter restrains me from writing them in advance by leaving me only 140 characters (right now, I’m over 100 words already). So, it saves two people’s time. As Winston Churchill, himself a graphomaniac, said, “The short words are the best.”

Short messages earn most interactions
Short messages earn most interactions (Source)

Like many other good ideas, this wasn’t the thing founders initially had in mind. They had to cut all messages to 140 characters to make them compatible with SMS and, thus, mobile. Later on, web services, such as Imgur, borrowed this cutoff. This time not as technical restriction, but to improve user experience. That’s an easy part.

The second part is difficult. Twitter is bad at prioritizing information. Tags and authors remain the major elements of structure. Search delivers unpleasant experience (maybe this made Twitter cooperate with Google). If you missed something in the feed, it’s gone forever.

This weak structure is partly due to initial engineering decisions. However, structuring information without user cooperation is difficult everywhere. And users won’t comply as twits should be effortless by design. It means engineers have to do more of hard work. In turn, it costs money and time. There must be strong incentives to do this. The incentive is not there because Twitter lacks competition.

Would anyone step in and fix it? Suppose, you’re taking a cheap way and ask users to be more collaborative. You can make Twitter for academics with all the important categories, links, and whatever helps researchers communicate more efficiently. This alternative will likely—if it hadn’t yet—fail to gain a critical mass of users. Even in disciplined organizations, corporate social networks die due to low activity. Individually, employees remain with what others use. The others use what everyone uses, and everyone uses what he used before. You need something like a big push to jump from the old technology.

Big pushes away from Twitter is more like science fiction now. Whatever deficiencies it has, the loss-making company priced at $30 billion dollars wins over better-designed newcomers. In the end, its 280 million users are centrally planned by Twitter’s CEO. That’s about the population of the Soviet Union by 1991.

It’s not new that big companies lock users in their ecosystems. The difference is, sometimes it’s justified, other times it’s not. For Twitter, it’s difficult to imagine any other architecture because major social media services all impose a closed architecture with third-party developers joining it on slavery-like conditions. To take the richest segment, most of iOS developers don’t break even. So, apart from technical restrictions that Twitter API has, the company doesn’t offer attractive revenue sharing options to developers that contribute to its capacities and, thus, market capitalization. For example, to address the structural limitations mentioned before.

All in all, interesting experiments in making communications more efficient end very quickly as startups reach traction. After that moment, they become conservative, careful, and closed. And this is a step backward.

Thinking Like Lee Kuan Yew

This is Lee Kuan Yew’s miracle everybody’s talking about today:

Data from Maddison
Data: Maddison

The rising green line includes better healthcare, education, security, housing, and other benefits of economic growth. A distinctive feature of Singapore—compared to virtually all developed countries—it hasn’t closed its borders after becoming rich:

Data: WDI
Data: WDI

Which shows how good institutions adapt immigrants and the country continues to grow in per-capita terms. “They’re stealing our jobs” and other forms of intellectual racism never look for the examples like this.

How much did Lee contribute to this success? Scarce evidences on personal contributions to economic growth (like Jones and Olken, “Do Leaders Matter?” [ungated working paper]) leave some space to leaders to affect history. But in specific cases, impact evaluation is informal. In this case, endorsements are also overwhelmingly positive—for the last thirty years or so. Then how did he do it?

Lee shares his executive experience in his well-known book From Third World to First. Perhaps, it’s a bad guide to development because readers may screen it for confirmatory evidences that reenforce their own opinions about economic policies. But the book has two valuable qualities that rarely coincide. First, it’s written by a top politician. Second, it’s written by someone who thinks hard.

In the book, Lee explains his decisions and their reasonable foundations. Why is his reasoning important for others? Because economic development is all about context. When a policy maker copies a decision without reasoning, for a start, he doesn’t understand the decision. Then he applies it to a wrong situation. Industrial policies in developing countries are full of this misunderstanding.

A politician rarely explains himself, and when he does, he is torn between embarrassment and empty words. In contrast, Lee has the point and refutable defense. His colleagues also recall that he’s okay to change his opinion. It seems trivial with all sophisticated economic research on topic; but when leaders lack these qualities, it’s irrelevant how much we know about development (or anything else, for that matter).

So, unlike most commentators of the day, I’d pay tribute not to what Lee Kuan Yew has done but how he thought about these things. The book is a good source to learn it.

Not So Free Facebook

The IT industry has two types of products: those that save time (think of Google Search) and those that waste consume time (like Facebook). Though both are free, time spent on Facebook is sort of opportunity costs, typically equal to the user’s wage or whatever he does instead.

Even if the consumer formally pays nothing for either of the services, his behavior is not the same. That’s because of demand elasticities. One marginally relevant example:

JPAL
JPAL

These are demand curves. Percentage shows adoption rates. Nevermind the goods on the right. These are not IT and even not the developed world, but this is the most illustrative data of this kind around.

Most goods have elastic demand here. The blue curve also shows the striking difference in demand between zero and any positive price. This is very much like web products: the user base shrinks rapidly when the price becomes positive. For the freemium models, the premium user base is south of 5%. That’s why startups avoid pricing users at early stages.

Facebook also likes to pose itself as a free product. But it’s not really free. According to stats, an average user spends 40 minutes per day on Facebook. Though overstated, such usage is equivalent to $13 paid each day with the median US wage taken as opportunity cost.

Facebook, unlike Google, can set nominal access fees. Users already pay a lot for it, and equilibrium is around inelastic zone of the demand curve. Paywalled Facebook would make its shareholders happier because its current evaluation at $200 per user skyrockets with the enhanced cash flow. The current ad-based model is a dead end for Facebook because its ads target cold clients (compared to Google’s and Amazon’s visitors). While current earnings are very low for such a big company, Facebook’s P/E ratio of 75 is what investors are ready to pay knowing the forthcoming switch to a viable business model—and the paywall is one of them.

The logic of low elasticity under positive opportunity costs is relevant for other time-consuming services. Major newspapers had got paywalls long ago, but for other reasons: they have fewer users and high labor costs. Genuinely scalable web services are reluctant to experiment with payments and settle with nicely looking “premium” prices, like $5 or $10, which are loosely connected with costs and nearby offers, but never look like empirically grounded. Generally, these services prefer rules of thumb to experimentation. Maybe that’s a miss, since when the monthly fee is way below the hourly wage, demand is expected to be inelastic, so revenue opportunities must be around.

And yes, that’s possible because the IT industry is basically many monopolies complaining a lot about competition which isn’t there.

It’s a Wonderful Loan: Economics of P2P Lending

The Financial Times wonders why big banks are going after P2P lending. Why do banks need companies like Aztec Money and Lending Club, which have negligible credit portfolios and messy business model? Well, banks themselves might say about their motivation in this case (so far they didn’t), but I can think of a good economic reason why they should pay attention to P2P lending.

This reason is older than the Internet, computers, and banks themselves. It’s information about the borrower. In between conspiracies against the public, banks do a very useful thing: they take off the lender’s headache about the borrower’s payback. Banks have to know their borrower well. And typically, they do and keep the net interest spread low. Here’s the rates for banks and credit unions:

banks_cu
Source

Credit unions have been in the industry like forever. They would fit what the FT names “democratizing finance” and have much in common with the ideology behind P2P technologies. Credit unions have higher deposit rates and lower interest in the table because they know more about borrowers. Unions lend only to trusted folks and the number of individual defaults decreases, so you see better rates. Better rates also mean an even lower probability of default, so it’s reinforcing.

The Grameen Bank (and Nobel laureate Yunus) played this idea brilliantly. They radically reduced the market interest rates in poor countries, where high rates coupled with high default rates had been strangling the economy. The Grameen Bank entered very much like a credit union. Borrowers had to provide references from local peers to get access to money. The interest rates have been reduced from 50–100% annually to a single-digit number.

The Grameen-type firms and credit unions are limited in geography and expertise. You could back only your neighbor and only in a very simple business. If he tells you he’ll buy a cow to sell milk, you’re okay. But if a guy on the other coast needs a credit line to build “radar detectors that have both huge military and civilian applications,” you want to know the risks better. That’s why in a complex economy, Grameen is no longer relevant. Each loan application requires more information about the borrower, his credit history, and, most importantly, the purpose of the loan.

The purpose is vital for business loans. Banks learned to dig information about the borrower and to come up with the individual probability of default (you can try to predict yourself). But they’re getting worse in knowing the client’s business. First, businesses are getting more complex. Second, banks reduce their human workforce and local branches, while local branches provided a lot of soft information on borrowers and their performance. Jimmy Stewart’s banking was about observing his little town’s economy and deciding what would be creditworthy there. Without this source, banks pool risks and set higher interest rates, deterring borrowers.

Here comes online P2P lending. When a nuclear physicist from CERN lends money to a nuclear physicist from NASA via P2P system, it tells something about the borrower’s project. The guy from CERN is the right guy to judge. He also throws his own money into this. And that solves both the complexity (you can always find a lender-investor with the right expertise) and neighborhood problems (an expert comes from anywhere). Plus it’s technically free. The CERN physicist has already done the job banks couldn’t do: he found the borrower’s project, evaluated, and approved it. It looks like an investor’s job, and it is. P2P lending platforms like Kiva do mix investing and lending. Users do informal research before lending money.

This info allows banks do P2P loan matching (like some VC and foundations do), buy individually-backed loans, securitize them, and so on. This is a rare example when new technologies are not eating someone else’s pie (like YouTube does to mass media) but create their own. Without this easy expert-loan matching, businesses face higher interest rates, often above their breakeven point, which means no business at all.

Still, P2P platforms themselves seem distracted from this advantage. Most reasoning behind them mentions phantom problems like “predatory interest,” much paperwork, and refused applications in traditional banking. These are not the problems. The financial industry is highly competitive even after the series of post-80s M&A. It evaluates the risks with huge volumes of data, hires good quants, and saves a great deal on scale. In fact, the low market capitalization of major banks indicates that they have no means to “exploit” customers (Google and Amazon do, though in a delicate manner, as here and here). So net interest margin declines:

quarterly_nim
Source

The bank’s paperwork and rejections are just the costs of low interest rates. It makes no sense for startups to “fix” banking in this direction because it’ll increase the rates—sort of getting the industry back into prehistoric times. The information flows between lenders and borrowers is the real thing to focus on.

How Google Works: Unauthorized Edition

Over the years Google earned a reputation as a unique workplace endlessly generating great innovations. This image of an engineering wonderland missed many important aspects of the company’s inners. You could expect Google’s management to be a bit more critical about this. But as Eric Schmidt’s new book How Google Works shows, it’s not the case. The book reestablishes all the major stereotypes, while paying little attention to the things that made up 91% of Google’s success.

Revenue: Auctions

The 91% is the share of revenue Google generates from advertising sold at the famous auctions occurring each time when someone opens a webpage. While an auction is an efficient way of allocating limited resources such as ad space, these ad auctions squeeze advertisers’ pockets in favor of the seller, that is, Google and its affiliates.

In economic terms, auctions eliminate consumer surplus:

Wikipedia
Wikipedia

That’s a “normal” market, when advertisers pay the equilibrium price. Instead, Google takes the entire surplus by selling ads in individual units—each for the maximum price advertisers would pay. The blue supply curve is nearly flat in this case, and the prices go along the red demand curve. Technically, advertisers pay the second highest price—the mechanism chosen by Google for stability (see generalized second-price auction and Vickrey auction)—but in intensive competition the difference between the first and second prices is small.

How does it work in practice? Suppose you are looking for a bicycle and just google it. When your AdBlock is off, you see something like this:

screenshot

Now, you click on “made-in-china.com,” buy whatever it sells, and have your bicycle delivered to you. Made-in-China.com pays about $2.72 to Google for you coming through this link (you can find prices for any search query in the Keyword Planner). This price is determined during the auction, when many bicycle sellers automatically submit their bids and ad texts attached to the bids.

The precise auction algorithm is more complex than just taking the highest bid, because the highest bid may include an ad that you won’t click on and the opportunity will be wasted. Also, since conversion rates are way below 100%, Made-in-China.com has to pay these $2.72 several times before a real buyer comes by. It increases the price of bicycles the website sells. Some insurance-related ads cost north of $50 each—all paid by insurance buyers in the end.

Though this mechanism would make no sense without users attracted by Google’s great search engine, the mechanism takes most out of customers—and transfers it to Google.

Retention: Monopoly

How does Google Search attract users? Well, first, by showing them relevant results. It sounds more trivial now than it was ten years ago. Now users expect Amazon.com to be the first link for almost any consumer good and Wikipedia for topics of general interest. These websites are considered the most relevant not because they’re the best in some objective sense, but again because of particular technologies that made Google so successful.

Larry Page and Sergey Brin’s key contribution to their startup was PageRank algorithm. PageRank is patented, but the underlying algorithms are easy to find in graph theory. The more links point to your website, the higher position your website gets in search results. When I google “PageRank,” I have Wikipedia’s article on the top. When I link to this article here, it becomes more likely that Wikipedia’s article will remain at the top. As a side effect, linking to the first page of Google results creates a serious competitive advantage for top websites. For Wikipedia, it may be a plus as more people concentrate on improving its pages. But strong positions in search results also secure Amazon.com’s monopoly in e-commerce.

Google’s search technologies are supported by its intensive marketing efforts in eliminating its competitors. Google paid Mozilla for keeping Google as its default search all along before Yahoo! outbid it in 2015. Four years ago, Eric Schmidt testified at Senate hearing about unfair competition practices by Google regarding search results allegedly biased in favor of Google services. The European Commission investigates Google’s practices in Europe. In mobile markets, Google demands from hardware manufacturers to install Google Mobile Services on all Android devices—so users go after their status quo bias and stay with Google everywhere.

There’re more fascinating examples of Google protecting its market share. They’re missing in Eric Schmidt’s book, which gives all credit to Google’s engineers and nothing to its lawyers and marketing people.

Development: Privileges

When a typical business creates something, managers carefully look after costs. They negotiate with suppliers, look for quality, build complex supply networks, balance payments, insure their company from price shocks. Google is the fifth largest company in the world, but it’s mostly free of these headaches. Unlike Walmart, ExxonMobil, or Berkshire Hathaway, Google employees make things out of thin air and outsource routines, like training its search engine, to third parties.

It ensures that even entry- and mid-level employees are extremely skillful. Not surprisingly, most Google legends concern its HR policies. These legends split into two categories: that make sense and that don’t.

The culture stuff is what makes no sense. It’s easy to see in non-policies like granting 20% of time to personal projects. This rule might mean something for car assembling jobs; but here it’s software development. An engineer’s personal projects may take 50% of the time if he’s done his daily job—or zero otherwise. It depends on his ability to deliver results expected from his salary. More importantly, his personal projects belong to Google, even if he delivers his daily projects in time but once edited his personal code at the campus.

The book also mentions the 70/20/10 rule: “70 percent of resources dedicated to the core business, 20 percent on emerging, and 10 percent on new.” Even if the authors could prove that the rule is optimal, most other companies are so limited in resources that they have to put 100 percent into the core business.

Neither real things make the Google culture different. Each employee must have a decent workplace, attention, and internal openness, but these things are not sufficient for a great company. We are not in Ancient Greece. Other companies also treat employees well: not much slavery around, meals are fine. Google just tends to be at the extreme.

Laszlo Bock, SVP of People Operations, tried to dissuade the public from thinking that good HR policies require Google’s profit margins. In his opinion, you can get much out of people with openness and good treatment alone. His examples include telling employees about sales figures. It’s sort of an alienated example. First, sales numbers aren’t always as optimistic as Google’s history. Ups and downs, you know. You have to learn how to communicate downs to employees and keep them optimistic.

Soberness appears in less fortunate startups. Evan Williams of Blogger had the moment when the money ran out and employees didn’t appreciate it: “Everybody left, and the next day, I was the only one who came in the office” (from Jessica Livingstone’s Founders at Work, a good, balanced account on early days at startups). It’s just one example that relationships with employees are not as trivial as Bock presents them.

If not culture, then what makes the difference? Quite trivially, the privileged access to job candidates. First, it’s not about money because Google easily outbids everyone else. Its entry-level wages surpass those of Wall Street firms, including major hedge funds, like Bridgewater Associates and Renaissance Technologies. Second, Google has the right of the first interview. That comes with exceptional reputation, low-stress jobs, secure employment, ambitious goals and resources to implement big ideas.

So What?

How Google Works understates the actual achievements of the company. The book is all about famous corporate rules making the business look simplistic. It’s not. The $360 bn business consists of hundreds of important details in each key operation, like hiring, marketing, and sales.

Keeping these things together is an achievement of Eric Schmidt, Laszlo Bock and other executives. However, Schmidt’s book should not mislead other entrepreneurs into thinking that the 20% rule creates great products and reporting sales numbers to employees increases sales better than ad auctions do. Google is a good role model for learning hardcore IT business, but readers will have to wait for some other book to learn from this company.