Software as an Institution

The rules of the game, known to economists as institutions and to managers as corporate culture, usually entail inoperable ideas. That is, any country or business has some rules, but these rules coincide neither with optimal rules nor with leadership vision. Maybe with an exception of the top decile of performers or something like this.

This inoperability isn’t surprising since the rules have obscure formulations. Douglass North and his devotees did best at narrowing what “good institutions” are, but with North’s bird-eye view, you also need an ant-eye view on how changes happen.

An insider perspective had been there all the time, of course. Organizational psychology and operations management organized many informalities happening in firms. In general, we do know something about what managers should and shouldn’t do. Still, many findings aren’t robust as we’d like them to be. There’s also a communication problem between researchers and practitioners, meaning neither of the two cares what the other is doing.

These three problems—formulation, coverage, and communication of effective rules—have an unexpected solution in software. How comes? Software defines the rules.

Perhaps Excel doesn’t create such an impression, but social networks illustrate this case best. After the 90s, software engineers and designers became more involved in the social aspects of their products. Twitter made public communications shorter and arguably more efficient. In contrast to anonymous communities of the early 2000s, Facebook insisted on real identities and secure environment. Instagram and Pinterest focused users on sharing images. All major social networks introduced upvotes and shares for content ranking.

Governance in online communities can explain success of StackExchange and Quora in the Q&A space, where Google and Amazon failed. Like Wikipedia, these services combined successful incentive mechanisms with community-led monitoring. This monitoring helped dealing with low-quality content that would dominate if these services simply grew the user base, as previous contenders tried.

Wikipedia has 120,000 active editors, which is about twice as many employees as Google has (or alternatively, twelve Facebooks). And the users under the jurisdiction of major social networks:

Source

So software defines the rules that several billion people follow daily. But unlike soft institutions, the rules engraved in code are very precise. Much more so than institutional ratings for countries or corporate culture leaflets for employees. Code-based rules also imply enforcement (“fill in all fields marked with ‘*'”). Less another big issue.

Software captures the data related to the impact of rules on performance. For example, Khan Academy extensively uses performance tracking to design the exercises that students are more likely to complete — something that schools with all the experienced teachers do mostly through compulsion.

Finally, communication between researchers and practitioners becomes less relevant because critical decisions get made at the R&D stage. Researchers don’t have to annoy managers in trenches because software already contains the best practices. Like at Amazon.com that employed algorithms to grant its employees access privileges based on the past performance.

These advantages make effective reproducible institutions available to communities and businesses. That is, no more obscure books, reports, and blog posts about best practices and good institutions. Just a product that does specific things, backed by robust research.

What would that be? SaaI: software as an institution?

Don’t Listen to Jack Welch (Only His Best Part)

Jack Welch advises executives to leave their rooms and find out more about organizations they manage.

I’m afraid, this is what executives will do. Why? Welch makes two points. First, he shows the problem, which is real for sure. Knowing your organization is important. Second, he suggests to solve it by visiting “stores, trading floors, regional offices, factories.” It’s also a good point, but not the best solution.

By taking Welch’s advice literally, executives will find no more than a mess of emotions, stories, suggestions, and demands. It’s like reading a morning newspaper: you really need a lot of prejudices to make sense out of this flow of information, when this flow doesn’t have any sense. It’s best at confirming existing prejudices. If you really want to know something about the world, you should do a comprehensive study on topic.

How does it look in management? If you want knowledge, organize it. Build an IT system that let your people talk freely (even if anonymously), send requests to supervisors, get feedbacks, and discuss ideas in a single place. Not face-to-face meetings of the king and His Majesty’s subjects (it always looks this way). It must be a distant platform. A person must know it’s for real and feel no pressure.

Computers are stupid but extraordinarily good at handling whatever comes out of this. Can a human delegate 8.5 million problems to 3.7 million solvers in milliseconds? StackOverflow does this routinely and arguably saved more working hours than YouTube wasted. It’s a matter of minutes to find popular problems, topics, and experts. It’s easy to find where your help is needed. This system shows what matters.

You can spend time traveling around “stores, trading floors, regional offices, factories” to declare, like Jonny Cash, “I’ve been everywhere.” Or you can systematically improve the system that delivers real information from real people right to your armchair. An IT system is better at everything that travels can do: moods, relevant problems, upcoming disasters, and best ideas. Exciting travels, as Welch noted, show that you’re not alone. But they are not for decision making.

Computer-driven operations at Amazon and Walmart have beaten flesh-and-blood shops around the corner. These systems know what customers want, unlike shopkeepers who talk to their customers for hours each day. There must be some sense of modesty regarding own abilities to admit this, but it would be one level up in business management. The creators of Amazon and Walmart could improve because they recognized their limitations and let machines do their work.

This transformation is slow in management because of the email reputation IT systems have. They’re something delivering tons of letters you have no time to read. It’s a failure of design. Emails came from the 70s and haven’t changed since then. ERM and other “management” systems often copy emails in asking too much irrelevant information. They lack human input and the sense of importance. But that’s how public web services looked in the 1990s. Since then they’ve changed tremendously; and so will B2B systems. Don’t miss this moment traveling.

Answering questions already answered on Stack Exchange

In one of the previous posts, the relative distribution of upvotes on Stack Exchange showed that adding one more answer to a question isn’t much demanded by readers because readers give at least half of upvotes to the first-placed answer alone:

(y-axis: the answer’s mean fraction of upvotes in a question; x-axis is the position of the answer)

But the absolute number of upvotes tells why answers still appear:

(y-axis: mean of upvotes; other notation is the same)

Questions with few answers happen to be unpopular in general (that’s why they have fewer answer in the first place). You can barely notice upvotes for questions with a single answer. But they grow as the graph says. The last frame describes the case of 16 answers, but it’s better to be careful beyond that because the original sample has too few questions with many answers.

The bottom line is that answering a question with many answers may be more useful to users than answering questions with no answers at all. That’s at least valid under the assumption of unconditional expectations. Controlling for time is the next most useful thing to do to understand how demand and supply operate in Q&A markets and what can be done to make them more efficient.

Stack Exchange and reward for being on the top

As mentioned in the previous posts, Stack Exchange has a very interpretable structure. It’s a market in which demand for answering a question meets supply, and supply is paid with upvotes. Such a rude interpretation is necessary for learning how knowledge exchange works.

I once looked into a demand side of Stack Exchange, but now a few points on the supply side. In general, we are interested in efficient allocation of resources. Given the fact that sometimes one answer is enough (especially for software development questions), many answers may be a waste.

And that’s the distribution of answers per question:

Well, it’s a peak at 2 with a long tail. The details:

number of answers Freq. Percent Cum.
1 2,123 18.35 18.35
2 2,601 22.48 40.83
3 2,138 18.48 59.3
4 1,458 12.6 71.9
5 967 8.36 80.26
6 674 5.82 86.09
7 461 3.98 90.07
8 325 2.81 92.88
9 190 1.64 94.52
10 135 1.17 95.69

About 80 percent of questions end with five answers or less.

The Reward for Being on the Top

But what’s the reward for having your answer on the top of the others? These are the means of fractions of total upvotes by the position a given answer occupies:

It says that the answer on the top have an stable advantage over all answers to a given question. You can see that after the fifth answer, adding more answers does not decrease total upvotes given to the existing answers. And the first answer gets no less that half of all upvotes.

That’s a huge bonus, since multiple other answers have to split the remaining half of upvotes. That may be discouraging for participants, as competition is high and the winner takes all.

Sample summary statistics

Variable Obs Mean Std. Dev. Min Max
upvotes 45463 5.390141 24.11624 0 1553
downvotes 45463 0.1868992 0.9164532 0 82
net (up – down) 45463 5.203242 23.94713 -19 1552
position 45463 4.449794 6.709625 1 114
total_answers 45463 7.899589 10.94102 1 114
relative position 45463 0.6272573 0.2922457 0.0087719 1
total_uv_b~q 45463 67.75934 221.0013 0 2488
frac_uv 44188 0.2440255 0.3074606 0 1

Top 1% on Stack Exchange, or inequality in Q&A markets

Yesterday’s review of voting activity showed how Stack Exchange users evaluate each others’ contributions. Contrary to some views, users don’t go berserk despite anonymity and in general balance their judgment.

Upvotes are a currency in a moneyless economy, like one of Stack Exchange. It doesn’t mean people do things for upvotes. In this survey, reciprocity comes first as motivation, while upvotes lag behind. But, like money, upvotes often measure one’s contribution to the community. Though it’s in unlimited supply, upvoting has its own costs (yes, costs of clicking on buttons), and can measure something. For instance, inequality.

Gini

Here’s the distribution of upvotes that a particular user got over time:

The long tail is wagging behind, but the inequality is very high.

The Gini coefficient for Stack Exchange is 0.85 for upvotes. That’s higher than in South Africa (0.63), which has the highest inequality among countries.

Top 1% and 0.1%

High inequality leads to the questions what does top 1% of users own. Well, they own 42.8% of all upvotes. Actually, it’s just about the same percentage that the 1% of richest Americans now control in the national economy.
Top 0.1% users own 15.4% of all upvotes.
Also, Pareto’s law nearly holds. Top 20% of users own 87.7% of upvotes.

How to get rich in this economy

In brief: by being rich. This is why:
(users with less than 10 votes in total are excluded as bypassers)
Until you earn about ten upvotes, the only thing that grows is your downvotes. That’s a mentoring stage when novices are downvoted as someone who do not read rules. But after you accumulated this minimum, upvotes grow relatively to downvotes. And they grow rapidly.
This result is robust to including bypassers back in the sample (see how the line changes the slope at ln(downvotes) ≈ 2):
So, in general, you have more when you already have enough.

Data Appendix

Analysis is done with a 1% sample of users. Data is available at data.stackexchange.com, while replication files will soon be posted online.

Unrestricted evil

Online services before Facebook were mostly anonymous. At least, no one required real names and SMS confirmations. You sign up and write anything. I mean anything.

There were scary fairy tales about writing anything without putting your real name on it. Various regulations imposed on the Internet, especially in non-democracies, rely on these tales. But is the opposite—real names and full disclosure—really necessary for a good-standing community?

Let’s check. The anonymous culture is still alive on some resources. Today is not about 4chan, but about Stack Exchange, a major Q&A website. Stack Exchange has an open data tool for querying data. The tool is quite useful for testing various hypotheses about human communities. It would be a service to humanity if other web services offered similar openness, but so far we have few.

Stack Exchange (SE) doesn’t ask names and so on. Though real names are common and some employers ask candidates for links to their SE profiles, the service is basically anonymous. We expect dirty things to happen.

One dirty thing is excessive downvotes for questions and answers others post. Kind of vandalism. And here’s the graph:

The graph shows net votes (upvotes – downvotes) for each user with 10+ upvotes or downvotes from a 1% sample (about 31K users). That small tick on the left is users who mostly downvote.

A slightly different perspective:

(The axes were log-linearized for easy reading.)

User behavior happens to be extremely balanced. Few users tend to upvote or downvote extremes. Most of them try to be honest.

So, showing your name is not necessary for good behavior. Online communities can manage themselves without references to the official world, regulations, and witch-hunting. It only matters what environment people want to be in, and then they’ll be able to recreate it online.