Experiments At Work

Outside Kay-Yut Chen’s economics laboratory at Hewlett-Packard in Palo Alto, Calif., the November air is unseasonably warm, even for California, and splashes of yellow and green leaves shimmer against the clear blue sky. But inside, in a windowless, fluorescent-lit room, the 12 visitors participating in today’s experiment sit patiently at their randomly assigned computers. When I point to the incongruity, Chen doesn’t miss a beat: “That shows one thing–the assumption that people like money is correct.”

This simple assumption goes to work for Chen whenever the HP principal scientist runs an experiment. The participants–mainly “starving students” from nearby Stanford University–will earn $25 to $75 or more, depending on how well they play today’s game. The experiment simulates interactions between sales agents and sales managers. At each period of play, the computer tells sales agents the current market conditions, and based on that information the agents must decide how much effort they will put into making the sale. Although effort incurs costs that take away from the total payout, effort also increases the likelihood of sales success. The agents’ total payout is simply the sum of the fixed payment and variable payment for successful sales minus the cost of effort. Managers, in turn, determine the fixed and variable payments they will offer–knowing their own payout will be total sales minus payouts to agents.

Chen explains these rules but says nothing about strategy. He doesn’t need to: through a little computer-mediated back-and-forth with their managers, most agents wise up to the fact that this game rewards sales but offers no incentive to tell managers anything. Similar compensation schemes in the real world explain why salespeople tend to sandbag their forecasts, making it hard for their companies to plan ahead.

Chen thinks he has solved the sandbagging problem: have each salesperson choose a personal balance of fixed and variable compensation. For example, the salesperson can choose a high commission percentage with no fixed salary or, at the other extreme, a modest fixed salary and no commission–or some combination in between. Each choice implicitly reveals how much the salesperson plans to sell, much as an insurance subscriber’s choice of deductible and premium reveals how sick she is. Based on a truth-telling mechanism from game theory, this design works on paper. But as an experimental economist, Chen will keep testing it empirically, comparing the emerging design with other available models, such as the one he is testing today.

Chen has successfully used that approach to help HP managers design good contracts with retailers and resellers, and he is starting to tackle other thorny problems for his employer: figuring out how to protect HP’s bottom line against international currency fluctuations and discovering ways for brick-and-mortar retailers and HP’s online store to coexist happily.

Kay-Yut Chen’s experiments showed that a proposed incentive program would backfire. So HP, his employer, scrapped the idea.

The science of experimental economics has boomed with the rise of personal computers–especially after one of its founding fathers, Vernon L. Smith of George Mason University, won the Nobel Prize in economics in 2002 (sharing the honor with Princeton University psychologist Daniel Kahneman). But the field has not caught on in what seems its most logical application: making business decisions within a company. When Chen started his lab in 1994, it became the first economics laboratory inside a corporation, and to this day no other firm maintains a lab like his. “He’s basically ‘Mr. Experimental Economics in Business,’” says Teck H. Ho, chair of the marketing group at the Haas School of Business at the University of California, Berkeley.

Some blame the absence of such labs on the shortsightedness of corporate America. Charles R. Plott, the experimental economist at the California Institute of Technology who pioneered the field, puts it this way: “A lot of us in academia have done business applications, but for someone to take a new science inside a big business that is just rife with competition for funds–and be able to create a science facility that can handle the basic research and the pressures of day to day–that’s a monumental task Chen has succeeded in.”

Were it not for Plott’s influence, Chen would not have become an economist at all. As a Caltech undergraduate, the Hong Kong native majored in physics and participated in Plott’s experiments only for the lure of the cash and the challenge of besting other players. But soon the science became the greater draw, and Chen briefly waxes metaphysical describing it: “Doing experiments on people is like playing a game with a higher being. Many people think they have free will, meaning you cannot predict what they will do, but the counterintuition is that people in a lot of cases are predictable.” Chen also recognized more opportunity for discoveries in this younger field than in high-energy physics–and Plott easily talked him into switching over.

As he was finishing his Ph.D. at Caltech, HP Labs was trying to invigorate its work in operations research through collaborations with experimental economists. Thanks to professors singing his praises, Chen, now 39, got tapped in 1994 to start the economics lab at HP, where he has been ever since.

HP managers say the benefits of lab-testing business ideas are immeasurable. “That way you’re not going to upset the business or your partners or the end users with a bad program,” explains Jukka Koskela, an HP sales operations manager who asked Chen to test the idea of rewarding retailers to be number one in sales. When Chen’s experiments showed this incentive would push too many retailers to either of two undesirable extremes–giving up at the outset or neglecting other HP goals–the company scrapped the idea. “You could waste millions of dollars implementing a program that isn’t good,” Koskela says. Chen’s tests showed that contracts that reward retailers according to how much business they bring in for HP produce far better results.

In general, a good contract aligns the interests of the principal (such as HP) and the agent (such as a reseller)–but in a business full of interdependent variables, that is much easier said than done. Think of the tangled web of business relationships: office equipment distributor IKON is an HP partner, but it also sells similar products from other firms and competes with other HP partners. In the lab, such mixed motives tempt some players to “game the system,” such as the time a student playing a reseller made vastly more than the average payout by hoarding products to lock out other resellers. This incident was an anomaly–and may be even less likely in the real world, where long-term relationships and other variables might prod people to honor a contract’s win-win spirit. Still, Chen wants to assure good outcomes even if people are at their worst, so he tweaked the contract to prevent resellers from ordering more inventory than they can really sell.

Such tinkering is anathema to pure theorists, who see experiments as a rickety crutch for those who cannot build good formal models. Chen takes a more pragmatic view, however, and he good-naturedly brushes off criticisms of applied experimental economics. Sure, student subjects aren’t exactly like real businesspeople, but if they’re bright they’re close enough–plus they’re affordable. And the experiments are a complement to theoretical reasoning, not its substitute. As Chen sums it up, “I would say this is two parts science, one part art.”