Has AI Improved Productivity?

"Does Machine Translation Affect International Trade? Evidence from a Large Digital Platform." Erik Brynjolfsson, Xiang Hui, and Meng Liu. National Bureau of Economic Research Working Paper No. 24917, August 2018.

Economists interested in artificial intelligence (AI) have been puzzled by what some call a "productivity paradox." The paradox is the gap between the optimistic expectations about the economic effects of AI and the effects that appear in the data. On one hand, predictive technologies like image and speech recognition have experienced breakthroughs in recent years. The multitude of potential uses for such technologies are why AI has been called a "general purpose technology" like electricity and the steam engine, whose diverse and far-reaching applications changed the ways we work and live. On the other hand, contributions from AI are nonexistent in measures of aggregate productivity.

The productivity paradox isn't new. A similar phenomenon accompanied the advances of information technology in the 1970s and '80s when the Nobel Prize-winning economist Robert Solow famously remarked, "You can see the computer age everywhere but in the productivity statistics." As productivity in IT-intensive sectors eventually picked up around the turn of the millennium, researchers proposed that the paradox was simply an issue of timing. It seemed that IT implementation required the development of complementary innovations and the reshaping of production processes before its effects could be fully felt. The AI productivity story may prove to be much the same.

If slow implementation of AI is responsible for its absence from aggregate productivity numbers, then those sectors that can most readily adopt AI should be the first to experience its economic effects. A recent National Bureau of Economic Research paper by Erik Brynjolfsson of MIT and Xiang Hui and Meng Liu of Washington University in St. Louis considers whether this may be happening.

Brynjolfsson, Hui, and Liu looked at the effect of machine translation on international trade. Machine translation is an AI technology that has become increasingly capable of producing near-human-quality translations. The authors focused on the 2014 rollout of eBay's in-house translation tool, eBay Machine Translation (eMT). The eMT's implementation in Russia, Latin America, and the European Union represented only a moderate quality improvement over the platform's prior translation tool; even so, the authors found its introduction was associated with a sizable 17.5 to 20.9 percent increase in trade flows between Latin American consumers and U.S. sellers over eBay.

Machine translation makes sense for early AI adoption because it can be easily embedded into a digital platform's existing production process. The eMT translation is automatic and requires no change in behavior from buyers or sellers; in fact, they need not even be aware of the tool's existence to use it.

Brynjolfsson, Hui, and Liu quantified the effects of the eMT rollout using a natural experiment research design. Much like a scientific experiment performed in a laboratory setting, a natural experiment identifies the effect of a treatment — in this case, access to the eMT — through a comparison with a control group. The authors identify the effect of the eMT rollout using U.S. exports over eBay as the measurable outcome. If the eMT reduces barriers to international trade as the authors predict, then consumers in the eMT treatment group countries should buy more from U.S. exporters relative to the control group.

Determining who exactly this control group should include is imperative to producing meaningful results with a natural experiment. A natural experiment employs a "differences in differences" methodology to isolate the effects of a treatment while controlling for confounding effects. In this paper, the first difference is a comparison of an individual country's consumption of U.S. exports over eBay before and after the eMT rollout, regardless of whether the country is in the treatment group. This controls for differences in the magnitude of trade flows by country that had nothing to do with the eMT. The second difference is a comparison of the results of the first stage. It measures the change in trade flows among eMT countries versus the change in flows among non-eMT countries. This stage controls for changes occurring over time that are the same for all countries, like a global expansion or recession that affects trade flows in both eMT and non-eMT groups.

The authors interpreted the increase in trade flows due to eMT as an indication of the obstacle to international trade imposed by language barriers. The effect on trade flows was even larger for buyers and products with higher search costs, meaning inexperienced eBay users and consumers in the market for nonstandardized products responded most to the eMT implementation.

Similar effects in other sectors may become apparent as the development of AI technologies continues to make leaps. Of equal importance is the adjustment of firms to technological change. Optimism surrounding the productive capabilities of AI may prove true after all: Radical changes to production processes just take time.