The Economist has a long story on the relative value of Big Economic Models -- the kind of general equilibrium monsters that are used to calculate how much the world benefits from a completed Doha round,or how much the global economy suffers from high oil prices.

The story does a good job of highlighting the sensitivity of these models to first assumptions -- while also pointing out their signal virtue:

[Leon] Walras was adamant that one could not explain anything in an economy until one had explained everything. Each market—for goods, labour and capital—was connected to every other, however remotely. This interdependence is apparent whenever faster car sales in Texas result in an increase in grocery shopping in Detroit, the home of America's “big three” carmakers. Or when steep prices for oil lead, curiously enough, to lower American interest rates, because the money the Saudis and the Russians make from crude is spent on American Treasury bonds. This fundamental insight moved one economist to quote the poetry of Francis Thompson: “Thou canst not stir a flower/Without troubling of a star.”

Such thinking now comes naturally to economists. But it still escapes many politicians, who blindly uproot flowers, ignorant of the celestial commotion that may ensue. They slap tariffs on steel imports, for example, to save jobs in Pittsburgh, only to find this costs more jobs in the domestic industries that use the metal. Or they help to keep zombie companies alive—rolling over their loans, and preserving their employees on the payroll—only to discover they have starved new firms of manpower and credit. Big models, which span all the markets in an economy, can make policymakers think twice about the knock-on effects of their decisions.

These models were, for example, a weapon of choice in the battles over the 1994 North American Free-Trade Agreement (NAFTA). The pact's opponents had the best lines in the debate—Ross Perot, a presidential candidate in 1992, told Americans to listen out for the “giant sucking sound” as their jobs disappeared over the border. But the deal's supporters had the best numbers. More often than not, those with numbers prevail over those without. As Jean-Philippe Cotis, chief economist of the OECD, has put it, “orders of magnitude are useful tools of persuasion.”

But how plausible were the numbers? Twelve years on, economists have shown little inclination to go back and check. One exception is Timothy Kehoe, an economist at the University of Minnesota. In a paper published last year, he argued that the models “drastically underestimated” NAFTA's impact on trade flows (if not on jobs). The modellers assumed the trade pact would allow people to buy more of the goods for which they had already shown some appetite. In fact, the agreement set off an explosion in the exports of many products Mexico had scarcely traded before. Cars, for example, amounted to less than 1% of Mexico's exports to Canada before the agreement. By 1999, however, they accounted for more than 15%. The only comfort economists can draw from their efforts, Mr Kehoe writes, is that their predictions fared better than Mr Perot's. A low bar indeed.

Dubious computations also helped to usher the Uruguay round of global trade talks to a belated conclusion in 1994. Peter Sutherland, head of the General Agreement on Tariffs and Trade, the ancestor of the World Trade Organisation (WTO), urged negotiators to close the deal lest they miss out on gains as great as $500 billion a year for the world economy. This figure came, of course, from a big model.

Even staunch free-traders, such as Arvind Panagariya, an economist now at Columbia University, thought these claims “extravagant” and “overblown”. They escaped scrutiny, he argued in 1999, because they emanated from “gigantic” models, which were opaque even to other economists. Why then did these models thrive? Supply and demand. “Given the appetite of the press and politicians for numerical estimates and the publicity they readily offer researchers, these models are here to stay,” Mr Panagariya concluded.