Benjamin Disraeli once said famously, “There are three kinds of lies: lies, damned lies, and statistics.”
I couldn’t help thinking of Disraeli’s comment when I saw the report issued by the city’s Department of Health last week claiming that Mayor Bloomberg’s controversial new anti-smoking law is having no effect on employment in Gotham’s restaurant and bar industry, despite howls from owners that their business is slowing down.
I wouldn’t quite call the report a “damned lie,” but it is a case of using statistics in the service of blatant political propaganda. In its report, the health department employs unreliable preliminary data—heavily massaged and likely to be revised (several times, probably)—to jump to an extremely questionable conclusion about an exceedingly complex subject.
In its release, the department says that employment at city bars and restaurants increased by 1,500 “seasonably adjusted” jobs from March to June. Since the new anti-smoking law went into effect on March 30, the report declares “the law has not had an overall negative impact on business.”
There are so many things wrong with this “report,” it’s difficult to know where to begin, but let’s start with the actual number of restaurant jobs in New York from March through June. First, we need to know that the government’s monthly employment data for New York City are not based on an actual count of jobs, which would be too burdensome to do every month, but on a projection made from a limited survey of local employers.
Because the government uses a small sample for these monthly reports, the data can be fairly inaccurate for individual industries, especially when it comes to picking up big employment shifts in one direction or another in turbulent economic times like this. The Bureau of Labor Statistics pointedly calls the monthly city job numbers “preliminary,” and the government frequently revises each month as more exact data come in.
What good are these monthly reports, then? Well, at the most they might indicate a direction the economy is heading, or something worth watching further as more exact data come in. But for long-term trends or studies, economists are much more likely to rely on job stats averaged over the course of an entire year for New York.
Not the health department. It used the monthly data to conclude something far more complex: the influence of a new law on an entire industry. For that you need not only accurate employment statistics but the ability to factor out other significant economic trends operating at the same time.
Moreover, the health department report says that the job numbers are “seasonally adjusted,” which adds yet another level of fiddling to what are already massaged data. Economists make seasonal adjustments to account for the fact that employers hire more workers during some times of the years, and fire them at others, in regular patterns. The seasonal adjustments are supposed to allow us to compare monthly job numbers in a way that smoothes out these temporary employment surges. But seasonal adjustments are themselves more art than science, and over the years I’ve seen so many different seasonal adjustment factors used for New York City that in the same month some economists can report jobs are declining while others will say jobs are increasing.
The city’s restaurant industry is especially subject to seasonal hiring patterns from March to June, when restaurants ramp up hiring for the tourist season. In every year since 1994, the city’s restaurants have hired somewhere between 7,000 and 10,000 new workers, in good times and bad.
To get a somewhat more accurate picture of what may be happening in the business right now, it makes more sense to look at the jobs data that are not seasonally adjusted from this year and compare each month to the same month last year, which reduces the impact of the seasonal hiring. Those figures show that in every month from March to June, the city had fewer restaurant jobs this year than it did in the same month last year. By contrast, in both 2000 and 2001, two years when the city’s restaurant industry grew, in every month from March to June restaurants recorded substantially more employment than for the same month the previous year.
If I were inclined to draw even the most tentative conclusions from the preliminary data, it would be that the industry is still struggling to get out of its recession. That’s hardly something for the head of the health department—or more egregiously, for the head of the city’s economic development department—to crow about. Nonetheless, both welcomed the jobs data in the health department study.
It will take a lot more time before we know what impact, if any, the anti-smoking laws have had in New York. But whatever the influence, one thing we know right now is that there is nothing in the extremely preliminary data we are seeing to suggest much good is going on in the city’s restaurant business. But don’t expect a press release on that from the administration any time soon.