Most Canadians by now are aware of the ‘erroneous’ report released by Statistics Canada according to which the economy had created only 200 jobs in the month of July. This week they have come out and acknowledged that it was a big mistake.  Foreigners and international workers who are intending to work in Canada under a Canada based work visa, work permit or a related immigration project should be especially comforted by this development.

The reports released by Statistics Canada have data on everything from beer consumption in Alberta to mushroom farming in Ontario. But it is the monthly numbers on employment that seem to have the most crucial impact on the economy.  These numbers are widely used by both the private sector and the government.

The international currency traders ended up pushing the Canadian dollar down by half a cent based on the July report and had to push it back up again after hearing of the error. Similarly, Ottawa has not been able to process any employment insurance claims and says they will do so only after they receive the revised numbers, since the eligibility for these claims depends on the local unemployment rate.

Considering the importance of this data to Canada, one wonders how such a huge error could be made.

According to Philip Cross, former Statistics Canada chief economic analyst, the reason is “summer vacation”. “When mistakes happen, they tend to happen in summer,” says Cross. “There’s less eyeballs and fingers pointing.”

Summer vacations also leads to fewer samples. In summer it is difficult to contact Canadians who are interviewed for the jobs survey as many of them are also away on holiday. “It corrupts your samples. You phone somebody and they don’t answer because they’re on holiday. So what do you do? You just have fewer samples,” says Cross.

However it’s not just the summer holidays that cause statistical imbalances in employment data. July and August are the months which see major changes in the jobs front. During these months, many high school and university students take up short-term summer jobs, and some switch from part-time jobs to full-time employment. Also several schools lay off their staff in the summer and rehire them in September in order to avoid dealing with the bureaucracy of vacation pay. For instance, in 2010, Statistics Canada reported that 68,000 teachers were laid off in July and retired in August. The problem is different for construction workers – their numbers go up in summer and drop in the winter.

Statistics Canada uses a process of ‘seasonal adjustment’ to update its data to compensate for these type of predictable changes. However if one looks at the unadjusted data from previous years, it shows how drastically things can change in the summer.

To understand how Statistics Canada made this big mistake, it is important to see how the agency collects its highly influential jobs data. To collect data for the monthly labour force survey, the surveyors have to go and knock on the door of 56,000 Canadian homes, which are chosen from the latest census data. They then quiz their inhabitants about their working lives.

Usually starting after the 15^th of every month, the interviews take place during the same week every month. The answers are then analyzed and compared to baseline data on population and job market and then transformed into “representative” samples of the entire country. The answers are “weighted” to ensure they accurately represent their age, gender, occupation and where they live.

It is at this point that there is a big room for error. So that 56,000 people can accurately reflect the working lives of more than 34 million Canadians, the surveyors have to compare them against a baseline set of data of the entire Canadian population. The census provides these numbers – local population counts, numbers of workers in different age groups, number of people working in various industries etc. Whereas the labour force survey is done every month, the census is updated only once in five years. As a result Statistics Canada has to estimate the changes happening to the population and the job market in the intervening years.

And when the new census comes out, it takes time for Statistics Canada to update all of its statistical models and past data. It can take several years from the time the most recent census is released to the time they can be input back into the employment data. And the further we get from the previous census, the more guess work has to be done by Statistics Canada. The monthly job surveys of 2014 are using data based off the 2006 census. This implies that Statistics Canada is comparing the answers of 56,000 Canadians in July to a situation in Canada eight years ago and then taking a guess on how things have changed. This process will obviously leave a significant room for error, especially in those parts of Canada where there have been major population changes or industrial upheavals.

According to Statistics Canada, here is how the switch from 2001 to 2006 census affected its employment numbers in the provinces at the time:

In 2010, employment levels were revised downward by 1% or more for New Brunswick (-2.3%); British Columbia (-2.1%); Newfoundland and Labrador (-1.5%) and Prince Edward Island (-1.0%) whereas estimates for Alberta were revised upward, by 1.0%.

Eventually, Statistics Canada does go back to revise all of its past data to adjust it to reflect actual population changes and thus make it more accurate. However this process takes time. The next such update was due in January 2015, when all Canadian jobs numbers were to be revised so that they would be based off of the 2011 census.

There is a chance that it was this update that caused last month’s jobs errors. According to Cross, such errors are few and far between and that Statistics Canada’s data has actually been getting more accurate over the years. He also refers to former chief statistician Munir Sheikh as someone who really improved the efficacy of the agency’s data verification process, including tying the error rate to job performance reviews. Cross says that those processes have continued to improve under the new chief statistician Wayne Smith.

Statistics Canada’s most prized divisions are the ones that produce the three most important numbers in the country: the GDP (economic growth), the Consumer Price Index (inflation) and the Labour Force Survey. According to Cross, these divisions were spared from budget cuts and the CPI is actually on a hiring spree. Even the labour force survey was not affected by the Conservative government’s decision to swap the mandatory long-form census for a voluntary survey, since it’s based off of the short-form census, which is still mandatory.

The error in July numbers however show much our understanding of the job market still depends on ‘guessing’ and is therefore subject to plenty of human error and assumption. This is true of most of the data collected by Statistics Canada. “There isn’t one major data point that over time I haven’t seen a mistake,” Cross says. “It’s a huge place. It processes a lot of data. It’s full of human beings and human beings make mistakes.”

Attorney Colin Singer Commentary:

International workers who may be interested to work in Canada under a work visa, work permit or a related immigration program are invited to contact us and receive a free consultation of their qualifications.

Source: Macleans