In the transcript of the interview, John Williams explains that his alternative measure of inflation is based on adjustments derived from the government’s own estimates of the effects of adjustments such as geometric weighting for substitutions effects, hedonic adjustments, and owner equivalent rent as a proxy for housing costs. Thus, Williams calculates his shadow inflation index by simply adding a constant to the government-published figures:
John Williams: [....] But if using the CPI the way it was before these games, you look at the changes made since 1990—and I look at it pretty much on an additive basis, based on the government's estimates of what these different changes have added or subtracted to the reported CPI inflation where right now the CPI year-over-year is showing one and a half percent inflation. If you went back to the way it was calculated in 1990, you would be up to something close to five percent; and if you went back to the way it was calculated in 1980, you would be looking at something up in the area of eight or nine percent.
Chris Martenson: Eight or nine percent. That is a full six and half to seven and a half percent higher than what we are being told it is right now.
John Williams: Right. The difference between my numbers and the government’s numbers, basically, is what is being lost in measuring the cost-of-living and maintaining a constant standard of living.
The problem is, regardless of what BLS statisticians might have said in the past, in 2008 the BLS published a rebuttal claiming that the effects are much smaller! To his credit, Williams published a link to the BLS article at his website. Here's what the BLS economists had to say in response to Williams:
http://www.bls.gov/opub/mlr/2008/08/art1full.pdf
One widely cited alternative index is based on an estimate that changes to the CPI since 1983 have lowered its growth rate by at least 7 percentage points per year. The use of the geometric mean alone is stated to have lowered the CPI growth rate by 3 percentage points, and other BLS changes, such as the use of hedonic models and OER, supposedly have lowered the growth rate by an additional 4 percentage points.
Each of these estimates of the impact of BLS changes is inconsistent with the empirical evidence. As noted earlier, the BLS has computed indexes showing that the use of the geometric mean formula has reduced the growth rate of the geometric mean of the CPI by only -0.28 percentage point per year, not 3 percentage points. Also discussed earlier, BLS analyses have shown that if the implementation of hedonic adjustment models since 1999 has had any net downward effect, it is very small. Hedonic adjustment models implemented subsequent to 1983, but prior to 1999, have almost certainly had an upward effect. Among the methodological changes examined in this article, that leaves only the shift to rental equivalence, and it is entirely implausible that its impact could be as large as 4 percentage points per year. Ear- lier, it was shown that from 1983 to 2007 the CPI for OER rose faster than an alternative index that, like the pre–1983 BLS homeownership index, is based on both house prices and interest rates. Another piece of evidence comes from an analysis published in the Monthly Labor Review in 1999 in which BLS economists Kenneth J. Stewart and Stephen Reed compared the historical published CPI-U with an index created in accordance with current BLS methodologies. For the years 1978–82, a period that witnessed very rapid increases in both house prices and interest rates, Stewart and Reed estimated that the use of rental equivalence would have had an average annual impact on the CPI-U of only –0.86 percentage point. Moreover, with house prices now declining in many parts of the country, one would expect that if the BLS were using the pre-1983 homeownership method, it would yield a lower, not higher, current measure of shelter inflation.
So the BLS economists are arguing that the effects of the methodologies in question totals to about 1.14%, not 7%. Now, I'm not saying I know exactly what the truth is here, but Williams can't point to the government as the source of his adjustments. Williams more or less admits as much -- in his response to the BLS article, he says:
http://www.shadowstats.com/article/special-comment
These issues will be fully explored and discussed in the upcoming academic study, including a careful analysis of inflation reflected in raw price data for various periods, versus the various CPI measures.
This promised study hasn't yet materialized, but another interesting data point is a survey conducted by George G. Paulos of Freebuck.com. His study looked at the change of prices of 48 consumer products over the interval from 1968 to 2004. His conclusion was that the BLS understated the rate of inflation by about 1% annually, which is remarkably close to the BLS' own figure of a 1.14% differential due to their statistical adjustments. However, Paulos makes the point that even a small annual differential can accumulate in a big way over time.
http://www.gold-eagle.com/editorials_04/paulos090904.html
The government CPI (Consumer Price Index) numbers may understate the true annual inflation rate by over 1% over the long term. Because of compounding, a relatively small difference in the annual inflation rate can magnify prices greatly over long time periods. The CPI is a critical number that is used to calibrate a plethora of payment systems including Social Security, labor contracts, inflation-adjusted bonds, and general interest rates. Distortions in this number can cause major long-term dislocations and economic inefficiencies. The same methodology used by the CPI is also used to calculate real inflation-adjusted GDP which is commonly used to make important policy decisions. Understating the CPI results in overstating GDP and can lead to poor decision making throughout the economy.
To me, the most alarming aspect is that the government may be inventing new ways to distort the CPI index, and that the divergence between the index and reality may be growing as we speak. Chris' discovery of the substantial under-weighting of health care is an example of something that Williams missed. Another point that Chris has made, is that commodity price inflation seems to be leading the way at this time. Raw commodities may make up only a small percentage of the costs of many consumer items at the grocery store or at the mall, but the producers of products like that may be seeing their margins squeezed. Thus, the commodity price increases may be a leading indicator of much worse problems to come. Also, people on the lower end of the global socio-economic scale spend much higher portions of their incomes on food and fuel in commodity form, so they see the worst effects.