Tracking Inflation: Consumer Price Index vs. MIT Billion Prices Project

My Money Blog has partnered with CardRatings and may receive a commission from card issuers. Some or all of the card offers that appear on this site are from advertisers and may impact how and where card products appear on the site. MyMoneyBlog.com does not include all card companies or all available card offers. All opinions expressed are the author’s alone.

AFter calculating the new savings bond rate, I noticed that from March 2012 to March 2013 the inflation rate per the Consumer Price Index was only 1.5% over the past year. Whenever you see the government announce a relatively low inflation numbers, there will always be people shouting “the government manipulates the inflation data!”. I looked into this previously with my post Does The Government Underestimate Inflation Through The CPI? Short answer: Yes they do, but maybe not in the way you think.

Usually, this is followed by the anecdotal argument “Does gas ever go down? Does your rent ever go down?”. It certainly feels like prices are rising quicker than that. My water bill just got hiked another 10%. The thing is, we always notice the increases, but tend not to notice when prices drop. When something is cheaper, we just chalk it up to being great bargain hunters. Truth is, gas prices did go down for a while.

Another way to keep an eye on inflation is with MIT’s Billion Prices Project (previous post) which tracks prices in real-time by grabbing them from websites. By checking on 50,000+ different prices daily covering everything from prescription drugs to clothing to real estate, this alternative inflation measurement has the potential to keep governments “honest” with their numbers.

Index Values: CPI vs. BPP

Annual Inflation (% change, last 365 days)

Charts taken on 4/17/2013. Over the past five years, the two indexes have actually tracked relatively closely. Just recently, the two are diverging a bit; BPP is showing a little over 2% inflation as opposed to 1.5% from CPI. It appears that the daily BPP updates run a little ahead of the monthly CPI updates, so perhaps an uptick in CPI is coming.

My Money Blog has partnered with CardRatings and may receive a commission from card issuers. Some or all of the card offers that appear on this site are from advertisers and may impact how and where card products appear on the site. MyMoneyBlog.com does not include all card companies or all available card offers. All opinions expressed are the author’s alone, and has not been provided nor approved by any of the companies mentioned.

MyMoneyBlog.com is also a member of the Amazon Associate Program, and if you click through to Amazon and make a purchase, I may earn a small commission. Thank you for your support.


User Generated Content Disclosure: Comments and/or responses are not provided or commissioned by any advertiser. Comments and/or responses have not been reviewed, approved or otherwise endorsed by any advertiser. It is not any advertiser's responsibility to ensure all posts and/or questions are answered.

Comments

  1. According to these Shadowstats charts, if you use the methodology the government used to measure inflation in 1990, current inflation numbers would be around 5%. If using the government’s own pre-1980 methodology, it would be around 9%.

    These are scary numbers. People believe inflation is some unavoidable natural phenomenon. It isn’t. It is a malicious hidden tax perpetrated by central banks and their fiat money for the benefit of those closest to the supply of newly printed money: gov’t & big banks.

    One of the few poignant quotes from John Maynard Keynes: “By a continuing process of inflation, governments can confiscate, secretly and unobserved, an important part of the wealth of their citizens. By this method, they not only confiscate, but they confiscate arbitrarily; and, while the process impoverishes many, it actually enriches some.”

  2. My total home payment (including escrow) actually went down 11% last year as a result of refinancing, and that is my household’s single largest expenditure.

    • no closing costs, andy, to reflect on the costs of that money (fixed or variable rates should also be factored in, just as teaser rates during the housing boom were no more an indication of low future inflation), which also prices in expected inflation? also, since variable mortgage rates are largely governed by the benchmark that is set, it’s hardly a good gauge. and since variable mortgage rates did fluctuate throughout the year it might be better to look at those as well.

  3. Thanks for the link! It really is helpful for a fact-check on the CPI.

    Also, I think you’re probably right-on that MIT may be a leading indicator of the CPI and that the government’s numbers are trailing indicators.

  4. The charts for CPI and BPP look remarkably close as far as I’m concerned. The differences there seem pretty marginal and likely within what I’d think the margin of error is.

  5. Brady, the claims by Shadowstats are honestly quite ridiculous. They claim that inflation is actually 7% higher than CPI. So when CPI is 2% the Shadowstats people say it is really 9%. That just doesn’t hold up to any examination. Has the price of EVERYTHING we buy doubled in the past 10 years? Do you spend 2x today compared to 2003? No, clearly not.

  6. @Andy – Nice! You may already know this but just wanted to throw out that rent is included in CPI and not mortgage payments. I’m not actually sure how BPP accounts for housing costs.

    @Jim – Good point, at a 10% inflation rate as the 1980s-version suggests, the total cost of a basket of goods should have doubled in the last 7 years. Makes you wonder what’s in that basket!

  7. Hi Jim:

    9% inflation compounded annually over 10 years represents a loss of 61% of purchasing power, not a doubling of prices. Inflation is a general overall rise in the price of consumer goods. It doesn’t necessarily apply to “everything”.

    Inflation doesn’t just manifest in higher cost of goods. It’s also the reduction of quality/quantity of product sold at the same price point. I’ve noticed this phenomenon with (among other things) peanut butter, diapers, and cereal — the price stayed consistent while the net weight or amount of product was reduced. A similar effect can also be had by watering-down drinks, using cheaper ingredients, etc., without having to change prices.

    No methodology is perfect. Shadowstats 1980-methodology numbers may be high, but I believe these numbers to be closer to reality than the govt’s. Also keep in mind SS is simply using the govt’s own previously-used calculation method. This SS article defends the methodology used. It states (emphasis added):

    The approach here is simple, and some argue that the inflation differential since 1980—suggested by the BLS’s own estimates—is too large to be realistic. The numbers are what they are, and refinement to the approach certainly is possible. Keep in mind, though, that the differences here are in weighting and in quality adjustments, not in the underlying surveying of raw prices. While some might argue the magnitude of the inflation-understatement, resulting from the historical changes, there is no question as to the understatement of inflation.

    It is in the govt’s best interest for real inflation to be high (profit, inflate away debt) while artificially lowering inflation estimates (less doled out for Social Security, GDP estimates are higher than reality, etc.). So it isn’t difficult to find a motive for misguiding the public about real inflation numbers.

  8. One inflationary feature I notice a LOT, mostly in grocery stores, is the tendency for food producers to cut the product quantity, rather than raise prices for a set quantity….think the government notices that? Doesn’t seem like it, looking at my grocery spending. (I don’t have supporting figures for everything, but there SOME obvious examples)

  9. Right, they use owner’s equivalent rent, which is what it would cost to rent a residence, not actual mortgage amounts, which are generally fixed in this country and if anything go down (as people only tend to refi when rates fall). To the extent that housing prices and rents increase over time and one’s mortgage payment stays fixed, CPI actually overstates the price inflation experience of most homeowners with regard to one of their largest expenditures.

  10. If you believe the US government significantly understated the inflation number, as those fire-on-hair shadowstats claimed, you gonna believe this country had been in recessions most of the time for more than two decades.

  11. Peng: Perhaps you’re referring to this chart which shows actual GDP adjusted for distortions in gov’t inflation numbers. It essentially shows consistent negative GDP growth since 2001.

    I believe it. GDP is, unfortunately, largely based on consumer spending. And household income in real terms has declined since 2000 (see here).

  12. LargeTalons says

    Lol Brady,

    That’s the thing about making up shadow stats, if you just claim the inflation rate is different than it actually is you can use those numbers to say anything you want about the economy. Did you miss the entire point of this post? Sure looks like the BPP tracks published CPI pretty closely. Or, are you trying to convince us that the net weight of peanut butter throws off those 50,000 prices enough to make BPP match CPI? Seems unlikely. Do you also want us to seriously believe that the quality of every single product has gone down in the last 10 years? I just bought a new TV, it’s 10x as good, and about 100x cheaper than a similar TV would have been 10 years ago. The truth is that if you don’t want to use the value of cash to inflation, you can just buy an asset, so why don’t you just do that already and stop complaining? Inflation is a necessary component of any monetary system, it represents the storage/opportunity cost of liquidity. If the value of cash just kept going up, nobody would buy real assets, because you could just hold cash and make money. That would have a real cost to the real economy, since everyone would value worthless pieces of paper over actual goods and services. This gold standard nonsense is giving me a headache, you Ron Paul people really need to take a look at the last 100 years of economic evidence and take off the tin foil hats.

  13. LargeTalons: Way to jump right into the convo with an ad hominem.

    You wrote:

    That’s the thing about making up shadow stats, if you just claim the inflation rate is different than it actually is you can use those numbers to say anything you want about the economy.

    That’s also the thing about making up government-sponsored stats, if you just claim the inflation rate is different than it actually is you can use those numbers to say anything you want about the economy.

    That CPI and BPP have so far tracked closely means what, exactly? That they’re far superior indicators of actual inflation? Please. Until the underlying BPP methodology is exposed, the numbers there are meaningless. From what little is said about it, it seems like it only tracks prices for online retail goods. What about housing? Energy? Utilities? Services? How about shipping charges? Since we don’t know, how can any faith be put into BPP?

    To your points about “every single product” and TVs, I’ve already addressed those previously.

    The truth is that if you don’t want to use the value of cash to inflation, you can just buy an asset, so why don’t you just do that already and stop complaining?

    It is difficult to decipher meaning from your question. I’ll assume you meant “lose” instead of “use”, in which case you’re saying buying an asset shields you from inflation. Some assets may provide that protection. But since we can’t pay for things with TIPS or commodities, the cash we have on hand is continually being depreciated through inflation. Same goes for new earnings. You can’t escape inflation entirely.

    Inflation is a necessary component of any monetary system, it represents the storage/opportunity cost of liquidity. If the value of cash just kept going up, nobody would buy real assets, because you could just hold cash and make money. That would have a real cost to the real economy, since everyone would value worthless pieces of paper over actual goods and services.

    That’s patently false. People don’t need it. A monetary system doesn’t need it. Inflation is only necessary from the standpoint of keeping the banking industry afloat. Take away fractional reserve banking and the ability of the Fed to print money and you have a system that doesn’t inflate the monetary base; ergo, a system essentially without price inflation.

    In the deflationary scenario you propose, I don’t see the problem. Let’s take your TV for example. Despite dramatically falling prices over the years, product quality and sales have continued to increase. This is because these TV manufacturers have utilized technology to become more efficient while still remaining profitable. All in the face of price deflation.

    This gold standard nonsense is giving me a headache, you Ron Paul people really need to take a look at the last 100 years of economic evidence and take off the tin foil hats.

    Thanks for the tip. Since you brought up gold and Ron Paul, I’ll address it. It is precisely because the Austrian school looks at historical economic evidence that gives them the perspectives they have. And if it allows me to stay objective, educated, and informed, I’ll keep my tin foil hat firmly in place.

  14. Hoosier1982 says

    “CPI and BPP have so far tracked closely means what, exactly? That they’re far superior indicators of actual inflation? Please. Until the underlying BPP methodology is exposed, the numbers there are meaningless. From what little is said about it, it seems like it only tracks prices for online retail goods. What about housing? Energy? Utilities? Services? How about shipping charges? Since we don’t know, how can any faith be put into BPP?”

    None of that is correct. The BPP is a measure that tracks millions of prices a week, across a huge range of categories.

    Unfortunately for you, the results invalidate your hypothesis.
    So you naturally have to both ignore and disparage the measure. If you even did a tiny amount of research on BPP you would answer your questions.

    It’s tough when data kills your argument, like it has here. But you need to man up and incorporate the BPP results into you next hypothesis.

  15. The ad hominem fest continues.

    The BPP is a measure that tracks millions of prices a week, across a huge range of categories.

    That’s what I’ve been told.

    Unfortunately for you, the results invalidate your hypothesis. So you naturally have to both ignore and disparage the measure. If you even did a tiny amount of research on BPP you would answer your questions. It’s tough when data kills your argument, like it has here. But you need to man up and incorporate the BPP results into you next hypothesis.

    Silly me. Perhaps it would be even better to anonymously disparage people whose ideas we don’t agree with, eh?

    Here’s what I’ll do instead. I’ll make up an index that measures inflation and call it the Supremo Inflation Index. It’ll measure a trillion prices from across a wide spectrum of… yadda, yadda. But I’m not going to tell you, exactly, what it does and doesn’t measure. But trust me, it’ll be too legit to quit!

    Sound good? Because that’s what you’re getting with BPP. Since you’ve apparently done the research, perhaps you can show me where the details of the methodology is published, because it certainly isn’t on the BPP website. All that’s there is a vague overview. Or maybe you can just answer my previous ponderings: does it track housing, energy, utilities, services, or shipping charges?

    Believe whatever you want to believe. I’ve spent long enough debating minutiae here.

  16. @Brady- I am not convinced you know what ad hominem means. Everyone who challenged you did so with ideas, not name calling or personal attacks.

    Shadowstats is just silly in its over-statement of CPI. The changes in calculations over the years have been improvements to counteract both low end and upper end substitution biases, increased sampling sizes, more accurate mimicking of consumer behavior based on biennial consumer surveys, using a more accurate geometric average formula, utilizing computers to collect data, instituting the Phone Point of Purchase Survey data, etc. Times change, consumer behavior changes, technology changes, and so do economic measures. In general, they are improved rather than ruined. MIT’s BPP is one piece of evidence that validates changes to the CPI over the years.

  17. Whatever ‘ad hominem’ means, I believe inflation figures have been massively understated over the past 10 years, based on what I’ve been spending to live. One thing that seems to have been ignored by everyone is the cost of health care and health care insurance. It’s obvious to me that it is not in the government’s interest to correctly state inflation figures. Our money DOES buy less (LOOK at package contents at the grocery) and those of us who try to save for a rainy day and/or retirement are having our pockets picked by 0.01% returns, graciously arranged by the same all-benevolent government.

  18. What about hedonics? The GDP is made up of about 30 percent fictional numbers by adding to the GDP free things like rent that a doesn’t pay; or checking accounts which are free but the government add the “cost” to the GDP …. LMAO .. The last time I read about a serious study into this phenomenon was done in 2003 which found that 30 percent of the US GDP was fictional numbers …..

    If you take hedonics out of the GDP … the problem would be clear!

  19. should read, “adding to the GDP free things like rent that a homeowner who has paid off his home doesn’t pay. the government makes up a fictitious “rent” and adds it to the GDP; or checking accounts

  20. Your comment about prices going down not being noticed as much is an interesting explanation…but is a temporary sale price or a one-time close-out deal really indicative of either the “psychological feel” or reality of prices? If the price of a certain product is going up 1%/month for 24 months and you get a 15% discount in month 24, have you actually gotten a discount? No!

    This is a note I sent to the BPP/PriceStats folks.
    I believe there are serious fundamental biases in their data…

    Interesting project. I’m very surprised at how little difference there is between your data and the official CPI. They move pretty much in lockstep and are never very far apart. how is it that your numbers are quite often lower than CPI? All other alternative inflation indices are substantially higher than the CPI at all times. Your low inflation rates are not at all similar to my experience as a consumer!

    How do you scrape food prices, since the vast majority of food is not sold online? What proxies do you use for food? How do you take into account sizes of products? Many food items have shrunk in size in order to maintain the same price, thus disguising as much as a 33% effective price increase. Do you include seasonality for fresh produce?

    Since your scraping reflects online prices, doesn’t this reflect/bias the decreasing prices of online efficiencies vs. the increasing costs & prices of brick & mortar retailers? Everyday staple items are generally bought locally, whereas disposable income purchases are more often purchased online which would further skew the data. Also, the lower-income & impoverished population buys almost exclusively locally, often at stores with zero online presence.

  21. I realize that economics is largely junk science, but I expect something sound from MIT. My question is how do you get “Billions” of prices from “Hundreds” of sources – so the Website states. Is this junk science from the superlative MIT?

Speak Your Mind

*