Understanding Margins of Error (MOE) in the ACS

The American Communities Survey (ACS) is the primary data source for communities since the demise of the long form from the decennial census process. For information about what the ACS is, and it’s relationship to the census, you might want to revisit this previous blog post.

Since the ACS is based on a smaller sample than the decennial census long form used to be, it has more variability in it’s statistics. Think of it this way, if you surveyed every single person in a community you would know with 100% accuracy how much money they made last year. If you only asked 1 out of every 7, you would still probably have a pretty good idea, but there would be a bit of measurement error in your number. So there would be some level of variability. But now imagine you only interview 1 out of every 50 people. You’d still have an income level to report but the amount of variance in your data would probably be a bit larger.

That’s the difference between ACS data and the decennial census data. The census “short form” was used to survey every single household. The long form only went to 1 in 7 households. Now along comes the ACS which only goes to 1 in roughly 50 households. The variability of the data is much greater when the sample size is smaller,

This is why there ACS provides “margins of error”, or MOEs. The MOEs provided in the ACS basically tell you that you can have 90% confidence that the ACTUAL number if you survey 100% of the community would be between the number provided and plus or minus the margin of error. Let’s use an example.

According to the 2010 One Year ACS estimate, the median household income in Oneida County is $47,286. The MOE (margin of error) is +/- $2,128. So that means that if you go above and below the median income level reported by $2,128, you can be 90% confident that the actually household income level is within that range. More specifically, I am 90% sure that the median household income in Oneida County is between $49,414 and $45,158 .

Why is this important to think about ? Well, numbers with really REALLY large MOE suggest that the data is less reliable. It might be interesting to know and give you a clue as to conditions for an area, but if the margin of error was HUGE it should also give you pause. Think about it this way – if in the last example the MOE for median income was +/- $40,000, that would mean we would feel confident that the median income was between $87,286 and $7,286 ! That really isn’t a particularly useful range of data for talking about median household income !

I’ll make a separate post on the importance of properly interpreting margins of error. In the meantime, keep the MOEs in mind as you look at ACS data ! And when they begin to exceed 20% of the base number being measured, you should at the very least take a moment to look at the data more closely and use caution in interpreting it.