The American Community Survey, 2014-2018, provides estimates of the characteristics of the US population. These estimates are based on responses collected from questionnaires sent by the US Census Bureau to a series of monthly samples of housing unit addresses, with approximately 3.5 million addresses surveyed each year. As with any survey, how this sample is selected impacts the accuracy and reliability of generalizations drawn about the entire population based on responses from the sample. As a responsible and authoritative statistical agency, the US Census Bureau follows a careful process in designing the sampling frame to help ensure and monitor the quality of the estimates produced. You can read more about this process in the Design and Methodology Report published by the US Census Bureau.
Certain metrics are also published with the American Community Survey results to help the US Census Bureau and data users assess the accuracy and reliability of the estimates released in the survey. For example, a statistic called the “margin of error (MOE)” is published with each estimate. The MOE indicates the likelihood that the ACS sample estimate and the actual population value differ by no more than the value of the MOE. For the ACS, MOEs are provided at a 90 percent confidence level, which means that the estimate is expected to contain the true or population value within a range defined by the associated MOE 90 percent of the time.
An example? Of course! Data Planet has published MOE values for the last two 5-year releases of the American Community Survey (2013-2017 and 2014-2018). We find in B19019 that the ACS estimate for Median Household Income in 3-person households reported in B19019 for the state of Delaware is $84,232 with an MOE of +/-$1,752:
By adding and subtracting the MOE from the estimate, we can calculate the 90 percent confidence interval for that estimate – meaning we expect the true population value to be within this range 90 percent of the time.
84,232 + 1,752 = Upper bound of the confidence interval
84,232 - 1,752 = Lower bound of the confidence interval
The chart below shows the results of this calculation, which was created by running a simple addition and subtraction formula using the Data Planet calculator:
Now watch what happens when we drill into narrower geographies in Delaware, here Places (includes cities, towns, etc.):
Notice how much greater the MOEs vary from the estimate than at the state level, resulting in a wider range of the values at the upper and lower limits of the confidence level. There is a logical explanation for this – review the data for Sample Count of Population in Table B00001 vs Total Population from Table B01003.
Calculate the percentage sampled and you’ll find relatively low levels of the estimated total population were sampled for these geographies. Because of the sample size, the estimates for indicators may indeed vary widely from the actual value.
For more on MOEs and the confidence levels of ACS statistics, see the Census Bureau presentation on MOEs.