Quick Note: ACS 2010
The Census Bureau has just released the American Community Survey numbers for 2010, using data calibrated to match with the 2010 census. At least, calibration is the best reason for why the ACS believes that New York went from 8,391,881 people in 2009 to 8,184,899 in 2010 (according to the new Factfinder). Because of such jarring discrepancies in results, people should under no circumstances directly compare numbers from the 2010 ACS with numbers from previous ACSes.
The best demographic survey in the US is still the 2009 ACS, which avoids the whopper claim that New York added more housing units than people at a time of skyrocketing rents, and should be used until it becomes completely outdated.
And even if 2010 census data is at all reliable, it’s still not directly comparable. Claims about absolute mode share or commute time are okay (the census after all only underestimated New York’s population by about 3%), but claims about change from 2009 are not. At best the 2010 ACS should be compared to the 2000 estimate base, and even that is strained – too much reliance on a census that doesn’t count everyone, insufficient reliance on years of rigorous statistical sampling.
Pedestrian Observations 
It should be mentioned that even the best Census would fail to count everyone. Various factors go into underreporting and overreporting (a college student, for example, being counted twice); as such, the Census too is a statistical sample. A remarkably accurate sample, but a sample nonetheless.
Ignoring the fact that the census is unconstitutional, with the exception of the question about how many people live at this address, the results that it produces are worthless. The only question that they should be permitted to ask are how many people live at this address.
The census was included in the constitution solely to record the number of people for congressional apportionment purposes. It was never supposed to record your gender, commute time, income, number of toilets at your home, etc. What we have today, with neighbors encouraged to snitch on neighbors, and census workers forcing you to snitch on your neighbors is what totalitarian states do.
The appropriate section of the constitution: “(Representatives and direct Taxes shall be apportioned among the several States which may be included within this Union, according to their respective Numbers, which shall be determined by adding to the whole Number of free Persons, including those bound to Service for a Term of Years, and excluding Indians not taxed, three fifths of all other Persons.) (The previous sentence in parentheses was modified by the 14th Amendment, section 2.) ”
Source: http://www.tenthamendmentcenter.com/2010/01/29/the-unconstitutional-census-gestapo/
Because the government wants to get the exact details of your life, some people willingly choose to provide fraudulent answers or not complete it at all. Urban areas with a substantial immigrant population are drastically under counted because the immigrants are afraid of potential repercussions if they disclose their true circumstances.
Fort Collins, Colorado was counted as having 136 ferry commuters by the ACS. Take a look at where Fort Collins is on a map. I am willing to bet that there isn’t a single ferry within 200 miles of here! With such glaring inaccuracies, how can that be scientifically sound? Today, there was an article in the local paper about the bicycle commute percentage (a measure fatally flawed as explained elsewhere) went from 9% to 4% in just one year. From my personal observations the number of bicycles on the road has held about steady or slightly increased.
A more objective way to measure mode split that counts everyone, not just commuters, would be to draw a line on a map and do a screenline count of everyone crossing that line. In the case of Fort Collins, a dividing line along the Spring Creek Trail would be a natural place to conduct it. Of course there are people who don’t cross the line.
In conclusion, the census is an excessive governmental overreach into our privacy with minimal benefits. If the census were to be scaled back to its constitutional restrictions, we would at least get an accurate number of how many people live where. The government has no right to force people to disclose their exact whereabouts to them, especially considering that the data ends up being complete junk.
It’s not just illegal immigrants. The homeless are not counted. Legal immigrants, the poor, and people with transient addresses aren’t reliably at home, and the Census Bureau workers don’t have time to do a complete followup of everyone.
The ACS is somewhat better – they send it to a sample, so that they know roughly how many people live in each area and what the social needs there are. The Census Bureau wanted to do additional statistical sampling to get a more accurate count, but Congressional Republicans realized it would fail to undercount Democratic areas, raised a stink, and passed a law banning it in the 1990s. Since then, the compromise has been that the ACS (using sampling) can be used for allocating spending, but not for Congressional apportionment. In 2010 the census didn’t even include those extra questions, since those are dealt with by the ACS – if I remember correctly it was just names, races, genders, and citizenship status.
The problem with the ACS: it’s based on a sample. It’s good for large areas, and bad for small ones. This is true of the census, too, since it should be thought of as a sample that has no sampling error (since nearly everyone is counted) and a sizable non-sampling error (since the choice of who is and isn’t counted is non-random). But if the bike commute percentage fluctuated this much in a single year, chances are someone compared the 2009 ACS to the 2010 ACS, or just didn’t read through the fine print and looked at data based on too few respondents to be statistically significant. It happens; journalists make mistakes.
Fort Collins’ ferry commuters is probably one of these sampling errors. In every area, there are residents who really spend most of their time elsewhere, and so count as commuting to an area that’s clearly not within ground transportation distance. See the county-to-county commute numbers and observe that 148 Laramie County residents were counted in 2000 as working outside the US, and another 900 in states other than Colorado and Wyoming. Chances are the ACS landed on two out-of-state commuters who really do use a ferry. In the 2005-9 ACS, they say there are 55 plus or minus 50 ferry commuters, which doesn’t sound outlandish.
The definition of “plus or minus” is such that 95% of results will fall in the range, and if you got the Fort Collins example from someone who was trying to attack the census, rather than from someone who randomly looked at Fort Collins’ commuting patterns and wondered, then chances are that this person looked at many numbers and picked the most outlandish one. If you toss a fair coin 10 times over and over, after a thousand trials you’ll probably get ten heads.
I got the 136 figure from a Coloradoan (local newspaper) report about mode split and the census a while ago. The number just stuck with me I guess.
Here is the report today in the paper about the bike commute rates. Take a look at the comments section in particular. The amount of blind trust in a government report is a bit startling.
http://www.coloradoan.com/article/20110927/UPDATES01/110927015/American-Community-Survey-Fewer-Fort-Collins-workers-biking-office?odyssey=mod|mostcom
Yes, it does fall prey to comparing the 2010 and 2009 ACS directly.
I try to avoid reading newspaper comment sections whenever possible, for sanity reasons. But yeah, no, this article is exactly why I wrote the post.
I am increasingly worried about government interference in good Statistics practice. However, I understand that in the case of US Census, because so much is at stake. It would make sense, in my opinion, not to count illegal immigrants, people on temporary visas and extended holiday tourists (something that sways numbers for small mountain communities in Eastern Colorado, for instance) for drawing political representation maps.
However, I don’t sign up for this “Gestapo” accusations. Non-anglophone developed countries in Europe all have some form of civic registration, and have had since mid-19th Century in some cases.
Mandatory notification to CDC of certain infectious diseases episodes is accepted as good health surveillance practice, but when such notifications were pushed in the 1970s, many saw a plot to “introduce mandatory euthanasia” that would be in full swing by the turn of the millennium.
What I do recognize is that not collection information that will be released in the public domain (thus making research and analysis much more difficult) serves very well a culture of political and administrative decisions made on slogans rather than facts. When you can’t compare, verify or measure on a study-event the effects of new laws, regulations and investment decisions, it becomes all too easy to push them based on feelings or “people know what is right and don’t need numbers to confirm that”.