Friday, October 24, 2008

By the Numbers













"Science consists in discovering the frame and operations of Nature, and reducing them, as far as may be, to general rules or laws — establishing these rules by observations and experiments, and thence deducing the causes and effects of things." – Isaac Newton


Earlier this week, Politico.com ran a story about media outlets “preparing for the possibility that their Election Day surveys could be skewed because of overstated support for Barack Obama, largely because of the enthusiasm of his supporters.”

Exit polling is a notoriously inexact science. After all, early exit polls in 2004 suggested John Kerry would be elected president. This year, multiple variables, including the zeal of Obama’s supporters, the candidate’s racial background, widespread early voting, etc. is making those whose job it is to track voter sentiment very nervous.

In theory, exit polls should perfectly match election results. But this is not always the case. After all, preliminary exit polls during the Democratic primary overestimated Obama’s strength in 18 of 20 states by an average error of seven percentage points. (Some have attributed this discrepancy to the Bradley Effect). This overestimation was most shockingly seen in New Hampshire. Experts credited this phenomenon to the youth and enthusiasm of Obama’s supporters. Eager to participate in polls, their numbers were falsely inflated. Older voters, more likely to be Republicans, often refused the request to participate.

So what are we to make of the current predictions, which show Obama ahead by a comfortable margin of error?

Back in early June of this year, just days before Hillary Clinton suspended her presidential campaign, my friend Neil deGrasse Tyson, an astrophysicist, author, host of “Nova scienceNOW” and director of the Hayden Planetarium in New York City, wrote an Op-Ed in the New York Times titled “Vote by Numbers.”

In it, Tyson stated that if the election were to be held that day, Barack Obama would lose to John McCain (and McCain would lose to Hillary Clinton). His conclusion was based on “a new method of analysis on the statistics of polls that has been accepted for publication in the journal Mathematical and Computer Modeling.”

The authors of this new way of examining statistical data, J. Richard Gott III of Princeton, and Wes Colley of the University of Alabama in Huntsville, are not political scientists.

They are astrophysicists.

For Tyson, this made complete sense. After all, “one of the tasks of scientists is to clarify the apparent complexity of the universe by using the language of mathematics.”

Gott and Colley discovered that “in swing states, the median result of all the polls conducted in the weeks prior to an election is an especially effective predictor of which candidate will win that election — even in states where the polls consistently fall within the margin of error.” (For a far more detailed description of the rules that went into creating the formula, CLICK HERE).

In a paragraph that was actually cut for space, Tyson wrote, “Normally when you think of an average, you invoke the elementary school definition of the word. But you later learn in statistics that there are three kinds of averages – the mean, the median, and the mode. And in the field of mathematical physics, the concept of average is further generalized into higher dimensions. They look for the median count of wins and losses in a set of polls for a state, which is the same thing as asking, “Who won the most polls?” Turns out, median statistics are especially effective in swing states, where the margin of error in a single poll is typically greater than the spread in votes. As such, it can be a far more accurate and robust predictor of who will actually win an election than most people’s politically informed commentary.

“If you ask three people what time it is, and one person says its five to ten, the second says its ten o’clock, and the third says its four thirty. You do not take the average, and declare the time of day to be early afternoon. You grab the time in the middle (the median) and proceed on the more likely assumption that it’s about ten o’clock.”

So accurate was this new method, in fact, that when the 2004 presidential election between John Kerry and George W. Bush was crunched, Gott and Colley correctly predicted the winner in 49 states (they missed only Hawaii with its four electoral votes)! This at a time when almost all political analysts said the race was either too close to call or leaning toward Kerry. The median method correctly predicted Bush as the victor, even months in advance.

Using Gott and Colley’s rules, Tyson applied the median method to the 2008 presidential race and discovered that without a doubt, Obama would lose to McCain and McCain to Clinton were the election held that day. Tyson was quick to stress that the analysis did not predict what would happen in November. Instead, unlike many fickle polls, the Gott and Colley method “describes the present better than any other known method does.”

Hundreds of people wrote the New York Times to take issue with Tyson’s reasoning. He was bombarded with commenters who, while they were quick to point out their admiration of his work, took great issue with his reasoning. Many assumed Tyson wrote the piece as a last ditch effort to help Hillary Clinton. Obama supporters especially thumbed their noses at the results. Tyson defended the political neutrality of the piece’s tone and content. In responding to some of his detractors, Tyson pointed out that while math can be abused, at its best math is politically neutral. There is no such thing as red math or blue math. Just accurate math and inaccurate math.

Over the months, Tyson has forwarded me updates. It has been fascinating — and not a little disheartening — to watch the numbers. While a vast battery of polls was showing McCain down and out and Obama surging, the median method disputed them all, showing McCain still held a comfortable lead. As much as I wanted to dispute the method’s findings, Tyson kept reminding me that people may lie, but math does not. Obama’s popularity, in this case, was irrelevant.

“Our system of electoral politics requires you to not only be popular, but popular in all the right places. While human behavior can be, and often is fickle, median human behavior is not. Otherwise, there would be no such thing as an advertising industry. Accuracy matters.”

Though the analysis admitted some gains (“Colorado is a runaway for Obama”) it also spelled out disastrous defeats (“Florida is hopelessly out of reach”). For every Nevada and Virginia it gave to Obama, the median method indicated that other states were drifting further and further away.

Going into September, there were only glimmers of hope.

“As of this morning, the median analysis show **no close states** (no swing states), in spite of the rants of analysts; Obama is beating McCain 273 to 265.”

It was not nearly enough. In effect, the best Obama could hope for was a draw. Tyson and the study’s authors indicated that Obama would need to win by far more than eight electoral votes to guarantee a victory. It would require a miracle, a sort of October surprise to fundamentally shake everything up.

Then came the financial crisis.

As the gravity of the situation bit into Americans’ realities, things began to shift on the median method map.

Obama’s totals surged to 291. Florida, once “hopeless,” now suddenly looked within striking distance.

The following is from an e-mail I received from Tyson just a few days ago. It contained the word “bloodbath.”

As of October 19, median analysis of polls for the previous 30 days give the following Electoral College count:

Obama 367
McCain 171

Obama has been ahead of McCain since August 22nd, when Nevada put him in the lead. He took Florida on October 14th, which has given him a significant lead since then. The enhanced Florida campaigning in that state has worked. The overall trend looks very bad for McCain and could lead to a landslide win for Obama, giving him the mandate he will require to fulfill the many diverse promises he has made to his electorate.

The current Blue/Red state division in the median analysis can be seen in the USA map below (see the image opening this blog post), with states scaled by area to the precise number of electoral votes they represent. This “Electoral Area Map” developed by my Princeton Astrophysics colleague Richard Gott, gives an immediate sense of who is winning and who is losing.

A reminder that states which many people still claim as up for grabs are not so in the median analysis. All states given to Obama (including Ohio and Pennsylvania) are, at the moment, rather secure.


It is important to remember that this method, like all polls, is nothing more than a snapshot of a day. However, this method has proven itself to be more accurate than anything else out there. If the election were held today, Obama wins easily.

Gott and Colley’s method is a robust engine, deciphering those things people normally think are difficult-to-predict variables. In June, when Tyson first publicized the findings in The New York Times, the data pointed, with mathematical certainty, to deficiencies Obama would have to surmount should he wish to overtake his contender. It appears he has done just that. It behooves campaign strategists to pay close attention to what the median method shows. Unlike those of his supporters who were in denial of the results, Obama’s campaign paid attention, and targeted the important states with their many electoral votes where he was behind. The unbiased, impartial data is also strategically useful information for the McCain campaign, showing exactly where the Republicans must make up ground. However, with only 11 days left before Election Day, McCain’s chances to turn things around seem insurmountable if not utterly impossible.

UPDATE: Politico now reports that there are no more swing states. Every one has now gone for Obama and is outside the margin of error.

7 Comments:

Blogger Rhonda said...

As Biden said in the rally I attended on Wednesday in Colorado Springs, we cannot be complacent.
It was a great time - fun, inspiring and moving.

11:59 AM  
Blogger Reacher said...

Sounds like LMSM (liberal mainstream math) to me.

5:56 PM  
Anonymous Anonymous said...

Across the electric wires, the hum is ceaseless: Give it up, loser. Don't go down with the ship when it's swept away by the Obama tsunami. According to newspaper reports, polls show that most people believe newspaper reports claiming that most people believe polls showing that most people have read newspaper reports agreeing that polls show he's going to win.

In the words of Publishers' Clearing House, he may already have won! The battleground states have all turned blue, the reddest of red states are rapidly purpling. Don't you know, little fool? You never can win. Use your mentality, wake up to reality. Why be the last right-wing pundit to sign up with Small-Government Conservatives For The Liberal Supermajority? We still need pages for the coronation, and there's a pair of velvet knickerbockers with your name on it.

Yes, technically, this is still a two-party state, but one of the parties is like Elton John's post-Oscar bash and the other is a church social in Wasilla. As David Sedaris put it in The New Yorker:

"I think of being on an airplane. The flight attendant comes down the aisle with her food cart and, eventually, parks it beside my seat. 'Can I interest you in the chicken?' she asks. 'Or would you prefer the platter of s--t with bits of broken glass in it?'

"To be undecided in this election is to pause for a moment and then ask how the chicken is cooked."

Well, to be honest, I've never much cared for chicken.

McCain vs. Obama is not the choice many of us would have liked in an ideal world. But then it's not an "ideal world," and the belief that it can be made so is one of the things that separates those who think Obama will "heal the planet" and those of us who support McCain faute de mieux. I agree with Thomas Sowell that an Obama-Pelosi supermajority will mark what he calls "a point of no return."

It would not be, as some naysayers scoff, "Jimmy Carter's second term," but something far more transformative. The new president would front the fourth great wave of liberal annexation – the first being FDR's New Deal, the second LBJ's Great Society, and the third the incremental but remorseless cultural advance when Reagan conservatives began winning victories at the ballot box and liberals turned their attention to the other levers of the society, from grade school up. The terrorist educator William Ayers, Obama's patron in Chicago, is an exemplar of that most-recent model: 40 years ago, he was in favor of blowing up public buildings; then he figured out it was easier to get inside and undermine them from within.

All three liberal waves have transformed American expectations of the state. The spirit of the age is: Ask not what your country can do for you, demand it. Why can't the government sort out my health care? Why can't they pick up my mortgage?

In his first inaugural address, Calvin Coolidge said: "I favor the policy of economy, not because I wish to save money, but because I wish to save people." That's true in a more profound sense than he could have foreseen. In Europe, lavish social-democratic government has transformed citizens into eternal wards of the Nanny State: the bureaucracy's assumption of every adult responsibility has severed Continentals from the most basic survival impulse, to the point where unaffordable entitlements on shriveled birth rates have put a question mark over some of the oldest nation states on Earth. A vote for an Obama-Pelosi-Barney Frank-ACORN supermajority is a vote for a Europeanized domestic policy that is, as the eco-types like to say, "unsustainable."

More to the point, the only reason why Belgium has gotten away this long with being Belgium and Sweden Sweden and Germany Germany is because America's America. The soft comfortable cocoon in which Western Europe has dozed this past half-century is girded by cold hard American power. What happens when the last serious Western nation votes for the same soothing beguiling siren song as its enervated allies?

"People of the world," Sen. Obama declared sonorously at his self-worship service in Germany, "look at Berlin, where a wall came down, a continent came together, and history proved that there is no challenge too great for a world that stands as one."

No, sorry. History proved no such thing. In the Cold War, the world did not stand as one. One half of Europe was a prison, and in the other half far too many people – the Barack Obamas of the day – were happy to go along with that division in perpetuity.

And the wall came down not because "the world stood as one," but because a few courageous people stood against the conventional wisdom of the day. Had Margaret Thatcher and Ronald Reagan been like Helmut Schmidt and Francois Mitterrand and Pierre Trudeau and Jimmy Carter, the Soviet empire (notwithstanding its own incompetence) would have survived, and the wall would still be standing. Sen. Obama's feeble passivity will get you a big round of applause precisely because it's the easy option: Do nothing but hold hands and sing the easy-listening anthems of one-worldism, and the planet will heal.

To govern is to choose. And sometimes the choices are tough ones. When has Barack Obama chosen to take a stand? When he got along to get along with the Chicago machine? When he sat for 20 years in the pews of an ugly neo-segregationist race-baiting grievance-monger? When he voted to deny the surviving "fetuses" of botched abortions medical treatment? When in his short time in national politics he racked up the most liberal – i.e., the most doctrinaire, the most orthodox, the most reflex – voting record in the Senate? Or when, on those many occasions the questions got complex and required a choice, he dodged it and voted merely "present"?

The world rarely stands as one. You can, as Reagan and Thatcher did, stand up. Or, like Obama voting "present," you can stand down.

Nobody denies that, in promoting himself from "community organizer" to the world's president-designate in nothing flat, he has shown an amazing and impressively ruthless single-mindedness. But the path of personal glory has been, in terms of policy and philosophy, the path of least resistance.

Peggy Noonan thinks a President Obama will be like the dog who chases the car and finally catches it: Now what? I think Obama will be content to be King Barack the Benign, Spreader of Wealth and Healer of Planets. His rise is, in many ways, testament to the persistence of the monarchical urge even in a two-century old republic. So the "Now what?" questions will be answered by others, beginning with the liberal supermajority in Congress. And as he has done all his life he will take the path of least resistance. An Obama administration will pitch America toward EU domestic policy and U.N. foreign policy.

Thomas Sowell is right: It would be a "point of no return," the most explicit repudiation of the animating principles of America. For a vigilant republic of limited government and self-reliant citizens, it would be a Declaration of Dependence.

If a majority of Americans want that, we holdouts must respect their choice. But, if you don't want it, vote accordingly.

1:39 PM  
Blogger Brandon said...

WOW! I had no idea that Mark Steyn of the National Review read my blog! Cool.

Wait a second, I'm beginning to think someone just cut and pasted Mark Steyn's article here without giving him the credit. No, none of my friends would be so shady. It must be the real Mark Steyn...

1:49 PM  
Blogger Brandon said...

I love how a post about numbers and reality has been turned into one about ideology and partisanship. That's not what this post at least was about.

Plus, I have a sneaking suspicion that all of these conservative pundits so angry at the sliding numbers never took umbrage with it when, say, Reagan took 49 out of 50 states nearly 30 years ago!

1:52 PM  
Blogger Reacher said...

Because Reagan was a god, and Obama is the devil.

Duh.

6:59 PM  
Anonymous Matthew Delahunt (POD) said...

It's a strange time to be a republican for sure, the apparent implosion of the party, the so called claims that the "Reagan" era is over - it is difficult to find a reasonable footing between the gauntlet of the zombie-adoring fans of Obama and the extreme bias that media has been given to his campaign this year. Heck, I just heard that the Obama campaign cut off a Florida television station because their reporter asked a "hard" question to Senator Biden... very disheartening to anyone who is searching for anything resembling truth in reporting the news.

Althought I could never vote for Obama on principle, I am beginning to believe that his election will be for the better health of the conservative movement and the Republican party from a long term view. It will force the party to regroup, get its bearings in place and move forward for the next election cycle.

It will be interesting to see how Obama governs. I am inclined to think that he will take the path of least resistance and not buck his party allowing a deluge of liberal legistlation getting passed without nary a veto.

I am just sick of the Republican Party being synonymous with "big government." The election of Obama may be just what the country needs to vindicate President Bush's tax and foreign policy legacy, as well as vindicate the notion that absolute power corrupts absolutely, as was what the case was with Clinton in 93 and Bush in 04.

9:59 PM  

Post a Comment

<< Home

Ut In Omnibus Glorificetur Deus