Blogs of War

Hizballah Cavalcade

Internet Haganah



Kremlin Trolls

Making Sense of Jihad

Selected Wisdom

Views from the Occident


American Terrorists

Anwar Awlaki

Al Qaeda


American Al Qaeda Members

Inspire Magazine

Revolution Muslim


News, documents and analysis on violent extremism

Saturday, May 4, 2013

Forecasting Terrorist Attacks With Big Data and the Wisdom of Crowds... Or Not

Back in January, I was playing around with Google Trends, the free service that tracks when specific search terms have been historically popular on a scale from 1 to 100. I noticed that in addition to its tracking function, there was a humble checkbox labeled "forecast" that predicted when the search terms would recur for some months into the future.

I was intrigued by this idea, since searches theoretically represent a mass conglomeration of factors that feed into an extraordinarily large data set. Out of curiosity,  I plugged in "terrorist" and "attack" and clicked the forecast checkbox. I tweeted my results, which I read as predicting a terrorist attack  around May 2013.

The actual attack came in April. A two-week margin of error isn't bad, all in all, especially when you consider that search traffic might not peak until after an event took place.

So the question now becomes: Is this ridiculous, or is it meaningful? Could the raw wild data set of people's search queries somehow tap into something that gives us a reliable window on what will happen in the future?

Significant terrorist attacks are outliers. They happen so infrequently and are so diverse as to defy easy statistical generalizations. The data surrounding them tends to be clumpy. So the fact that several attempted terrorist attacks since September 11 have been thwarted in the period between April and June, and in September through November, may not be meaningful but might shape the Google trends data enough to make it forecast May and September as likely peaks in searches for "terrorist attack."

September is also a consistent peak month for such queries, almost certainly because of renewed media interest in 9/11 anniversaries. And April results may be similarly skewed by the anniversary of the Oklahoma City bombing, which is also a significant date for domestic extremists on other grounds.

This may be reflected in the current Trends forecast, in which searches for "attack" spike sharply once more in April and May 2014, while "terrorist" rises less dramatically. "Attack" shows lesser peaks in July (the anniversary of two different attacks in London) and September, when "terrorist" also peaks, although less dramatically.

I waited until now to re-run the analysis in order to allow the noise from the marathon bombings to die down a bit and hopefully produce a more reliable forecast. Overall, I'd say these forecasts raise more questions than answers. But it will be interesting to monitor developments and see what's exactly what's going on in September 2013 and May 2014. And there's an earlier benchmark, which I also tweeted about in January:

As of today, Trends is still predicting peaks in August and January. Clumpy past data may again be contributing to the spikes -- Aurora took place in July 2012, and the Sikh Temple shooting in August -- but the past data is somewhat less clearly tilted toward those months.

It's intriguing to look at all of this and think there might be something to it -- that somehow the mass of all Google search queries, essentially crowdsourcing taken to its extreme, might somehow tap into knowledge or conditions that can predict serious events. One near-success is not nearly enough to go on, but it's interesting enough to keep track and see what the future holds.

Buy J.M. Berger's book, Jihad Joe: Americans Who Go to War in the Name of Islam


Views expressed on INTELWIRE are those of the author alone.



Tweets referencing this post:



", granular analysis..."

ISIS: The State of Terror
"Jessica Stern and J.M. Berger's new book, "ISIS," should be required reading for every politician and policymaker... Their smart, granular analysis is a bracing antidote to both facile dismissals and wild exaggerations... a nuanced and readable account of the ideological and organizational origins of the group." -- Washington Post

More on ISIS: The State of Terror

"...a timely warning..."

Jihad Joe: Americans Who Go to War in the Name of Islam:
"At a time when some politicians and pundits blur the line between Islam and terrorism, Berger, who knows this subject far better than the demagogues, sharply cautions against vilifying Muslim Americans. ... It is a timely warning from an expert who has not lost his perspective." -- New York Times

More on Jihad Joe


INTELWIRE is a web site edited by J.M. Berger. a researcher, analyst and consultant covering extremism, with a special focus on extremist activities in the U.S. and extremist use of social media. He is a non-resident fellow with the Brookings Institution, Project on U.S. Relations with the Islamic World, and author of the critically acclaimed Jihad Joe: Americans Who Go to War in the Name of Islam, the only definitive history of the U.S. jihadist movement, and co-author of ISIS: The State of Terror with Jessica Stern.


Newest posts!

J.M. Berger on Marathon Bombings

Marathon Bombing: Issues to Watch

Background on Marathon Bombers Tamerlan Tsarnaev a...

A day in the life of a terrorism analyst, 2013 edi...

On Comparing White Nationalists to Anarchists

Best-Liked on the Most-Wanted

A Closer Look at the SPLC's Hate List

Inspire 10: Still sucks

Bum Rap

Should we disrupt terrorists on the Internet?


New York Pipe Bomb Suspect Linked to Revolution Muslim

The Utility of Lone Wolves

Interview with Online Jihadist Abu Suleiman Al Nasser

A Way Forward for CVE: The Five Ds

How Terrorists Use The Internet: Just Like You

PATCON: The FBI's Secret War on the Militia Movement

Interview About Jihad With Controversial Cleric Bilal Philips

Forgeries on the Jihadist Forums

U.S. Gave Millions To Charity Linked To Al Qaeda, Anwar Awlaki

State Department Secretly Met With Followers of Blind Sheikh

State Department Put 'Political Pressure' On FBI To Deport Brother-in-Law Of Osama Bin Laden In 1995

FBI Records Reveal Details Of Nixon-Era Racial Profiling Program Targeting Arabs

Gaza Flotilla Official Was Foreign Fighter in Bosnia War

U.S. Had 'High Confidence' Of UBL Attack In June 2001

Behind the Handshake: The Rumsfeld-Saddam Meeting