Gathering Intelligence Faster than Real Time

by | Oct 4, 2012

Big Data Management

The real-time intelligence generated from the fastest-growing nation on Earth”the Social Media Nation”is incredibly valuable for analysts seeking to keep the peace, managing emergencies and fighting terrorism. But how can it be taken one step further? It will take an open source intelligence engine, uniquely tuned to the intelligence mission.

Lockheed Martin helps its customers merge traditional geospatial data with multisource data to support troops on the battlefield, intelligence agents worldwide and emergency responders at home.

The engine has to meet requirements for veracity and provenance of the source data, particularly with Open Source Intelligence (OSINT), which can be raw and at times unreliable.  It must have built-in discipline and sophistication to fuel mission-critical decisions. It must understand the subtleties of different languages and dialects, which can be crucial to divining the true meaning of online chatter.

The engine must have smart search and alerting mechanisms that seek out unique trends and tags to support analytical decisions. It must move from filtering and finding to smart aggregation through domain-specific ontologies and statistical models while tailored to the way analysts perform their tradecraft.

To go faster than real time, the OSINT engine must evolve beyond just crunching data into a system that can map emerging social, political and economic trends against human geography. It must enable the analyst to compare incoming chatter with an established baseline of attitudes, beliefs and stability in a specific region. As any analyst knows, data are meaningless without context”and the spatiotemporal lens provides that context.

Lockheed Martin is developing analytic applications that map out networks of influence based on regions, topics or people. These influence analytics help answer the critical question of whose voice has the most impact. It's easy to measure volume”to see who speaks the most”but it's far more important to know who can actually spark people to action.

We use a dual approach of a centralized Big Data engine paired with hundreds or even thousands of small, specific apps to give analysts the horsepower they want and the precision they need to sift through an ocean of OSINT. Our large-scale algorithms scour a massive amount of blogs and social media sites, but analysts only need the small fraction of data that applies to their job. So we crunch the Big Data problem with a large engine and filter it out with highly specialized apps. These apps are affordable and easy to produce, but need to be custom-tailored to a geographic region and specific domain.

With the sources identified and filtered, analysts can track messages and sentiments. The analyst must cross reference the relative influence of the individual with the power of his or her network, the response from others, the popularity of the topic and the significance of the message. We're building the algorithms now that will execute that cross-referencing instantly, giving analysts a constantly updated view of social media chatter.

With smart apps plugging into a powerful Big Data engine, analysts can watch those conversations unfold and glean more understanding of what's happening on the ground.

These conversations are key early warning signals of tomorrow's activity. If we can give analysts the tools and the data to monitor them as they happen, we can truly accelerate faster than real time and move into a mode of accurately anticipating future events before they occur.  That's Mission on Demand!

For more information, visit


May Issue 2024