Exploiting Time-Derived, Activity-Based Intelligence


A time-based metadata model correlates many data sources temporally.

Historically, geospatial intelligence (GEOINT) analysis focused on scheduled, precise, preplanned reconnaissance against known targets—countries or actors with the capability to threaten U.S. interests with military force. Today advanced communications systems and access to information make terrorism and irregular warfare more dynamic and unpredictable than our historical view of strategic threats. Emerging tradecraft for counterterrorism and irregular warfare requires integration of new sensors like persistent wide-area motion imagery (WAMI) with pervasive, timely, real-time sensors like full-motion video (FMV) and signals intelligence (SIGINT) to provide actionable intelligence. Time-derived, activity-based intelligence is a new perspective for understanding the activities of everyone, everywhere, all the time (real-time intelligence) and addressing fast, fleeting, hidden and unpredictable adversaries.

Correlating and integrating real-time sensor data in a common spatial and temporal reference frame is an enabler for the intelligence community to address emerging threats. By moving seamlessly between geospace, cyberspace  and activity-space, analysts can understand and exploit the dynamics of human behavior, activity, entities and systems that are nonphysical, such as transactions, interactions, social networks, human geography, etc., to enable a vision for activity-based intelligence. This approach requires treating precise location and tagging  of time equally with the notion of space.

Multisensor Intelligence

Military intelligence analysis traditionally focused on detecting, identifying and tracking physical objects of interest such as vehicles, aircraft and other physical entities where a static picture and some geospatial context was often enough to develop knowledge and understanding. Today the emergence of new real-time information sources and the explosion of open-source intelligence (OSINT) are driving a new forward-looking intelligence strategy. Multisensor intelligence consists of raw data sources such as audio, FMV or WAMI and prerecorded data such as static images or photographs, as well as derived products such as 2-D or 3-D models, SIGINT, ground moving target indicator (GMTI), video moving target indicator (VMTI) and human intelligence (HUMINT) data. OSINT, such as cell phones, Internet, Twitter, Facebook and other social media, is increasingly considered strategically significant to the intelligence picture.

These data sources are captured and processed on different time scales (some are real time, some are not). Furthermore, the utility of these sources—and their integrated products—depends on a mission-specific temporal need. The demand continuum describes a progression of temporal states, ranging from anticipatory, or before-the-fact analyses, to historical, long-after-the-fact analyses. This time continuum also suggests several types of activities that span from scenario planning and threat assessment to forensic analysis and war-games modeling. Managing data across this time continuum requires a digital multisensor asset management repository, which provisions assets to satisfy each of these time-sensitive, need-to-know situations. As you move from a historical to an anticipatory intelligence environment, dealing with real-time sensitive data becomes more important.

The solution to this activity-based intelligence problem entails significantly more than just linking platforms and assets to each other. It requires new methods to ingest, manage and process raw and derived metadata; new approaches to metadata correlation; and accurate visualization of these data sources based on time and geolocation. Temporal digital asset management (TDAM) is a well-established practice in the media and entertainment industry that leverages metadata to solve such problems. TDAM emphasizes the systematic extraction, processing and creation of metadata and enables reusable or referencible digital media objects that are bound by one or more temporal references.

It’s All About the Metadata

The definition of metadata is “data that describes other data.” Think of metadata as the data that provide the intelligence derived from a piece of content. Accurate metadata drive the primary value of any one piece of multisensor intelligence. The lack of key metadata or inconsistent sets of metadata undermines the value of multisensor intelligence.

One of the biggest impediments to correlating multisensor data sources is the lack of georegistered geospatial and temporal metadata, as well as contextually derived metadata, which limits multisensor data use. An extensible, time-derived data model addresses this problem and improves usability by integrating GEOINT from a temporal perspective.

Temporal Data Model

The concept of a temporal data model is common to the media and entertainment industry but not to the intelligence community. In the media industry, every video or image frame is treated like a container of information. TDAM systems time-tag each frame of imagery, providing a “frame accurate container” to correlate metadata. TDAM systems provide a powerful data model for the correlation of real-time or off-line multisensor intelligence data. Most data sources provide three methods of correlation of metadata: absolute time, mission time and frame time (note that not all data sources use frame time). The accompanying diagram demonstrates the basic concept of a time-based metadata model where many sources of data are correlated temporally. The diagram also represents the three most binding methods of metadata tagging: time, geolocation and events. The temporal-based event metadata tagging construct is based on the concept of continuous time. In this case, events are created as snapshots in time (such as a 9/11 terrorist event). Each event can have many sub-events, which can be backward or forward in time, and, within any given event, “eventlet-based temporal metadata markers” point to content relevant to an event from internal or external systems.

There are five critical workflow processes to a TDAM system: acquire, process, manage, distribute and consume. Critical to temporal metadata correlation is the ingest process of normalizing data sets to create accurate and uniform geospatial, time or event references. Once ingested, content is registered properly so it can be processed with analytic engines to create derived products and additional descriptive metadata. Each phase of the workflow process suggests a number of opportunities for automation and on-demand access to sensor data, digital information and the creation of intelligence packages. Workflow automation becomes the key to unlocking the potential to correlate multiple sources of information on-line or on-demand through user-driven analytics.

A TDAM solution for event-based tagging allows users to tag, manage and discover data and information based on temporal, geospatial and activity tags. While this dynamic, nonlinear workflow is new to the intelligence community, the “Facebook Generation” finds this tradecraft intuitive, easy and fun. By allowing users to interact with a time-based data model to get what they need from exactly where and when it happened, relevant integrated information can be more readily disseminated to handheld devices and low-bandwidth consumers. By leveraging best practices from commercial industry, TDAM solutions provide key capabilities to enable a time-derived, activity-based intelligence enterprise and put the power of real-time temporal GEOINT into users’ hands.

For more information, visit Harris at http://www.govcomm.harris.com/solutions/products/isr/geoint.asp