By Robert Mott, Intergraph Government Solutions (www.intergraphgovsolutions.com), Reston, Va.
s today's military, public safety and intelligence organizations fight terrorism and combat weapons of mass destruction and their proliferation, it's increasingly important that they expand their data collection and analytical capabilities into new and innovative methods. To this end, there's increased interest in incorporating full-motion video data sources, such as those from unmanned aerial vehicles (UAVs) and other unmanned aircraft systems (UASs), into strategic and tactical analysis workflows. UAVs/UASs already have been instrumental in providing real-time surveillance and operational support to military and intelligence organizations. Now they're being looked at more closely as a viable source for public safety organizations, with closer attention paid to post-collection analysis and integration with other forms of intelligence.
Recent advances in the development of lightweight, yet durable materials; advanced communication and navigation systems; computer hardware and software; smaller, yet more sophisticated sensors; and improved fuel efficiency contribute to the rapid growth of UAV/UAS technologies. This increased usage makes it more critical than ever to establish a reliable and robust infrastructure ground-side that can effectively collect and manage video and other data, then make it seamlessly available to a wide variety of consumers for integration, analysis and visualization. This responsible focus on the back-end system ground-side will substantially increase the value and applicability of these systems well into the future.
Not Just for the Military Anymore
The Global Positioning System (GPS) and the Internet are two important examples of military technologies that have transcended to the civilian space. These two technology sets have dramatically transformed government, business and personal lives. The UAV/UAS technology set is also expected to have a substantial impact in nonmilitary sectors and will experience rapid growth in the coming years. This includes widespread applications for law enforcement, firefighting, and numerous environmental and scientific purposes. The Federal Aviation Administration (FAA) currently is working to establish airspace safety regulations that would allow the systems to achieve wider usage. Some examples of civilian applications include:
¢ Natural disasters and other catastrophes (forest fires,
hurricane flood damage assessment, earthquakes, etc.)
¢ Search and rescue (land and sea)
¢ Monitoring natural resources
¢ Delivering relief supplies and other payloads
¢ Harbor, coastline and border security
¢ Monitoring and inspecting long expanses of pipelines and other forms of critical infrastructure
¢ Establishing ad hoc communication relay networks
High-Level Workflow
A typical high-level workflow consists of four key areas that must function in harmony to properly support the end-to-end requirements of the mission. Technology must be deployed prudently in each of these four areas to support the rapid transfer of data and analytical results to other areas. As shown in Figure 1, these areas are:
¢ Automated and manual capture of geospatial information, including imagery, video and other sensor data
¢ Management of enterprise geospatial content, including traditional vector data sets (layers, features), imagery, video and terrain models
¢ Integration and analysis of multiple overlapping sets of geospatial and nongeospatial information
¢ Visualization and dissemination through a variety of interfaces
It's important to note that standard formats for imagery, full-motion video and other data types are essential for the smooth transfer of information from one workflow segment to the other. This gives system designers improved flexibility as to what type of technology they choose to deploy at each stage. Proprietary formats limit the choices of technologies and force the selection of compatible technology elements at other points along the workflow. Many geospatial technology companies are embracing open standards for data storage as well as for open dissemination of geospatial information, such as through Open Geospatial Consortium (OGC) Web Services. This methodology encourages innovation throughout the workflow by allowing a seamless upgrade and integration of technologies.
The Power of Geospatial Fusion
Video-based data sources provide the most recent view of a situation and can augment other forms of geospatial intelligence (GEOINT)”such as satellite imagery and aerial photos”to provide a richer, more detailed view of the area of interest. To effectively use video as a source of intelligence, however, an analyst needs to seamlessly fuse the video with other types of intelligence, such as map features and annotations, as shown in Figure 2. This is highly beneficial, as these other sources can help orient the analyst's point of view and improve understanding of the activities occurring within the video by eliminating the tunnel vision effect of viewing the video in a dedicated video window.
Such analysis and integration can greatly assist in properly orienting the analyst and provide insight into actions and objects occurring within the video, as shown in Figure 3. In many cases, the UAV-based video represents the most recent information pertaining to the battlefield and so, when effectively integrated into an existing intelligence analysis workflow, it can yield a powerful analytical advantage and serve to make future planning activities more effective. It's important to understand the advantages and disadvantages of UAV video sources and to then effectively incorporate those sources into existing architectures.
The geospatial layer then serves as a backdrop for integrating the video sources with additional forms of intelligence, such as human intelligence, signals intelligence, and measurement and signature intelligence. The need exists to geo-fuse and correlate these disparate intelligence sources to analyze, understand, and visualize information and then derive intelligence value in innovative and exciting ways.
Then the blending of georeferenced video with other forms of geospatial information can allow the video source to be a source of geospatial data collection. It may be used as a source of updated information as to the location or portable assets or characteristics of existing features, such as the length of a runway. Analysts may use the source to update attributes of existing features, such as the operational status of an industrial plant or bomb damage assessment on a target of interest.
Geospatial Content Management
Other important factors in using video for analysis is the ability to easily query vast video archives for specific clips that meet an analyst's search criteria and to rapidly deliver the results to the analyst's exploitation environment. Some of these criteria may include sensor type, date and time of collection, geospatial extents of the video coverage and keywords that directly describe objects or activities occurring within the video.
The key to providing this type of robust query ability is establishing an enterprise content management system that handles the variety of video formats, along with other complex file formats, such as satellite imagery and elevation models. The end result is a scenario in which an analyst can quickly zoom to a part of the world”possibly even down to street-corner resolution”and instantly queue segments from collected video from the last 10 days that pertains to that street intersection. Then the analyst can fuse those results with satellite images and signals intelligence and effectively plan an activity, such as the routing of a convoy.
Although the growing volume of imagery, motion video, and other location-based sensor data provide greater coverage and information, they also create data-handling challenges. Traditionally, analysts maintain data in a file-based format through CDs, tapes and other electronic media. The data-management challenges that arise from this practice can easily overwhelm the analyst and impose limitations on data sharing. In some cases, analysts can spend more time using Internet Explorer to search through a file/folder hierarchy than they actually do analyzing the image. To further describe the problem, image data often are copied directly to an analyst's hard drive without important metadata that describe the image's life cycle (when it was acquired, when it was modified and by whom, and when it should be retired and archived), as well as duplicating storage and cluttering the disk space, requiring the analyst to spend precious time cleaning house.
Technology is available to automate the management of large amounts of satellite imagery, motion video, aerial photos, elevation data and other digital files that are essential to the GEOINT exploitation workflow. For example, Intergraph's commercial off-the-shelf (COTS) solution for enterprise image and elevation management, TerraShare, can provide multiple users with transparent access to large amounts of common imagery (Figure 4). This can greatly reduce the time from collection to exploitation while improving efficiency, collaboration and quality. This allows for the imagery to be indexed and cataloged, making that imagery discoverable to end users performing manual searches for imagery, as well as to image exploitation applications that need the most recent set of images covering an area of interest.
Just as imagery ingest and management are the keys to more effective imagery use in a time-sensitive environment, the same holds true for collected video. For the video to gain increased usage and value after real-time collection and surveillance, it must be tagged in a way that it can be easily retrievable in the future. For the video to integrate seamlessly, it will need to be tagged by date/time, as well as geographic context. In some cases, depending on the size of the video clips, some of this tagging may need to happen on individual frames rather than the entire video clip.
Automated Geospatial Content Ingest
Although today's image management systems provide robust solutions for storing and distributing data, many organizations still need methods and technologies that ease the administrative burden of finding, preparing and uploading the data into these archives. These processes can consume a significant amount of the operators' and analysts' time. As larger volumes of high-resolution data are being collected, and as the turnaround time for results is being compressed, it's essential the image ingest part of the overall workflow is automated to the greatest extent possible. Today's fast-paced, dynamic environments also demand around-the-clock monitoring and processing of new data. It's clear the only way to ease this burden and realize the full value of these extremely expensive and complex collection assets is to implement a mechanism for automating the ingest, organization and preprocessing of new imagery as it becomes available.
Several systems are available today that completely relieve analysts of the many tedious tasks required to effectively process, organize and exploit image and full-motion video data. For example, TerraShare Automatic Data Ingest automatically generates reduced-resolution datasets, histograms, thumbnails, footprints and necessary image metadata for images as they're ingested into TerraShare. As an additional upstream process, the system can be configured to process full-motion video. The result is that everyone in the organization has direct access to imagery and video that is ready to exploit upon arrival, minimizing the time required to search for and process imagery and maximizing the effectiveness and productivity of the entire organization. The confidence level in the analysis results and exploitation tasks dramatically increases as individuals know beyond any doubt they are working with the most recent and relevant data possible. This confidence results in significant time savings and more trusted analyses.
The ingest service allows the user to identify specific folders that are watched for incoming data. As data are received in these folders around the clock, they're automatically processed without human intervention and placed in a user-specified location with notification to the analyst that new data are available. The actual data file is stored one time only, and multiple references to the image are handled via pointers to the original image. This enables the maximum exploitation of time-critical imagery through automatic processing, relieving operators from tedious tasks and optimizing storage space. Tools allow the user to define the metadata configuration and auto-populate metadata from imagery headers and user-specific sources. This provides the maximum exploitation of current and historical imagery and video by capturing user-specific metadata that may be used as search criteria.
Forensic Video Analysis and Real-Time Quality Enhancement
In some cases, even with proper content management tools and powerful data integration and visualization tools, poor-quality video can hamper the analytical process. In some cases, the original captured video is of poor quality or is unusable due to flight path, altitude, tilting and buffeting of aircraft, and other factors. Therefore, it's important for analysts to use technology that performs real-time enhancements and corrections on the video, such as removing atmospheric distortion, correcting for shadows that affect brightness and contrast, and stabilizing jittery video.
Many of today's video analysis systems increase the usefulness of UAV video in a real-time and forensic capacity. By providing the operator with the ability to set thresholds, tolerances and other parameters, such video enhancement can be done in batch or interactively, providing multiple opportunities for handling the ever-increasing volume of video that needs to be processed for analysis.
Forensic video analysis can provide much-needed detail into the content within captured video. In some cases where the original captured video is poor quality”possibly due to poor lighting conditions or movement of a vehicle-mounted camera”post-processing is essential for gaining appropriate intelligence. Change detection between video frames can quickly direct an analyst to a specific point of interest within the video.
Generating Georeferenced Imagery from Video
A key step toward achieving the fusion of a video data source with other forms of intelligence is generating a georeferenced image as the result of stitching together or mosaicking hundreds or thousands of individual video frames. Several companies are developing applications that automatically generate this georeferenced image, which then can be seamlessly integrated with other forms of static data, such as aerial photos, satellite imagery or geospatial layers and features.
This process can dramatically improve the clarity and accuracy of the video, improving the ability to perform analysis on the video. Such video mosaic capability can provide a mechanism to glean additional details from the entire collection sequence that otherwise couldn't be obtained from individual frames. In cases where satellite imagery or aerial photos aren't recent enough, this process will provide a more current representation of an area that then can be compared with previous images to form a foundation for change detection from video sources. This also reduces the workload for analysts, as they can view the finished product more quickly and completely than by viewing the video in sequence. Because the resulting image is made up of multiple frames that overlap to some degree, the mosaic can provide a clearer representation of the coverage area, which is essential for exploitation. Figure 5 illustrates the process of generating a georeferenced image from UAV video.
It's important to retain the original raw video for additional analysis and integration, but this by-product can better orient the analyst and make the video source a viable candidate for geospatial analysis. The resultant video mosaic can serve as the basis for updating geospatial features, such as airfields, roads, bridges and other types of critical infrastructure.
Expanding Capabilities
As defense and intelligence organizations work to expand the use of motion video sources for broader purposes, it's important to appreciate that many civilian organizations also are incorporating video data sources into their existing processes and systems. Many civilian organizations of the federal government, as well as state and local governments, are establishing offices and programs to address unmanned aerial systems. As these systems become more reliable and economical, and as policies are implemented, a vast new array of new and innovative applications will emerge.
To make the most effective use of aerial video collection in a civilian and military context, it will be important for these organizations to implement the types of technologies that provide reliable enterprise data management, fusion with other forms of geospatial information, cleanup of distorted or jittery video and superior analytical abilities. The combination of these components is the key to providing the right information at the right time, achieving improved analytical quality, performance and superior decision making.
Similarly, military and intelligence agencies are faced with the need to adapt to wider-reaching demands and quicker response times than they have in the past. Furthermore, they're assimilating and analyzing more available data than ever before, such as high-resolution imagery, real-time video, and GPS-tracked objects. Never before has there been a greater focus worldwide on security and emergency preparedness.
Today's military and intelligence agencies must also meet the expectations of people and organizations dealing with natural disasters, an unsettled economy and devastating global events. Therefore, they need to quickly and effectively collect and analyze relevant information that helps make sense of current situations and reduce conflict around the world. With the continued evolution in technology, such as service-oriented architectures, advanced geospatial applications, mobile technology, and speed and method of transmission, now is the time to examine powerful and intuitive GEOINT solutions that can help military and intelligence agencies be more effective and cost efficient.
