A wide variety of airborne instruments can be
used to collect high-resolution, accurately georeferenced imagery
and terrain data. Combining datasets from several of these sources
to extract a specific information product to address an
application-specific problem is called “sensor fusion.” Fusion can
be as simple as combining data collected by different sensors with
photogrammetric, geographic information system (GIS) or image
analysis software. Think of this as traditional “data integration”
or “data fusion.” The term “sensor fusion” more specifically
describes multiple sensors mounted in a single aircraft that collect
diverse datasets simultaneously. The datasets then are processed
separately and again combined using software to produce value-added
information products. Finally, there are instances in which source
data streams from different sensors are combined in a “raw” state to
create a first-generation product.
No one remote sensing instrument can collect all the information
analysts need to answer an application problem. By design, remote
sensing instruments act in or are sensitive to particular regions of
the electromagnetic spectrum, usually in the ultraviolet, visible or
infrared wavelengths. Passive instruments measure ambient energy
reflected and emitted from the target surface. Instruments that
gather incoming energy across several broad broad wavelength bands
(blue, green, red and infrared) are referred to as multispectral.
Others that gather and record incoming energy across many narrow
bands within a broad interval of the spectrum are referred to as
hyperspectral. Active sensors, such as light detection and ranging (LiDAR)
and radar, emit energy then measure the amount of energy reflected
back from the target. This article addresses the fusion of digital
cameras, LiDAR systems and hyperspectral systems.
Evolution of Sensors and Enabling Technologies
Recent digital imaging and mapping sensors designed for airborne
platforms have evolved from prototype designs to mass-produced
operationally stable systems. Digital airborne systems provide
superior performance compared with analog predecessors, including
dramatically reduced turnaround times and significantly better data
quality in many cases. There is considerable variability in cost,
technical design and complexity. Detailed specifications for the
commercially available sensors cited in the charts below can be
found at the following links:
Technological advancements in navigation and
positioning of imaging sensors became available to commercial
operations during the 1990s. Airborne kinematic Global Positioning
System (GPS) technology and inertial measurement units (IMUs)
fundamentally changed airborne mapping, greatly reducing and often
eliminating the need for ground control. Direct georeferencing,
based on GPS/IMU integration, became the primary technology for
airborne sensor orientation. It is a required component for the new
active imaging sensors such as LiDAR, as well as for line-scanning
imaging sensors. Although digital frame cameras can be used without
direct georeferencing, the benefit of using direct georeferencing in
most mapping applications is so evident that GPS/IMU can be
considered a standard component for these cameras as well. The
integration of GPS and IMU with the imaging sensor is one example of
sensor fusion, where the GPS and IMU data are required to be able to
reduce the raw sensor measurements into meaningful geospatial data.
GIS and visualization software packages have quickly risen to the
opportunity afforded by the proliferation of digital images and
terrain data. Steadily increasing computing capacity allows for the
implementation of more sophisticated processing and analysis
algorithms, such as orthorectification, automated feature
extraction, spectral classification and 3-D rendering, thus
providing the end user with many possible ways to leverage the
abundance of information in digital imagery. A widely distributed,
variable scale geospatial data infrastructure is growing by leaps
and bounds, some of that growth organized through interagency
agreements and public/private partnerships as well as through
individual Web services. This infrastructure supports the delivery
of data from providers to end users, and allows the users to access
a wide variety of tools for data fusion, complex analysis and
information extraction. Decision makers with little or no specific
technical expertise can access imagery, terrain and intelligent
vector data quickly to help them plan and respond to a wide variety
of events.
Sensor Fusion Applications
There are many innovative uses of sensor fusion for decision support
and application-specific problem solving. For instance, as detailed
in “Transportation Planning—Cut Project Costs with Remote Sensing,”
an article that appeared in the March/April 2006 issue of Earth
Imaging Journal and is online at
www.eijournal.com/Transportation_Planning.asp, wetland
identification was performed with a high degree of reliability using
hyperspectral imagery, topographic LiDAR and digital soils data.
These data were acquired at different times by different aircraft,
but were all of comparable spatial accuracy. The hyperspectral
imagery was used to map vegetation type, percent cover and substrate
types. The topographic LiDAR data were used to help identify
vegetation species distributions in terms of their sensitivity to
elevation, drainage and periodic inundation. Furthermore, the LiDAR
data were used to derive drainage networks and stream channels over
the study area. Soils data then were overlaid on the vegetation and
drainage coverages to simulate the evaluation criteria used by
wetlands scientists to identify potential sites for protection or
mitigation.
Forestry is also an area where spectral imagery and terrain data are
well suited for fusion. Imagery can be used to delineate tree
crowns, as well as identify species type, species health and percent
canopy cover in two dimensions. Tree height is extremely important,
as is canopy structure and stem density. Much research has been done
on the use of waveform lasers mapping the tree tops, understory and
topography of the forest floor. Hyperspectral imagery can then be
overlaid on this wealth of information to answer many question
concerning forest health, structure and diversity, as well as to
qualify forest resources.
Coastal areas are also appropriate for fusion applications, as
coastlines are structurally fragile, environmentally sensitive and
subject to significant infrastructure development. Many early
innovative sensor fusion projects involving hyperspectral and LiDAR
data involved coastal studies. Hyperspectral imagery is used to
measure water quality and vegetation parameters, and bathymetric and
topographic LiDAR sensors map the land/water interface seamlessly,
particularly shallow water areas where traditional hydrographic
survey vessels can’t operate. Topographic data normally are
referenced to a geodetic datum, while bathymetry normally is
referenced to tidal datums. The sensor fusion approach makes it
possible to measure these datum differences directly and allows the
integration of topographic and bathymetric data from other sources.
An interesting approach to sensor fusion is the use of the
bathymetric laser backscatter to measure water attenuation. The
attenuation coefficient derived from the laser data is then applied
to process the hyperspectral imagery, specifically to remove the
influence of the water column and measure only light reflected from
the bottom.
Sensor Fusion’s Increasing Importance
Recent events of global impact—e.g., the 2004 Indian Ocean tsunami,
Gulf Coast hurricanes Katrina, Rita and Wilma, and the earthquake in
Pakistan—have fueled an unprecedented appetite for a bird’s eye view
of the world comprising imagery and terrain data as a background to
vector maps and geospatially enabled databases. The demand from
government agencies, as well public and private managers of
infrastructure and resources, continues to grow as situational
awareness and a holistic approach to decision making is becoming an
economic and political imperative. Sensor fusion brings accurately
co-registered georeferenced imagery to one’s personal desktop, where
simple user interfaces, enabled with visualization and real-time
manipulation tools, make it possible to use sensor fusion principles
to extract valuable geospatial information. Earth Imaging Journal
readers can expect to see great advancements in sensor fusion
techniques in the coming years.
