By Michael Ehman, CEO, Cutting Edge Networked Storage (www.cuttedge.com), El Cajon, Calif.
Organizations of all sizes and types are struggling to deal with a relentlessly rising flood of data. This challenge is made especially difficult by the shift toward more unstructured, file-based data, such as satellite image raster data and secondary vector data files, coupled with the need for longer retention times. Many organizations are grappling with a rapidly rising percentage of long-tailed data that must be stored for long periods of time but is rarely accessed after the initial file creation.
An example of geospatial long-tail data is the intermediate data files produced when processing image files into a globe presentation. The fused dataset for a high-resolution globe may approach 100TB, while the intermediate data files required to create the globe may exceed 300TB. Yet about 75 percent of the data required to generate and present the globe application are rarely accessed again.
The costs of deploying and running data storage systems are shifting toward increased long-term operating expense, which further exacerbates the problem of more data that have to be retained for extended time periods. Now the cost to provide storage system power and cooling actually exceeds the cost to acquire the equipment being powered and cooled.
In response to these challenges, there has been a move toward deploying unified storage solutions that provide access at both block level and file level, thereby giving users greater flexibility to handle the full range of data within the same storage systems. The development of today's advanced multitiered, unified green storage architectures in the cloud has helped lay the foundation for dealing with the ongoing data onslaught.
Driving Factors
Several factors are driving users to consider data storage technology in the cloud.
Data storage requirements are shifting toward unstructured files and longer retention. According to industry analysts such as IDC and Gartner, as much as 75-80 percent of data now being generated consists of unstructured file formats. Contributing to this trend are applications such as video surveillance, geographic information systems and video on demand, as well as the relentless worldwide explosion of user-generated information. In addition, as a result of regulatory requirements, more unstructured data is being retained for much longer periods of time.
Companies and IT service providers are shifting toward unified storage strategies. Unified storage is defined as a storage system that's able to provide storage access at both the block level and file level. Block-level storage has been traditionally provided by Fibre Channel via storage-area networks; now Internet Small Computer System Interface, an Internet Protocol (IP)-based standard for linking data storage facilities, also allows servers and users to access block-level data. Network-attached storage allows users to access files over an IP network using network shares. Unifying storage brings these two access methods together under a single platform. Traditional storage systems were islands of block- and file-level data, meaning that two sets of infrastructure and two skill sets were needed to manage the data.
Unified storage enables one footprint in the data center, providing power, space and cooling advantages, as well as a unified management platform, making administration much easier. The unified storage approach allows IT staff to provide internal and external business clients with a greater level of service, including faster deployment of new applications and more flexibility to handle the full range of data requirements.
Tiered storage has become a key enabling technology. The concept of tiered storage recognizes the various different data access and retention requirements by providing multiple strata of storage alternatives within the network. This sets the stage for more cost-effective tailoring of storage hardware to match actual data management requirements. An important key to success for any tiered storage system is the ability to intelligently and automatically make correct decisions regarding data archiving and to ensure transparency and undiminished performance for users.
Primary cost drivers are shifting from equipment acquisition to operating expense. The cost of provisioning power and cooling for IT equipment has been steadily growing to the point where it now exceeds the total cost of acquisition for the equipment itself.
With more than half of total cost now going to power-related expenses and an increasing percentage of those data storage resources being used for long-tail data, it has become clear that major improvements are needed to minimize the amount of power being used for storage data that is rarely accessed.
Planning for the Future
Looking ahead, organizations must have storage solutions that can scale up to meet escalating requirements for a wide range of data types. In addition, these solutions must bend down the rising cost curve with regard to system acquisition costs and power and cooling expenses. By combining a unified storage platform with multitiered archiving and power-saving hardware, users benefit from high-density, scalable storage that reduces power consumption for long-tailed data by up to 90 percent.
