Is information lifecycle management (ILM) another canny marketing acronym or genuinely a technology that can help companies with their exploding data
volumes? Jason Stamper investigates.
The amount of data businesses are storing is exploding. Estimates vary, but it is generally agreed that volumes are doubling every two to three years. Blame e-mail, more granular customer data retention, the Internet or even compliance initiatives; it really does not matter. The growth in storage is set to continue, analysts tell us, and companies are just going to have to deal with it.
So if the amount of information that businesses are storing is expanding rapidly, should they expect to be spending an ever-greater share of their IT budgets on storage? “Yes,” says Paul Brown, IBM system storage director for the UK, Ireland, Central and southern Africa.
“Although IT budgets overall remain flat, the portion for storage has grown from 13% in 2002, to a 22% average in 2006. More of a focus on process, organisation, technology and governance will enable companies to be more efficient and to align that increased spending with the business value of the information.”
Of course, one could argue that with the cost of enterprise storage per megabyte falling each year, companies can solve the problem by throwing more and more hardware at it. But as EMC’s chief technology officer, Jeff Nick, told CBR last month, that is not going to help the typical enterprise: “It is not just about the physical storage of the data. It is about how companies are going to find the data they want, how they are going to manage it, mine it, analyse it. It is about finding the right information, when you need it, for the right purpose.”
In response to this dual challenge – exploding data volumes coupled with the desire for faster, more relevant and contextual data – the technology industry has put its weight behind information lifecycle management (ILM). It is a term trademarked by StorageTek, but not enforced, so anyone can now use it.
As Chris Reed, a consultant at Morse, points out, “ILM as a concept has been talked about for years. But our experience is that it is still not that well adopted. Many companies move data from expensive storage to cheaper storage but they do not follow a true information lifecycle pattern.”
That is probably because ILM, in its earliest incarnation, was really an imitation of the hierarchical storage management of the mainframe world, adapted for open systems. With HSM, vendors proposed a system of tiered storage, from the highest performance and most expensive at the top of the pyramid, to the least responsive and cheapest at the bottom.
What dictates where a piece of data resides within that hierarchy is not, as in the class system, its birth, but rather its age, current relevance, last time accessed, business-criticality and a series of other variables. What moves it automatically between one class of storage and another should be a software platform enforcing policies set up by the company that owns the data.
HSM could do that on the mainframe many years ago, and ILM at its most basic still does that today. There is nothing wrong with that. It still has the capacity to save companies a great deal of money on their storage costs and make the management of different classes of information far easier.
But in more recent times, ILM has gone through its own metamorphosis. Faced with the fact that today it is not only a case of storing a large quantity of data and accessing some of it often, and some less often; vendors have evolved their ILM technologies to be far more sophisticated than just automated storage tiering platforms.
The essence of ILM
“Companies often move data from one tier to another in one or two steps,” says Reed, “but the huge growth in unstructured data and compliance means that companies have to look at data in a different way. You can no longer just archive to tape and forget about it. You need to be able to access it, manage it and even delete it at some point in the future.”
That, in essence, is ILM, and at least part of the policies dictating where each piece of data lives will these days be set by the need to comply with a set of rules, regulations and laws, varying by country, area of activity and other factors.
Now, many companies can offer tiers of storage hardware, from the aristocrat (Fibre Channel disk with full failover, high availability, mirroring and replication to remote sites for disaster recovery), to the ‘untouchables’ (open systems tape). Some will even claim to have the software needed to automate the migration down, and even back up the pyramid.
For example, if that e-mail from four years ago is material to a brand new lawsuit, it had better be copied from tape back to high-speed disk.
But as for the more holistic approach to ILM, which sees data managed according to its exact nature from its creation right through until its eventual archival or even deletion, that is still something of an emerging discipline, with different approaches from different vendors playing out against one another to see which gains the greatest market acceptance.
Most vendors are on their way towards this ILM nirvana, however.
As EMC’s Nick says, “ILM is definitely evolving to far more than HSM. ILM was initially seen as a way of automating data movement, it is true, but what has really changed is that now people need more intelligence.
“They need more programmatic ILM capabilities: for instance, classification in support of policy-driven actions; goals like recovery time objectives; securing the information in rest as well as in flight, and so on.”
According to IBM’s Brown, there are good reasons for looking to ILM, and the ability to lower or at least get a grip on costs is possibly the easiest to sell to the board. Says Brown, “ILM is proven to be able to reduce the dramatic rise of storage costs; reduce personnel costs associated with managing data (typically one head for every three terabytes); improve utilisation of storage assets (typically from below 40% to greater than 80%); help companies avoid financial penalties associated with regulatory non-compliance; and reduce on-going costs of data management.”
But another factor that has driven the interest in ILM is the great rise in unstructured data: namely the kind of data not usually found in the neat rows and columns of a relational database.
Unstructured data consists of e-mails, Word and other Office documents, graphics files and basically anything not in an RDBMS.
As IBM’s Brown points out, “More than 85% of all business information falls into this category. It is not that e-mails, documents or presentations are ‘unstructured’ per se; they just lack being part of relational database systems that provide advanced search and query features. IBM has led a new open standard called unstructured information management architecture (UIMA), that it has built into its technologies to enable such advanced search and management.”
Again, most vendors see the management of unstructured data and ILM as practically one and the same thing. So ILM is most definitely not just an evolution of HSM; it is also about how a company manages all of its unstructured data. How it manages dictates whether it is covered by security and compliance policies, and how it can be effectively delivered to those that need it, when they need it.
“There are more and more forms of data,” says EMC’s Nick. “So companies are having to take more decisions on how that data is accessed from other applications, how it is stored, what quality of service needs to be provided, policies, securities and more.
“Policy-driven is one of the big characteristics of ILM and the big driver here is definitely compliance. It is not just about keeping information lying around any more. It is also about being able to do a legal audit, the data’s discoverability, retention and more,” says Nick. “It is not just getting data to users any more. There is a significant burden for companies to provide compliance.”
As you can probably ascertain from that quote, there is far more to ILM than simply buying an ILM solution and hoping that the problem is solved. There is also a major human element, which is why analysts will tell you that ILM is not just a technology, but an entire discipline in its own right (see below, Analyst’s view).
IBM’s Brown agrees that, “It is definitely a discipline, a strategy for running a business more wisely. ILM is comprised of the policies, processes and tools used to align the business value of information with the most appropriate and cost-effective IT infrastructure.”
An extract from ‘Information Lifecycle: a Discipline Not a Product’, by analyst firm Butler Group:
ILM is not a discrete product, solution or technology. It is in fact a value-based discipline comprising systems, policies and processes. However, technology does play a significant role here, and so organisations must eventually consider products and technologies as well as information management theory. An organisation’s ILM strategy must take into account structured, semi-structured and unstructured information, as well as desktop, server, midrange and mainframe systems.
Business managers and senior executives must now put ILM on their agendas, albeit within a context that ensures understanding. With this group of users, IT management should avoid all talk of hardware and software. Instead, the focus should be on business information value management.
An organisation’s ILM strategy must at some point be reflected in hardware, software, products and technologies, but this should always be linked back to the organisation’s information management strategy.
Those responsible for implementing an organisation’s ILM strategy should avoid complexity wherever possible, even if that means sacrificing some level of functionality, as Butler Group believes that complexity inevitably leads to increased total cost of ownership.
Each vendor has a somewhat different view of ILM and exactly how it can be achieved. A former storage box-maker such as EMC, with its acquisitions of content management firm Documentum, security firm RSA and many others, shows just how seriously the market is taking ILM. EMC sees itself now as an information management company, not ‘just’ a storage firm. It is not the only one. Most vendors realise that the challenges of storage and information management today require a more modern approach to data management, from cradle to grave. For the time being, that discipline is being called ILM, and it is on the verge of becoming a ‘must-have’ for nearly every enterprise, while for some it is already long overdue.