For my entire career in Computer Science I've been fascinated with all things related to storage: File Systems, Magnetic Tape, CD, DVD, Paper Tape, NAS, SAN, DAS; it is all interesting to me. Thus it should come as no surprise that I've been asked a number of times on how to save on storage costs when using Livelink ECM-based solutions. When I was working at Open Text I was a bit hamstrung with my answers in that I had a responsibility to push particular Open Text products and solutions and so while my options were honest in the information I did provide, there were sometimes things I didn't say that I might have otherwise as it might not have lead directly to a new sale. But before I do give some of those alternate answers to this question, let me take a moment to describe the very good product that Open Text does sell called the Archive Server.

The Archive Server came from the IXOS acquisition where it was a product called ECR Server. Where Open Text pretty much from inception has been focused on what I call a "front-door application" where it tries to bring value directly to end-users (originally via searching, then to document management, into Records Management and today in many solution-oriented, specific applications like Well File Management or Internal Controls), IXOS started out as what I call a "back-door application", adding value to an existing front-door application. In particular, IXOS provided archiving solutions to the financial software heavyweight that is SAP allowing that beast to utilize less physical resources (relieving that database from content means that the resources required to run the database are lessened which means less software licenses for the DB and SAP -- not having to buy a new server is a BIG value to IT) but most end-users had no idea IXOS's solution was even available let alone being used. It's value is derrived though efficiency and cost savings, in other words on a Return On Investment in the archive server.

The Archive Server is a great product in that it provides a storage abstraction that allows for the use of tiered storage solutions and long-term archival solutions. It has some great, state-of-the-art storage reduction techniques (that aren't often used optimally) like Single Instance Archiving and, with the right front-end application the Archive Server can be used to actually move content from expensive Tier 1 storage through to Tier 2 (ie: online SATA arrays) and Tier 3 (long-term storage). It truly is a great product, unfortunately for the world at large, Open Text has chosen to price it at a point that makes it very difficult to justify for most shop looking for an archiving ROI (and, as I said, ROI is the only value statement in an archiving strategy). And the problem with large license costs is that it means your savings after implementation must be very large or your ROI is small or even negative.

Livelink Enterprise Server can utilize the Archive Server in various ways, and sometimes it has huge value like in Email Management or when a program of Intelligent Storage Management is put in place. Unfortunately for Open Text's customers, Open Text is woefully ignorant as to how the storage industry works and so has not made any headway into the storage market with their superior offerings. And it may now be too late for them to do so because far from being static, the storage industry has undergone and is continue to undergo massive change.

Storage was not always as it is today, as one of the very few mechanical devices within a mostly digital, solid-state domain, storage was traditionally the most failure prone component in any server and/or IT data centre -- it was virtually the only compoent that came with a listed Mean Time Before Failure (MTBF). With most storage devices it isn't a matter of if it will break but when it will break. Great fortunes have been made by the likes of EMC², NetApp, Hitachi, IBM, and others charging a significant premium providing redundant and fault-tolerant storage solutions using mostly commodity devices. Proprietary firmware, backplanes, and operating systems were built and marketed as mission critical and large sums of money started to go to these hardware giants. And, speaking as an IT administrator of the day, it was money well spent as data loss and downtime are two words administrators do not like!

There was no real reason to change this situation from the perspective of the storage vendors and the commodity device vendors (Seagate, Winchester, etc) because there was so much money to be made. Not only did the fault-tolerant nature of the offerings demand a hefty maintenance stream (so there was always recurring revenue), as hardware the devices would have to be replaced in 3-5 years but our appetite for digital content continues to increase dramatically. You'll appreciate that to be in the storage business in the 80s and 90s was a great thing to be!

In that period CPUs and other solid-state devices saw many orders of magnitude increases in speed and capacity (and similar decreases in cost), but storage capacity, speed, and cost seemed almost unaffected. What we saw this past decade was a large number of new storage vendors that found they could do in software what the big-guys have been doing in hardware, and they could do it just as well and for a lot less money! Today storage is nothing like it was 10 years ago; today $100 buys 1 Terabyte of externally available storage (not server class speed of course) and while the density of hard drives remained fairly low for a few decades, it is now commonplace to find 250GB SATA drives and a 2-Terabtye iSCSI-enabled, shared storage switch that provides replication and balancing, can be purchased for just a few thousand dollars ... often for less than the cost of the proprietary HBA boards required for utilizing tradtional Fibre Channel-based SAN solutions.

Things certainly have changed and all the economics have changed along with them. There is still a need for high-speed, fault-tolerant storage and there always will be, but the days of requiring expensive, proprietary hardware are now gone ... iSCSI is solid and fast (FibreChannel is dead technology), commodity servers are stinking fast, dirt-cheap and make great file servers offering software and hardware-based RAID (I'll point out that while you certainly can spend money on Linux support, it still remains possible and feasible to use this wonderful and capable operating system for free), the days of "special firmware" being required for the drives are long gone. In short, there are now options available for the small, medium and large business that do not require the use of the big boys of storage. Even the more complex and advanced techniques like replication, load-balancing, and single-instance-archiving are available from many different sources and has driven the cost of storage down which, of course, only makes it harder to justify yet another layer and cost like the Archive Server.

I'll follow up this post with another one putting these pieces together ... showing you how to use your existing LES with these new storage solutions and potentially saving a lot of money doing it.