Previously I talked how the Archive Server provided a wonderful abstraction to storage as seen by Livelink Enterprise Server but that the cost of the product made it a difficult sale because it's core value of ROI required large volume to justify the extra cost. The dirty little secret of ECM is that only some of the so-called managed documents are of the type that produce large volumes. The most notable exception is email but even there, once again the core value to an email archiving system is that of unburdening the email system iteslf. That's why archival systems really only exist for MS Exchange, Lotus Notes / Domino, and other groupware solutions -- it is those applications that require the help, not the underlying content. You don't hear about archival systems for sendmail, postfix and other Mail Delivery Agents because scaling is built into the service ... there is no database to unburden switching the entire problem to one of file systems ... but where email has a large quantity of objects, it does not have a large quantity of bytes (relatively speaking) and so file system based archival systems tend to not be justified.

So, unless you are a very large corporation or you already have the archive server or there is some other reason you have 10s or 100s of terabytes of data then using the Archive Server to save money on your ECM storage is going to be difficult. NOTE WELL, I speak of email archiving here, not email management (classification and controlled disposition) or email monitoring (compliance, legal discovery), those two products offer great value outside of ROI, in those cases the ROI is a bonus).

But all is not lost, Livelink Enterprise Server is still capable of saving money -- the problem today isn't so much that Livelink doesn't have the capability required but that Open Text doesn't understand today's storage market. It is created by developers used to DAS or at best SAN devices and those developers rarely have very much content to contend with in any case -- they have been spoiled with speed and do not really understand why archiving would be helpful. As storage is boring IT work and not high-end business work, Open Text doesn't believe there is enough profit in working that end of the market so there are no business owners with an eye to storage, thus no storage-savvy sales folk, and very few customer service personnel with the knowledge of the current storage technologies such that they can tell you how to save on storage. But I can...

Ever since Livelink 9.5 and the introduction of the Archive Server to the platform, Livelink has had the notion of Logical Storage Providers (the only one it knows about is the Archive Server) along with the traditional External File System. In oder to implement that change Open Text changed the design of Livelink such that it could support more than one type of storage simultaneously and as a consequence, intended or not, it could finally support multiple External File Stores (EFS) natively without the need for the Global Services Multi-Storage Provider addon module.

What wasn't intended and not well known is that there are now storage devices that can be attached to Livelink via the EFS that provide virtually 100% of the benefits of the Archive Server at a small fraction of the cost. If all you are trying to do is SIA (to reduce storage use) and tiered storage (to reduce storage cost) then there are plenty of Network Attached Storage (NAS) devices today that can provide Single Instance Archiving (or the similar Block Level Archiving), offer the use of cheap, high-density SATA devices, and even replication and load balancing ... there is no need to have something as heavy weight as the archive server. Look into something like Dell's EquaLogic line for instance, it is a game changer.

Now, what most people don't realize is that the archive server / enterprise server integration is actually quite weak; Livelink Enterprise Server has no real knowledge of the types of storage available to it from the Archive Server, only what the Livelink administrators have put in as descriptive text. The real "smarts" in the Intelligent Storage Management approach is the Content Move module, it is the one that can scan existing files within the repository and choose to move it to a different storage location based upon rules made up about the metadata surrounding the content. For instance, when it reaches a certain stage in it's RM lifecycle, when it reaches a certain age, if it is a certain size, if it is owned by a certain user or group etc. Unfortunately, as Livelink doesn't track access times except via audit records, it doesn't have the ability to archive according to last access time. It does, however, have the ability to archive according to the arbitrary attribute settings so, while it makes a poor fully-automated archive-only solution, it provides the worlds best MANAGED archival system even allowing end-users the easy ability mark things for archive (and back again) as required.

Content Move is an excellent module that is even smart enough to work within defined time windows so that you can utilize historically spare resources during the night and not affect those same resources during the work day. While Content Move is "an integral part of Livelink ECM Enterprise Library Services" it is an optional module for Livelink Enterprise Server (as of 9.7.1), ask your sales' rep for details on how to purchase/obtain it. Though Content Move is certainly a great tool for automating things, it is not strickly required to obtain significant cost benefits in most cases, but some local custom alternative will be required.

The cost benefits come from using cheaper storage; cheaper storage works because of the simple fact that most stored content will never be accessed again and the older the content is the less likely it will be accessed again. This is no less true for managed content as it is for unmanaged content, the only difference is that managed content should eventually be destroyed / disposed of whereas unmanaged content could live forever. If the storage device you choose to use utilizes Single Instance Archiving, for instance, then you will gain that benefit if you choose to store content on that device.

So, the simplest approach that will eventually lead to the greatest ROI is to simply setup a second and third EFS using different technologies for each -- I highly recommend a NAS solution; I think the days of dedicated storage are nearing the end, they are expensive and wasteful. Where a NAS solution can share a file system, in fact is designed to share a file system, a typical SAN does not ... it shares an hardware infrastructure while dedicating hard drives, backplane bandwidth, and physical interfaces to individual servers (not even services). One really must keep in mind Moore's Law and realize that computers are continually getting faster and more capable; whereas a NAS device from 10 years ago had difficulty with performance at the high-end; the file locking, file system work, and caching technologies have all benefited from both capacity and speed increases that make yesteryear's performance issues much less likely to appear.

You might even consider the ultra-low end external USB drives, at capacity ranges of 1.5 and even 2TB for often less than $100/TB they are cheap enough to replicate and offer an online long-term backup device; the bandwidth is terrible of course but they can be an effective part of your overall strategy as well. The biggest problem of archiving isn't the storage it is in finding what you have archived and that is why you need something like Livelink, something that can index whatever it is you want to archive. Now let's be clear, do NOT attempt to extract files from a USB archived file system in order to index them, unless you have only a few hundred files, the operation will simply take too long and you'll have almost no access to the drive while you do. You need to use the USB archive for content that is only indexed via the metadata.

Clearly there are limits to what can be done, USB does not provide the bandwidth to sustain much more than a steady trickle, but a steady trickle is often enough to reap large benefits! All you need to do is to feed the content into Livelink ... File System Archive is a great tool for that but again comes at a cost; using WebDav as an interface into Livelink, any webdav-enabled file system copying tool will do; a simple LAPI program run from a script could be a poor-mans feeder; and there is always the ability to use the Enterprise Library Services API and SOAP things over. Just ensure that the documents are put into the storage provider appropriate for the content and then let Livelink do its thing.