A funny thing happened on East Carolina University’s journey to creating a data-retention strategy: It developed a data-disposal policy.

As part of a compliance project launched about a year and a half ago, Brent Zimmer, systems specialist at the university, was working with attorneys and archivists to determine which data was most important to keep and for how long.

But it soon became clear that it was just as vital to identify which data should be thrown away.

Zimmer was aware of the importance of being able to quickly produce required information during litigation, “but the thing we never thought about was keeping data too long,” he says.

The risk is that by keeping data you wouldn’t normally be required to produce in a lawsuit, you could open the door for it to ultimately be used as evidence against you.

The university had its share of data that was overdue for purging. “We never made anyone throw away anything unless they ran out of space,” Zimmer says. The result: Some users had e-mail dating back to 1996.

East Carolina University is not unusual; many organizations hang on to more data than they need for much longer than they should, according to John Merryman, services director at GlassHouse Technologies Inc., a storage services provider in Framingham, Mass.

One reason is fear. “Companies are really sensitive because there’s a perceived underhandedness to purging data,” he says. “People might wonder, ‘Why aren’t you keeping all your records?’ ”

Another is the low cost of storage. Organizations have historically preferred to simply buy more disks rather than spend time and resources sorting through what they do and don’t need. “Many people would prefer to throw technology at the problem [rather] than address it at a business level by making changes in policies and processes,” says Kevin Beaver, founder of Principle Logic LLC, an information security services firm in Acworth, Ga.

But thanks to the e-discovery risk and burgeoning data volumes — a 20% to 50% compound annual growth rate for some companies — the tide is starting to turn, Merryman says.

Good thing. The average cost companies incur for electronic data discovery ranges from US$1 million to $3 million per terabyte of data, according to GlassHouse. And although you need to pay attention to retaining data, “all indications are that you need to be keeping less,” says Merryman.

A recent report from Gartner Inc. concurs. It states that the current explosion of data is outpacing the decline in storage prices, even before the resource costs for maintaining data are taken into account.

Estimating that the average employee generates 10GB of data per year, at a cost of $5 per gigabyte to back it up, Gartner says a 5,000-worker company would face annual costs of $1.25 million for five years of storage.

And considering that many companies maintain multiple copies of data (such as test data, operational data and disaster recovery copies, not to mention backups), “there’s an explosion of data in most companies,” Merryman says.

Aside from the costs, all those records, if kept indefinitely, can become a gold mine for attorneys looking for evidence.

By Mary Brandel, Computerworld