Deliver business-critical information to anyone, anytime, anywhere.

Rocky and Bullwinkle on the Information Life Span

Hokey Smoke! How Long Should We Keep This Stuff?

by Lou Washington - Cincom Systems

Rocky & Bullwinkle Hokey Smoke!

A few weeks back I wandered onto a ComputerWorld Blog that was asking if we should try to permanently keep certain kinds of information. I felt compelled at the time to throw in my humble opinion on that subject so I posted a reply.

My feeling was that most information was useless after a short period of time. That permanent retention of information should be limited to researchable scientific data and valuable cultural items such as Rocky & Bullwinkle cartoons.

After reading some of the reactions to this question and also to my own particular take on the subject, I thought perhaps a more detailed discussion was warranted.

My first career in the information management world was within a Records Management context. Records Management is a field that really solidified in the years following World War II. At that time, the U.S. federal government had been packing away information, mostly paper documents, for nearly 175 years. Most of this information was stuffed into any space not used for something else.

No one. Not one. No one!

No one ever threw anything away because no one was authorized to throw anything away. File cabinets were purchased by the train carload. When these were filled, department bosses would scratch their heads and look around for empty space. Files were pulled out of office environments, stuck in any old container that was handy and wheeled away to the nearest attic, basement, garage or stairwell where they would remain for eternity.

Most people don’t realize it, but the original design of the Washington Monument included file drawers from the sidewalk level all the way up to the viewing windows at the top. Another obscure fact is that NASA’s original mission to the Moon was to evaluate it as a potential records storage site. They figured, plenty of space and not a lot of moisture ... it was ideal! You won’t hear much about these ideas because the people responsible for them now work in Area 51 with aliens on their storage issues.

Finally someone had the presence of mind to ask the question, “ ... how long should we keep this junk?” Thus, the field of Records Management was born.

Initially the practitioners of this art were concerned primarily with paper files. This did change to some degree as other technologies were developed, such as the use of microfilm and the digitization of data. Unfortunately, many Records Managers did not actively involve themselves with their organization’s IT operations and hence they suffered from the perception that their only value was related to the maintenance of paper based data. Sort of a souped-up file room supervisor for the entire enterprise.

The Records Manager and the DP/MIS Director were frequently political enemies in competition for the same funds. Let’s face it folks, machines with blinking lights and spinning tape reels will beat out file cabinets every time. The glamour factor is powerful.

Sure, Records Managers brought us Microfilm. It had and still has some application today, but by in large it is extremely expensive and really doesn’t get the job done for many active information based processes.

So, our friend, Mr. Computer won the battle if there ever was a battle to win. Storage media quickly evolved with capacities that were and still are almost mind-boggling. This evolution continues and will continue for longer than I would care to predict.

Unfortunately, those Records Managers that were reluctant to become involved in this now essential aspect of most organizations, have no voice to deliver the answer to the question that is still being asked, “. . . how long do we have to keep this junk?” And, that is really a shame, because they have the answer!

People have come to not care how long we keep information because the cost to maintain that information was greatly reduced via high-capacity storage media and inexpensive storage facilities. But, as any competent Records Manager will tell you, storage costs are only part of the equation. Just because you can keep something around, does not mean you should keep something around.

All information, regardless of format or media type must be evaluated and maintained according to a corporate or organization-wide plan. The evaluation process must be a documented repeatable internal process. You cannot go out and buy a book that will tell you how long you must keep a particular type of information.

This is particularly true in light of the tendency many us of have to keep all of our “important” papers around forever, preserved for posterity. I know I have tons of stuff laying around that I just know future generations will want to pore over when visiting the Lou Washington Library of Stimulating Thoughts. My friends are planning to construct this shortly after I pass on to the hereafter.

The fact is that most information has a specific useful lifespan following which it becomes a liability.

The bane of all Records Management programs is the almost pathological need of some folks to store things permanently. This brings us to the original question posted on the blog. How long is permanent? Would, say 5,000 years be long enough? Don’t be ridiculous you say, Okay, how about 1,000 years? Come-on, get real! Can I talk you into 50 years? Can I get a show of hands for 25 years?

All of those choices are in fact way, way less than permanent. Permanent is forever, permanent is eternity, the end of time. You know what I mean here, the lake of fire and the seven-headed dragon. Permanent means you want your personal papers kept longer than the earth has been in business to date. We did have some stuff that was indeed scheduled for 25-year retention term. But, our permanent records fit neatly into a single, fairly small box. Most of them probably didn’t belong there.

Organizations that have successfully implemented programs to address this use a fairly simple set of criteria to determine the answer to the how long do we keep this questions. First you have to deal with information as a whole, not specific data points. You can’t think in terms of do we save this purchase order for five years and that one for two years. The question must be how long do we save purchase orders?

Here’s the ruler you use to make these decisions.

You need to establish three and occasionally four separate values for any information set you are working with.

These are the administrative value, legal value, vital value and finally the occasional one, the archival value of the information. Let’s look at each one separately.

Administrative Value – Simply put, how long is the information useful to the business or organization? How long must the information be available in order for the organization to conduct its business? What happens when the information is not available? We’ll take a look at how this works a bit later.

Legal Value – This is always the one people get worked up about. Yet, this is usually the one with the most latitude. Various government units such as EPA, IRS, SEC, FTC and the rest of the TLAs might have very specific performance criteria for your company or organization. These will relate to the regulations that they are responsible for. There may be specific compliance forms you are required to periodically submit to these agencies that document your performance under the body of law or regulations they maintain. The CFR or other publications might list specific required retention periods for the supporting data that backs up these submissions. Chances are they won’t. Especially non-federal governments, but that does not mean the feds are going to tell you exactly how long to keep something. The IRS was notorious for skirting this by requiring that information be maintained, “… until audit requirements are met” or something equally nebulous.

Archival Value – Here we are talking about historically important records. This value is highly subjective in nature and for many companies; this might not even be an issue. If you are Microsoft and you have a cocktail napkin with a few lines of handwritten code in Bill Gates’ hand and perhaps a note to call IBM about DOS ... that has archival value. The PO authorizing the purchase of a new chair for Bill Gate’s office would not have any particular archival value.

Vital Records – These are truly rare. This is information that the organization must have in order to operate. For companies, this would include things like the corporate charter, certain patents or copyrights and so forth. These are the only records that would be close to being permanent.

Okay, how does all of this work. We’ll pick something that is both sensitive, but still fairly universal in nature.

Employee attendance records. These records tell you what days your employees worked, took vacation, were out sick or took personal leave.  

How long is this information useful? Let’s say you revise vacation day allowances once per year and supervisors refer to these “records” on the employee anniversary date as part of the review process. So, let’s say from a use standpoint, this information is useful for a maximum of two years. This would cover the annual vacation carry over calculation plus the annual review that could occur up to 12 months following the fiscal year end. So, ADMIN VALUE = 2 years.

What about legal requirements? Is there anything that tells you that you must keep attendance records for any specific time? Probably not. Perhaps your General Counsel is cautious and decides to look at how long dismissed employees have to file a wrongful dismissal action against an employer. Let’s say the law allows a dismissed employee 36 months after dismissal to file a suit against an employer they believe dismissed them unfairly. So now we have other criteria to consider. Employees fired for poor attendance could legally dispute the firing for the three years following their termination.

LEGAL VALUE = 36 months following the fiscal year of the record or the dismissal date of the employee, which ever comes last.

Is this an Archival record? No. ARCHIVAL VALUE = 0

Is this a vital record? No. VITAL = 0

Most of time you will just look at the four categories, pick the longest time the information is considered valuable and go with that.

But, sometimes, it’s just not quite that simple.

Lets say you have a body of information related to the production of widgets. This information consists of the production date, the lot and serial number of the widget and the manufacture and distribution data for the widget all the way from the source of the supplies and raw material going into the widget to the end-user purchasing the widget.

You make and sell thousands and thousands of widgets every month. Your quality control is very good and ultimately only 1 in 5 million widgets fail. Even the most creative misuse of your widget would not result in any potential injury but under the assorted consumer protection laws you are responsible for replacing defective widgets for a period of five years after purchase.

After careful analysis, you find out that storing all the production data on your widget product for a period of five years will cost you 3 million dollars. The maximum number of returned widgets over the five-year period is 100 and each of those has a total replacement cost of US$10,000.

Are you going to spend three million dollars protecting yourself from a one million dollar liability? Of course not. Forget the legal requirement in this case and trash the production records at the conclusion of their useful life.

Here’s another thought. What liability do you create by keeping certain information around beyond its useful life? When something exists, and you know it exists, and it has some evidentiary value related to some sort of legal exposure, you can’t just decide to pitch that material and then tell the court that you can’t find the subject information. That is considered tampering with evidence.

But, what if you destroy specific records deliberately, according to a documented schedule as part of your normal business process? This is not unusual at all. Companies want to save money. They don’t want to store reams of paper, racks of tape or boxes of other media any longer then they have too. Why? It cost tons of money to store anything indefinitely.

Here’s another reason why

A recent President of the United States maintained tapes of conversations that took place in the Oval Office. Mid-way through an investigation of his actions as President of the United States, the existence of these tapes became known to the prosecutors involved in the investigation. Naturally they subpoenaed the tapes and the end result was this particular sitting president resigned his office.

Nowhere did it say he was required to maintain tapes of his office conversations. Nowhere did the law require him to keep tapes, once created, for any particular period of time. If he had established a practice of regularly destroying the Oval Office recordings every thirty days or every year or every single day, he would not have been in violation of any law regarding the destruction of evidence.

But, once the subpoena arrived, he was toast. He was compelled to produce the tapes. Any attempt to modify, destroy or alter the tapes would have been considered a criminal act.

This happens regularly to individuals, corporations and other organizations. It is critical to make sure that materials scheduled for destruction are indeed destroyed. There are many examples of legal actions being helped along by some employee who thought they were doing a good thing by taking a certain record or document home with them for safe keeping. Sometime during the investigation this employee feels compelled to come forward and produce the damning information. Thus their employer is found liable for something that they would otherwise not be proven liable for.

In closing, we don’t want to keep data permanently unless we want to be held responsible for every conclusion that data might support.

Would you want to be held legally liable for the actions of all of your fore bearers?

Great-great-great grand-dad the deserter during the Civil War?

Uncle Charlie the income tax cheat?

Great Aunt Wanda the carjacker. (yes, they are all related to me.)

Of course you wouldn’t.

Companies and organization are no different.

About Lou Washington

Professional Life: Started my career in Information Management from the somewhat misunderstood field of Records Management. Following four years of working for the University of Missouri System's Office of Records Management, I joined Tab Products Co. in 1980. Shortly thereafter, I became interested in the software business, PCs and how those systems would shape the enterprise of the future. We were transferred to Tab's then corporate HQ in Palo Alto, CA. I was the first Product Manager for Tab's Tracker systems software products, which utilized a PC-based bar-coding system to track the movements of everything from files to capital assets. I believe it was the earliest example of workflow automation available on the market. I was also peripherally involved in Tab's Laser Optics division, which brought to market one of the earliest business systems employing CD-ROM and WORM technology as an information-storage media.

In 1990, I returned to Cincinnati and joined Cincom Systems where I began to learn and work with mainframe-oriented products and systems. In those days, there was a real "split" between the mainframe forces and the desktop proponents. I always found this to be amusing since both had so many positive things to offer an enterprise.

My present role at Cincom involves a number of things including product security, pricing, finance packaging and industry research.

My wife, Barbara, and I reside in Park Hills, KY. I am a member of Blessed Sacrament Church and I am active in a local car club, Cincinnati Cruisers. We are a group of PT Cruiser owners who enjoy tricking out our cruisers and driving around annoying people who have to drive boring cars. I am the webmaster for the Cruisers, and I invite everyone to visit www.cincyptcruisers.com and check out our awesome rides! Barbara and I both enjoy photography, travel and our two four legged canine children, Chloe and Cookie.