When embarking on an IT project, stakeholders are always tempted to remove some objectives from the project plan to ensure only those items underpinning the desired functionality are included. One of the line items that is often cut back on—if not removed entirely—is written documentation. Pre-project, internal planning documents, and after-the-fact whole-project descriptions alike are susceptible to being left on the cutting room floor.
"It's expensive!" you say. Or, "I don't like to read it, and the developers don't even like to write it!" I understand. Good documentation can be time-consuming to write and boring to read. However, good documentation can and should serve as one aspect of what is the most important part of any IT project, be it custom software, Business Intelligence, or infrastructure—communication.
Documentation as Communication
This is the first place where things may fall down: Documentation winds up being a one-way communication apparatus. It should instead be a tool to facilitate two-way comms between project teams. Sure, at some point it will become one-way, as when the project is complete and the involved parties have either moved on to the next engagement or are consumed by other work (or even just on vacation). Now written documentation becomes the roadmap to those on the ground right now describing what should be going on.
It is this "future" role of documentation that makes the two-way discussions about the project and even discussions about the documentation itself so important. Just like the actual project, everyone involved needs to put in the effort to ensure the documentation is correct, understandable, and easy to consume by all intended audiences. If this effort, which requires plenty of discipline, doesn't happen, the documentation's usefulness will be limited, at best.
As a communication tool, documentation can take many forms and serve many discrete purposes. I tend to focus on a couple of those specific purposes when pointing out the good things documentation can bring to the lifecycle of any IT project.
These two purposes are "requirements" and a final, over-arching description of "what work has been done" (this is commonly called a "Technical Specification" document). Both of these purposes have two main audiences: the business folks who are dictating what it is they want delivered and who will be using the system when it is complete; and the developers or other IT/consultant people who will be making those requests happen.
As software development methodologies have evolved (and data-related ones, too), the classic, waterfall-style, write-once-never-touch-it-again Requirements Document has fallen by the wayside—and rightfully so. In its place have appeared User Stories and other tools which better serve the more iterative, back-and-forth nature of requirements gathering and development. I think this is generally a good thing; however, care should be taken to not let these tools fall into the same trap of being written and read once and then never reviewed by the requestor or the development team again.
It is also important to remember that one size does not fit all, and in data-related projects where there are many discrete data elements—each with independent processing rules—high-level user stories may not be sufficient to communicate the business needs. In these situations, it is important to ensure these nitty-gritty details are recorded and periodically reviewed with stakeholders and developers alike to ensure the project is still moving in the right direction.
Tech Specs are about the most useful piece of written documentation there is. As a complete picture of an IT project, a tech spec can be an indispensable resource in any number of situations.
Personally, I tend to use them as a bit of a design/planning document up-front: when the development effort starts, I will begin to write parts of the Tech Spec first, tackling some design decisions for how data should flow in a Business Intelligence project before I'm knee-deep in SSIS. This avoids the extra overhead of a discrete "Design document", which tends to only serve the needs of the dev team themselves. Additionally, as the Tech Spec usually includes deep-down business logic details of what the new system is doing, having these details written down ahead of time is an opportunity to discuss these rules on paper with the Business Analyst, Subject Matter Expert, or whoever else from the business would be interested in reviewing these rules before they get implemented. This helps to avoid rework in later phases of the project by ensuring the rules are interpreted correctly, right from the start.
Once the project is done, a detailed and complete tech spec really begins to shine. Here are some example situations:
In my world, Business Intelligence, having a clean, dedicated data dictionary section of a tech spec (or a related auxiliary document) for the data elements exposed on a dashboard or a reporting engine makes it easy to look up where those elements come from. This can become a handy reference for business users who have been involved in the project from the beginning and new users diving in for the first time alike.
For a business that has brought in a consultancy to do some work, having a specification detailing all of the logic and components of the system is useful for in-house IT to reference in the future for either troubleshooting or to answer questions further down the road.
These documents can fill the gaps when "the guy that wrote it" is either on vacation and unreachable, has left the company, or is otherwise not available to ask questions. This works for both future troubleshooting and easing the process of adding additional developers to an ongoing project effort.
Even as the person working on the Business Intelligence solution, sometimes questions are asked about specific data fields that are not at the forefront of my memory. It is much easier and faster to reference the tech spec instead of diving into the project's code to find the answer. Even putting an in-progress version of the document in the hands of the Business Analyst can get questions answered more quickly by having to involve fewer people in the process. Keeping the analyst very close to the development process in this manner has other benefits as well, like helping to keep ongoing, two-way communications open.
Documentation may look like a trivial blip within an entire project's budget, but its role as an indispensable communication tool, both on the developer side of the equation and for business users, now and in the future, should not be overlooked.