Software Project Management for Programmers

One of the many things that I'm not very good at doing is playing squash. I keep trying though, and meet up with a good friend of mine a couple of times a week to awkwardly run around a squash court missing the ball in exciting and innovative ways over and over again. I don't think I'll be doing any disservice to this friend when I refer to him as a "non-programmer". Indeed, he has devised a neat classification system that everyone in the world fits into: the world is divided into "geeks" and "normals". Geeks are people who pick up his PDA and go straight to the expansion port to see how much memory is in it. Normals are people who ask him if he can make phone calls on it.

I'm sad to say that I fit squarely into the "geek" category. I actually did find the expansion port the first time I saw his PDA and said "wow! 128 megs!". It should come as little surprise to you then that I have led a great number of completely failed software development projects. My concern is that it will come as a surprise to a great many of you. You see, all the programming expertise in the world won't help you if you don't understand what your program needs to do. As a professional programmer you need to find a way to talk the same language as your clients and find a way of keeping them abreast of the challenges you face without trying to draw these poor "normals" into your "geek" universe.

It's taken me a long time to start figuring out what's going wrong, and only now am I starting to find the solutions. I hope this article will at least give you some perspective on what's going on in your own software development projects. I talk here about web development, since it's all the rage, but I hope you can apply the same principles elsewhere.

The Problem

It begins with the phone call. Seemingly innocently it starts off with a "hey! how you doing?", and slowly more menacing "so, are you busy at the moment?" and on to "great! I've got this client that needs to set up a new web shop with streaming videos of his crocodiles dancing to Britney Spears and as soon as I heard about it I thought of you!". Great, you think, some work. Could sure do with some of that. Then there's the meetings. There's lots of them, but even then there's not enough. You get asked to build a "web store", you need to include "streaming video". It needs to "integrate" with the in-store point-of-sale system, it needs to support IE4 and it needs to be done by the end of the month.

So what's so bad about that? Here's some questions that I know will have been left unanswered:

• How does the point of sale system work?
• What data formats does it talk?
• How often will it need to "integrate"?
• Just what is "integrating" anyway?
• IE4? Are you for real?
• Streaming video eh? So who's going to setup the co-located box in America, buy the license for Real (or similar) and set it up?
• Who's going to maintain it?
• How is this streaming media going to get there?
• How many people will be watching this streaming media concurrently? Can you guarantee that?
• When can I expect to see the designs?
• Now when can I expect to see the second revision of the designs that bears no resemblence to the first one?

I could go on but I'll save you that agony. In short, there has really been no consideration to the complexity of the project -- the client doesn't have a good understanding of his or her requirements and they definitely don't understand what those kind of requirements cost.

I should point out that these conditions are very like a system I was asked to build just before I started at Anchor. They are in no way contrived. If they sound ridiculous to you, then you are a fortunate developer indeed.

Communication

Essentially what we're seeing here is a communication break-down. The client doesn't understand that a programmer can't program something they don't have full specs for. The designer doesn't understand that they simply can't keep changing their mind on fundamental decisions made weeks ago. No-one understands the point of sale system -- it was purchased off the shelf by the previous acquisition manager.

So who's at fault? The developer. Yep, it's all their fault. Inevitably they will be the only person in that room who understands all the issues at hand -- it's their job to communicate that to all parties concerned. You could argue that it's the project manager's job to do this, but unless your project manager is also a programmer, there's little point expecting him or her to get the job done. At the end of the day it's your neck and you're the only one who can save it.

The solution then is simple enough -- make everyone understand all the technical facets of the project. There's a couple of parts to that, but let's start at the start: to explain the technical aspects of the project you need to understand them yourself.

Understanding the project

Before anything else happens, you need to get a good feel for all of the project. You need to have a meeting with the project manager, and you need to have a meeting with the end client. You don't want to say anything at these meetings; you need to listent to their requirements, and understand their vision for the project. Occasionally you will find that there is disparity between the project manager and the client. As soon as you know about that, raise it with them both and find a solution. Here's some good exercises to use when you're in these meetings:

Use cases

Probably the most important thing at the end of the day is that someone's gotta be able to use this code to get a job done. Ask the client and the project manager to imagine that they're a customer, wanting to use this website (or application) and get them to tell you what they're doing, in detail to achieve their task. Then get them to tell you what business practices they're going to follow in order to make money from whatever transaction the end-user is partaking in.

You'll often find in working through this that there's a lot of unstated expectations. One response that I commonly hit was when asking the user to describe the ordering process was that I would be told "so I go to the search bar and look up the type of crocodile I'm after, and then...". That means there's got to be search functionality. I found it very rare that people actually said "the site has to be searchable" -- it just seems to be assumed.

At the end of the meeting, read back the list of requirements you've written down. Ask them if there's anything missing from the list, and ask them to have a look at your notes. The purpose of this is two-fold: first it serves as a check that you've been talking the same language and that you've been communicating effectively. The second reason is that it gives the client some feeling for just how much "stuff" they've asked for. It's inevitably more than what they initially envisaged.

Don't talk timeframes. Yet.

It's pretty common practice after clients learn just how many things they've asked for to want to know how long it's going to take. They've got a number in their head and they want to know if this meeting will make that number unachievable. It's almost certain this project will run over-time, no matter what number you give them so what you really need to give them is a bunch of dates and a plan for development. Unfortunately that will take more time and work than will be available at such a meeting, so you just can't give any dates. Tell them you'll look into it and get back to them in a week.

Then what?

After you've spoken to the main stake-holders in the project, you need to talk to any auxilliary parties. This includes people like sysadmins, or vendors for 3rd party software that will be used (such as the point-of-sale system). I won't say too much about this since the process will be very similar to what I just outlined. Make sure though, that you keep the communication channels open with these people. If they are affected by database design decisions that you make, then send them every update to your database schema, and ask for feedback, ask them what effect it will have on them. I assure you that these documents will go ignored unless you maintain a dialogue.

The Spec

In a perfect world, you'd get handed "The Spec" on day 1. "The Spec" would be a complete and authoritive treatise on everything you need to know to finish the project. These documents just don't exist. Trust me. That's not to say that attempting to write them is a fruitless exercise. The unfortunate fact is that the client -- the one who ultimately knows what they want, is hopelessly inequipped to write such a document. Here's what you need to include:

Problem statement

The first part of the spec should contain a few paragraphs concisely explaining the problem that needs to be solved. This is big-picture stuff. If you find it impossible to state the problem in a couple of paragraphs then this is the kind of project where you need a "specifications team" rather than a single author. Everything else you write should be judged against this problem statement; it acts as a test that the system, as imagined by the programmer, will do what it needs to do.

The subsequent sections will vary somewhat from project to project. Generally though, this document will address each logical "component" of the project and explain what abilities it will have, and what constraints (if any) are being assumed in its implementation. Any and all mechanisms for performing some task should be explained. For example -- the issue of dealing with "sales" should be dealt with. What mechanism will you provide to allow the discounting of items? (if any, and if there's not, mention that as a constraint)

We'll come back to the importance of a thorough spec, but for now treat it as "insurance". This will be the one document you will come back to when there are disputes about missing functionality in 5 or 6 months. Refusing to add any functionality the client expects is useless, but defining what will be included in your final price allows you to charge for the privelege of adding this functionality.

The Plan

Detailed implementation plans are almost as illusive as rigorous specs. Inevitably you won't exactly follow any plan that you write, but attempting to write a plan that goes from start to finish means you'll be able to give a good answer to your client when they once again ask you "how long will it take?". Here's my pointers for a good implementation plan:

Small time-slices

I can't honestly remember who originally said it, but when you're considering software implementation, you really need to chop up your activities into portions about 3 hours long. You don't need to mark all these 3 hour chunks on your plan, but make sure you've thought them all through, and be realistic. Your time is worth money, so it pays to get this right.

Milestones and versions

Clients and designers will want to see results imediately, so when you present this plan to your clients you want to give them a list of "milestones" -- functional achievements with tangible results. Milestone 1 (M1) might be a site with the ability to browse products linearly. Milestone 2 might add searching abilities to that. This allows everyone in the project to track the progress of it and feel assured at all points in time that things are getting done. Working in this methodical way through a list of achievements is a good way to program as well -- each stage can be tested and feedback gathered before problems arise that can't be solved later on.

As part of this milestone system, you should commit to weekly rollouts of your code to some testing environment. This increases visibility and lets you get your rollout process perfected long before release date.

It's wise to order your milestones such that after a certain point, the site could go live without any of the rest of the development being done. Of course it's not ideal, but there are always deadlines, and you're rarely aware of them all. Mark a particular milestone as "version 1" -- it will contain the minimum amount of functionality that you could reasonably expect your client to do business with, but without all the bells and whistles.

Invoicing

As part of your plan, you should include a number of sign-off points (such as "version 1") where you will bill the client for the work done so far. Though it may not seem so at first, this benefits everyone:

• It clears up cash-flow for your client
• It means you can walk away at any time without having lost all the income you may have received from the project

At each of your sign-off points you should arrange a meeting with your client to demonstrate the current level of functionality and judge the progress so far.

Meeting Two

Having prepared your spec and your implementation plan, you are left only with the problem of getting your client to sign off on them. But that will have to remain the subject of another article.

Happy hacking!