The Binary Fallacy

Two roads diverged in a yellow wood,
And sorry I could not travel both
And be one traveler, long I stood
And looked down one as far as I could
To where it bent in the undergrowth;
-Robert Frost, The Road Not Taken

Binary Thinking is the process of thinking of things in terms of two opposites. You can either do A or it's opposite B. But that's it. In Neuro-Linguistic Programming, it's a technique for manipulating someone into making the decision you want by framing the choice as a matter of either the thing you want them to do or the opposite, presented as a harmful choice. In Cognitive Behavioral Therapy it's a pattern to be broken to help people gain more control over their lives. In Software Architecture it's a warning sign.



If stuck between two different ways of designing something, the answer is always door number three.

Why do humans think like this? Why didn't Frost just keep walking straight through the woods and not worry about the roads? Why didn't The Clash stop worrying about staying and going and just Rock the Casbah? I don't really know. I'm not a psychologist. I'm just a software architect who's familiar with Analysis Paralysis. The phenomenon of getting nothing whatsoever done because you're locked up in the decision making process. And after years of trying to figure out how to avoid this, I came to one inescapable conclusion.

When you're in analysis paralysis it's because every path you're considering is wrong

In my opinion, this happens when you realize deep down that you're looking at the problem wrong and considering bad options. You can't decide because you know your options are both bad.

So what do you do? Well, that's the tricky part and adages about in which part of the box to think aren't as helpful as people who use them think they are. If thinking of something new was as easy as the realization that we need something new then Elon Musk wouldn't be quite the icon he's become. The important question is HOW? First, you just need to stop and accept the fact that you need to come up with something different. Your first decisions were wrong and you need to set them aside. Don't go back to them. Then I recommend you look at the great Stoic thinkers. You start by asking "What is this and what is it for?" I often tell developers to answer those two questions for every application, for every class they build, for every database and table they create. And you don't build the thing in question until you can answer those questions.

Then you ask if a thing is needed or just wanted. If it's not something you need, based on the answers to the above questions, then you leave it out. You strip away the irrelevant. Often I find myself locked up because I can't find a clean way of integrating the irrelevant into my design. Once I stop and realize the piece that doesn't fit doesn't need to fit, things get easier.

I've often said that software development is 90% thinking and 10% typing. If you understand how to think clearly, how to organize your thoughts, and how to tell the difference between the necessary and the unnecessary then there's little that can keep you from being a great software developer.

You've Got One Job

"If you choose to not deal with an issue, then you give up your right of control over the issue and it will select the path of least resistance." -Susan Del Gatto, Creating Balance in a World of Stress: Six Key Habits to Avoid in Order to Reduce Stress

Never forget that your only job is to provide a solution to a problem



Stick with any career long enough and you start to develop a set of principles that guide your approach to your job. Call it experience, call it being set in your ways, even call it wisdom if you dare. But we all do it. I've got about twenty of them, which I realized after an NCIS binge when I started writing them down Gibbs' Rules style. Rather than blasting them out all at once, over the next several articles I'm going to talk about each one. There's no particular order, other than the order in which I wrote them down. However, if I had to pick one to be the most important, it's the one I'm starting with.


Even at the beginning of my career, I never really thought of myself as a software developer. Rather, I thought of myself as a problem solver, albeit one who tended to approach solutions with code. But that idea has always stuck with me. Always be solving a problem.

Now, not all problems are created equal. Some are quite big. For instance, Linus Torvalds solved the problem of having to pay money for an operating system. Some seem almost insignificant. Recently I've been working on the problem of "I don't know Python as well as I'd like to". Something of import to almost no one other than myself. But that's the point. It's someone's problem, and it's getting solved.

Of course, what I'm really getting at is why you write software professionally. And while we all basically realize that we need to fulfill requirements, implement stories, or however we get the list of stuff we're supposed to write, it's easy to lose track of the big picture when you're in the weeds of the little picture.

The problem is that we, as software developers, don't just like to develop software. We like to develop cool software. However we define that, but usually as "the last new idea I read". Admit it- you do it, too. Goodness knows I do. We like to learn new things, we like to try new techniques, because we like, no we need to grow. This rule is to remind me not to take that too far. Ultimately, it's not about what I want, it's about what solves the customer's problem. Because if I'm not doing that, I'm wasting everyone's time.

I've also ran into situations where solving the problem didn't have anything to do with new software, custom software, or even software at all. I've on occasion been able to trim large portions out of a project with suggested business process changes. Or by pointing out that the requested software doesn't actually solve their problem, and on occasion might even compound it. A good software developer knows when to not develop software.

The idea of solving a problem is a big one, and one that spans several of the principles on my list. But this one gets its own item because it all boils down to this simple concept. You're not writing software to write software. You're solving a problem.

                                                                 

Move Fast and Break Stuff

“Now what is the message there? The message is that there are no "knowns." There are things we now that we know. There are known unknowns. That is to say there are things that we now know we don't know. But there are also unknown unknowns. There are things we do not know we don't know. So when we do the best we can and we pull all this information together, and we then say well that's basically what we see as the situation, that is really only the known knowns and the known unknowns.” –Donald Rumsfeld

I actually wrote this some time ago, got distracted, and forgot about it until I saw this tweet:

"It's not fail often, fail fast, It's learn often, learn fast" - @agilesensei #fastfeedback

— Melissa Perri (@lissijean) October 20, 2015

Okay, I admit that the title is slightly linkbait-ish in that I don’t exactly take the “Move fast, break stuff” approach. So let me propose a clarification: “Move well and don’t be afraid to break stuff”. Why? Because if you are building any sort of process, be it a software development project, a corporate business plan, or just a sales presentation, there are three critical pieces of information you must deal with. The things you know that you know, the things you know that you don’t know, and the things you don’t know that you don’t know. Failing to properly react to any of these introduces enough risk to sink your effort.

Things you know that you know

This is the easy stuff and any process plan should start here. You’re on solid ground and planning
becomes easy. But the common misstep here is to fail to act on what you know that you know. Your
domain knowledge is almost certainly incomplete, but you have something you can act on. This means you can begin working and solving problems quickly. You can gain measurable success sooner rather than later. And anymore, time wasted is success wasted.

Things you know that you don’t know

These things are also pretty straightforward, but are also a common trap. You know you have more
information to gather and additional analysis to do. But far too often the tendency is to stop here until you know all the things you know you don’t know. (If you need to, read that line again. I’ll wait.) Just like act action begets action, inaction begets inaction. The more you wait, the harder it is to begin. This is what “Move fast” means, but really you need to “Move well”. You know the things you don’t know, so while you’re acting, you take these things into account. You try as hard as you can to intelligently anticipate as much as possible. The idea is to have to redo as little as possible while being at ease with the fact that you will have to redo something. It's the acknowledgement that something is better than nothing. Except when it isn't. It’s a balance, and there are no right approaches here. The best piece of advice that I have is to be willing to draw a line and say “Factum dictum!” And that's really the point of "Move Fast, Break Stuff". If you wait until you have all the answers, you've waited too long.

Things you don’t know that you don’t know

This is the tough one. There’s always something you don’t expect. Something you don’t see coming.  You need to accept that as you proceed and accept the fact that something may very well happen that turns everything upside down. There’s both forseeable and unforseeable change. Understand that if your plan failed to protect you from unforeseeable change, that does not mean your plan failed. But also the whole point of “Move fast and break stuff”. If you continue to move, continue to produce, and refine your plan as you have more information, you will be in a better position to react to the things you don’t know that you don’t know. If you don’t, your best case scenario is that you’re only able to react to the unforeseen when you’re far enough into your process that it requires a major amount of effort to react. Worst case, you never move out of the planning and analysis stage because you delay action to account for the unforeseen.

“Move fast and break stuff” does not mean “Be reckless”. You’re process should not resemble a bull in a china shop. Move well and move with the understanding that projects, like life, can get messy. If you do what you can when you can, you put yourself in a far better position to succeed. And, as Melissa Perry pointed out, the learning has to come with the breaking.

A Tale Of Two Service Providers

This is an tale of two services that I recently had performed on my house. It's merely a generalized recount of things that have happened in my life, and not a commentary on DevOps, collaborating with business partners, or the #NoEstimates movement. --Editor [That's me]

We recently needed a new garage door opener, which wasn't a surprise. We knew it would need replaced since the old one had been less and less functional as time went on. We ordered a new one, selected certain other services, including installation, paid, and an install date was set. A date that did not make it onto the schedule and as a result, was pushed. At the new appointed time, the installer got to work. When he was finished he told me that there were problems with the setup. In resolving that problem, certain modifications had been made. He demoed the garage door, but it was using a cable that he then took with him, leaving the garage door opener non-functional at the end. But helpfully leaving instructions on how I could finish the work myself. The next day, we discovered that other contracted services had not been completed. Of the four support avenues we tried, all apologized for the inconvenience, albeit one only after trying to tell me that there had been no work order in the first place. A resolution date was set. A date that came and went with no resolution. We finally escalated the support call, informing the new people we talked to that we would be filing daily BBB and other consumer protection cases until the issue was fixed to our satisfaction. The dip in their ratings on these services were finally the stick needed to get a resolution that leaves us with a fully functioning garage door. However, it is good to know that the original dollar estimate will remain the same.

Somewhat less recently, we needed a new toilet installed. Our plumber came out and showed us his recommendations after listening to what we wanted. He set a work date and estimated price. When the installation was underway, the plumber pulled me aside and told me that there was something we needed to talk about. It turned out that the reason our toilet was degrading so badly was that our water softener was not working. Limestone buildup was corroding our fixtures. Replacing the toilet was necessary, but would not fix the problem in the long term. He told us that if we wanted to push the decision, it would not cause immediate problems. However, if we did not do this work, we would end up buying another new toilet. And probably faucets as well.

He told us his recommendations for a replacement, based on our water usage and gave us a revised quote. Making sure that we knew that the additional work was not necessary for fixing the immediate problem. He also showed us one other small problem, but told us that it was simple to fix. He quoted us a price for the part, but told us that the nearby hardware store had it for cheaper. Our decision. He also told us that the fix was too simple to bother charging for labor. Given all of that, we decided that it would be less of a disruption to our lives to have the work all done at once. We agreed on the new cost and time frame. The work was completed. The cost, time, and our level of involvement were all higher, but we knew why. And all steps of the decision making process were in our hands, with the plumber explaining the cost and benefit of all of our options.

Thank you for you patience with my tangent from the normal DevOps focus of this blog.

Dear Entrepreneurs

"Regard your soldiers as your children and they will follow you into the deepest valleys; look on them as your own beloved sons, and they will stand by you even unto death." --Sun Tzu

An open letter to entrepreneurs, from someone who has worked for quite a few of you.

Normally, I go through a lot of examples and points and then summarize at the end. This one is important to me, though, so I'll give you the summary now. Up front.

Your company is your baby. Not mine. You're invested, not me. To me, you're a job. Just another employer. Unless you do something to get me invested.

In my experience, a good number of entrepreneurs don't quite get this. I get that your company is your baby. You've worked for it, lost sleep for it, and probably skipped a meal or two for it. To you, this is personal. But it isn't to me. Not right away, and not just because you hired me. If I'm going to be personally invested in your company, in your idea, you need to give both reason and opportunity to do so. And it's worth it. If you're just a job to me, I'll walk away as soon as a better one comes by. If I'm invested, I'll have to be dragged out.

To that end:

Just because you believe, doesn't mean I do.

Most software developers I've encountered fall into one of two categories. First you have the mercenaries contractors. They work for the people who give them the best compensation. And I don't just mean money. Pay, benefits, environment, all goes into it. But as soon as they see a better place, they're a developer-shaped puff of smoke in the air.

Then, there are the developers who want to work for the company they work for. Because they want to be a part of what that company does. Believe me- those are the guys you want. But you have to give them a reason. And that's the bit that, in my experience, many entrepreneurs overlook. They think that the fact that they know how awesome the company is, then any employee should, too. Automatically.

Look- I'm a software developer and I'm familiar with the territory. A lot gets asked of us, and we take a great deal of professional pride in delivering quality work. But there's got to be a better reason than "Because I said so". If I'm working 50-60 hours a week just because "That's policy", then I'm probably going to have my resume out in a couple of months. Definitely before my one year anniversary. If I care, then I'll do what needs to be done, but you have to give me a reason to care. 

I've worked at both extremes of this. One entrepreneur mandated 50 hours a week, minimum, 24/7 support, and made us track our time down to the quarter-hour. For few benefits, and frequent communication that profit sharing and ownership would only come to those who deserved it. Which was, at least at the time, no one. I had taken the job in order to gain a specific skill set. Once I felt I had it, I moved on. I had no investment in the company because the founder hadn't given me a reason to care.

I went to another startup. The CEO gave regular meetings, not just on how the company was doing but why they were doing certain things. He told us quite frequently that his vision was to change a market in order to put more power and more control, and thus more opportunity, in the hands of customers. He ended every meeting with "Welcome to the revolution!" He believed. More importantly, he got others to believe. Nights? Weekends? No problem. Answer questions while on vacation- sure. Why? Because I wanted that company to succeed- still do, even though we've parted ways. And even so, I'm a touch sad about having done so. Because I believed.

Sprint Planning Pt. 1: Stop with the Points Already

"Our culture thrives on black-and-white narratives, clearly defined emotions, easy endings, and so, this thrust into complexity exhausts." --Caroline Knapp

I've been doing a lot of reading and thinking about story points lately, and over time I have been edging toward what I now see as a necessary conclusion. Story points are meaningless. Even... Wait for it... POINTless. Yes, I just did that.

To recap, when planning a development sprint, it's customary to assign a user story a point value as a measure of... Well, to be honest, I really can't tell you what it's measuring. I can recap what people say about what it's measuring, however I've become skeptical of whether or not points are measuring what people think they are. And even moreso of whether or not it's a good use of time to do so.

Definition of a "Point"

Here's the root of my problem with story points. What are they and what do they measure? These aren't simple questions, either. If you read here, you get the idea that points are a measure of complexity. However, this StackExchange answer tells the reader that points need to be a measure of size, not complexity. Both will tell you that points are not a measure of time, but everywhere I've run into the practice, someone has always defined a point something along the lines of "What one developer can do in one day". Which certainly implies that points are used to give some measure of time. So I have a hard time getting started here, because I don't even know what I'm supposed to be measuring. Time? Size? Complexity? The amount of effort it takes for two women to have one baby? 

What's the Point, anyway?

I promise- I'll stop doing that now. Maybe talking through why we need points, we can come to an agreement on what they are. We need them for estimating, right? But estimating what? The amount you can do in a two week sprint? The amount of time you need to finish a certain set of features in a product? After all, you assign points, add them up, and divide by about what your velocity has been, right? Because that works.

Except, of course, it doesn't. Addressing the latter poi ISSUE ::ahem:: first, we all know that doesn't work. Regardless of what some may say, points do not mean you don't need to re-estimate. Software development is a process that builds on itself with every build adding to the whole an impacting the effort/size/complexity/time needed for future changes. I'd be very dubious of the point value of any story not going into the next sprint. I'd be even more dubious of any time estimates derived from arithmetic on points, velocity, and time.

So, maybe it's a good way of determining what can be worked on in the coming sprint. Problem is, without definitive answers to the question "What does a point measure", using points to determine what stories can fit into a sprint is a terribly uncertain process.

If we take the definition of points being a measure of the size of the story, then we truly have no idea how long a story will take. In fact, the StackExchange user in the link I referenced illustrated this very thing. He gave examples of the complexity of a 5-point sized story varying by a number of factors, leading to wildly different completion times. I don't really see that as a practical way to plan a sprint. More importantly, I don't think the people who expect me to plan sprints will see that as a practical way of planning sprints.

If we take the definition of points measuring complexity, we run into a different problem. How do you measure "complexity"? It's not like defining the length of a metre or defining a joule. the only way is to compare the complexity of one thing to the complexity of other things until you get something that feels like a match. Inaccurate, sure. But that's why it's called "estimating" and not "precision". I'd be basically okay with this except for a couple of things:

1. That which is complex to one developer may not be to another. Planning Poker is supposed to account for this, but it doesn't. In fact, it ironically exacerbates the problem it claims to solve. My 5 point complexity story may be another's 2 point complexity. To assign it 5 or 2, then, is meaningless outside the context of the developer who will be working on it. And splitting the difference to make it 3 or 4 is just compounding the uncertainty of an already uncertain process.
2. Complexity has nothing to do with the amount of time it takes to do the task. Planning Poker assumes that every developer handles every type of complexity in the same way and at the same pace. Maybe one guy tackles a complex story by diving in and coding until he grinds to a solution. Maybe another takes time to read more on the subject, asking questions, and only starting when he feels he has a better understanding of the subject. Given this, complexity can only correlate with time if every developer approaches complexity the same way.

Either way you look at it, you're trying to objectively quantify a series of subjective data points. Logic 101 tells us that this is a highly inaccurate way of measuring anything.

Velocity

Finally, we get to Velocity, which I'm pretty sure is the whole reason we bother to keep track of points in the first place. Velocity is nothing more than the sum of the points of the stories in a given sprint. It purports to measure the amount of work done in a sprint so you can compare to other sprints and then plan a timeline for future features.

Where to start.

First off, all this gives you is a comparison of how you've been objectively quantifying the subjective sprint over sprint. But as a comparison of the actual work done? All you're doing is comparing arbitrary numbers to arbitrary numbers and pretending it's a meaningful measure of productivity.

Secondly, using this as a future planning tool is a recipe for disaster. In order to make that work, you'd have to give point estimates to everything in your backlog and then continually refine those estimates in light of what the previous sprints have accomplished. Do that for long enough, and sure. your velocity will become a more and more accurate tool for predicting the amount of work the team can handle. But at what cost? At what time commitment?

Conclusion (WHEW!)

I'm not saying we need to stop estimating anything. Projects, stories, releases, versions, whatever unit of work that is needed, all need to come with some idea of what it takes to accomplish them. What I am saying is that we need to re-think the whole point estimation bit and ask ourselves if this isn't just an overly complicated process. Effort, complexity, productivity- these are all complex issues that are impossible to quantify in a simple context. Maybe we need to stop trying. Because I really hate unnecessary complexity. I'm not even much of a fan of necessary complexity. I'm lazy and don't want to work any harder than I have to in order to solve problems.

Next article, what I see as the real root of the problem and what I have started doing about it.

New Challenges

"Be careful of what you wish for. You might get it".

Hard to say where that comes from, although some cite it as a misquote of Goethe. The image, of course, is from the amazing mind of Bill Watterson and his Calvin and Hobbes comic.

Regardless, that is kind of the source of why I haven't been writing much lately and why I'm starting again. I tend to write about what I'm seeing and experiencing. Which, over the last few years, has become fairly routine. After writing a good bit about my thoughts on Software Architecture, it became the task of taking my own advice.

So what changed? A chance to get what I asked for, namely a chance to go from helping insure quality in a software system to helping insure quality in a development team. The company I work for has a relatively new development team that operates fundamentally differently than our other teams. This team doesn't support a product. Instead, they handle smaller projects that have a more rigid "Definition of Done", as opposed to continued development and support of a product. This has come with more than a few challenges. Challenges that I have a pretty free hand to solve, but challenges that will take a lot of solving.

All this has put me in an interesting opportunity. Processes are getting built from the ground up. We have leave to borrow from other teams the processes we feel are helpful, as well as to ignore the things we feel won't benefit us. The end goal is increasing the rate of shipping quality software, and with very few exceptions I don't think the head of software development much cares how it happens. This gives me the freedom to take my biggest piece of advice to anyone designing software. Or, for that matter, anyone currently living and interacting with the world in any way:

“This, what is it in itself, and by itself, according to its proper constitution? What is the substance of it? What is the matter, or proper use? What is the form, or efficient cause? What is it for in this world, and how long will it abide? Thus must thou examine all things that present themselves unto thee.” --Meditations, Marcus Aurelius

In other words, "Question everything". Some ideas are worth leaving alone. Questioning the necessity of engagement between the business and development is a lot like questioning gravity. Feel free, just stay away from ledges. But how do you do these things when the business units aren't used to that engagement and don't understand the value in it. And that's not looking down on anyone- how can you know the value of something until it's been shown to you?

Beyond that, what traditional roles in Agile projects do we need? Which can we live without, and which should we live without? What do these roles mean, and how do we make sure everyone knows what an iceberg is?

What about the team? What common practices do we need? What offers us the best value in our situation. Not just what others are doing or what generally works well, but what works well for us?

These are the questions I'm asking myself and the ideas I'm exploring. I'm sure we'll go down some odd paths, and probably even the (hopefully) occasional bad path. And hopefully, my philosophy that understanding the world leads to understanding software still holds.

Preconceptions of Architects

"Science progresses best when observations force us to alter our preconceptions." --Vera Rubin

Been doing a lot of reading, been doing a lot of thinking. Having moved to a new company, I'm seeing my role in a different light, which is expected. Being the company's first architect, helping better define the role, and starting to work with development teams who are not used to working with an architect has really gotten me thinking. And reading some more.

I've read a lot of concerns about architects. Concerns that surprised me because they haven't been my experience, so I'm a little curious if these concerns are holdovers from days gone by (or at least going by) or if they are still relevant concerns. Either way, since the concerns are still talked about, I see a lot of value in looking at them and learning from them.

Architects are Ivory Tower Theorists

I've run into this before, and for good reasons. This is one have seen as a problem- designs getting passed on and the architect disappearing. Now, the disappearing act was often due to the fact that another effort had been dropped on an architect in a sorely understaffed group, but that doesn't mean the perception doesn't need changing.

Architects value theory over practicality

I'm kind of split on this one. There's a fine line here. Architects have to take the long view. I've sometimes said that I'm not working on this project, I'm working on the next one for this product. On the other hand, strict adherence to OO theory and complete Protected Variation (If indeed there is such a thing) are worthless if you can't get the project out the door. One of the biggest challenges of my career has been justifying a design that takes into account change that only might happen. I've found that YAGNI is a moving target.

Architects over complicate everything

Kind of an extension of number 2, but it goes a bit farther. A former manager of mine once said that an elegant design is one that makes the solution seem simple to the point of obviousness. I've often told developers that if I have to answer more than two clarifying questions about my design, then I made it too complicated.

So, what's the answer? I'm a pretty big believer in the idea that there isn't a "The" answer to much of anything, but I've been mulling over one answer. 

Architecture is an Agile project with the development team.

The parallels hit me recently. We provide a product for developers and should interact with them under the same principles under which developers interact with the business on a project.

Constant interaction

A sure sign that something has gone off the rails is an architect not knowing who is implementing a design. Just as an Agile development project requires constant communication with the client, so should we have constant communication and feedback from developers. A good development team doesn't need low-level guidance, be it from an architect, manager, or anyone else. But if the development team can't give feedback to the architect, then one of two things happen. Either they implement a flawed design or they ignore the design. Toss a coin as to which will cause more problems.

This doesn't mean an architect always has to incorporate developer feedback into the design. Sometimes you just have to say "No, this is the way we have to do it". I've done that once. Ever.

Factum non dictum

Deeds, not words. While thinking things through is important, even mandatory, at some point you have to act. And that point should be sooner, rather than later. Action is a funny thing. I've found that acting refines my thinking faster than thinking does. I've also found that action breeds more action. When I'm providing designs, rather than thinking about them, I get feedback faster (See the last heading) and thus work more efficiently.

But this isn't just about translating thoughts to deeds. How does one "act" as an architect? Over the years, I've come to three methods of communication:

1. For simple things, talk to the developers. More than that isn't necessary if things are simple. "We have a web service that manages this functionality. Let's use that rather than developing a new library." "Let's use SSIS for this task"
2. For moderately complex projects, I'll go ahead and use UML, but only if the resulting diagram doesn't look like a spider threw up a Visio diagram. If I have to actually follow a line with my finger to see what elements it connects, UML isn't helpful.
3. Code is best communicated using code. There's a reason we call them "Languages" and there's a reason we have the phrase "Lost in the translation". With some things, I'll create a library with stubbed classes and a few examples based on project use cases and acceptance criteria. I don't mean this as a final design to be carved in stone, but rather a communication of a thought using a language natural to the discussion.

They don't work for you, you don't work for them

Developers and Architects are two sides of a coin. And while Architects may be used to giving guidance and oversight to developers, we sometimes forget that developers can, and should, give guidance and oversight to Architects.

In his book Software Architecture for Developers, Simon Brown compares Architects to Master Masons, responsible for design and oversight of the work and the workers. I think there's a lot of truth to that, but that's just one part of the relationship. Architects, I've found, love to talk theory. I've spent hours comparing design patterns, arguing the possible effects of a given DI Framework, or just whether or not a particular class should be considered an Interface or an Abstract class. Which is all a heck of a lot of fun. But if discussion hinders progress then the development team has the oversight, and even the responsibility, to say "Knock it off and deliver something already!"

Check your ego at the door

Fortunately, there's one thing that makes this all far easier to manage. In a good development department, everyone's a professional working towards the successful delivery of a quality product. That goes a long way. If the development team accepts the realities of an architect and the architect accepts the realities of practical software development, then it's highly unlikely there will be significant problems. Which begs the question, when conflict arises am I the one holding too tightly to my ideas or not communicating them well? Part of why I'm writing this is to remind myself to ask these questions of myself.

The Tipping Point

"You can push that car just a little too far any Sunday afternoon. And if you break your neck in some damnfool wreck, they forget about you soon." --Charlie Daniels, "Stroker Ace"

I generally prefer to work at small to midsize companies. The reasoning is fairly simple, too. I like flexibility. Not a complete lack of defined policies, mind you. Policies are important. They give companies a road map for, if not efficiency, then at least predictable inefficiency. But smaller companies tend to have the flexibility to know when you need to stray off the beaten path. Large companies do not. And there's a certain logic behind that. The more you do, the more you rely on that predictability. Policies have saved my neck a time or two. Being able to say "I can't start this effort because our policies have not been followed" has forced projects to refine requirements and has gone a long way to prevent train wrecks. But there's a flip side, too.

At larger companies, it's harder to change policies that either no longer fit, or need refinement from the original draft in order to produce the intended result. There's also more risk in proposing new policies, I think. There generally tend to be more people that need convincing, which tends to mean more people that don't want to "Change what has been working for us". It often comes to a point where it's easier to just live with the inefficiency than try to change it. Or worse, at least in some ways, ignore policy and try not to get caught.

Okay. So far, I haven't told anyone something they don't know. This is one of the factors that need to be weighed when deciding whether or not to accept an offer or to leave a company. It's, hopefully, one of the factors you investigate when interviewing a company. As an aside, please note that I didn't say "interviewing with a company." You're as much deciding whether or not to offer them your services as they are offering you a position. Never forget, you're a contractor.

The point is, while working for smaller companies, I've noticed a reoccurring phenomenon, which I've started calling The Tipping Point. As smaller companies grow, they start to take on more work and start to do more. At this point, I have observed two results.

If the company does not at least have some policies guiding their work, the company tends to collapse. There comes a time when it's simply too late to gain control of source code, whether because nothing is in source control or there's no structure. When it's simply too late to implement good project policies because stakeholders aren't interested in becoming properly involved in projects and that disinterest is too entrenched to change. When projects are dangerous to deploy because no one knows what anyone else is doing. At this point, while the company may struggle on, it's no longer possible to fix the problems, and the only smart move is to leave.

The second result I've observed usually happens when there are a modicum of policies in place, and is usually hastened by bringing in An Expert. Perhaps a new CIO, perhaps a contractor. The buildup, again in my observations, tends to start slowly. No deploying projects without communicating with other groups. Nothing terrible there. Common sense, in fact. Issues, bugs, and defects need to be logged. Again- why would you not? And we need to get a sense of how much we're spending on projects, so we need people to log time spend on projects. Not my favorite activity, but I can't deny the importance.

But then things grow. Communicating deployments becomes getting signoff from other groups before deploying. The bug tracking system becomes a full-on change management system designed to integrate with every step of any process. Except yours, inevitably. Rather than turning in time tracking spreadsheets, either a web app gets build, or you're to use a built-in feature of the change management system.

Finally, comes the tipping point. I wish I could say this is an exaggeration, but in one case, I was entering time into an application that had a database lookup for projects, yet my projects never seemed to show up. I also had to email the same to my manager. The time tracker was in half hour increments, but my manager wanted 15-minute increments. Every change, even to dev environments, needed to be entered into five different systems (again- not exaggerating) and cleared in a bi-weekly Change Control Board meeting.

And all this doesn't even take into account well-documented problems when you mix in an Expert Scrum Consultant. I think I've made my opinions on the subject pretty clear, but it's pointless to deny what often happens when Scrum policies get blindly applied. And that's the problem- policies getting blindly applied. The "Monkey See Monkey Do" style of "Best Practices".

It's a rare gem when you find an organization with the self-discipline to recognize a policy that needs refining or simply removed. Or temporarily ignored on a non-precedential basis. A good set of policies are a balancing act. Too many and you get buried under their weight. Too few and you get buried under your own weight. That's The Tipping Point.

Dear Recruiters

Start with what is right rather than what is acceptable." --Franz Kafka

An open letter to recruiters. Mainly recruiters for software developers, but I suspect much of these are reasonable requests for all sectors.

1. Please stop saying "Leading provider of". Sure- you want to give the idea that your client is A Major Player. Fine. All good. But using rehashed buzzwords doesn't convey that idea. The phrase is like the paintings in a hotel room. If you even notice them, they have no real meaning. Instead, tell me why your client leads their industry. What have they accomplished? How do they lead? Is it through use of new technology? New innovations? Say something meaningful. Something that will have an impact.
2. Please stop telling me how many years your client has been in business. This may be useful information for clients, but potential employees aren't looking to retain their services. They're looking for a culture that fits. Generally, the amount of time a company has been in existence is irrelevant. Instead, talk about the culture. Not everyone wants to wear a suit and tie and interact with a rigid corporate structure. But not everyone wants to work in shorts and sandals, sitting on a bean bag, and walking into a Director's office for casual conversation. Laying this out will help find candidates that are a good fit for the company.
3. Proofread your communications. Twice. When I see "I think you'd be a good fit for [Job Description]", I delete the email. Same with blatant misspellings. Anymore, any text entry application comes with an automatic spell checker.
4. Read the resume. The most recent position is where your candidate has moved in his career and it's reasonable to think that that the position on purpose. If your contact is in a leadership, planning, or strategy position, it's probably a waste of time to offer direct development work. Your contact has moved on to something else and it's fair to assume that, barring unemployment, moving back isn't likely to happen.
5. Be specific. "Maintaining large applications" or "Excellent communication skills" doesn't tell me anything. Again, these are hotel room paintings. What I want is a good snapshot of my day-to-day activities. Does "Sr. Application Developer" mean "Application Developer with a lot of experience"? Does it mean "Responsible for the work of a development team"? If I don't know, I'm less likely to take a chance. And that's exactly what you're looking for. Someone who will take a chance on your job offering being better than the current one.
6. This is likely somewhat out of your control, but stop with the "X years of experience". While I realize that your client is telling you that, under the misapprehension that years of experience is an accurate measure of anything, you shouldn't have to tell that to candidates. If you can't tell from the resume how many years of experience the candidate has in the necessary technologies, then the resume has been badly written.
7. Same with this "Degree needed" or "Degree or relevant experience needed" silliness. Aside from this not being any measure of anything important, it's pointless to include in a communication to a specific candidate. If your candidate doesn't have the necessary degree, don't bother. Likewise with "Degree or relevant experience". If your candidate doesn't have either, they clearly aren't a good fit.

So, what's the bottom line here? Much like the fact that a good candidate needs to stand out in order to be seen as a good candidate, recruiters need to stand out in order to be seen as a good recruiter. You can't afford to be a carbon copy of everyone else. You can't just be hotel room art, and you can't hand over a vague description with ubiquitous buzzwords and expect to spark any interest. Instead, you need to capture the candidate's attention. You do this through clear and concise communication of what your client is looking for. You do this by standing out.

A thought to keep in mind. All the advice recruiters give to job seekers about how they present themselves through resumes and interviews? That should apply to your communications with candidates. Especially your first contact.

Agile Isn't Dead

"Best practices are useful reference points, but they must come with a warning label : The more you rely on external intelligence, the less you will value an internal idea." ― Gyan Nagpal, Talent Economics: The Fine Line Between Winning and Losing the Global War for Talent

Agile isn't dead. It's not even on life support. However, to extend an analogy that everyone in the world should stop using, it does have a couple of CAT scan anomalies it should get checked out.. I've read a few tech opinion pieces to the contrary lately, so I thought I'd throw my hat in the ring.

+Hayim Makabee wrote an excellent piece on the subject, but one that I feel missed the mark somewhat. Jim Bird wrote on a finer grained level on the subject, and if you take them together an interesting picture starts to emerge. Starting with Jim Bird's piece, he lists several "Agile" practices that aren't needed. Skipping by a few points on which I disagree, his overall point is sound. Not all common practices are necessary. Hayim Makabee takes this a step further, rightly pointing out how "Agile" or "Scrum" consultants "over-simplify the software development process and underestimate the real complexity of building software systems". And I'm not really arguing that this "one size fits all" approach is a problem. I believe that the problem's source hasn't quite come out yet.

In short, the problem is IT decision makers. Not Agile, which is just a set of principles. Not Agile consultants, who are just selling a service people have asked for. And while we can discuss the moral culpability of someone "just selling a service", the fact of the matter is that people are buying the service. And those people are IT decision makers in pursuit of a concept that has made me cringe for some time now. Best Practices. "Best practices" are like the Dark Side of the Force. Quick, simple, seductive, and once you walk down it's path, forever will it dominate your destiny. "Best Practice" is shorthand for "What everyone else is doing", and while it's the sort of thing one ought to take into account, I've seen enterprise after enterprise implement iron-clad procedures based solely on "Best Practices", or rather "Just do what the neighbors across the street are doing". And before too long, you have Citogenesis, that curious phenomena where people think something is correct because people think something is correct.

Agile is a set of principles, and as far as principles go, they're pretty darn good. Agile isn't dead because these principles are still the best way to create software. But it really needs that dark spot checked out.

Life is Agile

"It's not what happens to you, but how you react to it that matters" --Epictetus

I'd like to pose a question to any parents reading this. When your child was born did you plan out every detail of your child's life? Did you attempt to account for every problem and develop a contingency plan before even leaving the hospital?  Do you parent in a vacuum, having no contact with other parents?

Similar question to those of you with active careers, be it in the job market or at home. Yes- being an at-home parent is a career. Did you plan out every step of your career before you began working? Do you work without interaction with others in your career? Did you plan for every problem to the most minute detail?

Of course not. To approach such a major endeavor in life is more like planning to fail. And this is why Agile is the only way to successfully complete a software development project of any consequential size. Because it mimics human behavior and human capabilities. At it's core, the Agile methodology is an admission that you can only do so much to plan for the future. It's a system built on the foundation of communication with others. It does not merely recommend regular and effective communication, it states that failing to do so will cause overall failure. It is built around doing what you need to now, while keeping an eye on the future. Similar to how you eat an elephant (one bite at a time), so do we manage our lives. Why should we assume that managing a development project should run so completely contrary to our fundamental behavior?

The converse is also true. It's been a very long time since I've had to defend Agile vs. Waterfall/BDUF. The reason appears very straightforward- the proof is in the doing. Agile has proven itself. It's not theoretically better, it simply is.

How, then, can we extrapolate this to our lives? Should you find yourself working at a company where you have no future, what do you do? Do you scrap the idea of working at your best capacity while you look for another job? Agile says differently in that a development team has the discipline to finish a development sprint, i.e. the work in front of them.

If you want something in your career, do you just keep working and hoping it happens? One of the foundations of Agile is communication. Do we keep grinding and hope, or do we approach others for advice? And then do we approach management and clearly communicate what we want?

Life is Agile is life. The lessons we learn from each can be applied to the other.

Healthcare.gov failures in leadership

"Good management consists in showing average people how to do the work of superior people." --John D. Rockefeller

"Good management is the art of making problems so interesting and their solutions so constructive that everyone wants to get to work and deal with them." --Paul Hawken

Yep. I'm back at this well. Partially because it's a great way to boost hits (I don't use ads, but I do have an ego) but mostly because I've been there. Not in a project this size, but I've lived the nightmare. Maybe someone that can affect this boondoggle

reads this and listens. Likely not. I have an ego, but I'm a pretty small fish. More important to me is that the developers and IT staff affected by projects like this understand the failings so that they know when to update their resume.

This one is coming from a New York Times article titled Inside the Race to Rescue a Health Care Site, and Obama. And again, whether Ms. Stolberg and Mr. Shear realize it, they paint a picture familiar to many IT veterans.

Failures of Testing

I've written about the technical failures of the development teams but what has been written in this article serves to underscore something all development teams know, but fewer do. Testing. From the article:

"To do that, they would have to take charge of a project that, they would come to discover, had never been fully tested and was flailing in part because of the Medicare agency’s decision not to hire a "systems integrator" that could coordinate its complex parts." 

"The website had barely been tested before it went live, so a large number of software and hardware defects had not been uncovered." 

"'There’s so much wrong, you just don’t know what’s broken until you get a lot more of it fixed,' Mark Bertolini, the chief executive of Aetna, said on CNBC."

Regression testing. Unit testing. User Acceptance testing. I can't think of a better way to talk about the dangers of not budgeting time to properly test your project.

"In Herndon, as engineers tried to come to grips with repeated crashes, a host of problems were becoming apparent: inadequate capacity in its data center and sloppy computer code, partly the result of rushed work amid the rapidly changing specifications issued by the government."

Code reviews and proper architecture concern would have prevented the "sloppy code" issue. The rest of this segueways nicely into my major point, thought.

Failure to understand a software development project

Clearly, project management didn't know how to manage a large-scale project. I've said before that the evidence could not be clearer that this project was managed Waterfall-style. And while the far-preferable Agile development style can also struggle with changing requirements, the iterative workflow, combined with constant checkpoints with the business stakeholders allows for better options for dealing with the fact of life that is spec change. Here are a few examples from my playbook:

"No, we will not change specifications in mid-sprint. If you want this change, submit it to the Icebox and give it a priority. We will then do the design and estimation work and add it to a future sprint."

"No. This is a two week sprint. We will not finish early. We will not rush. We have a planned amount of work that will take two weeks to deliver properly."

"No, we will not add to this sprint. The sprint covers an amount of work that can be accurately delivered in two weeks."

"No, we will not include this work into a sprint until there is proper user acceptance criteria attached. The delivered code will then conform to the acceptance criteria listed. No more, no less. I'm here to help you write good acceptance criteria."

Enforcing this kind of project discipline is crucial. It should be well-communicated up front. Everyone should know what an iceberg is. And once the expectation is set, that expectation is enforced. But even with that, the problem runs far deeper. Some quotes from the article really chilled me:

"Out of that tense Oval Office meeting grew a frantic effort aimed at rescuing not only the insurance portal and Mr. Obama’s credibility, but also the Democratic philosophy that an activist government can solve big, complex social problems." 

"'We’re about to make some history,' she (Ms. Sebelius,) said.

"...reveals an insular White House that did not initially appreciate the magnitude of its self-inflicted wounds"

As any regular readers know, my overall philosophy comes from Marcus Aurelius:

“This, what is it in itself, and by itself, according to its proper constitution? What is the substance of it? What is the matter, or proper use? What is the form, or efficient cause? What is it for in this world, and how long will it abide? Thus must thou examine all things that present themselves unto thee.”  

By itself and in its proper constitution, Healthcare.gov is a website that allows people to purchase health insurance. It is not a justification of a political philosophy. It's not about making history. Those are incidental. You cannot make good decisions if you don't understand the context for those decisions. You can't make good decisions about a development project if you see it as anything other than a software development project. And don't think for a second that minimizes its importance somehow. Most developers I know care a good deal more for their code than they do your politics. But if management mistakenly sees a project as some kind of ideological movement or as setting their legacy then they aren't making project management decisions.

Management culture

Finally, what may be the worst failing of the management of this project. It's culture. The standards management sets for its operations. The only place I've ever heard of management leading so poorly is in Dilbert. For example:

"For weeks, aides to Ms. Sebelius had expressed frustration with Mr. McDonough, mocking his “countdown calendar,” which they viewed as an example of micromanagement."

Mocking other stakeholders. Ignoring, for a moment, the notion of expecting professionals to not act like spoiled children, why was this behavior considered acceptable? Management sets the standard for how people act. A good manager understands this. A good manager sets an expectation of professionalism. Complaining is one thing. Open mocking should be the sort of thing people are embarrassed to do, or even to listen to.

"Contractors responsible for different parts of the portal barely talked to one another, hoping to avoid blame."

There's enough written about how a management culture of blame creates a toxic environment. Problems aren't addressed. People pass the buck rather than address problems. The concern is to not be left holding the bag, rather than making sure work is done well. Fear does not create quality. And clearly, this is not known here.

 "Mr. Obama, meanwhile, was under assault. After years of telling Americans, “If you like your insurance plan, you can keep it,” he was being accused of lying. On the night of Oct. 28, Ms. Jarrett, one of Mr. Obama’s closest confidantes and a guardian of his personal credibility, took to Twitter to defend him — and to shift the blame. 

“FACT,” she wrote. “Nothing in #Obamacare forces people out of their health plans. No change is required unless insurance companies change existing plans.”

The tweet touched a nerve; it was not the first time the Obama White House had used the insurance industry as a scapegoat. Ms. Ignagni’s (chief executive of America’s Health Insurance Plans,) members were furious. “Here it comes — we knew it would happen,” one executive recalled thinking."

 The Obama administration built support for the ACA by building hostility to the insurance industry, with whom they must now work. So, in essence, the Obama Administration has taken every opportunity to publicly condemn the insurance industry and is now relying on that same industry to launch not only it's centerpiece website, but the validation of their political philosophy. A management team that does not treat their vendors with respect will find themselves very lonely once they need their vendors. UnitedHealth and CIGNA have already "mostly shied away from the online marketplace" (From the article). To assume this has nothing to do with the administration's clear contempt for their vendors is folly.

I have a difficult time summing up my shock at the sheer number of leadership failures involved here. Not just in knowing how to run a software development project- clearly Medicare and the Dept. of Health and Human Services considered this a "Fire and Forget" project and that they were absolved of all responsibility to be involved once the project started. More disturbing, though, is the culture that management allows. Childish behavior and finger-pointing should not be acceptable. And that attitude starts from the top.

Dedication and Vision

"If you do not have an absolutely clear vision of something, where you can follow the light to the end of the tunnel, then it doesn't matter whether you're bold or cowardly, or whether you're stupid or intelligent. Doesn't get you anywhere." --Werner Herzog

Short article today.

Ran across something that really made me think about software development projects. Three developers in San Francisco built an alternative to Healthcare.gov in two weeks.

Two weeks.

Now, you can't purchase- just search and compare rates and plans. But again. Two weeks.

No complicated procurement process. No contractors. Certainly less than hundreds of millions of dollars. Just three dedicated guys with a clear vision.

A G+ friend of mine +Chaka Hamilton took this a step further:

I bet if the gov offered a darpa style challenge, they'd have a working website in half the time, and cost. They obviously have learned nothing from open source / crowd source community.

 Indeed. What if they had.

A New Perspective

"I believe everyone should have a broad picture of how the universe operates and our place in it. It is a basic human desire. And it also puts our worries in perspective." --Stephen Hawking 

"Everything we hear is an opinion, not a fact. Everything we see is a perspective, not the truth." --Marcus Aurelius 

This  article falls more heavily under the "Musings" title of my blog. I'm less making a point than I am thinking out loud. As always, feedback and insights are always welcome.

My job responsibilities are changing. I phrase it like that because I doubt my actual title will change, merely the meaning of that title. I rarely handle project level work anymore. Rather, I'm more involved with enterprise level architecture decisions. I haven't implemented a design pattern in quite awhile. I find myself, instead, setting the standard of what patterns are best to use or avoid in certain situations. Or what frameworks we will be using or whether we will use an off the shelf solution or build our own. In other words, my implementation decisions are becoming less important than my opinions and experience with those decisions. I find this a very new perspective and more than anything else, I find myself more and more writing about new perspectives on old ideas.

For instance, take our current investigation into unit tests. The discussion started with "What mock object framework should we use?" We quickly boiled down to "Which framework will be unduly burdensome to the development staff?" This actually eliminated a couple of frameworks at the beginning. But when we settled on two that are, more or less, of equal use the conversation quickly changed to unit test standards. I have a few strong opinions on the matter. I believe that unit tests should cast a wide loop so that behavior consistency can be assured. If that means mocking Internal methods to assure their consistency, then so be it. If that means only using strict mock objects, despite their fragility, then so be it. In fact, I like fragile unit tests. If the behavior of the class changes then the tests should break. I realize that my opinions are not shared by the community at large, and I'm okay with that. I'm always open to debate, but I approach things somewhat differently than normal. I think I've made that clear. Now, of our enterprise level architects, one disagrees with me and the other is still weighing arguments and that's great. That kind of conversation is a new perspective for me.

So what's this new perspective and what does it give me? Because I'm no longer considering patterns and practices for a given set of circumstances, but rather to be followed in the enterprise, I have to more seriously consider the pros and cons of those patterns and practices. I find that thinking of effects of standards on the enterprise at large makes me think differently about the effects of my design decisions at the project level. Not just "Does this work here" but "Would this work in other, similar, situations and if not, why not?" If the answer to the second question is "No", then should I reconsider my decisions at the project level. Note that I'm not offering concrete conclusions here. I'm expanding my perspective and thus expanding the pool of questions I ask myself before making a decision.

Maybe that's my point here, although I expect I'll be getting comments about my approach to unit tests. That's fine, too. The day I stop listening to others is the day I stop being useful.

The Unambiguous Measure of Success

"[T]he presence of an unambiguous measure of ex-post success (profit) serves to harness the natural tendency toward overoptimism that otherwise would almost certainly be present when someone else’s money is being spent." --Robert Wagner, "Economic Policy in a Liberal Democracy"

Every once in a while I'll come across a quote or an article that makes me think about software development. Often that's because I tend to read a lot of material related to software development, but sometimes it's not. Such is the case of Donald Boudreaux's Quotation of the Day for October 23rd. The full quote is:

[T]he presence of an unambiguous measure of ex-post success (profit) serves to harness the natural tendency toward overoptimism that otherwise would almost certainly be present when someone else’s money is being spent.  The necessity of putting one’s money on the line and of being responsible for the ultimate outcome surely has a sobering effect on the assessment of the prospects for such projects [that governments typically undertake], an effect that is weakened when tax money is used in a setting where no judgement about profitability has to be faced.

I'm no economist and I don't pretend to be one. And this isn't a post about economics, anyway. What caught my eye was the idea that putting one's money on the line and being responsible for the ultimate outcome by setting an unambiguous measure of success. Even though we all know, often through painful experience, why clear and unambiguous project goals are a necessity, I think it's interesting to look at it with an economist's viewpoint.

Instead of "someone else's money is being spent", let's use "someone else's resources are being spent". In other words, not just the salaries of development staff but also time and infrastructure. If the burden of this is borne solely by the development staff, then the tendency of the customer stakeholders is toward overoptimism. Features, both initial and scope creep, and timelines all trend toward pushing the limits of what the development staff can reasonable accomplish. At least, that's been my experience.

A couple of things happen when the customer is expected to give a clear and unambiguous measure of success. Not just in terms of a requirements document, but close involvement in the development process both in defining clear acceptance criteria for user stories and in reviewing the results of development sprints. The customer is now spending their resources on the project. Their staff has to be available for clarifying requirements. Their staff is has committed their time to insuring that the "measure of success" is being met. And their staff also has to budget, and therefore use effectively, their time for the project in relation to the time needed for other tasks. The tendency of those with a stake in the game is to be more careful with how those resources are spent and to insure that resources aren't wasted. I guess this makes sense, really. People tend to spend money more frivolously when not using their money, or when it doesn't look like they are using their money. It's why managing credit can be tricky and it's why casinos use chips instead of currency. Why should spending resources on a project be any different?

I like this quote. It's a truism of software projects, for that matter projects in general, that you can't finish a project if you don't know what "finished" looks like. I'd never stopped to think about how a clearly defined measure of success affects the customer in a project.

Health Care Exchange Project Pt. 3

"One test is worth a thousand expert opinions." -Wernher Von Braun

And finally the last piece of the puzzle. As a software developer, I find the previous types of project problems maddening. However, this last category boggles my mind. Perhaps I'm just an idealist, but I truly want to believe that this sort of thing doesn't happen anymore. Sadly, I read The Daily WTF far too often to really believe it. We are now down to technical failures.

Again, quoting from From the Start, Signs of Trouble at Health Portal

"Others warned that the fixes themselves were creating new problems, and said that the full extent of the problems might not be known because so many consumers had been stymied at the first step in the application process."

"'So much testing of the new system was so far behind schedule, I was not confident it would work well,' Richard S. Foster, who retired in January as chief actuary of the Medicare program, said in an interview last week."

We all know that maintenance, especially bug fixing, is the true bulk of any software development work. And we all know that testing is the heart of finding, and therefore fixing, bugs. This is not under dispute. However, I've been associated with so many development projects that ignore this basic principle that I want to weep sometimes. And it's always the same. "We don't have time to test because we're busy building features". Or "We'll focus on testing later". Or the worst, "We'll worry about bugs when they're reported by users". (Yes- I've been told that)

To be clear. Unit tests insure that a unit of code has a consistent result at any point in time. It doesn't insure that the code does what it is supposed to. It insures that what the code does hasn't changed due to other factors. Unit tests are how you do regression testing. At least, how you do it without resulting in Cthulhu-level madness.

User Acceptance Testing insures that the users can actually perform the tasks called for in the specifications. This doesn't happen at the end of the project. This happens at planned stages throughout the project so that testing happens on a manageable set of features. A set of features that can be easily documented, easily described, and easily managed. Failure to do this step before rollout is inexcusable.

And while we're at it, what about Exception handling testing? What effect does any given exception have? How is it reported, both to support and to the user? How are exceptions tracked? Testing isn't just about making sure the application works well. It's about insuring that it fails gracefully.

"The biggest contractor, CGI Federal, was awarded its $94 million contract in December 2011. But the government was so slow in issuing specifications that the firm did not start writing software code until this spring (Em. mine- MO), according to people familiar with the process."

I'm pretty okay with most of this, but the failure is so bad that it's worth mentioning. The award amount doesn't bother me. I'm also pretty okay with two years of requirements. This isn't like turning on a switch and watching everything work. This is a serious development project- far more serious than anything I've participated in. I would have been more surprised to see the award amount or the planning time significantly lower.

But read the bit I emphasized. Development didn't start at any point during planning. In other words, a project with this level of work and complexity was attempted Waterfall-style. Not Agile. Waterfall. In a project like this, the technical leadership deliberately passed on the ability to easily reach to changing requirements. And the ability to work on completed requirements as they become available. And on increased involvement between development and stakeholders. And continuous testing.

I'll take the heat for saying this- Agile isn't a buzzword. It isn't a topic for bloggers to discuss. And it isn't an alternative methodology. It's the only sane way of approaching a development project of any more than a trivial size. You simply can not anticipate everything ahead of time, and attempting to do so harms the project more than it helps. Case in point.

All of the problems the NYT article describes are serious. Any more than one or two of them will probably sink a project. The fact that people are reporting this many fundamental mistakes makes this an example everyone familiar with software development should understand. If only to protect your career.

Health Care Exchange Project Pt. 2

“No matter how good the team or how efficient the methodology, if we’re not solving the right problem, the project fails.” - Woody Williams

In my previous article I started talking about the New York Times article From the Start, Signs of Trouble at Health Portal. See previous article for the disclaimers that hold here as well.

In Part Two, I want to talk about the parts of the article that, to me, describe a major failure in project leadership. Not to be confused with executive leadership. In this case, the described failures in those responsible for the actual project leadership.

"Failure to plan is a planning to fail". It's a cliche for a reason. Starting a development project without a plan, or with an obviously flawed plan, is a massive waste of time and money. From the article:

"Dr. Donald M. Berwick, the administrator of the federal Centers for Medicare and Medicaid Services in 2010 and 2011 'The staff was heroic and dedicated, but we did not have enough money, and we all knew that,'"

From the beginning, we have a serious issue. The project wasn't funded.  The only way this works is if you have a plan to scale back what you can't pay for. The quotes in the previous article regarding executive leadership make this an impossibility, however.

 "Some people intimately involved in the project seriously doubted that the (Medicare and Medicaid) agency had the in-house capability to handle such a mammoth technical task of software engineering while simultaneously supervising 55 contractors."

"The political people in the administration do not understand how far behind they are." 

Well, now we have some insight into some of the executive leadership issues. Project management is supposed to be responsible for insuring that the correct groups are responsible for units of work and are responsible for reporting progress. It very much sounds to me like the project management team fell flat here. This is by no means unique to ACA, nor even to government development projects. I've seen, far too often, project managers who think their job ends with the kickoff meeting. Or who only schedule meetings and do little else. A good sign of a sinking project is negligent project management. Worse is project management that is unfamiliar with what the role entails.

"A round-the-clock effort is under way, with the government leaning more heavily on the major contractors"

"Worried about their reputations, contractors are now publicly distancing themselves from the troubled parts of the federally run project."

"Senior executives at Oracle, a subcontractor based in California that provided identity management software used in the registration process that has frustrated so many users, defended the company’s work. 'Our software is running properly,' said Deborah Hellinger, Oracle’s vice president for corporate communications."

How often have you seen this story play out? Lack of planning and poor leadership lead to "crunch times". As a result, development staff is required to work late. Demands raise past what is reasonable and soar up to the ceiling of what is possible. The result?  Discontent and CYA. The quotes above tell me that the contractors have already given up on the project and its leadership. Worse yet, the "blame game" has gotten into full swing. Blame doesn't happen when the project staff sees the project as salvageable. Blame only happens when the project is seen as a loss and people only want to salvage their careers. In a very real way, the blame game prevents problems from getting fixed.

Project management problems are a warning sign that is difficult to see. Oftentimes, poor project leadership isn't obvious until the project is well under way. At that point, recovering the project can be problematic. Tasks have already been assigned inappropriately, Reporting is either far behind or nonexistent. And at the worst, although I didn't see any evidence of this in the article, poor project management often involved a lack of clear project goals. This last is, in my opinion, the worst way project management can fail. If a project without clear end goals is allowed to continue, failure is the only possible result.

Health Care Exchange Project Pt. 1

“I have witnessed boards that continued to waste money on doomed projects because no one was prepared to admit they were failures, take the blame and switch course. Smaller outfits are more willing to admit mistakes and dump bad ideas.” - Luke Johnson

Let me start off saying that this isn't political commentary. If you want to talk about whether or not the Affordable Care Act (ACA) is a good idea or should be repealed, please go elsewhere and don't pollute my blog with political commentary.

Earlier today, I came across NYT article headlined "From the Start, Signs of Trouble at Health Portal" and written by Robert Pear, Sharon LeFraniere, and Ian Austin. Before going on, I highly recommend reading the article. I also found a very readable companion piece written by Megan McArdle.

The reason it caught my attention isn't that it is about ACA but rather the amount of project failures that I can relate to. The signs of common project management mistakes are so obvious, anyone experienced with software development projects will find themselves nodding their head with nearly every paragraph. This article doesn't describe a mere failing project. The failures described here are so blatant, so numerous, that I have to wonder if we're looking at deliberate sabotage. Or possibly the article is merely social satire.

The point of this isn't to slam the ACA, let me make that clear. The desired end of the project isn't what caught my attention. Nor am I trying to "discredit" the ACA. This is a real world study of how to run a software project into the ground and guarantee its failure before the first line of code is written. This article should, absent all else, serve as a warning to those undertaking software development projects, and indeed projects of any kind.

As this is far longer than my usual blog posts, I'm breaking it up into three parts, by what I see as logical categories of the project's failures. For the first one, lack of executive leadership.

It's frustrating when software projects are crippled by executive-level politicking. Executive-level leadership is foundational to a project's success. And like a building with a flawed foundation, poor executive leadership will destroy a project in unexpected ways. I have been involved in several projects that lacked executive-level leadership. All collapsed. Quoting the NYT article:

"Politics made things worse. To avoid giving ammunition to Republicans opposed to the project, the administration put off issuing several major rules until after last November’s elections. The Republican-controlled House blocked funds. More than 30 states refused to set up their own exchanges, requiring the federal government to vastly expand its project in unexpected ways."

"Administration officials dug in their heels, repeatedly insisting that the project was on track despite evidence to the contrary." 

"Mr. (Henry) Chao’s (Chief digital architect) superiors at the Department of Health and Human Services told him, in effect, that failure was not an option, according to people who have spoken with him. Nor was rolling out the system in stages or on a smaller scale, as companies like Google typically do so that problems can more easily and quietly be fixed." 

I see a few serious warning signs here. First, not all stakeholders were on board with the project. I don't care what rationale you use, if the stakeholders aren't on board, the project is doomed. This is as close to political as I want to get here. Neither the reasons the Democrats had for steamrolling the Republicans on the ACA nor the reasons the Republicans have for trying to kill it matter in this context. The unassailable, unarguable, truth is that if a major stakeholder want to kill a project then the project is dead. Ideology does not change this, nor does intent. Whether the result of the project is necessary, helpful, or detrimental is also irrelevant. And it is inexcusable for executive leadership to begin a project like this, through intent or ignorance of This Law, with such high level opposition.

Just as bad, however, is the unwillingness to acknowledge that the project is in trouble, but rather relying on the "Failure is not an option" method of resuscitating a project. Take a moment and count the number of times someone told you "Failure is not an option" or "Just make it happen" and it actually helped. Go ahead. Raise your hand if you got a number above zero. Anyone out there have their hand raised?

Didn't think so.

Attention all managers- these phrases do not solve problems, nor do they create an environment in which problems can be solved. For the love of all that is holy, stop using them.

Several reasonable suggestions are on the table. Postponement. Small scale rollouts, Google-style. The reasons given for ignoring these options are just about the worst possible. Executive-level politics.

Executive leadership enables development projects in ways that many people on the project never see. Because of that it's difficult to see when executive leadership is failing. Sadly, the results are less hidden.