The Binary Fallacy

Two roads diverged in a yellow wood,
And sorry I could not travel both
And be one traveler, long I stood
And looked down one as far as I could
To where it bent in the undergrowth;
-Robert Frost, The Road Not Taken

Binary Thinking is the process of thinking of things in terms of two opposites. You can either do A or it's opposite B. But that's it. In Neuro-Linguistic Programming, it's a technique for manipulating someone into making the decision you want by framing the choice as a matter of either the thing you want them to do or the opposite, presented as a harmful choice. In Cognitive Behavioral Therapy it's a pattern to be broken to help people gain more control over their lives. In Software Architecture it's a warning sign.



If stuck between two different ways of designing something, the answer is always door number three.

Why do humans think like this? Why didn't Frost just keep walking straight through the woods and not worry about the roads? Why didn't The Clash stop worrying about staying and going and just Rock the Casbah? I don't really know. I'm not a psychologist. I'm just a software architect who's familiar with Analysis Paralysis. The phenomenon of getting nothing whatsoever done because you're locked up in the decision making process. And after years of trying to figure out how to avoid this, I came to one inescapable conclusion.

When you're in analysis paralysis it's because every path you're considering is wrong

In my opinion, this happens when you realize deep down that you're looking at the problem wrong and considering bad options. You can't decide because you know your options are both bad.

So what do you do? Well, that's the tricky part and adages about in which part of the box to think aren't as helpful as people who use them think they are. If thinking of something new was as easy as the realization that we need something new then Elon Musk wouldn't be quite the icon he's become. The important question is HOW? First, you just need to stop and accept the fact that you need to come up with something different. Your first decisions were wrong and you need to set them aside. Don't go back to them. Then I recommend you look at the great Stoic thinkers. You start by asking "What is this and what is it for?" I often tell developers to answer those two questions for every application, for every class they build, for every database and table they create. And you don't build the thing in question until you can answer those questions.

Then you ask if a thing is needed or just wanted. If it's not something you need, based on the answers to the above questions, then you leave it out. You strip away the irrelevant. Often I find myself locked up because I can't find a clean way of integrating the irrelevant into my design. Once I stop and realize the piece that doesn't fit doesn't need to fit, things get easier.

I've often said that software development is 90% thinking and 10% typing. If you understand how to think clearly, how to organize your thoughts, and how to tell the difference between the necessary and the unnecessary then there's little that can keep you from being a great software developer.

You've Got One Job

"If you choose to not deal with an issue, then you give up your right of control over the issue and it will select the path of least resistance." -Susan Del Gatto, Creating Balance in a World of Stress: Six Key Habits to Avoid in Order to Reduce Stress

Never forget that your only job is to provide a solution to a problem



Stick with any career long enough and you start to develop a set of principles that guide your approach to your job. Call it experience, call it being set in your ways, even call it wisdom if you dare. But we all do it. I've got about twenty of them, which I realized after an NCIS binge when I started writing them down Gibbs' Rules style. Rather than blasting them out all at once, over the next several articles I'm going to talk about each one. There's no particular order, other than the order in which I wrote them down. However, if I had to pick one to be the most important, it's the one I'm starting with.


Even at the beginning of my career, I never really thought of myself as a software developer. Rather, I thought of myself as a problem solver, albeit one who tended to approach solutions with code. But that idea has always stuck with me. Always be solving a problem.

Now, not all problems are created equal. Some are quite big. For instance, Linus Torvalds solved the problem of having to pay money for an operating system. Some seem almost insignificant. Recently I've been working on the problem of "I don't know Python as well as I'd like to". Something of import to almost no one other than myself. But that's the point. It's someone's problem, and it's getting solved.

Of course, what I'm really getting at is why you write software professionally. And while we all basically realize that we need to fulfill requirements, implement stories, or however we get the list of stuff we're supposed to write, it's easy to lose track of the big picture when you're in the weeds of the little picture.

The problem is that we, as software developers, don't just like to develop software. We like to develop cool software. However we define that, but usually as "the last new idea I read". Admit it- you do it, too. Goodness knows I do. We like to learn new things, we like to try new techniques, because we like, no we need to grow. This rule is to remind me not to take that too far. Ultimately, it's not about what I want, it's about what solves the customer's problem. Because if I'm not doing that, I'm wasting everyone's time.

I've also ran into situations where solving the problem didn't have anything to do with new software, custom software, or even software at all. I've on occasion been able to trim large portions out of a project with suggested business process changes. Or by pointing out that the requested software doesn't actually solve their problem, and on occasion might even compound it. A good software developer knows when to not develop software.

The idea of solving a problem is a big one, and one that spans several of the principles on my list. But this one gets its own item because it all boils down to this simple concept. You're not writing software to write software. You're solving a problem.

                                                                 

Finding Valrhona, or Habits of Effective Architects

I have very strong opinions on the subject of software, architecture, and quality in general. Coors is not beer. A Hershey bar is not chocolate. Neither Velveeta nor Kraft Singles are cheese. Starbucks does not serve anything I identify as coffee. "Cowboy Coding" is not software development". This really isn't about my quirks in food quality. More a list of items that I find helpful in making sure I'm helping to deliver Valrhona.

Understand what you need to deliver

Before you select technologies, before you start with the design patterns, and certainly before you put hand to keyboard, make sure you understand what pain point you're relieving for your customer. Software development is about solving problems. So many times, I see projects skipping right over the problem to be solved and heading right to implementing a solution. Oftentimes, talking through the problem to be solved makes a murky solution obvious. If you're stuck, ask yourself "What problem are we solving?" If you don't know, you know what to do next.

Solve for realistic problems

If you don't need a full enterprise-y solution with distributed widgets, Something As A Service, and Fully Configurable Everything, don't build it. This is kind of an extension of the previous point, but for every architecture decision you make ask yourself "What value is this adding to the solution?" if you don't have an answer then you don't need the Thingy.

On the other hand, understand that no software project is ever finished and that no set of requirements stay static. Especially once development starts. As Helmuth Graf von Moltke said, "No campaign plan survives first contact with the enemy". There are well established patterns for solving common problems. A thorough understanding of the Gang of Four's design patterns will go a long way in helping you avoid common pitfalls. Don't use them just to use them, but don't avoid them just because they're common. They're common for a reason.

Stop. Collaborate and Listen.

Okay, for those of you who get the reference, I apologize. Note there's no link. I don't want to infect those of you who don't get it. But, origin aside, it's good advice for the architect. Even if you're sure of yourself, get feedback. A development team is more than the sum of its parts, and several smart developers working together produce far better results than several smart developers working separately. Capitalize on this. Ask for comments and then listen to them. Especially the criticism. The worst that can happen is that you'll feel more confident in your design.

Along those lines, keep up on the current trends in software developemnt. I'm not saying you have to be KanBanAgileScrumTDD just because others have written about how shiney they are. But you won't know how these concepts can, or can't, help if you aren't familiar with them.

Strive for Elegance, but understand what it means

To me, an "Elegant" solution is not necessarily overly-clever. It does not have to solve problems in a new way. And it certainly doesn't take Donald Knuth to understand. To me, "Elegance" makes the solution look easy. Sure, maybe you come up with a better way of solving a problem. But maybe you recognize that some techniques are "Tried and True" for a reason. Either way, your result shouldn't look like a bunch of work. It should look obvious.

Know when to say when. And when not to.

Understand that at some point in your career (or, in my case, at some point in your day), the pursuit of higher quality will conflict with the overall effort in such a way that the pursuit does more harm than good. be able to recognize that time and let go.

Understand that at some point in your career, you will be expected to sacrifice quality for the overall effort in a manner that does more harm than good. Don't dig in. Don't get stubborn. Learn to present your case in terms that the decision makers understand. You will not always get your way, but you will become known as an asset that is always looking out out for the overall project.

Note that there isn't a lot of actual code advice here. Sure, I could tel you that if you're instantiating different types of classes depending on context, consider an Object Factory or even an Abstract Factory. Or if you have a somewhat complex process that other processes interact with, or a subsystem that might change, consider a Facade. I could give you the ol' "Design from the Interface" advice or even tell you that if you find yourself considering recursive queries maybe you should step back a bit. But I think that if you really take the above to heart, everything else is just details.

Clay Pots

"Perfection is not attainable, but if we chase perfection we can catch excellence." --Vince Lombardi


I tried to find a link to the experiment, but could not. Perhaps it’s allegorical. However, I read once about an experiment done by a pottery teacher. She divided the class into two teams. She told the first team to make the perfect clay pot. She told the second to simply make as many clay pots as they could. At the end of the experiment, the perfect clay pot was indeed made, but not by team 1. As it turned out, the constant iterative practice by team two trumped the careful work of team one.

NOTE: Thank you to +Dave Aronson for pointing me to the link I couldn't find!
http://kk.org/cooltools/archives/000216
He also has some thoughts on the subject at http://www.dare2xl.com/2010/08/just-do-it.html

This is not an article about getting better at software development the more you develop software. If you’re reading this, you already know that. No, this is about software architecture and building the perfect design. Which, as I explained in my first article, doesn't exist anyway.

Every Software Developer/Architect/Engineer/Whatever that I've worked with has shared a couple of characteristics. They want to get their work done right, and they take time to think through what they're doing before they do it. Both of which are commendable. The problem comes when this leads to Analysis Paralysis. When the process of thinking things through in order to make the perfect design deadlocks the developer and he can't move on.

When that happens to you, remember the clay pots.

Agile methodologies such as Scrum and XP were developed, in part, to avoid analysis paralysis at the project level. With a focus on action and testing the results, agile methodologies seek to create the perfect pot by creating pots until they get it right. As it turns out, this technique works just as well at the individual level.

Sometimes the best way to break through design indecision is to just start writing it. Build the class stubs, make them interact, and build unit tests around them. How well does it work? How badly doesn't it work? Then consider what worked, what didn't, refine your ideas and start over. Wash, rinse, and repeat until you’re happy. Or at least satisfied. Or at least still on this side of “I’m so frustrated I’m about to throw my laptop through a window”. Seeing how the design plays out and forcing yourself to refine and retest can often lead to better results than trying to think through every detail in advance so that you create the "perfect" design the first time.

Don’t get me wrong. I’m not advocating against careful thought. I’m not saying “Don’t plan” or “Don’t think”. And I'm certainly not saying you should just throw code against the wall until you get something that looks workable.

Consider the T.V. show "Dr. House". His beliefs that there is one absolute right way of handling a problem is completely detrimental to software development. But one of the few things that I agree with Dr. House *in practice* is his insistence on thinking through a problem before acting on it. But if you remember the series, he follows the clay pot model. Think. Do. Refine. Think again. Continue until done. You won’t get it right the first time, and you should be very suspicious if you do, so don’t grind on it. And I love his attitude that making mistakes is expected. No one cares, as long as your end result is solid. 

Here’s the thing I tell architects and developers alike. There are no points for style. No one is counting code check ins, no one is counting compilations, no one is counting design iterations, and no one cares as long as the end product is a good one. Until then, if you have to slam it in with your knees, do so.

Often, you don't know what works until you've seen something that does not.

Layering It On

Once you know who you are, you know what you have to do.” --Steve Perry, The 97th Step

We all know that building your application in layers is important. Portability, separation of concerns, extensibility, and blog articles are all highly dependent on proper application layers. The problem I see isn't a lack of understanding the importance or disagreeing with it. The problem I consistently see is people not understanding how to layer their applications. Part of this is, of course, practice. My first attempt at building an application with a 3-Tier architecture was an epic disaster that would have made the Titanic step back and say, “DAMN- I thought this was bad.” My second one was also pretty terrible. But better than the first.

Practice become easier with understanding, though. Tips, circumstances, and examples are all limited in scope in that they only give you a small slice of the whole picture. But once you understand what a layer is, why it’s important, and how to look at it, the rest is just reinforcing your understanding with practical experience. As any regular readers will know (Have I been around long enough to have regular readers?), I see software architecture as applied philosophy. I know I've used this one already, but:

“This, what is it in itself, and by itself, according to its proper constitution? What is the substance of it? What is the matter, or proper use? What is the form, or efficient cause? What is it for in this world, and how long will it abide? Thus must thou examine all things that present themselves unto thee.” --Meditations, Marcus Aurelius

I originally used this in understanding classes and properly understanding what they do, but it applies to application structure as well. Once you understand what something is, be it a class, a layer, a carburetor, or a hammer, you know what to do with it. So let’s take a pretty typical application stack- Presentation, Controller, Model, and Persistence. We start by considering each layer as a real-world entity, with things it knows about, actions it knows how to take, and actions it does not know how to take. Then we ask ourselves Aurelius' questions about these entities.

Presentation

What is a Presentation layer, in itself and by itself? Not to put too fine a point on it, but the presentation layer presents data, both to the end user and to the model. That’s what it knows how to do. It knows how to arrange data in a way that makes your application usable and useful. It knows how to sort and filter data so the user can get to the important data without wading through the unimportant data. 

Is your presentation layer interpreting data for any reason other than how to properly display it or properly send it to the lower reaches of the application? Then your presentation layer is doing something it doesn't know how to do.

Controller

Of all the application layers, I've seen more misunderstanding about the Controller than any other. And this is a prime example of why understanding needs to come first, because this one is easy to get wrong if you don’t understand it. The Controller is a Switchboard Operator. Okay- there are a ton of more recent comparisons that are just as good, but I’m going with switchboard operator. The controller routes requests from one place to another, and that’s it. It knows where a request came from and based on that, it knows where the request goes next. A controller that routes the request to different receivers based on some conditional logic with the data itself is interpreting and attaching meaning to the data. It doesn't know how to do that.

Model

In and of itself, what is a Model Layer? What's its purpose? The model knows what the data means, how it should be interpreted, and how it should be used. Which is, admittedly, the meat of the application but there are a few things this layer doesn't do as a part of its purpose. It doesn't know where data comes from. It doesn't know where data goes when the it is done doing what it does. In this way, it’s a lot like an assembly line worker. A widget  shows up and the model performs a task on it. Then the widget moves on. Where it came from and where it goes next are not important. The task performed is the only thing that is.

Persistence

What is the form or efficient cause of the Persistence layer? Sure, this layer interacts with data, but the question is "What is the... *efficient* cause". In its most efficient form, the persistence layer retrieves the data it’s asked for and stores the data it’s told to. It doesn't know how to do anything else. If, for instance, you've asked your persistence layer to tell the model if the correct data has been retrieved, then you’re asking your persistence layer for something it doesn't know how to do. If, as is common, you’re asking your persistence layer to know whether or not data is correct before storage then you are also asking it for something it doesn't know.

Although this becomes much easier with practice, the underlying key to application layering is knowing what you want your layer to do, and making sure that it doesn't do anything else. Thinking about your application layers as specialists helps greatly in keeping in mind what they should, and shouldn't, be doing. You don’t call your pediatrician when your car dies and you don’t call a ticket box office when your roof leaks. Don’t call a model layer when you need to know how to display data.

What Is a Software Architect

"That's incidental. What's the first and principal thing he does, what need does he serve..." --Dr. Hannibal Lecter

As any regular readers know, (Thanks, by the way!) I pretty strongly believe that understanding a thing tells you want to do with that thing. And since I've been writing for some time now about being a "Software Architect", we should take some time and ask the question "What does it mean to be a Software Architect?" As I've said before, not agreeing on definitions leads to communication breakdown.

On the surface, a software architect designs software. he is responsible for the design, even if he doesn't necessarily go on from there to build it. Although he might go on to do just that, as in many organizations the architect and the developer wear the same skin. So, the architect creates not just the class diagrams but often the technologies as well, thus shaping what the final product will look like.

But, as Dr. Lecter would say, that's incidental.

In order to design a software application, an architect needs to know two things. What is needed of the application and what is the best way of fulfilling those needs.Without these pieces of information, the design will fail somewhere along the line. So first of all, the architect needs to be familiar with the business requirements at hand. Secondly, the architect needs to have a broad enough field of experience to know the proper technologies to implement the requirements. If the architect only has a shoe or a bottle to pound in a nail then the final result can't help but fail. So, a software architect is starting to sound a lot like a general contractor. You get the request, you choose the materials, plan your work, and then hand things off to subcontractors to fulfill.

That, too, is incidental.

Software doesn't exist in a void or for its own sake. It exists to fill a need. A real-life need that is both concrete and immediate. Whether the application is as innocuous as Angry Birds or as complicated as the Linux Kernel, it exists because someone needed it. In order to design a useful application, you need to understand that need. When you understand a need, you can provide a solution. If you provide a solution, then you can provide a spec doc that can actually be fulfilled. If you have a spec doc that can be fulfilled, then you can choose the right tools and build the proper design.

This is not incidental. This is the first and principal thing that you are and the need you serve. You're a problem solver. This has been my employment Elevator Pitch for years.

"What do you do?"

"I solve problems."

Foreseeable and Imaginary Design

“Architecture that protects against foreseeable change has foreseeable value. Architecture that protects against imaginary change has imaginary value.” --Office saying

Way back when I was a young web developer I worked for a small, very specialized, health insurance company. The business model was fairly simple. Provide health insurance to U.S. citizens traveling to foreign countries where their regular health insurance wouldn't be usable, and then get independent insurance agents to sell it. The IT and Development departments were fairly carefully run, in that decisions were not made lightly and there was a great deal of thought put into infrastructure- network, database, and application. Central to much of this was the all-important Customer record, containing all the necessary information about an insured, an insured’s spouse, and links to the insured’s dependents.

Eventually, as often happens with a small company looking to become a medium sized company, they branched out their business model. Rather than just selling to travelers, they bid for and won a contract to become the official health insurance provider of a small island nation. Excitement abounded, as this contract was, from my understanding, worth as much as the current revenue of the company. To put it mildly, much effort was put into implementing the needs of this nation.

That’s when a problem arose. Polygamy, specifically men with multiple wives, was not only allowable but a common occurrence. The Customer record represented a customer as it related to a purchased insurance policy and the data model only allowed for one spouse for a customer that purchased an insurance package. This, of course, could be changed but this relationship was so central that changing it would have been like throwing a rock through a window. There would be obvious damage to both the flagship application and the website, but once that was fixed there would still be tiny invisible cracks in the infrastructure. Cracks that were invisible until they suddenly caused more damage. Ultimately, the level of effort was deemed too much.

As architects, it is our job to anticipate and protect against foreseeable change. The key word here is “foreseeable”. Architecture that protects against foreseeable change has foreseeable value. Architecture that protects against imaginary change has imaginary value. The two worst sins in application architecture are missing the obvious and accounting for the unreasonable. Of the two, the latter is the more subtle mistake. When you account for the unreasonable you not only extend the amount of work that needs to be done, but you also increase the complexity. Do you really need a calculator that allows client applications to define their own implementation of a number? Certainly not. However, say you’re designing an application that is meant to move data from one repository to another and it’s likely that this process will not be needed elsewhere. Or at least not many other places. As an architect, it’s your job to say “SSIS package”, or something similar based on your preferred platform. If you find yourself designing a webservice that takes an interface implementation that defines a recordset, determines a queuing database to save the recordset to based on a data manager received from a factory and then notifies a Windows service to process the queue then you really, really need to ask yourself if you’re working too hard and making things too complex.

Which makes the most important job of the architect not implementing design patterns or creative ways of solving problems. Not even, should your organization task architects with this, providing accurate time and effort estimations for development projects. These are all secondary to finding answers to the questions “What changes are possible and which of those changes are foreseeable”. Clearly it is not the architect’s job to answer those questions. That falls squarely on the business stakeholders. But it is the architect’s job to ask and to account for the changes that are foreseeable, and to ignore the changes that are imaginary. I've actually created a “Mental Warning Flag” list. If I find myself implementing something on that mental list, I stop and ask for another set of eyes to give feedback. To use a phrase that I will almost certainly be used again “Two developers working together are more than twice as smart as two developers working separately”.

In the insurance company’s case, the data model seems reasonable for the original intended purpose. A Customer represented someone who had purchased an insurance package. The purchaser can have, at most, one spouse because the target customers were only legally allowed one spouse. Not even divorce and remarriage were relevant because the record represented the relationship as it applied to a purchased insurance package. The marital status and spouse for a given insurance package had no impact on past or future spousal relationships.

Did an architect ask the two most important questions? I don’t know. I wasn’t there. If so, was the possibility of eventually covering people that could legally have multiple spouses considered foreseeable or imaginary? I don’t know that either. What I do know is that if you are acting as an application’s architect, be it a defined role in your organization or because the task falls on you as the application’s developer, it is your job to ask those questions and continue asking until you are satisfied with the answers.  And that next time I model a data relationship involving marriage, I’m going to ask about the possibility of polygamy.