Best Practices

"The cart before the horse is neither beautiful nor useful." --Henry David Thoreau

You're doing account signup forms wrong.

Okay, I should say "If you're doing account signup forms, you're doing them wrong." and while logically (if not grammatically) more accurate, I thought it made less of an impact as an opening statement.

The problems with most signup forms are myriad. Why do sites force you to enter your email address twice? Mobile platforms have figured out that masked password input boxes aren't always necessary, and give you either an option to turn it off or give limited clear-text viewing of your input. Websites haven't gotten the memo. And if I see one more site serving up a picture of what can only be described as a dust storm and telling me to type in the letters in the box to prove that I'm human, I'm going to flip and write a blog post about it. Just. Stop. It.

Now, as I hope I've made clear in my writing, I don't care about forms, CAPTCHA, or even UX issues. Or rather, I care about them only in the context of the thought process that went into them, and that's where account signup (and often account management) process fall flat. It's due to a very insidious concept known as "Best Practices".

I hate best practices. As soon as that phrase is first used in a requirements gathering meeting, I step on it like I would a roach. "Best Practices" used to refer to process that the industry had adopted, formally or not, as the best way known at the time to approach a problem. That, I have no problem with. What I have a problem with is the fact that "Best Practices" doesn't mean that anymore in software development. Anymore, it means "What is everyone else doing?" Which leads to lazy planning. Which leads to bad results. And I really hate bad results.

"Best Practices" are insidious. Because it's assumed that these practices are used because they're the best way of approaching a problem, people stop thinking about solutions as they apply to their specific needs. Any time you start implementing solutions without considering whether or not that solution is actually a solution to a problem you have, you have at best added unnecessary complexity to your project. At worst, you end up implementing code that hurts you in the long run.

Worse, though, is that no one ever advances that body of knowledge when "best practices" are applied blindly. Since everyone is using the same solutions as everyone else, no one thinks up new ways of solving problems. In the UX arena, this results in carbon-copy sites that don't stand out and present all the same inconveniences that the other sites present. In the web security arena, it's even worse because you are implementing that worst kind of security. The kind that makes you feel secure without necessarily offering any concrete benefits. Since the "security" principles that are labeled "Best Practices" are applied blindly, you don't know if the implemented solution solves the problem at hand, much less whether or not the problem at hand is one that needs solving in your context.

As an example, let's take password masking. Many mobile devices briefly show in clear text the latest character typed into a password box. This is not new. In fact, I've read people claim that Apple innovated that idea for the iPhone, but my Palm 600 did that. This idea is ten years old and the web world still hasn't caught on. It's become an expected inconvenience because everyone is looking at everyone else's paired "Enter Password/Reenter Password" boxes. Even worse, this is billed as a security measure without asking whether or not this is a *necessary* security measure. Does it really hurt anything if a site offered a way of viewing your password in clear text, thus avoiding the usual paired password box routine?

Am I saying that these measure are all unnecessary? Absolutely not. Except when they are. And if requirements are being gathered in the context of your needs and your solution, then it becomes obvious what you do and don't need. The problem is that "Best Practices" are applied backwards. The solution is selected and the problem lays unexamined. Software development is merely implemented problem solving. You can't solve a problem you have not examined.

Painting Fences

Imagine this exchange, if you will:

Homeowner: We need to to paint the outside of our fence white.
Contractor: No problem.

(2 weeks later)

Homeowner: I know we asked you to paint our fence white, but now we need it painted red.
Contractor: You don't like the white?
Homeowner: It's not working for us, and we'd like to try red.
Contractor: No problem.

(2 weeks later)

Homeowner: Okay, sorry about this but now we want our fence green.
Contractor: Sure, I can do that, but I have to ask. Why the color changes?
Homeowner: The neighbor's dogs bark all night and we're trying to find a fence color that calms them down.

Silly, isn't it? Ridiculous, in fact. And yet, a conversation in which I've participated more than a few times in my professional career. It stems from the business side of a software development project telling the development team what they want, which should be avoided at all costs. Yes. I just said that. The business shouldn't be allowed to tell development what they want. The reasoning is fairly simple, too. They don't know what they want and even if they did, they don't have the technical vocabulary to communicate it.

What the business does know is what it needs. There's a pain point, a failing, an inefficiency, or a breakage somewhere that needs to be resolved. That's what they should be talking to you about. They need to show what the process is now, what they don't like about it, and what it should look like when the problem has been solved.

There's a subtle difference between business needs as requirements and implementation details as requirements. Subtle enough to go unnoticed sometimes. Subtle enough to even seem reasonable. It seems almost reasonable to say "The account creation form needs to validate the format of the email address." Or "I want to be able to delete accounts". But it's like two lines that aren't quite parallel. Extend them out for a long enough distance and they end up in very different places.

Let's take a look at the two examples above. Validating an email address seems reasonable. Necessary, even. After all, you don't want users accidentally putting in a bad email address, right? If we're going to have any success at all, we're going to need a regex.  Something along the lines of

bool isEmailValid = Regex.Match(emailAddress, RegexString).Success;

Problem is,writing that regex is harder than it looks. Bigger problem is that it likely doesn't solve the underlying issue. Note, I said "likely" because we don't actually know what the real issue is. All we know is that we need to validate an email address. There's no mention of why. But if the underlying issue is making sure that the user enters a valid email address, which seems reasonable, then this doesn't actually solve the problem. It's very easy to write a regex (okay- let's face it. Search for a regex and copy/paste it. C'Mon- you know you do it.) that doesn't properly validate an email address, and prohibitively expensive to write one that does.

This solution falls even shorter of the actual need if the need is to make sure that the user enters a valid address that they have access to. Which may be what the business meant. It also may not have been what they meant, but would agree this was the proper direction if they had communicated their need, thus giving the architecture team the chance to respond and ask questions. So, due to a solution disguised as a requirement, we have a solution that will likely either validate malformed email addresses or reject valid addresses and does nothing to make sure the user has access to the email address. So what, in fact, have we actually accomplished?

The second example seems pretty reasonable, too. After all, do we really want dead accounts laying around? And besides, what if there's a user that we don't want accessing their account anymore? It's not like this will be an every day thing, but just in case. The problem here isn't whether or not a solution can be implemented, it's the repercussions that are not being considered. What do we do with the order history of a deleted account? If we keep it, to what do we associate the orders? If we delete it, how do we explain the discrepancies in financial reporting? Or inventory levels vs. newly adjusted sales numbers? What if you need to process a refund? Good grief, what if you need to undelete the account? This requirement is like an early 80's Buick. As soon as you fix something you uncover two more problems.

There's no easy solution to the problem. The business side needs to be careful to communicate needs. User stories are supposed to help with this, but they only help if used properly. Here's where a good BA will come into play. A good BA can make sure that user stories aren't reduced to endless copies of "As the product owner, I want {technical solution} so that the product works properly". The three parts of the story are necessary because, when used correctly, they define a need instead of a solution. And the architect needs to ask himself, for every requirement, "What is the need behind this?" If the answer isn't immediately obvious, there's a good chance that you're dealing with a solution disguised as a requirement.


(Ed. note: If the customer requesting the fence painting is Mr. Miyagi, just do it.)

Foreseeable and Imaginary Design

“Architecture that protects against foreseeable change has foreseeable value. Architecture that protects against imaginary change has imaginary value.” --Office saying

Way back when I was a young web developer I worked for a small, very specialized, health insurance company. The business model was fairly simple. Provide health insurance to U.S. citizens traveling to foreign countries where their regular health insurance wouldn't be usable, and then get independent insurance agents to sell it. The IT and Development departments were fairly carefully run, in that decisions were not made lightly and there was a great deal of thought put into infrastructure- network, database, and application. Central to much of this was the all-important Customer record, containing all the necessary information about an insured, an insured’s spouse, and links to the insured’s dependents.

Eventually, as often happens with a small company looking to become a medium sized company, they branched out their business model. Rather than just selling to travelers, they bid for and won a contract to become the official health insurance provider of a small island nation. Excitement abounded, as this contract was, from my understanding, worth as much as the current revenue of the company. To put it mildly, much effort was put into implementing the needs of this nation.

That’s when a problem arose. Polygamy, specifically men with multiple wives, was not only allowable but a common occurrence. The Customer record represented a customer as it related to a purchased insurance policy and the data model only allowed for one spouse for a customer that purchased an insurance package. This, of course, could be changed but this relationship was so central that changing it would have been like throwing a rock through a window. There would be obvious damage to both the flagship application and the website, but once that was fixed there would still be tiny invisible cracks in the infrastructure. Cracks that were invisible until they suddenly caused more damage. Ultimately, the level of effort was deemed too much.

As architects, it is our job to anticipate and protect against foreseeable change. The key word here is “foreseeable”. Architecture that protects against foreseeable change has foreseeable value. Architecture that protects against imaginary change has imaginary value. The two worst sins in application architecture are missing the obvious and accounting for the unreasonable. Of the two, the latter is the more subtle mistake. When you account for the unreasonable you not only extend the amount of work that needs to be done, but you also increase the complexity. Do you really need a calculator that allows client applications to define their own implementation of a number? Certainly not. However, say you’re designing an application that is meant to move data from one repository to another and it’s likely that this process will not be needed elsewhere. Or at least not many other places. As an architect, it’s your job to say “SSIS package”, or something similar based on your preferred platform. If you find yourself designing a webservice that takes an interface implementation that defines a recordset, determines a queuing database to save the recordset to based on a data manager received from a factory and then notifies a Windows service to process the queue then you really, really need to ask yourself if you’re working too hard and making things too complex.

Which makes the most important job of the architect not implementing design patterns or creative ways of solving problems. Not even, should your organization task architects with this, providing accurate time and effort estimations for development projects. These are all secondary to finding answers to the questions “What changes are possible and which of those changes are foreseeable”. Clearly it is not the architect’s job to answer those questions. That falls squarely on the business stakeholders. But it is the architect’s job to ask and to account for the changes that are foreseeable, and to ignore the changes that are imaginary. I've actually created a “Mental Warning Flag” list. If I find myself implementing something on that mental list, I stop and ask for another set of eyes to give feedback. To use a phrase that I will almost certainly be used again “Two developers working together are more than twice as smart as two developers working separately”.

In the insurance company’s case, the data model seems reasonable for the original intended purpose. A Customer represented someone who had purchased an insurance package. The purchaser can have, at most, one spouse because the target customers were only legally allowed one spouse. Not even divorce and remarriage were relevant because the record represented the relationship as it applied to a purchased insurance package. The marital status and spouse for a given insurance package had no impact on past or future spousal relationships.

Did an architect ask the two most important questions? I don’t know. I wasn’t there. If so, was the possibility of eventually covering people that could legally have multiple spouses considered foreseeable or imaginary? I don’t know that either. What I do know is that if you are acting as an application’s architect, be it a defined role in your organization or because the task falls on you as the application’s developer, it is your job to ask those questions and continue asking until you are satisfied with the answers.  And that next time I model a data relationship involving marriage, I’m going to ask about the possibility of polygamy.