Bad Form(s): A Brief Rant

A lot of interaction on the web is achieved through the use of forms, panels of text and fields used to input, classify and store data. Forms have been around pretty much since the beginning, and at first were something resembling sorcery: getting a form to look good, fields aligned, was tricky. As HTML and CSS became more sophisticated, along with improved Javascript capabilities, forms have become became both easier to build and more complex to debug. And the temptation to add unnecessary glitz is frequently the culprit behind the worst forms out on the web.

Input Fields are the workhorses of forms. They can restrict the number of characters input and the type of data entered. They can choose to display or mask the input as in the case of passwords. While there are far more options than these, this small set of variables is more than enough for poorly-implemented forms to become nearly unusable, leading to frustration and outright silliness.

Some of this silliness has declined over the decades because developers have gravitated toward some standards of behavior. Sadly, one still to this day encounters forms with some of the same lameness that we saw in the 90’s or 00’s — and we can’t blame the COGs (Crusty Old Guys) for all of it. Too often, new developers fall into the same traps of bad form design, disregard of user experience, cool-philia, laziness or the “I am IT and I decide” mindset that has consistently been focused on the mechanics of a site even when in opposition to the success of people using a site. For some reason, I encounter these issues most frequently on banking and government sites, where some noob has clearly discovered the modern equivalent of the blink tag and can’t resist inflicting their new-found toy on users.

Standard Information

Data like phone numbers, ZIP codes, Social Security numbers are pretty standardized in the US. Phone numbers, for example, are expected to have 10 digits, but they may be formatted in a number of ways:

  • (800) 555-1212 (traditional)
  • 800-555-1212 (business traditional)
  • 800.555.1212 (standard geek)
  • 8005551212 (machine version)

While a company may want to format a phone number in a particular way, all they actually need is those 10 digits — they shouldn’t really be concerned about how a person might format their number since it’s so easy to parse out the digits and store just that; storing digits only allows more efficient sorting anyway. A number can then be reformatted as desired on subsequent read and display. Still, I constantly find forms that throw errors if a user includes hyphens, periods or, gasp, spaces in a phone number — or doesn’t include one of those. Some forms simply limit the phone number field to 10 characters and leaving the user to guess what’s going on, adding no benefit but inflicting a developer’s idea of proper form entry.

ZIP Codes

The US Post Office instituted ZIP (Zone Improvement Plan) codes in 1963, and extended those codes with the “5+4” format in 1983. That’s all before forms existed, but the 5+4 format isn’t widely used even now. But it still manages to cause some forms (or the developers behind them) indigestion.

Synchronicity is Key

One particularly idiotic situation occurred when I was logging for the first time into an account that had been set up through an separate process. To verify my identity, I was asked to enter my ZIP code, at which time my verification was rejected. Checking back with the administrator, I found that they had my ZIP code had been recorded as a 5+4 value. The web form, however, only allowed 5 digits, making a match impossible unless the developers actually used some common sense. I’ve seen this occur as well when a form on a web page allows or requires something different than what is allowed on that same form on a mobile app.

The Future is Here (Occasionally)

“The future is already here — it’s just not very evenly distributed.” – Michael Gibson

I sometimes run across forms that ask for the ZIP code first, then use that information to automatically populate the fields for city and state. Smart, and rare, a form that actually helps you fill in the blanks. With technology that’s been in use since at least 1998 and information that’s been available since 1963…

Divertimento: Non-Regional Postal Codes

From Wikipedia:

In Canada the amount of mail sent to Santa Claus increased every Christmas, up to the point that Canada Post decided to start an official Santa Claus letter-response program in 1983. Approximately one million letters come in to Santa Claus each Christmas, including from outside of Canada, and all of them are answered in the same languages in which they are written.[13] Canada Post introduced a special address for mail to Santa Claus, complete with its own postal code:

SANTA CLAUS
NORTH POLE  H0H 0H0

Gotta love Canadians.

Form input and the Database

Nearly all data input through forms ends up in a database. To store data, most databases receive those data in the form of an SQL (Structured Query Language) query. For example:

insert “xyz1234pplr” into table passwords where user = “John Yaya”

[Note to geeks: these examples are simplified for a lay audience; this is not a coding class, so please back off.]

That query is evaluated by a parser and the value is stored. That parser evaluation, however, is were we can get into trouble.

Code Injection

Code injection is one of the oldest, most common and most insidious forms of hacking. Parsers look for meaning as part of their evaluation process, and some characters can cause some really interesting things to happen. Take our example from above, and let’s add something destructive:

insert “xyz1234pplr|‘/bin/sh unlink /'” into table passwords where user = “John Yaya”

The vertical bar character “|” is used in some systems (like Unix variants) as what’s called a “pipe“, a way of passing the output from one program to be input into another program. The inserted phrase:

|‘/bin/sh unlink /’

Means “pass the command to the main system to unlink the filesystem root.”

In times before developers and operating systems were careful about input (actually this is still a problem), this query might have been executed, resulting in the effective deletion of the entire filesystem on the database server. These days, rather than directly destroy a filesystem, someone might try to use injection to give themselves administrative privileges, and from there they can get into all sorts of trouble.

Input Character Restriction

Sadly, some IT groups continue the ancient (by IT standards) and lame (by any standard) practice of trying to protect against injection by restricting what characters can be entered into a form. So things like “|” or “>” might be disallowed. Problem is, good passwords these days make use of all sorts of punctuation.

Institutional Arrogance : Personal Capital IT

Recently, I opened an trial account with Personal Capital (PC), a Mint-like web site where you enter all you financial accounts and it advises you about your investments (I’m intentionally not including a link out here because I can’t with good conscience make it easy for anyone to go there). But I was unable to add some of my accounts because the passwords appeared to be getting rejected. After some back and forth, I was informed that the IT staff at PC reads your passwords when you add a financial account and strips out any characters they think are unsafe — even if those characters are perfectly fine with the financial institution. To be clear: 

  1. You create an account with a bank, establish a login and password.
  2. You create an account with PC and add the bank and login credentials to your PC account.
  3. PC reads your password when you supply it and then strips out any characters they feel are objectionable according to their internal policy. You are never informed about this action, and the now-corrupt password is stored rather than the real password that you and your bank agreed on.
  4. PC uses the corrupt password to attempt to log into your bank.
  5. The bank rejects the corrupt password.
  6. On login rejection, PC displays a message to you indicating that the bank doesn’t like your password.
  7. If this happens enough, the bank locks your account. It is then up to you to resolve the mystery with your bank, because at this point neither you nor your bank know why someone has been trying to log into your account with a bad password.
  8. At no time does PC let you know they caused the entire problem.

PC’s support personnel were very defensive about their actions, maintaining that they’re following OWASP guidelines and are justified in their policy. They informed me that the solution was for me to change all my passwords to my banks to conform to their (PCs) requirements.

One of the most myopic and arrogant IT abuses I’ve seen in a while. 

Input Escaping

A better method of defending against injection attack is by input escaping. Basically, before you parse any input from a field, you “escape”, or defang, any characters you feel are dangerous. Escaping can take a lot of forms, but in essence it means flagging a character in a way that tells the system “this is just a character, don’t use it as something else.” For example, the vertical bar character “|” in some environments is used as a “pipe”, a way of passing data from one program into another program. If that character is treated as a pipe when entered into a database, it might be interpreted as an operation, and text that seemed innocuous might get executed rather than simple entered as text. Our nefarious query from above becomes:

insert “xyz1234pplr\|‘/bin/sh unlink /'” into table passwords where user = “John Yaya”

The added backslash “\” tells the parser that the vertical bar is just a vertical bar, and the pipe is neutralized. Smart systems store the escaped string in the database so that later retrievals are safe as well.

It Really Isn’t Rocket Science

Finding a well-built form, while not rare, does continue to be uncommon in my experience. As an engineer, I’ve embraced the rule of parsimony (or Occam’s Razor if you prefer) from early in my career, which effectively means “don’t make it fancier than it needs to be.”

Modern techniques can provide all sorts of functionality to the user experience. The trick is keeping a firm eye on the goal: that technology should be used to enhance the experience, not provide a place to show off at the user’s expense.

Annealing design and technology

My grandfather left me his pocket watch, an 1896 Volta repeating chronograph. Under the crystal, the face and hands are exquisite – checking the time is a delightful experience. This watch has a crystal on the back as well, offering a look into the technology behind the experience, as gears turn and tiny hammers strike tiny chimes: the timepiece is both a beautiful design and an engineering marvel.
One of the things that most attracted me to frog design was the idea that, as an engineer and occasional artist, I’d be working for a design firm. I confess that I hadn’t fully formed an image of how that would work – mainly I just loved the idea, the people were incredibly smart and motivated, and they were clearly going a lot of cool directions all at once; my kind of place. Technology has become an increasing part of the process at frog, as solutions are more tightly integrated into systems, and clients want to see functional prototypes.

Technology as the naysayer

In design projects, there can be a tendency to bring technology into the mix later in the project, to validate the feasibility of the concepts being proposed. This usually means that the concepts have been refined quite a bit, and that significant resources have been allocated toward that refinement. It also means that technology is frequently placed in the position of saying things like “that’s a wonderful concept, but you won’t be able to actually do it. Sorry.” Whether web-based interactions that were simply beyond current states of the art, or visual transformations that might bring mobile hardware to its knees, or cloud-based features involving subtly convoluted security issues, there is an implied veto. And since people don’t generally appreciate their concepts being vetoed, members of the technology group might be a little less welcome on the next project, meaning that they might be invited to the table even later in the process than before.

Engineers really do love being able to say “yes, you can actually do that.”, or even proposing solutions that the design team may have assumed were out of reach, enabling the team to go big. frog embodies design, and that should be the driving force behind what we do. Form may follow function, and form may follow emotion; delight frequently follows seamless integration of form and function. This requires a stimulating, sometimes argumentative environment built with patience and trust and respect between all members of a team. I’ve had strategy people say things that I felt were completely, utterly off the mark. But they were patient with me, debated with a genuine desire to find truth, and they gave me the time I needed to realize that they were actually completely, utterly right.

“We need more ‘technology projects'”

As an innovation company that touts considerable technology prowess behind the design, leaders at frog are interested in ways of displaying that prowess. If a project is perceived as “technology-centric”, the technology group might take an increased role early on. The result may be something entirely feasible, but may not produce the desired level of UX sophistication or user delight (yes, we do use that term): technology people tend to be very clear on functionality, but may not visualize the sort of UX that grows out of the rigor our interaction analysts can bring to bear. The twin seductions of technology coolness and feasibility will influence, and may actually overwhelm, solid design. And regardless of how amazing a new feature might be, twelve clicks, six screens and three feats of mental gymnastics is asking too much of the average user.

“…and now for our next trick…”

There are teaching methodologies that suggest one should be reaching all members of a group, regardless of their preferred learning modality, and presenting to a client is a form of education. The synthesis of design and technology can unearth new possibilities when we deliver our work to the client. Some people learn visually, some by watching repeated examples, and some are kinesthetic; they need to touch and feel a concept. Dealing with technology-savvy or even technology-centric clients is similar in the sense that, in a given room, there will be some who gravitate toward a more traditional, storyboarded design approach, while some may prefer simply reading a specification in silence. The challenge lies in reaching everyone during a single presentation. Too design-focused, and we may get “it’s too conceptual for me – what do I do with this?” Too technical, and the designers in the room may perceive the presentation as mere documentation.

Indeed, when a client group tends toward the middle of that continuum between design and technology, it may be unclear whether frog is acting as a design company enhanced by technology or a technology company enhanced by design – and that’s okay: what matters is the alloy resulting from design and technology contributions: a new element. This is the area that is becoming more fascinating as we explore ways to reach all audiences, not merely supplying separate deliverables to satisfy different learning types, but by inventing new methods for reaching everyone through different aspects of a single concept.

SXSW: interactive 2010

dfp_20100313_SXSW_0017

SXSW Interactive 2010 is over. I spent most of that time in Austin, and came away with some surprising impressions. On the flight down from Seattle, I’d say 80% of the passengers were headed to SXSW. Lots of “design-y” folks, obscure t-shirts, spools, mustaches, tats, hats and obligatory hair. It was a cool crowd, not the hyper let-me-prove-myself-to-you vibe one finds in crowds headed to purely tech events.
Still, SXSW Interactive has a lot of tech to it. Which made it surprising that the tech side of the event seemed like it needed life support, or at least a triple espresso. The website was a bit tangled, the official SXSW iPhone app never fully synced with the data on the site (twitter hashtags never made the jump). And there was the “profile merge” problem: if you set up a my.sxsw account before you actually registered, the photo you uploaded never made it to registration, and your account via the iPhone app never recognized you as an official attendee, meaning no QR scanning, or messaging, or any of that coolness. To fix this took asking specifically for a “merge” wherein your data were munged together. Then manual account setup at 1-2 other external sites to get the QR stuff online. Argh.

Once at the event, there were lines for everything.  Coffee, bathrooms, swag, entry into talks, drinks, dinner, clubs, music. Granted, a lot of these things were worthwhile, but you get tired of waiting. At the same time, the constant thrum of thousands of people circulating in the same space to talk about design and tech is intoxicating.

Content-wise, SXSW Interactive was a mixed bag. Few of the talks made much use of the big screens, leaving them to roll through the sorts of advertisements one normally finds when waiting for a movie to start. Speakers ranged from the awesome (Clay Shirky rocked it) to the mundane. And while I’d expect a some level of lack of preparation from newbie speakers (the zero waste panel had a member who actually said “what the fuck”), it was surprising how flat the keynote speakers were, people you’d normally expect to bring it.

Favorites:

  • Again, Clay Shirky kicked ass talking about changing the systems to enable changing our paradigms.
  • Robert Fabricant inspired with his examples of world-changing processes in action right now like Project Masiluleke (full disclosure: I’m a frog, but I think Project M is cool).
  • Making snow outside the convention center.
  • Sampling single malt Macallan at the frog party.
  • “Is WordPress killing design? No, lack of imagination is killing design.”

Further thoughts:

  • “Futurist” presentations should be 45 minutes rather than an hour – and no Q&A; these sessions seem especially prone to the “do you actually have a question?” phenomenon.
  • People posing as having questions when they’re standing up to promote themselves/their companies should be gently encouraged way from that  (LifeSize).
  • Biofeedback is not a form of mystical kinesthetic awareness.

Favorite quotes:

  • “augmented mindfulness” – Robert Fabricant
  • “believability is an extremely ductile process” – Bruce Sterling
  • “rainbow unicorn mode of sharing” vs “jackhammer sharing” – Clay Shirky
  • “abundance breaks more things than scarcity” – Clay Shirky
  • napster == “bit torrent that didn’t work very well” – Clay Shirky
  • word for “not sharing with someone when it would make their lives better” == “spiteful” (in relation to the recording industry institutionalizing spitefulness) – Clay Shirky