Events Methods Research

Usability testing is all practice, not magic

I just returned from Westminster Hub where I ran a workshop called “Usability Testing How To’s for Small Teams (and Little Budgets). The workshop is based on my experience setting up the usability testing arm at Future Workshops. As one UX person with fifteen developers, I learned a lot about getting by on a tight timescale.

The group of attendees tonight was great! I was so impressed with how eagerly everyone learned, and how natural they were at facilitating tests. A few things that came up in the discussions were very insightful, and echoed a lot of my experiences running usability tests:

It can be done (today!)

As one of the workshop participants cleverly concluded, there’s no magic involved. Usability testing can be done with more budget, more time, more trained staff, better recruitment, better equipment. The question is whether it’s done at all. It’s entirely possible to hold cheap and quick usability tests and still learn a lot. There wasn’t one group during the workshop that didn’t discover a serious usability problem in just one hour!

Soon after the test part of the workshop began, I started hearing familiar sighs across the room. Cringing soon followed, and then hushed group conversations about things to definitely put in the fixes list. Some attendees had never done usability testing before but were able to facilitate a test and find useful results right away. It can be done!

Workshop photos on Flickr
Instructing a test participant on her first scenario – all smiles so far!
Workshop photos on Flickr
Testing a website with a very specific audience group

It’s hard to stay quiet

Facilitating a test well takes some practice. It always sounds easy on paper, but as the workshop participants quickly realized – theory is different from practice. It’s tough sitting across from a person and watching them struggle with an interface without interfering.

I recognized a lot of the questions from when I was first learning how to facilitate: when do we interrupt the participant? What do we do when they get stuck? How do we ask questions without leading? My formula: if they’re not upset, stay quiet. If they are, tell them something nice.

Challege: DIY recruiting with a specific audience

When a website or app have very specific audience characteristics, recruiting can be a pain. So what do we do? One of the work groups decided to test their own product: a website with data on how many people are killed in drone attacks in conflict areas. They had to make due with participants from the rest of the group attending.

The key for them was setting the right context. A real user would be knowledgable and motivated to find out about drone attacks. To simulate this, they asked participants to read an article about the issue, mentioning their website, and gave them specific instructions on what information to look for. They discovered some usability problems to fix in the process.

There will be another run of this workshop on 22 January, 2014 in London. Would love to see you there!

Meanwhile, my slides are on SlideShare:

And some photos on Flickr:

Empathy Principles Research

What Having a Spaceship in Your Closet Has to Do With User Experience

Originally posted on Medium.

When I was a kid, I used to play a lot of pretend.

I had a detective’s office, a spaceship, a courtroom, a school, a halfway house for distressed teens, a tax office, a secretly-trained assassins’ bureau, a chemistry lab, a movie production studio, a radio station, and a doctor’s office.

Adults told me I had a great imagination, and my friends enjoyed playing with me. We left our 8-year-old bodies and became nurses, judges, detectives, and talk show hosts. In our play, we were made to feel, act, and speak differently from the way we would in our real lives.

We went through great lengths to make everything as realistic as possible. The spaceship was perhaps our most distinguished effort. My friend, Hadas, and I worked tirelessly on it while my mother was at work. We used a roll of aluminium foil we had purchased with our pocket money and taped it to the inside walls of a closet which was used to store winter clothes.

We printed the astronomy section of a CD-ROM encyclopedia and stored the pages inside a foil “computer” that spit out useful information about the planets on which we landed. Irresponsibly, we attached a desk lamp to the ceiling of my closet and sat in the terrible heat as we escaped into space and into our missions.

Naively, we thought these objects and environments made our games real. We worked hard to collect all the necessary props: stamps, forms, hacked school boards, stationery, fax machines, and tape recorders. When we grew up and started using all these objects at our real, grown-up jobs, we realized it was nothing like we’d imagined it to be.

This is because, during play, we set our own constraints and decided our own rules. In the “real world,” we had to do things that were useful for the job, not just the things that were interesting or enjoyable. Working in an office was nothing like playing, neither was being a teacher, and especially not doing taxes.

So what does this have to do with user experience?

Guessing how to design or build something for someone else is like pretending to be a nurse or a detective. When we mentally try to put ourselves in the shoes of our users, we are still us. We can try our best to imagine their experiences and needs, but we will always be free to choose the parameters we think apply to those experiences.

During a recent research project, I was interviewing people about the process of deciding where to go out on weekends. We thought this would be really simple. After all, how much complexity can be involved in rounding up some friends to share a cocktail or a game of bowling?

I was entirely unprepared for what I was going to hear. When I asked one woman how she plans get-togethers with family, she told me a story I could never have imagined myself. She lives in a community of a dozen, non-English-speaking family members, who rely on her to arrange how they spend their free time together. Planning family activities is one of her core roles in her family, she told me.

Each outing is a project she proudly orchestrates: she finds a place that would be interesting to the children who are coming (depending on their age), she takes into account the financial situation of everyone attending, she phones in advance if the group is getting larger. She also keeps a vast mental and physical collection of future outing ideas. “I take great pride in knowing I’ve set up an enjoyable time for everyone who came,” she told me, “everyone in my family relies on me to organize our time together.”

Her role in her family and community was tied to what I could only imagine as a frivolous activity, or an afterthought. Without speaking with her, I would continue to make product decisions based on that universe of assumptions. And I would be wrong.

Our imagination is grounded in who we are. Sure, child play and business assumptions share only a thin thread of similarity. But imagination works all the same: it transports us from one corner of our mind to another. It cannot transport us into the minds of others with any accuracy we can rely on.

The only way to truly understand the constraints, emotions, philosophies, and behaviors that guide our users is to get in touch with them: speak with them, observe them at work or play, and seek to find out how they’re different from us.

Otherwise, with the best of intentions, we’re only pretending to design for others.


Image credit:

Empathy Research

On ideas, research, & augmented reality

Would you rather design an app with a fashionable feature, or an app that someone truly needs?

Mobile apps (and websites and their digital friends) are often confused with advertisements. The mentality goes something like this: “we’ll build something with our logo on it that has cool features, looks pretty, and it will reflect well on our company.” But these projects often end up with pointless features or important things lacking.

Methods Research

Practicing Your Interviewing Skills With Strangers

It’s important to practice interviewing because it’s a skill that’s different from just having a conversation. Asking open questions, not trying to fill silence with more ideas, finding the right balance between listening and sharing – all these things are a learned skill. And learning often means making mistakes.

But how do we make mistakes without putting our projects at risk?

Last summer I read a blog-turned-book titled Talk to Strangers. The book is a collection of autobiographical posts by an anonymous  author who goes by “People Person”. The author is a shy, lonely man in Los Angeles who realizes that a few years out of university, he has no close friends and feels entirely lonely in the big city.

What he does next is pretty extraordinary. Even though he describes himself as nothing short of an antisocial a-hole, he decides to make friends by initiating conversations with complete strangers on a daily basis. What follows is a collection of hilarious, touching, and sometimes embarrassing stories that eventually land him in a completely different life to the initial lonely planet he lived on.

His successes and failures (and there are plenty of failures), made me realize this might be the most convenient, low-investment way to practice interviewing. Just like People Person, you can do this anywhere: in line at the store, on the bus, on a flight. Anywhere where you have to be in the company of people for a few minutes or longer. Be warned, however: apparently elevators are a poor place to start.

Mission: Find out one meaningful thing about a stranger

There are many detailed resources on the skills you need for effective interviewing. For these short encounters, I would suggest focusing on one thing only: finding out something meaningful about the other person. People are great at making small talk but we know that design work is best aided by meaningful insights about people, like motivations, philosophies, and attitudes. How do we find someone’s motivation? We start with a meaningful action that was driven by it.

On a flight from NYC to London, I decided to find out how the people I met chose where they lived. I started with a simple question: Where are you from? Here are some of the stories I heard.

The hair stylist who doesn’t like NYC. The woman who sat next to me on the flight was from Poland but married an American and moved to New York with him. She left on a bridal visa, had a child, and they got divorced since then. She doesn’t like NYC but is a hair stylist and doesn’t want to go through getting a license in a different state. She also loves her client base. When her daughter is old enough, she will definitely move to avoid the NYC school system.

This story is a bit more detailed than your usual small talk. It goes to show that most people are pretty happy when asked about their lives or what they do for a living. It didn’t all come from “where are you from?” I had to ask some follow up questions:

  • How did you end up here?
  • Do you like it?
  • Do yo think you’ll stay here forever?
  • Where would you move next? Why aren’t you moving there?

The Flight Attendant Who’s Never Moved. When I got up to use the rest room, I saw that the flight attendant standing near the lavatories had a band-aid on his forehand. I asked what happened and whether he’s having a good break (he wasn’t on a break!). Then whether he was from London. He was, in fact, born and raised in Glasgow and still lives there because he travels a lot and loves the feeling of a stable home base.

Here’s the entire inventory of questions I asked him:

  • Are you from London?
  • How long have you lived in Glasgow?
  • Why have you decided to stay in one place?

I didn’t get as much out of this conversation. It was pretty ambitious to ask several open ended questions while he was rushing through serving the morning tea. Eventually, he seemed to be somewhere between irritated and busy. It wasn’t the best timing.

The Student Who Has Friends Everywhere. While eating lunch at the airport, a young woman asked to sit next to me because there were no other seats available. I agreed and immediately asked to borrow her phone charger even though I had one (you gotta do what you gotta do). She politely declined and explained that it is her lifeline for the next several hours since she has two 12 hour connections. I had to ask where she was going.

She turned out to be an American college student who wanted to travel so badly that she postponed her graduation to do it (talk about a meaningful action). As a rule, she chooses to travel to places where she already knows people. What is even more impressive is how easily she opened up and began talking about her life. It took perhaps two minutes.

Mission Accomplished

This experiment was useful but also fun. It felt a little less foreign and lonely to spend ten hours in airport halls. I got to share a bit about myself and spend a lot of time listening to people’s stories. Best of all, even when it didn’t work out – I could politely get back to my reading and try again later.

What’s important here is how much more I could learn about them beyond just demographics. These were people who based their life decisions on their family, who were desperate for a constant place to call home, and who put aside a normal life plan for new experiences. Now, that’s something we can work with when thinking about personas, user journeys, scenarios, and how to plan features.


Empathy Research

What We Reject Is Who We Are

To do good design, it’s not enough to ask what someone likes, we need to ask what they do. We need to ask what they never do. And when they tell us, ask why. And ask why again.

Likes and opinions are too fleeting to drive design. When doing user research, talking about product strategy, or trying to understand our colleagues’ obscure decision-making process, we have to dig deeper than just superficial opinions or generalizations.