This podcast features Julia Keren-Detar, UX Designer at Untame, and her Lightning Talk, “Usability Testing & Development on a Shoestring Budget” from the design leadership conference Prototypes, Process & Play on August 10th, 2017.

Prototypes, Process & Play presentation podcasts are sponsored by Balsamiq – with Balsamiq Mockups, anyone can design great software.

Julia Keren-Detar – Lightning Talk

UX Designer at Untame

Julia Keren-Detar is a game designer and developer based out of Chicago, IL. Currently she is the Creative Director for Untame, a studio shared by her and her husband. She is working on Mushroom 11 with Itay Keren, Simon and Kara Kono, contributing to the game design, UX and marketing. She’s worked on several casual and indie titles including Untame’s award-winning mobile game “Rope Rescue” and Arkadium’s hit Facebook game “Mahjongg Dimensions Blast”. At Arkadium, she was a senior developer at the R&D department, prototyping and designing dozens of games. Julia is also an avid quilter and a hobby game historian.

For more, keep up with Julia on Twitter as @quiltingcrow.

Usability Testing & Development on a Shoestring Budget

As an indie game studio, we had a shoestring budget for developing Mushroom 11, an award winning video game on desktop and mobile. This talk will explain how we used prototyping to make fast iterations on our design and how we used trade shows and other marketing events as ways to collect user data which informed our design process throughout development.

Presentation Transcript

Please note:

Podcast transcript below.​ Please note: Transcription was recorded live; there may be errors (typographical and contextual), as well as omissions or other content gaffes.

​Additionally,​ there was microphone feedback that happened in the room from time to time, and we did our best to minimize it in the podcasts. We apologize for any disruptions to your listening experience that this may cause.

Julia Keren-Detar:

I’m Julia Keren Detar, I’m a game designer. I work on Mushroom 11. Desktop and mobile. This is sort of what it looks like. And it won a few awards.

And there was the Google framework this year, which I missed. But this is the team. So as you can imagine, we bootstrapped most of the funding for this project. We didn’t have a lot of for development and testing. And I’m going to tell you about how we tried to solve the problem of doing all the stuff without a lot of money.

And hopefully you can take away some of the stuff that we learned.

So the game ideaitself came out of the global game jam. It’s basically a hackathon. It’s a very, very short period of time to build whatever you’re building. So it’s an annual event that happens at the same time all over the world. You get to feel like you’re participating in this really cool event. But they haven’t just for small teams, a lot of – use these, as well. The most famous one – So I mean lots of hackanons. The most obvious is time constraints. And having really low expectations allows for experimentation. You’re not going to get great software in 48 hours. You’re just not. This frees you up to play around with really crazy ideas.

If you had more time and more resources, you would be able to participate in.

It’s also a really great way to try to untap if you want to switch to a different engine or a different programming language, it’s a really great way to grab money.

And then for larger game companies, I used to work at a game studio where I implemented game jams.

And the biggest takeaway was cross‑disciplinary learning, especially people across development. Having them on teams, even days. It allows them to sort of see how it gets made. And not only do you feel more integrated as part of a team, but we also have a better understanding of what they’re promoting and selling.

So game changing is another type of prototyping. And prototyping is basically how our team was able to really quickly get development costs down because we were able to do iterations very, very quickly. So a lot of times before, you know, people even write a lot of code and software, they’ll do prototyping with something that they have around. For us, we use prototyping throughout the entire development of the game. We don’t care about the art. We just grab whatever we have. We don’t care about optimization or whether it looks great. We’re just trying to figure out is this idea worth pursuing like as fast as possible?

This interactive game design is really important. For games, it’s such a wide design space. It could be any kind of experience. It could be any kind of game system. And small changes can really impact that player’s experience. So it’s really important to answer like a lot of questions and get to those answers as fast as possible.

One of the questions would be like does the player know what they’re doing, which goes beyond just U.I. Yes, U.I. is really important and we need to know what button to press to make my guy go. But more importantly, where do people go when they have a wide open space, or do they know what items they need to crack to put together the weapon that they need. These other kinds of systematic problems are just as important and you need to figure that out really quickly.

Another thing that games deal with a lot is something called difficulty ramp up. If you make a game too hard too fast, you’re going to frustrate users. If you make it too easy, they’re going to get bored. And the interesting design challenge for gamers is you have a time scale along with this. You have beginners and expert players. And they occupy the exact same space. You have to design a product so you’re making sure that the beginner has a good experience and you’re making sure the expert player has a good experience.

And then of course fundamentally like is the game itself fun? Is the action that you’re doing interesting enough that you can expand that out to, you know, 30‑hour game play or 15‑hour game play, however long you want. That has to be interesting.

The problem with these questions is they need outside verification. The game designer can come up with a pretty good solution of what might happen. But unless you get users to interact with it, you just don’t know the answer to those questions.

For example, these are just some of the iterations we have to go through to design a dynamic help system in our game. This is one small aspect of the game that half of the players won’t even see because it only pops up when players are really struggling in the beginning of the game.

So user testing, this is really, really important with every iteration we do. For us, we didn’t have any way of paying the users. So one thing that we did really early on is we had a lot of friends and other co‑developers that we would co‑op with or work with or have local meetups where we would share our products we’re working on. And this allows us to get really fast feedback and expert feedback. So their feedback is really valuable. But the only problem with this is you run through your users really, really quickly.

So if you want to test like a question that relates to a new person, if they’ve seen your game and they’ve seen your product, they’re not a new person anymore. You need to find somebody else.

So meanwhile separately we were applying to all these festivals and contests so we could get into these shows so we could get promotion and talk to the press and stuff like that. But we found out that these shows are really, really great for user testing, as well.

Most of the shows, the games are not up yet. They’re in development. There is this expectation that they can be tweaked and they’re not final. This allows us to do things like AV test different stations. We can do a physical test where we can compare versions and see which one performs better.

Another thing about shows is there are so many people at these shows and there’s so much hype around it that your userses have like no buy‑in whatsoever. I’m like paying someone to come into your lab and interact with a product, these users don’t care. As soon as they’re confused about your product, they’re going to go out and leave. It’s really easy to see where there are design flaws in your game. But it’s also really important to know who your audience is. These shows were able to give us a lot of demographic variance and regional differences. This is important when observing your players.

For example, we showed it at South By Southwest and the people who go there don’t necessarily go there for video games. They go there for music or art. But they’re not traditionally gamers. And they’re going to have a vocabulary that is different from non‑gamers. How should a game look or feel. And those people may be stuck at different points of interaction. And then you can get a better sense of what may be the problem.

Another one is identifying when a lot of people get stuck. Specifically for game designers. Because game designers tend to be like they are stuck because they just haven’t mastered this awesome challenge that I made. How to conquer the solution. Really they haven’t figured it out yet. They don’t realize that it’s probably not that. It’s probably a design flaw in your product. They don’t realize what they’re doing or a game puzzle.

Another thing is unexpected behavior. This is what fascinates me about user experience. Oh, people totally understand. It’s really important to try to get into their heads and understand why they’re doing this behavior that you didn’t expect for. Can you use it to better improve your design? Or do you have to scratch it and come up with something else? Don’t lead the witness. Right? This is really hard to do especially in conferences specifically because you have press and journalists. What is your game? How do you play your game? Give me your pitch. No, you should figure it out. I’m not going to tell you how to play it or solve the puzzles, because once you do you sort of ruin that user.

And then the other really important thing, the last part that I have is collecting context to build a fan base. It’s really important to have a group of people that you can come back to after the show is over or when you have different versions of the product in the future, you can reach out and ask for information about how they feel about it.

Your users are really excited about your product. They’re expert players. You have to take that into consideration and understand that they’re not going to be your normal user. So that’s it. Thanks!

[Applause]

About Chicago Camps

Chicago Camps, LLC (chicagocamps.org) was founded in 2012. They plan multiple low cost, high-value events primarily in Chicago.