The Future Proof Podcast 016
Melissa Avery-Weir 0:21
Hello, and welcome to the Future Proof Podcast. This is our monthly podcast where we chat about stuff we've been working on and anything cool, we're planning. I'm Melissa Avery-Weir.
Gregory Avery-Weir 0:30
And I'm Gregory Avery-Weir.
Melissa Avery-Weir 0:32
And today we have a special guest, Dr. Lucy Arnold of Liminal Consulting is here to talk about the survey that we've run for the last couple of months.
Lucy Arnold 0:42
Hi, thanks for having me.
Gregory Avery-Weir 0:44
Happy to have you here. We're going to chat with you a little bit about some of the survey work we've been doing.
Gregory Avery-Weir 0:48
Before that though, we've got an interesting little task we've been working on for Exploit: Zero Day.
Melissa Avery-Weir 0:55
Yeah.
Gregory Avery-Weir 0:56
We've talked in the past, I think, about the user blocking system we've had in place where we want players to be able to say, "Oh, I don't want to hear from this player," just for whatever reason, right? You want to be able to block people on the internet. But what we didn't have is we didn't have that system in the game itself, where you could hide people's levels from your notice. We didn't have that hooked up to the forums.
Melissa Avery-Weir 1:19
Right.
Gregory Avery-Weir 1:19
So that you also block people on the forums when you block them in the game. We're working on the code, and we think we've got it to, to make those two things go together. So when you block someone, you block them on the forums, and when you unblock them, they're ignored on the forums.
Gregory Avery-Weir 1:34
But we ran into a thing with Discourse, where-- Discourse is an opinionated piece of software, and what we mean by that is it's a, it's software that where the developers have decided that there is a right way of doing things. There's a right way of running a forum. And their, the software is designed to enable that approach and not really interested in supporting alternative approaches. I think generally we like that sort of way of doing code.
Melissa Avery-Weir 2:05
Like, we, we certainly like products or you know, anything from games or whatever that that actually live their ethos, right? Like, yeah, say you're here for something, do that thing.
Gregory Avery-Weir 2:17
Yeah. And on the flip side, if you say you don't have an opinion, and you don't have a way of examining something you actually do.
Melissa Avery-Weir 2:25
You do.
Gregory Avery-Weir 2:25
And you're just not surfacing it. But it meant that, like Discourse assumes that you only ever want to ignore someone, you should only be able to ignore someone for four months or less, I think was the number?
Melissa Avery-Weir 2:40
Which is bizarre. A third of the year? What? Where does that number come from?
Gregory Avery-Weir 2:44
And also you need to be a relatively well established member of the forum in order to start ignoring people.
Melissa Avery-Weir 2:49
Which, again, what?
Gregory Avery-Weir 2:51
Does not match the way that we imagine things. Their, their idea is like, "Discourse requires communication and and requires, you know, good, good community--good community building means that people have to be able to hear other people's points of views." Not how we feel. I understand that.
Melissa Avery-Weir 3:07
Or any social media network I want to be part of.
Gregory Avery-Weir 3:09
But it meant that I actually had to go in and create a plugin for Discourse that sort of reach into where ignoring is turned on and off and pull out bits and allow other bits to be customized. So I've finally done that. And we'll probably--when we do this next deployment, whenever we do, it will probably link to that. No support provided; it's really simple. But if anyone runs into a similar thing with, with Discourse, they can certainly use that.
Melissa Avery-Weir 3:35
It's definitely one of those things that was like, okay, we think now that we know that we can hook in with the API to set a user as ignored... We think this task might be an hour or so right? Plus time to test, like, something like that. And then it was like, Oh, snap, you cannot do it permanently. Or it you know, indefinitely, I guess.
Gregory Avery-Weir 3:54
So it ended up being more like four hours of work including a lot of research and reading through forum posts and so on.
Melissa Avery-Weir 4:02
So that was unpleasant, understandable, but also reveals the opinion of the developers and designers of Discourse.
Gregory Avery-Weir 4:13
Yeah. But speaking of community building,
Melissa Avery-Weir 4:19
Yessss.
Lucy Arnold 4:19
And social capital.
Gregory Avery-Weir 4:20
Yeah.
Gregory Avery-Weir 4:22
Lucy, let's talk a bit about the the, I think we called it a community survey but it was also to help us direct our marketing of the games we make and of ourselves as a... as a... I was... ourselves as a brand.
Lucy Arnold 4:39
Brands are hard. I guess for me, marketing has that kind of unethical, "This is something you don't actually want that I'm trying to persuade you how to purchase" vibe to it. So I prefer the idea of community building, because we're all in a capitalist system, but communities don't have to operate within that capitalist system.
Gregory Avery-Weir 4:59
Yeah, I know that Lissa have often felt kind of grimy, trying to do "marketing" or "PR" for that sort of reason.
Lucy Arnold 5:07
Well, that's actually when we first started talking about this. Lissa was expressing that kind of feeling about griminess associated with PR.
Melissa Avery-Weir 5:15
Yep.
Lucy Arnold 5:16
And I think, you know, people, businesses want to be able to do good work and find communities who want that work and who need or want to be a part of that community. And so creating that kind of alignment between people, to me is better than thinking about it as marketing.
Melissa Avery-Weir 5:35
Yeah.
Lucy Arnold 5:36
And also maybe more accurate, right?
Melissa Avery-Weir 5:39
Yeah.
Lucy Arnold 5:39
So what you're doing,
Melissa Avery-Weir 5:40
When I think of community, I think of people in a single place, right? I think of people on Reddit or people on a forum. When I think about marketing, I think about a dispersal of information. And so, part of the reason my mind didn't connect those things automatically, I mean, not that people--Obviously, people have communities on Twitter, or various other more separated and asynchronous locations. But the reason I have struggled with how to link those two is that is a forum the answer, right? Like.
Lucy Arnold 6:13
Yeah, I mean, marketing does have that sort of one way feel to it and thinking about it as a discourse--not that I'm referring to the same thing that you already been talking about--I think is really helpful. And I think a survey is a good starting place because it encourages a conversation. It encourages you to think about what you're creating and how people are thinking about what you're creating and what kind of like what kind of use people have for that. So you have a survey now!
Melissa Avery-Weir 6:50
Yes, so it ran from end of October, late October, until December 10 if I recall correctly. So about a month and a half give or take, is that correct? Yes. That's how time works.
Gregory Avery-Weir 7:04
Yeah. And we certainly, like, gave you some ideas of what we wanted to, to know and so on. But, but you, Lucy, you did a lot of the actual design of the survey yourself. And what was that like? How do you do that? How do you go about that that process? What's, what's your?
Melissa Avery-Weir 7:23
If that's not the magic sauce that's like, the whole business, you know.
Lucy Arnold 7:27
I mean, it is the magic sauce, but it's a craft, right? Something that you have to practice doing because--you all had a lot of ideas about what you wanted. And so a lot of the start of that process was to listen to you to try and figure out 1) What, what did you really want and then, 2) what of those things that you want are actionable, right? Because some of the things you know, some things you're curious about, right? And you'd like want to do some inquiry on it, but if you don't have a use for that information, you might not always want to put in something like a survey.
Lucy Arnold 8:03
Now, I mean, if you guys were this were an academic survey that might be a little bit different because sometimes we do really long, really complex academic surveys, but you don't want to send something out like that to your community, because then people are not going to spend the 30 minutes or longer it's going to take to fill that out.
Lucy Arnold 8:21
So you wanted it to be something short and engaging for your community. Because that's the other thing about it is it's not just a survey for you to learn from. It's also an opportunity for you to say, "here's how we're thinking about our community. Here's how we're thinking about the people who play our game." That's a piece of rhetoric, right?
Gregory Avery-Weir 8:39
Yeah.
Lucy Arnold 8:40
So it shows how you frame the people in your community. And also plants a kernel of thought.
Melissa Avery-Weir 8:49
I was thinking about that as I was looking at results and some of the questions we asked were "Have you heard of this game?" Right? Like have you heard of "Silent Conversations," which is an old Flash game that is hosted on futureproofgames.com. And by virtue of putting that question there, people are gonna be like "I hadn't, no, I haven't heard of that." Which is fair. Will they go look it up? Right? They now at least know that name of it or have heard the name of it that they might not have ever heard before. So yeah, I was definitely thinking about that, as I was looking at the results. It's like, hey, folks just learned about six new games they hadn't heard about.
Lucy Arnold 9:20
And that's great. Yeah, I mean, that's a good use of the tool that you're doing. And then you're also learning something about the people who are taking it. So that's good. I think the big thing that came out of the creation of the survey was really listening, getting all of our ideas down. You know, we had a shared Google Doc, where everybody was contributing, and we were really thinking about it.
Lucy Arnold 9:45
But then in the end, we really had to think about the scope and clarifying your questions, so that you could get what you wanted out of it. So that was, I think maybe my role in that part of it was figuring out how to turn those into questions that we're really getting at. Because a lot of times, you can't just come out and ask the thing that you really want to know, because people often don't know how to answer.
Melissa Avery-Weir 10:08
Right.
Lucy Arnold 10:09
Those kind of questions.
Gregory Avery-Weir 10:10
I mean, part of what we're hoping for from the survey is "how do we actually get to sell stuff?" And that's not really like--
Melissa Avery-Weir 10:18
But some of it is come play our free stuff, right?
Gregory Avery-Weir 10:20
Yeah, yeah. How do we get our--how do we get you to play our stuff? But that's not, that's not a good way to approach interacting with someone is to say, here's what I want from you.
Melissa Avery-Weir 10:29
And also, like, it's like if you, if you make a game and you're like, "give me feedback on the game" or "tell me what you want to see in this game," you get a lot of armchair game design that is not useful. I mean, it's, that's really, I don't want to be harsh.
Gregory Avery-Weir 10:45
But yeah, there's there's a there's an adage of that's, like "feedback is very valuable from players, but players often don't actually know the solution to the problems that they identify because that's not their job."
Melissa Avery-Weir 10:57
Right.
Gregory Avery-Weir 10:58
But they are very good at knowing what's wrong.
Melissa Avery-Weir 11:00
Yes
Gregory Avery-Weir 11:00
Or, or what, how they feel about something that's wrong.
Melissa Avery-Weir 11:03
Yeah. So this seems like a very comparable type of situation where you want feedback, but folks probably can't give you the solution.
Lucy Arnold 11:12
Right. Yeah. Well, so we can talk about--this is the exciting part because we can kind of talk about what actually happens. So, yeah.
Gregory Avery-Weir 11:21
So you've completed 100% of all the analysis you're going to do on the data already?
Lucy Arnold 11:25
No, no, no, no. I'm going to do a much deeper dive into the data. This is just a sort of initial taking a look at it.
Melissa Avery-Weir 11:34
So to be clear, for for our listeners, we close the survey very late on December 10, because we had some issues with the survey provider, and it is now the afternoon of the 14th. We have just finished a role playing game for several hours. So I'm sure that Lucy has done all the analysis in the last several days.
Lucy Arnold 11:55
Well, so there will be a deeper dive but just a few interesting things to sort of start. One is about data collection that I just think is really interesting. So, you know, you sent out this digital survey. So people were mostly public taking it on mobile and on their computers, and it was disseminated through social media and their email. So a lot of times surveys don't even get clicked on. So that's one big issue with it is even getting surveys out there. And even once they get clicked on, a lot of people don't complete them. So one thing that immediately stood out to me is that you guys had a 73% completion rate, which means that 73% of the people who clicked on it finished it. That's really good!
Melissa Avery-Weir 12:38
Holy shit! I have worked at a lead gen company and 73% would make their day.
Lucy Arnold 12:44
Right!
Gregory Avery-Weir 12:44
We have very awesome folks who took the survey screen.
Melissa Avery-Weir 12:47
Thank you. Oh my God.
Lucy Arnold 12:49
That's amazing. I will say the last survey I conducted which was an academic survey, the completion rate was significantly lower. So I was immediately struck by that. And I think that that speaks to like how you got out there and also to the engagement of your communities. So yay.
Melissa Avery-Weir 13:09
That's awesome.
Lucy Arnold 13:10
Yeah, that's that is awesome. And then just sort of top level looking at results. So you talked about branding, going into this.
Melissa Avery-Weir 13:18
Yeah. Quote unquote.
Lucy Arnold 13:20
Yeah. So you, in a way, you're offering part of your branding with the survey. Because it's what you think you know, your brand is.
Melissa Avery-Weir 13:27
We gave options.
Lucy Arnold 13:28
Yeah, exactly with options.
Gregory Avery-Weir 13:29
So we said something like, "what, what do you think of when you think of Future Proof Games or when you think of Future Proof Games works?"
Melissa Avery-Weir 13:35
Yes. It was transhumanism and quirky and something like that?
Lucy Arnold 13:38
Well, quirky is, yeah, that's the one, right? That's the highest result like: you're quirky.
Melissa Avery-Weir 13:44
Yay! It's hopefully in a mostly good way.
Lucy Arnold 13:48
I would think so. I mean, for one I think quirky is one of those words with a great connotation, right?
Melissa Avery-Weir 13:54
Yes.
Lucy Arnold 13:54
It's not "esoteric." That sounds bad. But quirky, that's good.
Melissa Avery-Weir 13:58
Yeah.
Gregory Avery-Weir 13:59
So unfortunately, it's kind of hard to leverage, like, it's hard to be intentionally quirky.
Melissa Avery-Weir 14:03
Yeah.
Gregory Avery-Weir 14:04
I guess it means that, you know, if assuming that the deeper analysis holds this up, we can be comfortable being weird. We don't have to go like, "Oh, we don't want to go too far afield."
Melissa Avery-Weir 14:13
Right.
Lucy Arnold 14:14
Well, yeah, that's actually one other thing that I like just some of my initial impression from looking at the results about how to think about the quirkiness, maybe that people like is: I think one thing that your respondents are saying is that you have high expectations for players of your games, and they love that. They dig it. So...
Melissa Avery-Weir 14:36
Interesting.
Lucy Arnold 14:37
Yeah. I mean, you know, I can show you in some of the qualitative analysis when we take a look at the results, but it comes up very consistently, you know, people say things about cleverness and the quality of the writing and that kind of thing.
Gregory Avery-Weir 14:51
You can't see this, but Lissa and I are dapping.
Melissa Avery-Weir 14:53
Yes. That was the soft sound of a dap.
Lucy Arnold 14:57
Where is the explosion?
Melissa Avery-Weir 15:02
*explosion noise* Thank you for this editing nightmare.
Gregory Avery-Weir 15:06
Oh, this is all staying in.
Melissa Avery-Weir 15:08
It must, because I cannot cut around it.
Lucy Arnold 15:12
Excellent. Two things that came up, two things that came up from the responses both in the close-ended questions and the open ended questions were: puzzles and meaning. Those are two obviously very different things.
Melissa Avery-Weir 15:28
Or not?
Lucy Arnold 15:29
Well, I think they're probably two different things. At least in this context, I'm not trying to talk about language. I mean, I am a language professor. Like we could go there. But now I will temper that by saying that a high majority of your survey takers, were some combination of Majesty of Colors, players and Exploit: Zero Day players.
Melissa Avery-Weir 15:52
Well, so yes.
Lucy Arnold 15:53
Do take that into consideration.
Melissa Avery-Weir 15:55
Yeah.
Lucy Arnold 15:55
When you think about both of those games, right.
Gregory Avery-Weir 15:58
Yeah. On the other hand, that does mean that they are people who have played our stuff, right, like, these are, these are existing members of our community.
Lucy Arnold 16:06
I think that's also good to know.
Melissa Avery-Weir 16:08
Yeah.
Lucy Arnold 16:08
That those are the two games that at the top of the list for a lot of the respondents.
Melissa Avery-Weir 16:13
Yeah.
Lucy Arnold 16:14
And then the other part of the results that I would just very quickly touch on would be the issue of social media and how to communicate with people. I think that's something we're gonna want to dig into on a little bit of a deeper level.
Gregory Avery-Weir 16:30
It's something that we've definitely struggled with for like years. For a while we were like posting to a whole bunch of networks until we realize it like probably no one wants to hear from us on LinkedIn.
Lucy Arnold 16:41
Uh, your survey confirms.
Gregory Avery-Weir 16:43
Okay.
Melissa Avery-Weir 16:44
Yes.
Lucy Arnold 16:44
Do not use LinkedIn any further for Future Proof Games.
Melissa Avery-Weir 16:47
And then there's the question of, do we only, like one of the things we've struggled with is: is there any harm and broadcasting everywhere even if we don't expect engagement on LinkedIn?
Gregory Avery-Weir 16:59
Yeah.
Melissa Avery-Weir 16:59
Like, is there harm there? If not, then?
Gregory Avery-Weir 17:03
There's a cost, right? There's our own effort and, and sweat and tears and pain.
Melissa Avery-Weir 17:09
Yeah, that's true.
Lucy Arnold 17:09
So that is important. I mean, those, that effort is actually important.
Melissa Avery-Weir 17:13
Yeah.
Lucy Arnold 17:14
Because you only have so much energy that you're putting into a thing. And you do need to think carefully about what you're putting that energy toward, in my opinion.
Melissa Avery-Weir 17:21
Yeah.
Lucy Arnold 17:21
And then I think you also have the other thing to consider: I think a lot of the stuff going on with Facebook was not as big when we first began this survey. But there's also the ethical question of whether you want to engage with something like Facebook.
Melissa Avery-Weir 17:35
Yeah, if not for anything I do that's related to development, like the fact that we have like Facebook login, and things like that. If not for that, I'd have closed my Facebook account a hot minute ago. Like it's only for that dev stuff that I have it.
Lucy Arnold 17:48
Yeah, I mean, so those are all really legit questions. And I also think it's interesting that this was actually one of the lower responding areas in the survey. So we left the survey so that people did not have to take any of the questions that they didn't want to. So you can also see which of the questions were most engaging to the people who took the survey. Writing a survey is sort of like game design, because you do want a survey to be engaging for the people who are taking the survey and you don't--
Melissa Avery-Weir 18:18
I have taken many boring surveys.
Lucy Arnold 18:20
I'm a bit of a connoisseur of surveys. So you definitely want your survey to have that sort of engagement. And it doesn't have to be fun, but you do want it to be engaging, right? And create some kind of meaning for the people who are taking it.
Lucy Arnold 18:33
And so we also have that level of analysis which is thinking about which of the questions did people take and a lot of them did not even respond to some of the social media questions and, and that's an interesting fact, you know, like looking about what that means.
Melissa Avery-Weir 18:47
That makes it tougher. If one of the important things we wanted to know was how to, where to engage with people?
Lucy Arnold 18:53
Well, and they did answer some of the other social media questions. So we'll, we will have some data to take a look at in terms of where, where people want to engage with you. So that'll definitely be something we'll be able to dig into. I definitely think it'll be interesting to have some conversations about streaming and Twitch. And--
Melissa Avery-Weir 18:55
Oh, yeah.
Lucy Arnold 18:57
That kind of thing.
Melissa Avery-Weir 19:14
There's that.
Lucy Arnold 19:16
Do you guys have any other questions for me right now? Try not to make them hard ones.
Gregory Avery-Weir 19:22
Was there anything else that particularly surprised you just glancing through? Even if, even if it's something you're not sure about from, you know, you haven't had time to analyze the data. But was there anything that popped out to us like, this is a weird thing?
Lucy Arnold 19:32
I think the thing that surprised me the most, which I had to give a little bit of background to. A lot of times when you're doing a survey where you want high response rates, you keep it to multiple choice, or some kind of closed questions, and you don't often have required open answers, that kind of thing because it takes more time for people to write things. And people don't always want to take that time to write things. You a lot of respondents who wrote things.
Melissa Avery-Weir 20:00
That's cool.
Lucy Arnold 20:01
That is interesting.
Melissa Avery-Weir 20:02
Yeah.
Lucy Arnold 20:02
There was a high level of, I would say written engagement with your survey, and some of it was quite lovely to read it.
Melissa Avery-Weir 20:10
I will admit I went through it I was like, aww these folks are so sweet.
Lucy Arnold 20:14
Now I mean, not even just kind of a lot of it was very kind, but also just really lovely. So.
Gregory Avery-Weir 20:20
Like well written?
Lucy Arnold 20:21
Yeah, I mean, you some, some smart, creative people taking the survey.
Gregory Avery-Weir 20:25
Well, that's kind of what we want to inspire. So that's good to, good to hear.
Melissa Avery-Weir 20:29
Awesome. Well, thank you, Dr. Arnold, for coming on the podcast to talk about this with us. We are really, really excited for, like, the deep dive and working through like--My mind tends to focus a little bit on what's actionable, but also I'm interested in sort of the broad sense of things, right? Like the softer side, for sure.
Lucy Arnold 20:51
You're just hyped for that slideshow and those charts and graphs.
Gregory Avery-Weir 20:54
Oh, there's going to be a slideshow?
Melissa Avery-Weir 20:56
Excellent.
Lucy Arnold 20:56
I've got my laser pointer!
Melissa Avery-Weir 20:58
Now will it be PowerPoint, or will it be Prezi?
Lucy Arnold 21:03
I said don't ask me hard questions!
Gregory Avery-Weir 21:06
Picking a favorite child.
Melissa Avery-Weir 21:07
I know. So Greg and I have been working on something else interesting that we'll be talking about in January's episode, which is that we have been doing sort of a broader lifecycle planning process for our games.
Melissa Avery-Weir 21:21
So this is instead of like, normally, we would do like a launch timeline, right? So here's what here's what the month leading up to launch looks like, here's what a few weeks after it looks like. It's like emailing press and social media and update the charts. And this is like, what do we do with The Majesty of Colors over the next couple of years? What are interesting things we can add to it? What are kind of points of interest for players, things that we can add?
Melissa Avery-Weir 21:46
And so as we're, we're looking at The Majesty of Colors first. We'll cover probably maybe Rosette Diceless, maybe another one, just kind of... It's a lot of meetings. So we don't like, you know, we also want to do other kinds of work. And so in January, we'll talk some about kind of how this looks. Like if we take a couple of our games and we look at this, what the next couple of years might bring, and then we kind of overlay these on top of each other. That kind of helps us set priorities for what our work looks like in the next, in the upcoming months.
Melissa Avery-Weir 22:15
So we're excited to talk about this. We'll see kind of how things look. So that's look forward to that next month.
Gregory Avery-Weir 22:21
Yeah, I mean, we don't really know exactly how it's going to end up yet. So once we do we'll, we'll be able to talk about it a little more clearly.
Gregory Avery-Weir 22:29
So Lucy, I would highly recommend that people consult with you for survey design and analysis and and other advice surrounding that, that sort of community building stuff. If folks are interested as well, how would they get in touch with you?
Lucy Arnold 22:44
Thanks. You can find me on Twitter at @LucyReadsEvery1. That's the numeral one.
Gregory Avery-Weir 22:52
And we'll link to that in the show notes.
Melissa Avery-Weir 22:54
Yep.
Lucy Arnold 22:55
Or via email at zen.lucy.arnold@gmail.com.
Melissa Avery-Weir 23:02
Excellent. And you can find all of our stuff over at FutureProofGames.com. We're over on twitter at playfutureproof and we'll probably be staying there. And on Facebook as Future Proof Games, which... you know, we'll see. Hit us up with questions or comments over on our blog or social media. Our theme music is "Juparo" by Broke for Free which is available under a Creative Commons Attribution 3.0 license.