Author: marthasadie

Why I’m fed up with digital projects (and why I’m not): a rant

I’ve felt a little rant bubbling up in me over the last few months: a sense of disquiet about digital, a jaded annoyance about wasted time and resources and opportunities squandered. Today I was reminded about an old project that was the epitome of digital idiocy, one of those thoughtless knee-jerk “we must have an app!” projects that make me want to throw a toddler tantrum, kicking and screaming “but who is it for?” until someone agrees to at least do a bit of audience research or string together a minimally viable set of objectives. And that reminder seems to have brought it all to the surface, so here goes.

I am fed up of seeing people and organisations produce digital rubbish: poor apps, clunky games, badly designed microsites and other half-arsed online, mobile and technological systems and whatnots. I am fed up of people who are smart about digital, who think about users, who ask the right questions at the start, who embrace technology without fear and understand how to apply it, being over-ruled by people who’ve just bought an iPad for their kids and now assume that everyone has them and that this is all the justification they need to insist upon spending £50k on a new app for their organisation (an app for who?! For what?!). Moreover, I am fed up of projects that fail being brushed under the carpet, with no evaluation or debrief that enables everyone involved to learn from mistakes and build something better the next time.

It makes me sad to see money and time being wasted, when it’s usually avoidable, and even more so if nothing is then learned from this. It makes me especially sad, because the technology we have available to us today is AMAZING. My phone is able to do astonishing things and the computers I use have incredible firepower (I still remember how exciting it was to get a floppy disc drive for my BBC micro, now look at them!).  It feels like there is so much potential and so much more that the technology we have could allow us to do.

It’s not like there aren’t enough examples of brilliant products, apps, games, and other applications of these technologies out there to learn from. Too many to list. But many many many more that have been an utter waste of everyone’s time from the beginning because they just didn’t ask some basic questions about who and what their product was for, and test it with that in mind along the way. Isn’t that a bit sad?

I know it’s not easy. That there is a degree of luck in rising above the huge amounts of competition online or on on the app store, that things beyond your control can go wrong during the development of a project, and that there are some times you need to go on your gut instinct, and that’s not infallible. Goodness knows I’ve made every kind of mistake creating digital things over the years, as have most people working in this constantly changing field. And I will continue to make mistakes, as will most people etc.

So I’m not claiming there is a perfect process, but there are better and worse ones, and there are definitely massive howling alarm-bell-ringing clangers that should be sending up the red flag right from the start. The “we need an app, never mind who for”, the “we’ve got all this content, let’s just stick it online, people will come to us”, “let’s create a whole new portal for something that already exists”, and so on. And sometimes it feels like we aren’t moving on from this, these whack-a-mole stupidities keep popping up over and over again and just will not die.

Why is that? In a recent conversation with a fellow person-who-works-somewhat-indeterminately-within-“digital” we discovered we’d both been feeling this slight ennui with the field. This sense that it wasn’t meeting its potential, that a lot of rubbish was being made, and that it was hard to see how to make things better. We wondered if it it might be that digital is constantly being driven by the hardware makers, by Apple and console or computer companies and that their vested interest in creating new stuff for us to buy means we’re always just playing catch up. Why do we let this dictate the pace? Where is the chance to take stock and develop best practice?

My experience as a freelancer over the last few months, applying for funding and responding to ITTs, has also been eye-opening. The systems and funding in place for making digital stuff are really broken. If you were trying to design a process that would result in crappy digital products, you’d make people (who may not be particularly digitally literate) scope everything out and pin it down before they’d ever been able to build anything or ask their audience about it, only then engage an agency who are the real experts but can’t change the scope, and then make the project team go through a month long change request process if they want to develop the project in a different direction from the scope based on the evidence as they start building and testing things. Which is exactly the process in place in a frighteningly high number of places. The old models for procuring new infrastructure etc just do not work for digital projects, and it’s hamstringing them from the very start.

Plus money, of course. Not enough money, or not tailoring the project to the actual budget available.  I’ve had this rant elsewhere. I seem to spend a lot more time these days telling people not to make a digital thing. Would it work just as well as a piece of text? As a printout? As a physical activity? Or a really simple website? Don’t overreach if you just can’t afford it, do something straightforward and proven.

But at the same time I love working on and seeing others working on brilliant digital projects. The reach you can get, the amazing response in terms of both scale and thoughtful quality, the sense of shared global experience, the playfulness and the utility, when done well. So, how do we do more of these?

How to make games on a low budget

So you’re convinced, as I am, that games are a great way of reaching new audiences and engaging them meaningfully with your message or content. But you have little or no money, what to do? It’s all very well when you have upwards of £40k to spend (and ideally even more), but what about when you don’t? Are you excluded from creating or commissioning games?

Well no, I don’t think you are. But you’re going to have to do things a bit differently. You may have to give up a certain amount of control, and be relaxed about where games end up. But you probably should be anyway, the point is to get your message out there, right?

What not to do

I’ve recently heard about two separate organisations that were looking for proposals for educational games for between £500 and £2500, which made me gasp, I must admit. Taking this approach to doing games on a budget – just not paying much for them – is a very bad idea, for a number of reasons. I speak from experience, believe me.

Low budget projects are ALWAYS the projects that cause the most problems and take the longest. Your game will naturally take a back seat with the agency or developer when higher paying jobs come their way, and fair enough. As the commissioner, you won’t have much clout or sway over how the project develops, and aren’t likely to have much opportunity to change it as changes=time=money. Unless the agency/developer is bad at handling that equation, in which case they are likely to go bust trying to complete your game.

It’s the old cost/time/quality triangle. If the cost goes down, either the amount time goes up or quality goes down. Or, more likely, both. And since time=money, as established, the true cost actually becomes much higher for everyone involved. No-one wins and it’s bad for everyone’s business.

Instead, maybe try one of these options.

Find more money: other funding sources

There are a number of sources of funding for games out there. My old employer, the Wellcome Trust, is keen to encourage more games on biomedical subjects and offers a number of potential grants (this development grant, for example). Other public engagement funds are also likely to cover games projects, even if they don’t explicitly state that. Look at what people like Nesta or the Arts Council are currently offering in the way of funding. IdeasTap appears to have quite a lot of funding bodies listed on its website too.

I’ll add in more examples here as I come across them, but please do suggest any you know of.

A tip. Always always always, with any grant application, get in touch with people at the funding body to find out more and get advice on whether your project is suitable, or find out what you might need to do to make it suitable. Public engagement funding in particular is likely to need some sort of decent evaluation of impact built in, so don’t just say you’ll count the number of hits, give it some thought.

Find more money: partnerships and co-investment

You might not have the money, but presumably you do have something great to offer – content, domain expertise, a well respected brand etc. All of these might well be appealing to someone who does have money. A games agency might be interested in developing something in return for profit share. A company might see a good fit with their aims and want to sponsor your project. Another similar organisation (arts, cultural, educational?) might be interested in a partnership, or perhaps a group of you could club together and pool budgets. A broadcaster such as the BBC or other online platform might be interested too.

Partnerships of this sort can be tricky, true. All parties need to be very clear about their roles, where IP rests, where the final sign-off lies and so on before getting too far along with it. Get an agreement in place as soon as you can, and you may have to be prepared to relinquish some control. Regular communication is obviously really important too.

There are potential benefits beyond just the extra budget though. Your partner might also have additional expertise or resources, for example in marketing, that could be very handy. They might have their own large audience which you would then have access to. Choosing partners for what else they can offer is therefore wise as well.

Use existing game creation tools

You don’t necessarily have to start from scratch when creating a game, there are a number of tools out there that can simplify the process. Perhaps you or someone in your organisation could even have a go at making one yourselves.

I haven’t tried many of these tools, so can’t vouch for them. I know GameStar Mechanic has been successfully used to get kids building games, as has interactive fiction creator Quest (which I am a big fan of), which should be a good indicator of their ease of us. GameMaker: Studio by YoYo games is another possibility.

Googling for game creation tools throws up loads more options, Sploder, The Game Creators, Game Gonzo and more. Anyone used any of these and had a good/bad experience?

Run games jams or competitions

Games jams involve inviting lots games designers or games development team to spend usually 24 or 48 hours rapidly building a game on a particular theme. They’ve become more and more popular recently. Done well, this could be a good way of getting lots of games about your subject matter of interest out there. Done badly, though, they can feel exploitative and yucky.

It’s probably wise to work with someone who has run successful games jams in the past, and understand what it takes to make them work. Or work with a regular games jam event. You need to make sure people have a reason to be there, that their time is valued and that they are in some way compensated for it. Oh, and that they keep hold of the IP. The ones that Wellcome ran simply paid developers for their time, and then ran it as a competition so that the winners got money to develop their game further.

Update: This “Hack Day Manifesto” provides some useful advice for running events such as these (via @oonaghtweets and @dannybirchall on twitter)

Game design students

There are many game design courses out there. I haven’t actually tried this myself, but another option might be to work with a university or other teaching organisation that has a games design course to provide a brief for students.

Are you involved with a game design course that would be up for doing something like this? Let me know in the comments!

Try something different

Does it have to be a slick online game? Could you create a paper or card game and then make it available online? Add new rules to an existing game a la the brilliant Boardgame Remix Kit? Maybe if you want to get it into schools, you should just do something simpler like provide pictures and lesson plan suggestions and upload it to TES. Or do the same with instructions for a live or pervasive game that students could play?

And finally

Thanks to Sharna Jackson, Phil Stuart and Kim Plowright for their suggestions, which I’ve incorporated into this post. If you have any more thoughts, funding sources etc, please do add these in the comments and I will update the post.

Update

Updating to add some very useful additional thoughts from others following a discussion on the LinkedIn Games Based Learning Group.

Dustin Chertoff: The more the commissioner has completed up front, in terms of both art assets and game design, the cheaper it will be to actually develop the game. But that also means the commissioner is likely to be more resistant to the inevitable change requests from the developer saying “such and such feature doesn’t work or is too complicated.” A co-design between the commissioner and the developer is very useful in this regard, unless the commissioner already has a strong game development background (which is usually not the case).

As far as keeping project costs down, engines such as Unity3D are a huge development boon. Devs can start working on game system development much faster and there is a huge library of tools and assets available that further cut down development time.

Even so, from my experience it still takes a team of 3 (2 devs, 1 artist) working full time around 20k USD about 1.5-2 months to design, develop, test, and polish a relatively simple game. It does become faster and cheaper to add more content once you get the core game systems developed and tested though. But the low-cost commissions I’ve seen never have money beyond that first version.

Mathew Georghiou: Budget is always an issue and most people do not understand the complexity and effort required to develop a game as compared to other types of applications. There has been some industry research done that suggests the average mobile app costs $20,000-$40,000 to develop, and that seems to mirror my experience and that of Dustin’s comment above.

Some very basic apps can be done for less, but there will always be some significant compromises required as you have identified in your article.

The best advice is to always consult with someone with experience before designing your specifications or budget, and certainly before issuing an RFP, so that you can make sure to develop a plan that is feasible.

Peter Stidwill:  Although not a tip, this visual document from the Games for Impact academic consortium here in the States has some ballpark figures for professional game development. This is helpful to point to when trying to set expectations about what can be achieved.

http://gamesforimpact.files.wordpress.com/2012/09/gamesforimpact-bestpractices.pdf – see ‘Production cost estimate’

Evaluating online video

Whilst I was Multimedia Producer at the Wellcome Trust I began a programme of evaluation for our online videos. I left before I was able to see most of the results from that, but in this post I’ll share the plan of action I had for the evaluation, which used a number of different approaches. I was building upon work that Danny Birchall and I did in games evaluation, using some of the same methods, but evaluating video presents a somewhat different set of challenges. If you have any thoughts about other techniques one could use, or ways of improving on my methodology, I’d love to hear that.

Wellcome Trust YouTube Channel
Wellcome Trust YouTube Channel

Why evaluate videos?

At the Trust we (me and the other Multimedia Producer, Barry J Gibb) were creating short documentaries about the research funded by the Trust, educational films, and so on. The audience varied, but was usually considered to be an interested lay audience or a GCSE/A-level student. Our video dissemination strategy was fairly straightforward, we posted them on our site (where they frankly got a bit lost) and on YouTube (which I always considered to be far more important especially because of the much greater chance someone would find them there, and the ability to share).

I set the YouTube channels up in early 2009, but had no formal plan of evaluation (oh! to turn the clock back) and simply kept an ad hoc eye on analytics and feedback. The analytics were sometimes interesting, but commentary on YouTube, you may have noticed, is rarely that worthwhile. When we got lots of it, it was generally polarised and sometimes unpleasant. When we got none, which was usually the case, it was like putting the videos up to the sound of tumbleweeds and crickets. We had no real idea who was watching our videos, whether or not they liked them, how they were using them, and therefore it was hard to know how to improve them.

That was the key, for me. How could we develop a strategy for the future when we didn’t really know what had worked or not in the past? But all was not lost. A lot of information was just sitting there, in the form of analytics, and to find out more, we just had to ask.

My approach

I split the project up into a number of different areas:

1. Analytics

2. Staff feedback

3. Audience feedback

4. External organisations

Analytics

My plan was to track several paramenters for the lifetime of each video, ideally broken down into years. I had decided to go for calendar years rather than years since posted to be able to compare videos across the same period, but was somewhat in two minds about it. Youtube analytics offers information about views, traffic sources, and playback locations, which I had planned to include, and demographics and audience retention, which I hadn’t. Demographics on YouTube are only tracked for those people who are signed in and have added those details to their profile, so are largely worthless. I guess they may be less so now that it’s owned by Google and more people are likely to be signed in, but I still wouldn’t place much stock in them. The audience retention graphs are interesting, but hard to compare.

The analytics presented the biggest challenge for the project, though.  The first challenge was that the analytics were split between Google Analytics for the videos on our website and Youtube’s own analytics. The latter are much more restrictive than the former, though they give more detail on the attention patterns, for example. GA depends very much on how it’s been set up for video, but thankfully when GA was set up for the Wellcome Trust site at least someone had the sense to track the number of actual video plays as events, rather than rely on pageviews. But still, marrying GA and YouTube together wasn’t going to be easy.

The second issue was that getting data out of YouTube isn’t easy. You can download spreadsheet reports for one parameter over one time period at a time, but when you have well over 50 videos and want to try and track several parameters over several years, well, I didn’t fancy doing that kind of epic legwork. There is a solution, but it’s not ideal either. Google have an experimental API for YouTube analytics, and our IT department were as I left on the case of trying to write an application that would use it to built reports.

I did have a couple of initial findings from my more ad hoc monitoring of stats over the years, though these aren’t going to be hugely robust and I had hoped to interrogate them further with more data.

  1. On YouTube, most people were arriving after searching for relevant keywords *in Youtube*, (ie, not through Google organic search), or from related video links. The latter were definitely responsible for hits on our most popular videos, that were coming up as related links to other high profile videos from others.
  2. Many more people were arriving directly at video pages rather than the Channel page.

Staff Feedback

The Wellcome Trust has several hundred employees and, as with our external audience, we didn’t know what all of them thought of our video output. We didn’t even know if they were all aware of it, that they could use it or even commission their own films. Their opinions and needs were also very important to developing our video strategy. I put together a survey in Survey Monkey, had it checked by our UX expert (always worth doing)  and posted it on our intranet. I wasn’t around for the results of this one.

Audience Feedback

This was the bit I was most interested in, qualitative feedback from our audience. For our games evaluation work we had put up a survey at the end of the game, followed up with telephone interviews, and also looked at community commentary. As we’ve established, for Youtube the latter is not particularly useful, and we also didn’t have comments open on our site. But the survey was still an approach we could take, except instead of one game, we had ninety three videos. And instead of several hundred thousand plays over the launch weekend, most videos would have a few hundred views over their lifetimes.

This meant that we would need a much longer data gathering period as only a small fraction of people will answer a survey. As always, we ran a prize draw with this to help the numbers, but even so, it seemed wise to assume it would be six months (at least) before we had enough responses to come to any conclusions.

I drew up some survey questions, with the help of colleagues and again had them checked over by Nancy Willacy, our UX expert, to make sure they were understandable and that we were asking the right questions. There is lots out there on survey design, including some points from our own experience in this paper on games evaluations that I wrote with colleagues at Wellcome, The Science Museum and the National Space Center. Being very clear about what you are trying to find out is very important, it’s easy to creep beyond that and try and find out everything, but nobody wants to answer a huge survey online.

My intention was to add the survey as a link on our site next to all videos, and also to put them up as linked annotations on YouTube videos. I did the YouTube links myself before I left, which was a pretty tedious job. You can see an example at the end of this video. I have a feeling that you may have less options in the annotations if you aren’t a YouTube partner, and that adding links like this might not be possible. I guess you’d have to add one in the text below, but that’s not ideal.

External organisations

Finally, I wanted to know what other people were doing with video at similar organisations (and also what they thought of our output, if they’d seen it). This was to be much less formal, simply identifying other organisations and writing to them with a list of questions, at least in the first instance.

And you?

I hope some of that is interesting or useful, but it would also be great if other people share their experiences of evaluating video. What do you do in your organisation? What could we have done better here? Even better, does anyone have any results they can share?

An encounter with Patrick Moore (people are complicated)

A story I would like to share, on hearing that Patrick Moore passed away today (or perhaps yesterday). Already twitter is filled with a mix of sadness at his death and discussion of his positive legacy for science, but also reminders of his sexist comments and fairly extreme right wing views. I think it’s OK to talk about both these things, challenging the latter and celebrating the former.

When I was much younger I was a huge astronomy nerd and Moore was obviously an icon. When I was perhaps around 11 or 12 I was at an astronomy conference in London with my mother, who would escort me to astronomy weekends, observatory viewing sessions, and events like this. Yeah, I know, I was a weird kid. During a coffee break at the conference, Patrick Moore was there just sort of hanging out, and my mother encouraged me to go say hello.

I did, and Moore was charming and friendly. He was obviously pleased that I was so enthusiastic about the subject, apologised that he couldn’t talk for longer, and invited me and my mother to come visit him in Selsey. We arranged a time to do so via post (imagine, using the postal service to arrange a meeting!). I even kept his typewritten notes to me, see below, so I must have been a bit starstruck (pun intended).

A message from Patrick Moore
A message from Patrick Moore

We went to visit during a holiday nearby, and he was a great host. He showed us his telescope (no jokes please, seriously) and various bits of astronomical equipment and garden observatory. Finding out that I played music, he played his xylophone and recordings of some marches. We had tea and cake. He was clearly keen to encourage my interest in astronomy and science. We left after a couple of hours of visiting and that was the end of any correspondence, beyond a thank you letter from me of course, but the encounter left a huge impression on me.

Years later, I was immensely disappointed to discover his political views, and especially disappointed to hear his sexist comments about women ruining TV and so on. How to square this with the avuncular character I’d met who was so supportive of my enthusism for science? People contain multitudes, I guess. His views are clearly more complex than the impression you get from what is reported, but this isn’t a defence of them. It’s disappointing that he wasn’t challenged more on this in his lifetime.

We sometimes forget that people in the public eye are as nuanced, messy and complicated as any of the rest of us, and we shouldn’t expect them to be otherwise. We can be grateful for Patrick Moore’s kindness and great work in popularising astronomy and angry about his views at the same time.

On the BBC #FusionGames Summit, changing the tune about educational games, and Susan Bloody Greenfield

As I sat in the audience at a session called “Learning To Game or Gaming To Learn?” at last Wednesday’s BBC Academy’s Fusion Summit on games in Salford, I became rather frustrated. The session had been billed thusly:

How far should broadcasters use games as a vehicle for learning? Join some of the finest minds in the field as they wrestle with the future of learning games and the controversial subject of gamification.

On the panel were Mark Sorrell (Hide and Seek), Carlton Reeve (Play With Learning), Tom Kenyon (NESTA), Phil Stuart (Preloaded), John Milner (Bitesize, BBC Knowledge & Learning), and they were being questioned by Kate Russell of BBC Click. Undoubtedly a great line-up, yet it was all rather unsatisfying, and also rather familiar.

This is my attempt to explain and unpick this frustration, which is actually a more general frustration with the way educational games are so often treated with scepticism and distrust. This isn’t really a criticism of those involved in the panel, since it was just playing out in the same way that these sorts of discussions always do, and probably are always expected to.

Starting the session with a discussion about gamification didn’t help. Russell acknowledged that it wasn’t going to go down well with some of the audience, and winced as she said it (as I did writing it, ugh). So much has already been said on this subject, whether gamification is just pointsification, whether it mistakes the extrinsic trappings of gaming for the reason why people enjoy games, whether it should be reclaimed as just meaning adding game mechanics to content and so on.

Most people do seem to understand it to mean pointsification, and I can’t see this having any more than limited value. In this context, it’s also a total distraction. This isn’t really what people are talking about when they talk about games based learning in my experience, so it’s a shame it took up so much of the panel’s time. In fact, it’s a shame gamification has taken up so much time on so many conference panels and sessions over the last couple of years, can we possibly move on from this now?

But it was the next few questions that troubled me more. Russell asked “do educational games work?” And “where’s the evidence?” Now, it’s not that I think we shouldn’t ask these questions, but it seems that these questions are all anyone ever asks about educational games. The implication always seems to be that one should be hugely sceptical of such an outlandish and possibly even NEUROLOGICALLY DANGEROUS (more on that later) concept, and that educational games exponents had better have some seriously good evidence up their sleeves if we are to countenance allowing their nutty ideas into our schools and homes.

This makes me weary. It is really such a leap to see that an activity so absorbing as playing a good game, could be harnessed for learning of some sort? An activity whose very essence is about learning, as you must do to improve in any game? Does it trouble people that much that it might be possible to have fun whilst learning? (Note: not that learning in and of itself can’t be fun, but if there was a better way of educating children that all of them would really enjoy, shouldn’t we be really happy about that and keen to explore it further?) Some of the panel did indeed make some of these points, but were rather on the back foot in the face of this slightly negative questioning. Asked to think of evidence off the top of their heads, they were unsurprisingly unable to cite any academic papers in support of their position.

Gran Turismo Academy was mentioned (by Mark Sorrell IIRC) where players were put in real cars, and performed brilliantly, despite only ever having played the game before.  It’s a great example, but in the rest of the discussion little other evidence was mentioned and the conclusion was that there wasn’t much out there. I couldn’t think of any either off the top of my head at the time, but a google search shows there’s a fair bit out there. I’m pretty sure the military wouldn’t be so keen on using games in their training if they didn’t have some good evidence, for example. But maybe all of us involved with games for educational purposes should be better versed in the literature (and there is definitely a discussion about better dissemination about this sort of research and evaluation to be had at some point).

Testing the efficacy of games in learning is always going to be tricky though. For example, testing existing games may show that some of them are poor learning tools, but you couldn’t conclude from that that all games are poor learning tools or that it’s impossible for games to work in this way. And perhaps this is a rather back to front approach anyway for those making the games. Using what we know about the science of learning and good gameplay to make great educational games, with the majority of the testing taking place in formative stages would surely be more effective?

This week I read the Nesta Decoding Learning report, which makes a similar point about starting from good learning principles when creating digital educational tools, and which I highly recommend. I’ve also had a number of discussions recently with people who are indeed working on this basis, which is great, and it may be that this line of thinking isn’t news to many people working on educational games and technology.

So why do we always end up on the back foot in discussions like this? Why is the default position apparently one of scepticism? Responding with examples of the odd good game based learning initiative or stats about how the games industry is huge, broadly equal in terms of gender, not just played by teenage boys in their…snore… doesn’t appear to be making a difference. I hear this defensive tone so frequently and, hands up, have definitely been guilty of doing this myself in the past. Especially to audiences I assume will be sceptical (science types, for example). I promise to stop doing that now. But I have heard this discussion about whether or not educational games can work so frequently in conferences, articles and from people outside the “industry”, and it never seems to move on.

Perhaps we’re doing it wrong. As I tweeted at the time, perhaps all this defensive navel gazing is counter-productive. Perhaps it’s merely reinforcing the impression that the scepticism is right. I think many others in this area, like myself, know in our hearts that there is much potential here; that there is something in games which could work really well for increasing people’s understanding of subjects, situations, and systems. There may well be aspects of learning that games are not good at too, of course, but I don’t think it’s so far-fetched to think they could be very powerful educational tools.

Let’s not allow the likes of Susan Greenfield and her Daily Mail pleasing nonsense about video games set the agenda around this. She might not be able to point at any actual evidence for her claims, but we can. (On another note, if she thinks games have so much potential for evil, they must be powerful things, and therefore have potential for much good too. And of course they change the brain, as any repeated activity will, and this is not necessarily a negative response, as well she knows. But I digress). Let’s not sit back and wait for others to confer respectability on this area, let’s set examples, continue to do great work, and let’s talk instead about how to deal with the real and meaty challenges facing educational games: reaching teachers, funding projects, being heard above the noise, and so on.

On that note, we do actually get into the genuine issues around games based learning at the London Educational Games Meetup group (LEGup). Please do come along to talk games, share learnings, tell me I’m full of crap, or tell me how you think we can change the tune on this issue.

Bring out your games evaluations

Games evaluation page on the museum games wiki
Games evaluation page on the museum games wiki

So, the games evaluation page on the museum games wiki is looking a little sparse. The only thing up there at the moment is the High Tea evaluation. I should be getting a couple of others up there soon, but I’m hoping there are others out there we could add.

Do you have any sort of games evaluation that you’d be happy to have published/linked to from here? It doesn’t have to be a museum game to be useful, any sort of game evaluation would probably be relevant to those developing museum games. It could be a formal document, blog post, presentation. Just something that talks about what you did with your game, what your aims with it were, and what you did to test whether it achieved those aims.

Let me know in the comments or via twitter, where I’m @marthasadie. Thanks in anticipation…

Notes from #SXSWi: Public Lab: Mapping, DIY Activism & Civic Science #PublicLab

I’m attempting to write up every single session I went to at SXSWi. Will be mostly about games, but also how tech can kill, neuroscience, digital anthropology, civic science and more.

[youtube=http://www.youtube.com/watch?v=aIWMSmKwDns]

A video introduction to the Public Laboratory of Open Technology and Science.

Public Lab: Mapping, DIY Activism & Civic Science

http://schedule.sxsw.com/2012/events/event_IAP9516

This session was an introduction to an extremely interesting organisation: The Public Laboratory for Open Technology and Science (PLOTS). In their own words, they are “a community which develops and applies open-source tools to environmental exploration and investigation”. They provide simple tools and DIY solutions that enable people to collect data about their environment. The applications of this are absolutely fascinating and potentially very important.

As the panel explained, much environmental data is effectively owned by agencies and major organisations. Moreover, the standard tools for collecting this sort of information are expensive and proprietary, requiring serious investment in hardware or software. This creates an imbalance, especially in communities where the environment is under stress as a result of the action of big companies, think of the BP oil spill (more on that later). Those communities don’t usually have the means to collect their own data. PLOTS aims to change this.

They call what they do “civic science”: enabling projects which are community developed and community owned. They were keen to distinguish this from “citizen science”, which they defined as crowdsourcing data that then goes back to a big agency or research team, out of the hands of the people that collected it. The tools they develop are low cost and open source and information is shared under a creative commons share-alike license.

To give an example, during the BP Deepwater Horizon oil spill, people along the Gulf Coast wanted their own monitoring tools to see what was really happening to their shores. They clearly mistrusted the official line, and wanted to see for themselves. So PLOTS provided the information and tools to do this, particularly in the form of aerial mapping, and funded the project via Kickstarter. The rig for the aerial mapping technology is deliberately basic, uses balloons, a soda bottle and a camera, and yet can produce data “an order of magnitude better than Google maps”, they claimed. You can see the rig here, and more information about the Gulf Coast project here.

The aerial mapping tools have also been used by protestors to monitor demonstrations. For example this article on The Verge has pictures of balloon rigs being used by Occupy Wall Street protesters. This and other mapping projects can be found on the Grassroots Mapping site. From the PLOTS website:

Maps are often used by those in power to exert influence over territory, or control territorial narratives. “Grassroots mapping” attempts to invert this dynamic by using maps as a mode of communication and as evidence for an alternative, community-owned definition of a territory. To date, our tools have been used to contest official maps or rhetoric by enabling communities to map sites that are not included in official maps. In Lima Peru, members of an informal settlement developed maps of their community as evidence of their habitation, while on the Gulf Coast of the US, locally produced maps of oil are being used to document damage that is underreported by the state.

Mapping can be hugely political, and as they say, this has traditionally been a tool only for those in power to wield. It’s exciting to see the tools being made available to try and redress this imbalance, but the challenges don’t end there. Firstly, it is important, they said, for people carrying out aerial mapping activity to consider how what they are doing might be perceived. After all, people may be unsettled by unmanned flying objects taking pictures of them if they don’t know what it’s for.

More significantly, though, there is the hurdle of getting your data recognised by authorities. Particularly, I imagine, if it is data that contradicts the official information. They have to be careful about the chain of custody for data to make sure that its legitimate and can’t be called into question. Also, they have to go through a legal process for advocacy which they described as “frustrating”.

But it sounds like they are doing good work, and it’s an inspirational idea. I did a bit of digging, but haven’t turned up any similar projects here in the UK. Anyone heard of any?

The whole session was recorded and you can listen to it here. You can also follow PLOTS on twitter: @publiclab.

Notes from #SXSWi: A New Culture of Learning: Gaming, Tech, Design #newlearn

I’m attempting to write up every single session I went to at SXSWi. Will be mostly about games, but also how tech can kill, neuroscience, digital anthropology, civic science and more.

[youtube=http://www.youtube.com/watch?v=sCWmy0T73d4]

Trailer for TiltWorld

Heather Staker, Nicole Lazzaro and Scott Stropkay: A New Culture of Learning: Gaming, Tech, Design

http://schedule.sxsw.com/2012/events/event_IAP9607

This fascinating session invited us “to imagine a future of learning that is as powerful as it is optimistic”. It was a discussion on “exploring play, innovation, and the cultivation of the imagination as cornerstones of learning” with three speakers:  Heather Staker, of the Innosight Institute, a “a nonprofit think tank devoted to applying the theories of disruptive innovation to problems in the social sector”, which sounds pretty interesting; Nicole Lazzaro, founder of XEODesign, a “Player Experience Design” consulting company; and Scott Stropkay, a “Lateral-thinker, envisioneer, strategist, designer, prototyper, builder” and founder of Essential Design. You can listen to the whole thing on the link above, and I recommend it, but here are some of my highlights from the session.

Nicole Lazzaro introduce us to her 4 keys to fun, or principles of game design, available as a poster. It sounds like XEO design do a lot of interesting research as part of their design process, including something called facial emotion coding (this?). She also mentioned the work of Clayton Christensen, author of Disrupting Class: How Disruptive Innovation Will Change the Way the World Learns. And she introduced us to the new release from her company, Tiltworld, a iPhone game that aims to both educate players about the environment but also have a direct impact in that points gained result in trees being planted in Madagascar. Games in general were much discussed as effective learning tools.

The panel discussed innovation in the classroom. A good example, apparently, is work being done in Singapore where large scale efforts are underway to change the way kids learn. Another was the Carpe Diem school which offers online as well as campus learning and puts the emphasis on students taking control of their own learning. I liked the concept of teachers as coaches, helping individuals to “get unstuck”. The Carpe Diem school is an example of something called Blended Learning, which is based around using a mix of different learning environments.

Design based learning also came up, which sounds intriguing. It appears to be based around setting students problems to solve, which “empowers kids to think of themselves as creative problem solvers”. That from Scott Stropkay, if I remember rightly. This article seems like it may explain more. Somewhat inevitably, the incredibly successful Khan Academy was also brought up.

But currently, this innovation isn’t making it into the mainstream in the US, as Stropkay pointed out. Heather Staker agreed that it’s very hard to break into the traditional learning system, as it’s so entrenched. She also noted that we shouldn’t forget that actually not all students want to learn in this way.

Lots of food for thought, then, in this session, which opened my eyes to the possibilities for taking education in new directions. Listen to the whole thing here.

Evaluating games: learning through play? (SCC2012 panel) #SciCom12

[slideshare id=12921875&doc=summativeevalmh-120514035048-phpapp02]

Above are the slides from my talk about summative evaluation of games for a panel on different approaches to evaluating games for this year’s Science Communication Conference. The panel also featured Hannah Clipson, from the Science Museum, and Helen Kennedy, from the University of the West of England and the Digital Cultures Research Centre.

I haven’t added more notes for these slides, because my talk was almost entirely based on the evaluation we did for High Tea, which you can read here on the Museum Games Wiki.

We also recently presented a paper at Museums and the Web 2012 on games evaluation of all types with colleagues at the Science Museum and National Space Center,  entitled Levelling Up: Towards Best Practice in Evaluating Museum Games.

Hope you found the panel interesting, do feel free to feedback in the comments.

Examples of experimental/art games

Screenshot from Every Day The Same Dream
Screenshot from Every Day The Same Dream

We’ve been playing The Company of Myself and Dys4ia in today’s Games Club at work, run by Tomas Rawlings. This sparked off a conversation about experimental/art games (which I’m not going to try and define here, I think you’ll get the gist) and some really great examples were mentioned. I thought it would be useful to collate these somewhere, not least because every time this subject comes up I have to try and dig up the examples I dimly remember to send people links, “oh it’s this game about suicide or something by an, um, Daniel someone, I’ll go look it up”.

So here’s my list, with several examples via Danny Birchall (updated to add examples from Phil Stuart).

Update 22.09.12 Some more examples from Mathias Poulsen, who has two lists for “Poetic” and “Newsgames“, I think I like “poetic” and “news” better as descriptions.

Anyone got more good or interesting examples I can add to the list?