Skip to main content

Collecting real life gameplay data with Parchment

There was a lot of talk during PAX East and elsewhere about collecting real life data from people playing IF. I’m proud to announce that I’ve hacked the Parchment web interpreter to do just this: it saves transcripts to the server every time a game is played.

Try it out yourself: Click here to start a game and then open this page to a new browser window. Enter a command in the game and see how the transcript in the live feed updates. (If you don’t see your game in the live feed page, reload it. It shows the three latest games being played at the time the feed page was loaded.)

And then there’s the beef: I’ve made a rudimentary statistics page that shows some interesting information using the collected data, including most used commands and average turn count and playtime.

The obvious use for this is of course collecting information on how people play the game and using that information to make improvements for the next release. You could also use this for closed beta-testing — give the web address to your testers and Parchment saves the transcripts automatically. The testers don’t have to remember to start the transcript or send it to the author.

Statistics from other games

First I want to stress that at the moment this is just a demo of a hack. It can and will behave strangely especially if the game bypasses the standard parser and takes input directly (menus etc.) That said, if you want to try other games than the sample game, you can use either http://parchment.game-testing.org/demo/parchment.full.html?story=xxx or http://parchment.game-testing.org/play/parchment.full.html?story=xxx (replace xxx with the full url of the game you want to play.) The difference is that the latter doesn’t have the live transcript feed and only has the statistics page (note a different address than what the demo site uses). You can’t single out individual players from the statistics so there’s no disclaimer notifying the player of the data collection (which might unnecessarily scare off some players.)

The statistics page collects information and displays the urls of all games played this way, so don’t use this for trying out something you don’t want the whole world to see.

How it works

I’ve added AJAX calls inside Parchment’s UI code at appropriate places where it sends the player’s command and the game’s response to the server. On the server a PHP script receives the data and saves it into the database. This is all done behind the curtain so for the player the gameplay experience is exactly the same as with unmodified Parchment.

The statistics page collects some extra information about Inform 7′s internal error message and library message codes. This requires some modifications to the game file itself. The Raising the Flag on Mount Yo Momma game used as the example has been modified to include this:

Before printing a parser error:

say “#ERROR:[parser error code]#”.

To decide which number is parser error code:

(- etype -);

When play begins:

change the command prompt to “#LIBMSG:[library message number]#>[clear libmsgnumber]“.

To say clear libmsgnumber: change the library message number to 0.

When play begins:

say “#PARCHMENTWARNING: This is a special story file designed to work with a modified Parchment interpreter. Seeing this message means that you are using some other interpreter or a wrong version of Parchment to play this game. It’s advisable that you download another version of this game, or you might see between normal text some control strings that are not supposed to be shown to the player.#”.

That code shows the error codes and the library message ids after every turn in the prompt, for example #LIBMSG:40#>. When the modified Parchment sees these strings it saves the information in them and removes them from the output so that the player doesn’t see them. Played on any other interpreter the game would of course display these, so the last line of code warns the player that they shouldn’t be using this game file if they can see the control codes.

Unfortunately the error codes and library message codes aren’t very reliable; some parser errors don’t seem to have an error code and some library messages don’t have an id. Most noticeably the “unknown verb” error doesn’t have either, so it doesn’t show in the statistics. I don’t actually know what the codes really do, I just noticed they’re there and used them; future development would include using custom codes for different situations in the game.

Future plans

  • Test it more and fix bugs so that it could actually be reliably used.
  • Clean up and comment the code so that it could be packaged for people to install on their own servers.
  • Create a service where people could have saved transcripts and command statistics behind a password so that they could improve the game for later builds or use Parchment to save beta-testing transcripts.
  • Add more information and charts to the statistics page (suggestions welcome.)
  • Try to convince Parchment developers to include this functionality into the main build or provide hooks so that this could be added as a plugin. Right now I’m just modifying Parchment’s code directly, so whenever Parchment is updated the files have to be edited by hand again.

What would you like to see done with this?

Comments

  1. Oh, cool. I’d made a start on this with Parchment as well; you can see where I got to at http://github.com/sgranade/parchment before life swamped me a while back.

    I’d taken a slightly different tack so as not to need AJAX or output modifications: I was hooking into player input internal to Parchment and sending only those commands, not the game’s response, to cut down on data traffic and storage requirements at the other end.

  2. Dannii

    Cool!

    I had been thinking I should write up a public summary of the discussions we’d been having before on the intfic forum – this would be good to standardise so that it’s not Parchment specific.

    The development version of Parchment that I’m slowly working on will make this easier. You’ll be able to go $(document).bind(‘lineInput’, handler) to listen in to all of the commands, without needing to hack the core source code. You could do that with another JS file, or even inline in your parchment.html.

    The current proposals had only been about logging commands. Your idea to specifically log errors and library messages separately is an interesting one. We could perhaps do that by detecting Parchment based on interpreter ID… at least then nothing would happen for normal terps.

  3. I implemented a similar feature in the Silverlight version of Secret Letter, although slightly differently.

    I save every command and the resulting main output by location to a database with user session information. We also had a sign on screen that asked for basic information like school, location, age, and gender.

    Then I created reports that allowed you to drill into any location and review commands. It’s very useful to see that a particular point in the game breaks down with either guess the verb issues, or in the case of newbies, when they started to lose interest and type insane things.

    I also had reports for a session, which is essentially a full transcript.

    I think saving transcripts is an excellent first step, but building a detailed database of command, location, output, and any other game state information is extremely helpful for improving the final product.