Saturday, September 28, 2013

TiledStack Update - Developer workflow and Tiled integration

I'm continuing on with a project I am calling 'TiledStack' which is a set of libraries, web services and clients that assist in the development and running of games that utilise the Tiled map editor.

This past week or so I have focused on 'developer workflow'. How developer(s) make changes to a map and how those changes get incorporated into the iteration and feedback loop workflow of the developer(s) involved.

If you've read my previous post on this library, you'll remember that I got simple messaging working between multiple MonoGame clients. This worked by all clients updating their map via messages received from the server and the server repeat each request to each client. I was generating these messages via a bound key, not exactly a real world scenario. What if we wanted to get changes to a map from an arbitrary source just by comparing the original instance and a new instance? Like updates from Tiled map editor itself?

Developer Workflow

Since the Tiled map editor is the common part to every developer using the Tiled map format, I want something that integrates with workflows that developers already have. This functionality originally came out of my own desire to not have to restart/rebuild my own game instance just to see how the game would play/look with the new map layout/map objects/layers/tilesets. This functionality might sound fairly insignificant, "It'll look the same as in the editor, get over it!". However, in my own experience, I found it a great source of motivation to be able to see the new changes running in the actual game, preferably on the actual device being targeted (eg, Android/Windows Phone/PC etc).

To get a simple proof of concept up and running, I'm using a simple console application to watch for file modifications in a specified directory that contains Tiled map json files. After sorting out some (apparently well known) issues with watching for file modifications in .NET using the 'Filewatcher' class, I had to be able to compare the old to the new and pick out the differences in the two.

Modifying Tiled

One of the main problems I came across when building the file watcher client was handling multiple deletes/creates and updates in one file change. This was due to, by default, there are no unique identifiers on arrays or layers, objects or tilesets. I tried quite a few ways of creating an id every time the map was loaded, but in the end it was based off the index number. I ended up getting the latest version of Tiled from github and changing these objects to produce a uuid on creation (Fork here). This has made solving the problem of difference checking much simpler and also gives an opportunity to manage objects differently at runtime. Below is a snippet of how I'm generating the different CRUD lists.

This might not be the fastest way to check what has differences with the two instances of map, but being about to do these simple statements in LINQ makes short work of it.

So we have lists of differences, now we can create simple requests from the containing the new data.

Now, if our map has 200 changes, sending out 200 requests is not a very efficient way of doing things, but for now this is what I'll be doing. A 'bulk' request functionality will be needed, but for now, it's a premature optimisation.

Send all the requests via the TiledStackProxy and ..

Updates straight from editor the running game :). This allows me to update tilesets, layers, map objects and properties and see how they look/act in the game at run time. Now that I'm running the latest version of Tiled which supports Map Object rotation, you'll notice some position problems. Positioning map objects before was accurate, so I'm going to leave my rendering code as is. Once I've updated Tiled itself to support turning these guid elements on and off in the UI, I hope to make my first pull request to the project. Hopefully this or similar functionality makes it into Tiled, as this will make passing the data around a lot simpler.


Now that I have changes going to a MonoGame client, I want to be able to stop the game, fix some code and continue where the map left off or run the latest changes if some one else is updating the map. There are a number of problems to solve for this functionality, the first of which is persistence. I have started with MongoDB for saving the map changes and holding a copy for a user. MongoDB is really easy to get going and for what I am currently doing, does a nice job. That said, I actually plan to move to Redis. This is because speed is going to mater and the source of truth of the map is not held solely remotely. The volatility of using a in memory database doesn't concern me in this situation and I've also been meaning to use Redis more in personal projects. This seems like a good fit.

For reference though, this is one of the services that is updating the map.


I was writing a HTML 'map viewer' if you will that would also have the same live updates, but after trying to get 'MelonJS' to update the map for a few hours, I gave up on the idea for now. I was up and running with a static map of MelonJS very quickly, but found the code base quite difficult to extend (I might be coming from the wrong angle to do so though). I also very briefly tried 'canvasengine' , and although I didn't get canvas engine rendering, the project is very well architected and I may give it another go in the future.

I do want to have a HTML viewer (even if it is just a static latest version of the map) to help give distributed teams a common place to have a quick look at at the latest level looks like. I also want it to be a part of a larger set of web tools around hosted maps that people can share with the indie game dev community. If a developer decides to make a map public, people can have a go with other peoples map designs and maybe extend them. This is all in an ideal world and not sure how far I'll get with those features as I still have a lot to do around the basic developer workflow. For now, HTML will be stalled while I work on other features.

I'm currently working on 'versioning' via Tiled command, a way to mark and persist a version of the map as 'latest' for developers who want to use the latest version, but don't want objects popping into the game while they are debugging. I'll talk about those options and others in my next update.

Sunday, September 22, 2013

C# Extension Methods. Abuse of static or not? Pros and Cons

Recently in my projects I've been finding more use for extension methods in C#. If you're not familiar with extension methods, they go something like this.

Basically some syntax sugar. Instead of writing 'Extensions.AsScoreString(myclassInstance)', you get to write 'myclassInstance.AsScoreString()'. To me, the later reads a lot more clearly. However, does the fact 'AsScoreString' is now not within the 'MyClass' object itself, muddy the waters or improve things? Here is the code for a simple (but arbitrary) use.

Example of use

Although it is just a 'nice to have', I've found them very useful for various reasons However they can make things confusing for the flow of code etc. Here are a few of the pros and cons I've come up with whilst using them in different scenarios.


Code Readability

First and foremost, readability can be improved quite a bit if your extension methods are well named. Lets say you want to convert from one data structure to another like the above example shows. For someone skimming over a code base, what the code is doing is conveyed very quickly whilst hiding the how away from either data structure, making them also clean and easy to read.

Structural Separation

For functionality like conversion between one data structure and another that represents same information, you could  use a Decorator pattern, why bother with extension methods? Using a Decorator would tie data structure 'MyClassB' to 'MyClassA' which may not be desirable. If I wanted to use the MyClassB data structure in another library, I would take a dependency on 'MyClassA' with me. Having an extension function to perform this operation within a library that needs the conversion, avoids this dependency, gives me clean syntax and leaves the original data structures untouched. A Factory could also achieve this independent conversion, but now messes the readability of the conversion step itself with the factory object in the middle.

Avoids Relying on Internal State

Since these methods are static, they can't rely on internal state of say a factory (or other) object to help with the logic. This obviously has it's pros and cons (and ways out), but I have found this to be mostly a pro as functionality is contained and isolated away from various changes being made to a system.


I See static People

This grates with me a lot and is the source of why I am a bit wary of the over use of extension methods. Seeing so many statics screams at me that I am missing an object or design pattern. Other times however, these additional objects or interfaces add a lot of friction to the development process.

No Internal State

Like many things, it's a double edged sword. One thing I have come across while using extension methods is the unavailability of techniques like Dependency Injection (DI) and Inversion Of Control (IoC). Since these methods are in a static class as well as being a static method, I can't use IoC like in other areas of my code.

Where's that code going?

If you haven't got access to the source of the projects you are working on, extension methods can make the flow of code feel very confusing, jumping from seemly unrelated static extension classes and back. Also while reading the simpler code can leave you a bit worried that you don't know what the extension method is doing. This can give the feeling of 'a bit too much magic' and I don't like 'magic' in programming.


Extension methods are very useful. However, can be easily abused. Although I do use them, I try to be aware that they are can be a symptom of problems with design depending on the situation. 

Saturday, September 14, 2013

TiledStack - Tiled map rendering, messaging and support project for MonoGame

Recently in my spare time, I have been playing around with idea of using a simple HTTP services for messaging between game clients, specifically updating Tiled maps. Traditionally, networking between games has been done by TCP sockets as it is a far more efficient and light weight tech to use for games.However, as an enterprise developer by trade and games developer hobbyist, I'm not too fussed by the perhaps heavy handed approach of HTTP. Also, it's the tech I (and many others) know and use on a daily basis.

I also want to create these services to assist in the production/testing on Tiled maps as the game was running to create a short iteration cycle and tight feedback look into a developers workflow when using Tiled and MonoGame.

Continuing with my use of Tiled map editor for creating levels in my game, I first looked at how to move away from the Content Pipeline of XNA (a build time process) to a runtime process for loading of these Tiled maps. The original Content Pipeline I used for Tiled was written for XTiled library written by Michael C. Neel (ViNull on twitter). I was going to first look at using a lot of what he has done and just move the parsing code some where else, but I then remembered that Tiled has a 'Save As' function supporting various formats, including JSON.

JSON is a really easy data format to parse and thankfully there are many good .NET frameworks that handle this nicely. ServiceStack.Text is one of my current favourite serialization libraries in the .NET world. I made a quick set of DTOs (Data Transfer Objects) and proceeded to write an adapter from Tiled DTOs to a slightly modified XTiled's data structures which I still like for rendering etc. However, although I like the data structures, there was a lot of behaviour on these objects that I just didn't want there and also wanted to replace with my own rendering and processing code that I've written before. The main reason behind going straight from JSON instead of the native Tiled XML format (TMX) is mainly to save time and lines of code. Note: I intend to support TMX format once I have some more of the core functionality working.

Once this was done and I incorporated some rendering and updating code, a few unit tests to confirm parsing the data was working as expected, I gave it a quick test.

Yay, up and running again. Loading at runtime, check! Wired up some simple input for moving around to confirm everything is rendering where it should be in an Isometric map.

OK, server time. One of the best things about ServiceStack is the sheer speed of getting up and running with simple web services. I like to separate my data objects from my request objects because they aren't always exactly the same and I don't want to litter my data objects with request/response meta data.

So I've got empty placeholder services with endpoints for a couple of the Tiled map updates that I want to be able to communicate with various clients, however ServiceStack doesn't really help me with telling other clients what has happening on another client, we'll need another framework to make this happen with as little code as possible. SignalR, if you haven't used it, is pretty cool library for distributing messages to connected clients, take a look at Scott Hanselman's post on SignalR with nice examples showing how little server code you have to write for simple messaging. It supports various different clients including .NET and Javascript, both of which I plan to utilise. A little bit too much magic? Maybe. Takes care of a whole bunch of plumbing I'm not interested in at the moment? Definitely!

So, how do you wire up ServiceStack and SignalR together is a very clean way? Checkout Filip W.'s  post of using incoming and outgoing hubs via attributes , and it's exactly what I did to get this going, write a very simple hub (literally just repost the request to other clients) and presto! We have a server redistributing messages to all clients (including the original requester). 

So now we are receiving messages, we should do something with them. I have decided to use a simple Blackboard design to deposit incoming messages and make the collections accessed in a lock as we are using separate thread for handling messaging than we are for the game loop.

Messages are deposited and checked every frame for updates. This will probably be optimised to be a controllable frame interval as we might not really want to check this every frame. Once the remote requests is found, we process them very much like the original parsing. Wire up some simple input to a key to generate/send map object requests and 'boom goes the dynamite', map updated from another client!

Though this is not the most efficient way to do messaging with games, it is an easy to to get some pretty cool functionality with some great frameworks that are already out there. I have quite a number of features I'd like to play with some time in the future like persistent changes on the server, updates straight from the Tiled map editor and hopefully I will be able to get these working in the near future. I do plan to open source this project in the near future once more of these core features have a nice structure to expand on.