Writing games in HTML5 and CoffeeScript combining Box2D and EaselJS in an OOP/MVC architecture

Crap. That was a mouthful.

There seems to be a huge amount of stuff on Her Majesty’s Interwebs about all of these topics, but very little explaining how to pull them all together into something coherent. There’s a shit-ton of bad code to copy to get something bouncing around on screen, but none of it cares about scaling well into anything complex.  And there seems to be a massive dearth of diagrams showing how anything connects together, whether it’s classes within libraries, or how you’re supposed to fit them together in your own code. I don’t know about you, but I can’t design anything without diagrams, so expect lots here.  In fact, let’s have one now.

A diagram about diagrams

There. I feel better already.

(If you want to sketch diagrams like this quickly to help you design your code, try Pidgin. It’s a cool little tool that lets you write out relationships and it does the drawing-y bit.)

Now, I’m fairly new to Javascript having spent most of my (front-end) programming life mucking around with languages designed explicitly for Object Orientation, not bastard sons of webpage scripts. Javascript was a bit of a shock for me. Not an entirely unpleasant one, I admit: I discovered the language was far more powerful and flexible than I had realised – but, having delved beyond dabbling with it, and faced with the challenge of creating something substantial that needs to combine two powerful 3rd party libraries like Box2DWeb and CreateJS, I have definitely decided Javascript is not for me. Its syntax is horrendously ugly and hard to read, its “gotchas” many and dangerous, and writing OO code that will be easy to maintain and DRY is just expecting too much from it.

Or perhaps me.

CoffeeScript

Enter CoffeeScript. It’s JavaScript, The Good Parts with a Ruby-like syntax. Excuse me a moment while I just fetch some Kleenex.

Let’s look at JavaScript versus CoffeeScript syntax around Inheritance for example. I’ve stolen this from the CoffeeScript site showing what you would write in CoffeeScript (on the left) needed to do what JavaScript is doing on the right:

CoffeeScript versus JavaScript for Inheritance

Okay, the death text might have been me.

There’s no contest.  Given this syntax (JavaScript’s) any benefits provided by Inheritance are just eaten up by the unintelligible, un-maintainable gobbledygook you have to write to get it.  We don’t need inheritance; we can always use composition to achieve re-use (and we still will), but there’s something very elegant about having your game entities extend an underlying class that (say) understands about being dynamic in a 2D world.

CoffeeScript comes with it’s own cons of course: it’s compiled, not run natively in browsers, so that means you have to have a build process. You can either have the build done by the server so you actually deploy CoffeeScript, or you can build as part of testing and deployment.

EaselJS and Box2D libraries

If you’re just hacking around and trying to get your head around either EaselJS, and/or Box2D, and you’ve never tried CoffeeScript, you might want to just go ahead and get something working first using JavaScript (that’s certainly what I did). To write CoffeeScript, you need to understand JavaScript anyway (it compiles to JavaScript after all), and you have enough on your plate to learn as it is. It’s a challenge to take powerful libraries for display and physics and combine them in a way that won’t make you cry when you look at your code in a year’s time (also what I did). I’m going to assume that you care about writing code that’s nicely architected and easy to maintain. If you don’t, you’re reading the wrong blog post.

I’ve chosen to work with EaselJS (part of the CreateJS suite) and Box2Web (for now) but the real challenge here is actually independent of the choice of libraries. It’s not about which one you choose to render graphics and which one you use to handle physics, but how to combine them well. A decent game architecture should allow you to switch either fairly easily, and also port your game to different platform, like a specific mobile device (iOS/droid) or a platform-independent SDK such as Corona.

The Model-View-Controller Pattern

This is where MVC comes in. Model-View-Controller is the architectural design pattern adopted by web frameworks like Ruby on Rails and CakePHP. It’s more of a principle than a specific architecture, since it can take different forms depending on what you’re actually building. A game is not a web application built around a relational database, so the architecture of a game isn’t going to look exactly the same as a web system. However, the real principle at the heart of MVC is Separation of Concerns.

Now, I’m not actually an architecture purist – I firmly believe there is a balance to be had between writing beautifully architected code and getting something built and shipped. I am a big fan of Lean. There is absolutely nothing wrong in my view, in hacking some hideous Frankenstein of code together to get something working and out there. Software architecture isn’t like building architecture; its beauty is not on display for all to admire. I do think though, that once you have something out there, it is worth thinking about how you can refactor it so that the code is easier to maintain and scale. (It’s not even about code re-use. To paraphrase Matz, it’s not about achieving perfection, it’s about how you feel when programming. I don’t mind hacking prototypes to get something working or to learn how something works, but I hate working with that code afterwards.)

Productivity not purity

MVC comes with its own overhead in terms of number of classes, complexity and messages being thrown around. It’s interesting that MVC hasn’t been widely adopted (apparently) in games industry projects is cited as A) performance (probably true) and B) the fact that the Model in games is often “the same as the View” (partially true e.g. if you have a dedicated GPU for rendering then you may well rely on its ability to other fast vertex operations like collision detection which breaks MVC completely). In a web game though, the model can be properly separated from the View.

Remember though, it’s not a perfection or purity of this paradigm but getting the benefits of the underlying principles of Separation of Concerns.  Architecture is a function of project scale, not just the technologies involved: If you’re planning something tiny, it’s certainly overkill to go for a fully OOP and MVC design, but you should still about separating concerns and writing modular, DRY code. The great thing about libraries like EaselJS and Box2DWeb are that they are already dealing with separate domains. EaselJS is all about the View. Box2D is all about the Model.

So, is your game code the Controller? Hmm, no. It’s all three, and mostly Model, in fact.  We need to understand what MVC really means. What is each part concerned with and how do they interact? I keep seeing diagrams showing MVC that are wrong.  Let’s fix that now:

MVC

Note: those diamonds are not arrowheads. They’re UML… the lines point away from the diamond.

However the responsibilities of Models, Views and Controllers, the dividing lines of the separation of concerns is very different in a game than it is in a web-application.  The decision about what goes where. For example, visual effects, like particles, will probably use the physics library but really they’re View code because they’re not truly part of the game Model.  The game is independent from them. They’re visual gloss. They could be rendered completely differently without affecting the game.

Controllers

Controller classes are all about handling action input from the User. This can get a little fuzzy when we consider that this can include input from elements from the screen, e.g. like on-screen buttons, but remember even when the user taps a button on screen, it’s the touchscreen on top of the View that’s handling the touch.  Controller classes are responsible for responding to the User’s desires and making the internal game state change appropriately. This includes creating the game world in the first place when the User clicks ‘play’.  Controllers don’t necessarily fire events or call methods on Models – they can also be polled to get their state.  You can use both: i.e. your game start button’s click event can call Game.start() but during the game, you might want to know if the left key is down or not during a game tick to decide whether to apply a force left to your hero. You poll the keyboard controller for this information.

Models

The Model classes are all about representing the internal state of the game and the logic about changing state. So that’s the physical world itself and its contents, and all game elements in play.  This is more than just the physics; physics is just part of the game logic.  Notice that although the Model is not concerned with rendering, it does “update” the View.  All this means is that it is responsible for the telling the View what’s changed.

Views

View classes are only concerned with rendering things so the user can see them.  We should also include in the View anything sense-related actually, so that means audio too.

To MVC or not to MVC…

If you’re planning to port your game to run on different platforms, or you’re not sure about which libraries you’re going to use for physics or rendering, then you will probably want to be very clean about keeping these domains decoupled.  In other words, you will want write your own classes to wrap around any library classes.  Remember, we’re not just Separating Concerns for fun, we do it because it will make life easier and coding more enjoyable. If it is going to make things harder overall, don’t do it!

In a (business) web application, separating the model and view is easy because there’s usually nothing visual about the business model. Games are intrinsically visual: the view and model are coupled, and there’s no escaping this.  It’s a question of loose-coupling versus tight-coupling.

If, for example, we create a BallModel class that contains a Box2D ball body, and BallView class that contains the EaselJS Bitmap. Any calls from BallModel to update the view go via BallView, so your model is now competely agnostic about which library you choose.  If we have these inherit from base classes that do the common stuff, then we can benefit from Polymorphism.  So we associate our base ObjectModel class with a base ObjectView class and ObjectModel.update() can call ObjectView.update() which will do standard transformations of the physics coordinates to pixels (say).

Now if we decide to switch to a different graphics library, our code changes are all (hopefully) contained in our View classes. The same goes for your physics engine.  You will need one class at least that knows about both in order to convert from model units (kg/metres/seconds) into view units (pixels). This could be a helper, but it’s also reasonable to have it handled in the View. It’s the View’s job to render the Model.

MVC in games therefore is going to look broadly like this:

Model-view-controller pattern in games

Controllers will kick off the game and handle user input during it.  The models will concerns themselves with the (internal/invisible) world of the game, and the views will make it all visible to the user.  Each of these domains (MVC) should have a nice class hierarchy for nice modular code, not just our own code, but 3rd-party library code.

This is all very abstract, let’s have look with some concrete examples to see what the challenge is working with real libraries.

First off, EaselJS.  EaselJS is a brilliant library that makes graphics and sound in HTML5 really easy. It interacts with a <canvas> element in your HTML document.

How EaselJS works in an HTML doc

 

I’m using “is” here to show inheritance.  I’m only showing the core visual classes here, there are dozen more that do all sort of fancy things like image filters, event handling etc.  I’ve also included some other libraries provided by CreateJS as you’ll almost certainly need them.  You can just include the EaselJS library on its own if you prefer.

Easel has a Stage which represents the Canvas in your HTML doc.  It is (inherits from) a Container which both is and can contain DisplayObjects. There are classes like Bitmap and Sprite (animated bitmaps) which are DisplayObjects. The documentation for EaselJS is excellent although lacking lovely diagrams like this one to help see how it fits.

Now let’s look at Box2D.  The architecture of Box2D is pretty much platform-independent so it shouldn’t matter which port you use, or whether you read the (excellent) C++ manual even though we’re using JS/CoffeeScript.  Box2D is a fully-fledged rigid body physics engine. It’s complex. I’m going to assume you’re fine with that. I’m not going to be covering all the details of Box2D here, only the bits needed to work out how to create an HTML5 game architecture with it and EaselJS.

So, while Box2D is actually in three modules (Common, Collision and Dynamics) I’m just going to look at a small part of the Dynamics to get started.

Box2D basics

Box2D worlds are the corollary of our Stage in EaselJS (but can be bigger).  Then we have bodies which are rather more complicated than our Bitmap / Display Objects from EaselJS.  Box2D separates physical concerns into three classes for objects.

Bodies are concerned with having positions, moving about, rotating, responding to forces etc. but they’re a bit like ghosts; they don’t have a shape or material.

Fixtures add material properties to a body (so it’s like telling a body what it’s made of, how dense the material is and how springy for example).  Fixtures also have a Shape, which in turn tells the body how large it is.

Shapes are all about collisions.  They determine the edges and extent of bodies and therefore how they interact with other bodies/shapes.  Shapes are also the most tightly-coupled concept in our Model to our View, because normally our graphics will need to match our shapes. There’s no escaping this. It’s nothing to worry about.  Our models are responsible for creating and updating their views.  I.e. initiating and causing those events. They’re just not responsible for the details of what happens inside them.

Notice that a body can have more than one fixture.  I.e. you can create compound bodies by fixing (say) a rectangle and circle to a body.  These two shapes will never move with respect to each other.

We don’t actually need to add anything else to get a basic physical system going.  We could add Constraints and Joints to our world to remove degrees of freedom or join bodies together, but let’s leave that for now. It’s detail we’re not concerned with yet.

The Box2D world doesn’t come with its own ticker.  Once we setup the rules (gravity etc.) we call world.Step() with some parameters to run it one timestep.  This ticker is provided by EaselJS, so here the GameApplication View will be asking the model to update itself.  The model in return will update all of it’s entities and they will all communicate those updates back to their Views.

Hopefully, you’re starting to get a clear sense of how to organise your code.

In part 2, I’m going to get into the details of a class hierarchy that brings all this together with examples.

How to write DRY SQL in MySQL – Part 2: Stored Procedure Hacks

In Part 1 we looked at using VIEWS to keep our SQL DRY. Unfortunately, there are cases when VIEWS are either not allowed or perform terribly and can’t be optimised.  Stored Procedures are, of course, the mainstay of writing DRY SQL, and optimisation.  Unfortunately, you can’t treat the result set of Stored Procedure as a query; therefore you can’t join stored proc results in queries and so developers often end up having to copy their SQL leading to WET code. However, there’s a neat little hack which effectively lets you treats stored procedures like views.  It’s a little more work, but it’s worth it.

There are some very common queries that MySQL doesn’t support as VIEWs:

1. Any query that contains a subquery in the WHERE clause. E.g. where myfoo not in (select foo from bar). Fortunately, you can usually rewrite these using a left join and then filter any rows with nulls on the right hand side.

2. VIEWs that contain group/count aggregate functions cannot be made to use ALGORITHM=MERGE.  THIS IS BAD, especially if you naively forgot to set the algorithm, or just didn’t know about them: MySQL will happily build the view using ALGORITHM=TEMPTABLE instead of merging it with your query at runtime. If you have a generic query that covers a large result set, you won’t be able to use a view without bringing down the whole system!  You might not even notice this is happening if the project is new and as your data grows it’ll come back and bite you in the ass.

It’s pretty likely anyway that you will want to encapsulate, or already have encapsulated, complex queries in parameterised stored procs, so making them work like normal queries is incredibly useful and keeps your SQL DRY.

In order to make sprocs joinable, we can take advantage of temporary session tables.

Temporary tables stick around as long as the session is active, unless you explicitly drop them. So, instead of returning the result set from a stored procedure, we just put the results into a temporary session table, and whatever calls the sproc will have access to that table.  We can parameterise the sproc to keep the set small, and we can have queries as complex and procedural as we want!

This little hack will even work with your middleware code so long as it’s capable of running complex SQL (i.e. more than a single statement).  This is almost always the case.

Here’s an example:


DROP PROCEDURE IF EXISTS sp_create_tmp_myquery;
CREATE PROCEDURE sp_create_tmp_myquery(  )
BEGIN
-- this sproc creats a session temp table for use OUTSIDE the procedure 
-- essentially allowing DRY reuse of the query 
-- if the query or sproc calling this doesn't drop the temp table it'll be dropped in the next call 
-- to this sproc or when the session closes
  
  drop temporary table if exists tmp_myquery; 

  create temporary table tmp_myquery
  as
  -- some complex sql. Knock yourself out.

END;

-- to use the sproc:
call sp_create_tmp_myquery();

-- work with the results 
select foo
from   bar
join  tmp_myquery on <some condition>;

-- or just select them
select * from myquery
-- drop the table explicitly; not necessary but cleaner
drop temporary table if exists tmp_myquery; 

You’re not limited to a single data set either. If you want or need to, you could create multiple tables in the sproc and work with them after.

How to write DRY SQL in MySQL – Part 1: Views

Writing DRY code is a fundamental principle of software development.  DRY, or Don’t Repeat Yourself, was popularised by the 1999 Pragmatic Programmer coding practices book (the alternative being WET code, meaning Write Everything Twice, or possibly “We Enjoy Typing”).

DRY code is optimal, not just because it’s less typing, but because when you come to refactor code (make changes to it), it’s much easier if there’s a single place where that change needs to be made.  If you’ve repeated the same code blocks all over the place, you’ll need to go edit all of them.

The wider principle of DRY is, “Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.”  Given that one of the fundamental design principles of good Relational Databases, Normalisation, is also about removing redundancy and repetition, you’d imagine that relational database would support DRY coding practices well.

Unfortunately, writing database server code (SQL queries, stored procedures, triggers, etc.) in a DRY way, is surprisingly hard in most Relational Databases I’ve come across.  So much so, that modern frameworks like Ruby on Rails and CakePHP tend to do far more of the database work than older systems.  But languages like Ruby and PHP aren’t set-based languages and they can be very inefficient and slow, especially if they have to make lots of round trips to the DB to get data.

There are several techniques though that anyone working with a database can learn to help DRY up DB code.

Use Views to DRY Common Joins

If your data tables are properly normalised (to at least 3rd or Boyce-Codd Normal Form which they absolutely should be) then you will no doubt have primary tables that store data linked to various ‘type tables’ which store category or attribute data for of entries in your primary tables.

Let’s say, for example, you have a table called books to store details of published and unpublished books.  Some attributes of books are many-to-one relationships, meaning a given attribute might have many entries but only one of them applies to each book.  Two examples of such attributes might be:

  1. publishers (let’s assume each book has just one publisher for this)
  2. publishing_states (a book has one state, “published”, or “unpublished”, or any other state we want to define, such as “work in progress”)

Attribute tables like these tend to be one the most common types of join that you need to write in your SQL queries and so these are perfect candidates for DRYing up.

You might also have a table called print_formats to store possible formats such as “Hard Cover”, “Kindle”, “Paperback”, etc. (a book may have several formats) so this is a one-to-many relationship.

Finally, of course, you’ll also store many-to-many relationships such as authors (a book can have many authors and an author can have many books).

Let’s say in your first iteration with a small sample of data your tables look like this:

books

id title publisher_id publishing_state_id author_id
1 The Looking Glass Club 1 1 1
2 Harry Potter 1 1 2

publishers

id name
1 Sencillo Press
2 Bloomsbury Press

publishing_states

id name
1 published
2 unpublished

authors

id name
1 Gruff Davies
2 JK Rowling

Given these tables, you find yourself writing SQL code something like this all over the system.

select *
from books b
join publishers p on b.publisher_id = b.id
join publishing_state ps on b.publishing_state_id = ps.id;

When you want authors and books you’d write:

select *
from books b
join publishers p on b.publisher_id = b.id
join publishing_state ps on b.publishing_state_id = ps.id
join authors a on a.id = b.author_id = a.id;

You’ll no doubt be joining to other tables in different combinations as well as these, but the this triple join appears in all of them.  You’re repeating yourself, which is kind of a “bad code smell”, (which happens all the time in SQL, unfortunately).

These joins end up being used in your non-DB server code (PERL, PHP or RUBY) and stored procedures you write for efficiency, and inside queries that you use for administration and testing.  It’s everywhere already.

Now, let’s say you start adding some more book data, but this time you’ve been given some book titles that haven’t yet been published.  So now your books table is now:

id title publisher_id published_state_id author_id
1 The Looking Glass Club 1 1 1
2 Harry Potter 1 1 2
3 Supernova null 2 1
4 Wonderland null 2 1

You go to run the system and none of your standard queries bring back these any of these new titles.

You scratch your head for a while and then realise that when you did the first iteration, you only had published books in the system and it didn’t occur when writing your queries that unpublished books wouldn’t have a publisher_id yet.

Oops.

All those triple joins are written as inner joins when the publisher join should have been a left join in order to bring back rows with no publisher.

You now have go search and edit every single query in the system and all your testing and admin code and fix the joins. You might be even forget your own admin code because you don’t run it except when needed and one day that will come back and bite you because you won’t spot the error.

Views to the rescue

As soon as you realise that you’re typing the same joins all the time (you usually know in advance with many-to-one attributes that you’ll be doing that), you can write a VIEW to do the join for you.

So, in iteration 1 of the system, (in MySQL) we’d have written this early on:

CREATE OR REPLACE ALGORITHM=MERGE VIEW books_publishers_publishing_states
AS
select *
from books b
join publishers p on b.publisher_id = b.id
join publishing_state ps on b.publishing_state_id = ps.id;

I’ve called the view the name of the tables concatenated. Yes it’s long, but it’s fully descriptive and it’s far shorter than writing the full triple join.

Very Important: MySQL has two types of view and you always want to use type MERGE where possible. MERGE views are just SQL statements and get merged at runtime with any query they are used in. If you don’t define the algorithm as MERGE, MySQL can decide to use the other type, TEMPTABLE which will evaluate the query, selecting all the data and then join the resulting temporary table with whatever data set your query returns. This could be A LOT OF DATA if your view has no WHERE clause like this. It can bring the whole system down.  Trust me, I’ve done it.

Now we have a VIEW, we can use it anywhere where we would before have written out the full triple join making our code lovely and DRY.

So we can write:

select *
from books_publishers_publishing_states;

or, our “all books and authors” query looks much simpler to:

select *
from books_publishers_publishing_states bpps
join authors a on a.id = bpps.author_id;

Now, in this case, when we add our new data, the same problem occurs, but when we realise our join is wrong, we only have to edit it in one place.

We refactor the VIEW with the left join:

CREATE OR REPLACE ALGORITHM=MERGE VIEW books_publishers_publishing_states
AS
select *
from books b
left join publishers p on b.publisher_id = b.id
join publishing_state ps on b.publishing_state_id = ps.id;

Once we run this against the DB, everything works again with unpublished books.

Now, you might think this example is contrived (examples usually are), but we still save on typing and get to reuse code here. And I promise you will want to refactor your views as the system grows.

A more realistic example of such refactor is that you might want to add a where clause to filter certain data sets globally and change these quickly across the whole system. Views make this sort of refactor extremely quick to do.

DRY limits with MySQL VIEWs

Unfortunately, you can’t always create a view for every case. If you query uses a subquery in the where clause for example, MySQL can’t create a view for that case. Or, you might something more complex than a single query that you want to keep DRY. Fortunately, we can handle these cases another way.

In my next post, I’m going to show you a nice little hack that effectively lets you joins to the results of stored procedures as if they were tables or views.

The inability to treat stored procedure results like queries is another main source of WET database code.