Data Driven Game Architecture

You hear the term "data driven" bandied about reasonably often in gaming tech circles. Why is this? Here I'll present a summary of my experiences in the matter.

Goals:

• Laziness. We want to write the least amount of code to deal with each instance of a feature of our game. Why on earth write a whole class for a new bad guy if we can get away with a bit of data instead?
• Laziness part 2, or maintainability. As a consequence of laziness, we want to end up with the most compact form and the least possibility for errors in our instances.
• Quick iteration. Reloading a bit of data in a running game is much quicker than compiling and restarting. We want to spend time on the game, not the structure.
• Encapsulation. Games change during development, usually quite a lot. Data is easy to add and remove from projects whereas code can all to easily get entrenched.
• Devolution of responsibility. The power to actually carry out game implementation begins in the hands of the programmer, but not all aspects of implementations belong there. Data, and tools to edit said data, let you shunt that power to whomever requires it.

Bonuses:

• Better modularity. We end up partitioning functionality in our code more naturally when we're thinking data driven. As a result, you're also more likely to get recyclable code for next time.
• Easier serialisation. Writing your save game module can be made easier by a nicely data driven framework: the data to save is usually easily found and easy to mark out.

How to go about it?

• For a given feature in your game, collect a definition that encompasses all instances of the feature. For example, if you're making an RPG you'll need NPCs. Each NPC might want to be able to engage in a conversation, query information from the world state, be able to retain and be queried on its state, etc.
• Figure out what portions of your definition will be the variables between your instances, e.g. each NPC will have a different name, a different description, a different storyline, etc.
• Figure out what sort of data will best facilitate describing those variables.
• Start out right! If you already know you'll need this, invest in the time to get it right. Sure you can hack in something hard coded to start with, but you'll later pay the price in lost momentum when you least want to.

There are two broad divisions on methods of data driving a game.

• On the one hand, we have the simple property bag. This is passive data that simply sets up a few variables for game objects. Simple RPG weapons, for instance, where each definition is a name, a description, damage information and some art resource references. My preference for this sort of data is a nice XML file. XML is robust and easy to parse, there are plenty of decent raw XML tools to leverage (especially if you write a schema to go with it), and there are plenty of mature XML code libraries to work with. That last part is particularly important as it makes a painless transition to a binary format, should it prove necessary, trivial. XML isn't perfect, but it's easy and it's there.
• On the other hand we have the more complex behavior bag. This is an extension of the property bag where we also add some scripted behavior specialisations to each of our instances. Enemies in a shooter for instance: where each enemy is defined with some art resource references, and callbacks that contain code fragments to describe their AI. There are a lot of ways to implement this, a virtual cornucopia of choices. You have all manner of embeddable scripting languages like Python or Lua that are quite mature now, and plenty of experimental smaller ones if that's your bag. Depending on your exact requirements, rolling your own might even be an option. There's a lot to say for domain specific languages.
  

Things to watch out for.

• Don't get carried away. Do your homework first, and make sure that you really do need to data drive a feature. There is some overhead to pay in terms of implementation time, and it's not a cost worth paying if all you'll do with it is define the one instance that never changes!
• Remember performance. You should absolutely be willing to pay some performance decrease in exchange for data driving, as obviously you're trading that for a quality upgrade from the faster design iteration. Be careful not to be disastrously generous with this trade though; as with all things be pragmatic about the end result. Above all do not fall into the "we'll fix it later" performance trap. The industry is littered with horror stories where teams have leaned on a scripting language only to spend half a year backpedaling out of it to get the frame rate back to 30, or get the memory footprint back down to the release configuration.
• Beware the lure of generic! When building your data driven system, it is intuitively tempting to continue to generalise the implementation. In fact, as you continue to work, you'll keep getting struck with "Hey, this is just a slight specialisation on that other thing!" Be careful here! All problems in your game are implementable in an ultimately generic structure: the programming language you're writing the game in. Hold fast to the idea that what you're doing is in fact specifying groups of solutions to remove the redundancy of implementing their variations generically! In other words: if you end up embedding a scripting language into your C++ game and then writing nothing else but script for every feature of the game... all you've done is swapped programming languages. This is not necessarily bad (and possibly a topic for another time), but it's not an action towards the ideal we're discussing here.
• Continuously evaluate where you are. The whole point here is that you're optimising how you write the game. If you end up haemorrhaging more time than you otherwise would have, you've made the wrong choice. Often it's too late to back out, but learn all you can from the experience.