Kindle RSS Ripper

After trying out blogs and periodicals on the Kindle, I'm left reasonably impressed. It's really neat to pick up the Kindle and find that one of the blogs you watch has an update you haven't read yet. The push functionality is cool, and I dig what amounts to the pay as you go approach to the EVDO bandwidth. I mean, someone has to pay for it somewhere, right? And so what if Amazon are playing the margins for profit, the ARE a business, right?

But what about RSS feeds that aren't on the Kindle service yet? For that matter what about RSS feeds that you don't care about quite so much as to be that up to date? I have a few blogs that I read once in a blue moon, and entertained as I am by them, I can't see shelling out cash for them. Clearly Amazon don't want to trumpet free content when they're offering their upgraded cool stuff for pay, but they've graciously left a trail of breadcrumbs for anyone so inclined to follow.

First off, the format of Kindle documents appears to just be a wrapper around Mobipocket files. These are compiled binary files created by tools available at the Mobipocket site. Believe it or not, there are free versions of the tools there, including a command line compiler.

What are the input files? Turns out they're just plain old HTML. Mobipocket recognize a few extra tags aimed at books (e.g. a page break tag), but otherwise they're just HTML. Reference for the whole thing is available at the site.

So we have XML (RSS) feed documents on the one hand, and a command line compiler that takes HTML files and builds our desired output. Piece of cake!

I've whipped up a little command line utility of my own in C# that will process a list of rss feeds into a set of .mobi Mobipocket books. I'm posting it here for anyone else to try, but I make no guarantees about whether it'll work for you, nor will I offer any support or additional features. I don't have a lot of free time these days, and I'm only likely to work on it further should I need something more of it. Hey, if we're lucky this will inspire someone else to go make a fancier version!

Having said that, this does work reasonably well. So far I've tried it with Kotaku, Slashdot, GayGamer, XKCD.com and a bunch of other sites. If it fails for you on some other feed, I'd be inclined to suspect that it's yet another weird variant of what it means to be an RSS feed. I didn't know that the whole concept was such a mess!

To run this:

• Download and unzip the RSSKindleSync.zip from here file to any directory of your choice.
  
• Download the mobigen utility from the Mobipocket site and copy the actual mobigen.exe file to the same directory where you left RSSKindleSync. You should now have 4 files there: RSSKindleSync.exe, RSSKindleSync.exe.config, feed list.xml and mobigen.exe.
  
• Now edit the feed list.xml file in that folder to add all the sites you're interested in ripping. Just copy and paste the existing tag and change the URL parameter for each.
  
• Run the RSSKindleSync.exe file and away you go!
  
• You should end up with a pile of .mobi files in a /books/ subdirectory under that exe file, one for each RSS feed you listed. Just copy these over to your Kindle's documents directory and you're done.
• Whenever you'd like a refresh, just rerun RSSKindleSync.exe and all your RSS books will be rebuilt.
  

Enjoy!

Zap! Pow! Biff!

A few things have happened since last time:

• I now have the different firing modes in place. Each of Tad's (Tad, his name is Tad for now) weapons have 3 firing modes: tap for a quick shot, hold and release for a "shotgun" and hold for a while for a berserk blast. The fire gem has all 3 working now.
  
• Added support for a bigger variety of effects. Mean, mean friends were saying it just wasn't pretty enough.
• Built a cave environment.
• Made froggies.

The vid above shows the attack modes. I'll post another soon with the new pretties.

I think I've reached the limits of the built in content pipeline plugins and it's time to branch out. As soon as I hit another wall, I'm off to write my own importers and processors.

I've grown to like mesh animation. It's crude, but appealing to me. Normally you'd rough out an animation by defining key poses along the timeline, then iteratively pass over the whole thing again and again adding overlapping motions and follow through to get a natural looking motion. Animating on a budget generally means animating pose to pose, and leaving it at that, and rightly enough that suits the mesh animation look better. I simply model the poses I require (using skinning in Max, if it's a complex mesh like Tad), then string them along a timeline in Stutter, duplicating appropriate hold frames.

With my own content processors, I could pack all those mesh frames into the minimum amount of data necessary, and the overhead should be pretty reasonable. Especially as I'm going low poly anyway. Certainly the runtime processing cost is laughably small compared to a skeletal animation.

Live scripting language is absolutely the bomb. Almost all of the logic bar collision is written in Stutter now, and the joy of being able to write player control while actually in the game is intoxicating. It would be quite a bit more work to get a professionally robust version going, but I think I'm going to start recommending it anywhere I work. It's just so worth it. I just need to see if there's some easy way to hook up live asset reloading (in this XNA version) as well. As it is, adding/changing assets still requires me to drop out and rebuild in VS.

In case anyone is wondering why bother with the XNA and C# when I'm off traipsing through my own language... how on earth do you think I got it all up and running so quickly? I'm still well inside month one, and I have the whole kit and kaboodle chugging along. In fact, I had a model on screen, being manipulated by Stutter inside the first few days. This is a product of C# and .Net's excellent reflection abilities coupled with the accessible nature of the XNA libs in GSE. All I've done is written a layer of abstraction above these that lets me speak directly to game concepts.

GLH out.

PS Music for the vid is by Cornelius (god please please please someone make him write music for games), the track's name is "2010".

Baby lisp

So the rpg has come on a little bit. I really ought to come up with a name at some point. For now, it stays RPG Zero.

New since the last update:

• Tried to refine the look of the thing. I'm going primary colours, sharp contrasts, simple forms, abstract surfaces and everything easy to put together quickly. Also leaning a lot on vertex colouring for the low frequency shifts.
• Our protagonist can now be hit by badguys, and keeps track of his health.
  
• First blush at a HUD, and UI framework to back it up.
  
• A chest! What's an RPG without a treasure chest, eh?
• Refactored the player control to remove some annoying kinks, like sliding while idling.
• Wrote a *secret* story outline!
  

I mentioned last time that the scripting language in this thing was a bit... odd. Well, here's a look at what a typical dev session is like.


Seem oddly familiar? It's not really lisp, but it does share some characteristics with it. Chiefly: code is data. I can pass it around, chop and change it, store it and call it later. The scripting language is interpreted, which means it inherently has the ability to inject novel code into the currently running session. Godsend that, when working on a game. No compile waits, you're right there, in the game, where you can see what you're doing immediately.

It's called Stutter.

It's implemented as a runtime interpreter that is plugged into a reflection based dispatcher. I have a class that contains all the intrinsic functions. I then have a few intrinsics for dealing with .Net types directly, cnew (new a C# object), cset and cget (set/get properties or fields) and call (invoke a C# function). These are for... well cheating. My character entity for instance, is a C# based PhysicalObject, that gets initialised and bandied about by Stutter, but lives in the Game.Components update loop.

Why not real lisp? Well, I couldn't find a handy implementation, and I've still a lot to learn about building and implementing the support for an embedded language, so it seemed like the right time to learn. And the right time to experiment. I've taken the things I like about lisp, and screwed around with the formula to give me the things I've always wanted for games. Will it work? Will it be fast enough? Will it be expressive enough? Who knows? That's the whole point of trying.

I did mention this whole thing is one big bout of self indulgence, didn't I?

P.S. Today's vid is higher res than the last one, so you might want to click on over to the bigger Google Video player to see it. Today's music is Billy Bragg and Natalie Merchant doing Way Over Yonder in the Minor Key. Totally love that tune.

Shooter Zero: A data driven sample

So, in the last programming post I tried to dump what I know about data driven architecture onto a rather dense page. Here I'd like to share with you a sample application that might shed some light on what I was trying to say.

The ShooterZero.zip file over on my google pages page contains an XNA project that is a 2D shooter game framework.

The code describes a series of classes that together provide services that a shooter will need:

• Resource management and a game loop with debugging functions (even a simple console!)
  
• An object model, where all elements of the game are Object Instances that are created from Object Definitions
• A scripting language to define the behaviour of Objects
• A set of services that the Object Instances can employ, e.g. intersection, rendering, input, etc.
• A parser that can create Object Definitions and Object Instances

The project comes with 3 XML file definitions for 3 different kinds of shooters.

The implementation here is admittedly very slapdash for lack of time invested, but I believe the principal features of the kind of data driven system I'm trying to demonstrate are evident:

• Game is setup with parameters read from a data file - one engine supports a variety of expressions by implementing common primitives.
  
• Objects in the game are defined using data driven templates, both properties and behaviors - data is parsed into logical collections of primitives.
  
• Object instances are created from data - separate structure from content.
  
• All the Data is reloadable at runtime - faster iteration time when it comes to implementing content.
  

So, if you wish, download and have a play with it. Questions are welcome, and if there's enough interest I could write a few more posts that break out the details.

A word on XNA performance from the horse's mouth

Curious to know which bits of information on managed code performance on the 360 are FUD versus truth? The .Net Compact Framework Team's blog has a 2 parter on performance considerations on the beast.

There are some interesting tidbits on where the XNA and the standard Compact Framework diverge. Shame that it doesn't yet take advantage of the Altivec stuff, but oh well.

They mention memory management, of course. I'm still of the opinion that the best way to minimise the effect of memory allocation/deallocation on your game's frame rate is the same way to minimise the effect of anything on your frame rate: don't do it. Allocate what you need at the start of the most convenient logical block you have (game start, level start, etc) and recycle. No sense in deallocating a bad guy when it dies if you'll just need another in a few seconds!

As a bonus, the nice chaps at the .Net CF blog also link off to what they consider to be good advise on managed code performance in general.

All very much worth taking note of.

Hello, XNA

So I've been writing tools in C# for building games for almost two years now, but I've not really attempted to write a game there yet. I had a brief flirt with MDX for the novelty of it, but without being able to use it at work I just haven't been able to find the time. A bit of time to myself and the launch of XNA though, has prompted me to finally investigate.

Turns out, XNA is wonderful.

To start with what I know: C# is ace. It's a reasonably clean language that in the MS implementation comes with a great set of libraries (.Net). It's easy to learn, easy to read and easy to debug. Not to mention the built in honest to goodness reflection. Man, does that come in handy.

What a joy to find that a good chunk of that stuff shows up in the XNA Framework as well. For the uninitiated, the XNA Framework (based, as I understand it, on the Compact Framework) is the cut down version of the full .Net libraries designed to run on both Windows and the XBox 360. The plan is that if you stick to these bits, you should be able to run your game on the 360 with just a recompile. If you're resolutely windows bound though, you're free to additionally use any other libraries that strike your fancy.

Getting into the actual game libs, XNA provides runtime managed wrappers for DirectX graphics, input handling, audio playback, and some other miscellaneous bits and bobs. My experience so far with the graphics portion has been sublime. Not only is it laid out in a wonderfully clean and concise manner, but the performance hasn't been shabby at all. Given the overall advantages of working in managed code, I can happily see developing a game this way.

Offline, XNA also provides a content pipeline for organising your data builds. Constructed as a Visual Studio plugin, the content pipeline does the usual dependency checked source to binary platform target conversions. The system is designed to take plugins for each conversion type, and comes with standard converters for .x, .fbx, model files .fx effect files and a brace of standard image files. These default plugins worked well enough, and I had practically plug and play data out of 3D Studio Max in no time.

There's plenty still missing, but if this is version 1 then I'm happy to invest the belief that it'll come eventually. Following is my initial laundry list of missing XNA bits and bobs:

• Animation. Preferably built in such a way that makes animation compatible with but not limited to models. Reasoning for this is that data over time can be rather useful for any number of things besides moving an articulated skeleton of bones. A channel based architecture, I'd wager, would be appropriate. Bonus points for supporting some generic form of triggering data as well, e.g. sound triggers.
  
• Game Structure. The current Game class takes away a lot of the tedium associated with starting up a robust directX window (actually, amazingly so. Hello XNA World is an astonishingly small amount of code), but it doesn't really point much in the way of game structure. It would be nice to see, if only as good practice examples, modules from Microsoft for graphics scenes/contexts, state machines, etc.
• UI. There are only so many ways to build the back end code for a UI system. I know, I've written a few myself. I've yet to see any really compelling reasons to pick one over the other, and frankly if any of them were available off the shelf I'd jump at using it. Microsoft have already built the great Forms system for Windows programming, I'd love to see them apply some magic to game UIs.
  
• Online Support. Nothing in the XNA about playing online yet.

So week one of the XNA and I'm quite happy that I've spent the time on it. To be honest, I don't think that this'll take the dev world by storm just yet. Many of the advantages XNA offers we already have, the content pipeline in particular. What dev house doesn't have one? Writing in C# at the moment is also a no go for anything but the Windows platform. Even for the 360, we have to wait until the middle of next year for the professional version of XNA to surface.

No, I think where XNA really lives, for us devs anyway, is in promises for the future. If Microsoft can demonstrate that they're in it for the long haul, then they might win over the more staid among us when it's time to upgrade from our current setups.

Then perhaps I'll finally get to write some runtime managed code!