Today I’m going to talk a bit about a tech demo I was working on recently. It’s Sprite Lamp-related (at time of writing, Sprite Lamp has three days left to reach its next stretch goal, by the way). This is part of a series of blog posts about more advanced techniques that fall into the category of “hand drawing and dynamic lighting” – some make heavy use of Sprite Lamp, others are merely suggestions for things to use in conjunction with Sprite Lamp. Just to be clear, the following examples make use of Sprite Lamp, but they are not possible using only Sprite Lamp. This is an example of something you can achieve with a dedicated graphics programmer on board. So yeah, basically, don’t support/buy Sprite Lamp because you want this effect in your game, unless you’re willing to do a lot of the programming tasks described here.

With that out of the way, the project was Project Lonsdale, and it was an attempt to create a full hand drawn scene with full dynamic lighting and shadowing. The plan was to put it to use in a point-and-click adventure game, and perhaps one day that’s what will happen. I’m showing this off now as a demonstration of some of the deeper uses of Sprite Lamp – this isn’t currently planned as a product launch, it was just a thing I played around with a few months ago.

[youtube=http://youtu.be/uV0wGBylkXs]

So there you have it. Explaining how this was all done will take ages and I don’t want to get too distracted from working on Sprite Lamp, so I’m going to do it in parts. Today I’m going to give an overview of what’s involved in creating the scene, both in terms of art and in terms of processing. Next time, I’ll give an overview of what goes into rendering a frame. Then over time, I’ll write more posts that flesh out bits of these overviews. Hopefully in the end, I’ll have a detailed technical write up.

Note that this tech is currently abandoned (though I may one day pick it up again), and that’s largely because the pipeline was just too involved for Halley and me to realistically create a whole game with. It’s really convoluted, seriously. Fully developing Sprite Lamp has made me realise a few ways in which it could be improved – what I’m describing here is the process, as far as we got.

Creation of a scene

I mentioned in passing that this involves not just Sprite Lamp, but also a bit of 3D modelling and quite a lot of programming. Here’s a quick rundown of what was involved.

1. Make a 3D model of the scene. This was actually a really easy step, because the 3D model didn’t have to be super sophisticated. In fact, for the first scene in the video, I (Finn, the programmer, and not a skilled artist at all) did the modelling in about a half hour. We really just couldn’t think of a good way to do proper shadow casting without some kind of mesh. Alas, the dream of not having to use modelling software at all dies. Here is what our mesh looked like at that point:

2. Load the mesh into the Lonsdale tool, and manually position the camera where you want it. This is for framing the image. Then, you export a bunch of stuff. Lonsdale at this point rendered out a 16-bit depth map, as well as a very basic preview render to be used as a guide for the artist.

3. Now it’s the artist’s turn. Halley would draw a whole bunch of stuff at this point. She’d draw the lighting images, as per Sprite Lamp’s requirements – the whole scene, approximately matched up to the render of the mesh, lit from all directions. Details that don’t need to cast shadows (such as textures on a surface, embossed lettering, etc) get added here. These would go into (the early version of the thing that eventually became) Sprite Lamp to create a normal map. Meanwhile, she would also paint a diffuse and a specularity map. Finally, she would draw something called a silhouette map. This is important, because it is simply a greyscale image where distinct edges define the outlines of the objects she’s drawn in the scene. There is always going to be a disparity between the rendered depth map and the hand-drawn everything else maps – the silhouette map helps resolve this disparity. More on that later.

4. And, back to the Lonsdale tool. This would do an edge detection on the silhouette map to figure out the shapes of Halley’s hand-drawn objects, and then jigger around with the rendered depth map to create a ‘fixed’ depth map. It would then combine the fixed depth map (red and green channels) with the silhouette map (blue channel), because the silhouette map is used in the shadowing algorithm. Now we have the depth/silhouette map, the normal map, the diffuse map, and the specularity/glossiness map. Not to mention the mesh. Everything we need to start rendering.

Coming up next time… rendering the scene

This is almost as involved as preparing the assets, but it does contain a cool screen-space shadow blur that I think might have some applications in more traditional rendering, too. Stay tuned.