Tag Archives: Sprite Lamp

Graphics, Programming, Sprite Lamp

Engine integration so far

8 Comments

Hi folks! I’ve been spending some time lately investigating a few of the most commonly requested engines for Sprite Lamp integration – namely Unity, Game Maker, and Construct2. The results so far have been pretty positive – I’m hoping to get at least some support for these three engines made public before I head to GDC. With this article, hopefully people who have been wondering about these three engines will come away with a better idea of what will be possible.

Unity

So, Unity takes the lead as most commonly requested (even though I think mostly people assume it goes without saying). Fortunately, despite it being used for 2D games very often, it is a 3D engine, and that means it has a lighting engine, which means this will be both a smooth integration and a feature-complete one.

As some may have seen, this article by Steve Karolewics of Indreams Studios details an initial attempt at getting Sprite Lamp lighting working in Unity. Steve (along with everyone else who has ever used Unity for anything) knows more about Unity than I do, and I’ve taken his work as a base for the full shader (currently a work in progress). So far, I’ve been working on the hardest part – self-shadowing. Here’s progress so far (hint: it’s working).

The light on the left is closer to the stone wall than the one on the right, which affects shadow length and attenuation.
The light on the left is closer to the stone wall than the one on the right, which affects shadow length and attenuation.

The big part that remains to be done here is that the Sprite Lamp shader has to know the resolution of the textures (for per-texel lighting and for shadowing). At present (for the above screenshot) it’s just hardcoded, but obviously that’s not a long term solution – I’m going to write a little script that runs at load time that sets up the relevant shader variables automatically, so the user doesn’t have to worry about it.

The important thing to note with the Unity integration is that, as mentioned, it already has a lighting system. Light entities already exist in the game, and they will work with Sprite Lamp as you’d expect. Multiple lights work as expected, and Unity will automatically handle stuff like calculating which lights affect a given object. You can also place lights in 3D space (that is, vary their depth position) and have everything work as expected, because as mentioned, Unity is at its heart a 3D engine.

Basically, everything that can be done in the Sprite Lamp preview window is going to be achievable in Unity. Other features that I will most likely add at some point will be support for tessellation (making use of the depth map) which might be cool for the sake of parallax or 3D, and (pending further investigation) support for the deferred rendering pipeline.

Game Maker

I often say that I don’t have a clue about Unity, and that’s pretty much true, but I have at least used it before. That is not the case with Game Maker – I literally opened it for the first time the other day to investigate stuff. It’s therefore possible that I’m missing obvious things – if so, please let me know!

That said, the results of this investigation have been positive. The short answer is, lighting is go:
GameMakerSL

This might not look like much, and it’s not as far along as the Unity integration, but the important thing is that I can load my own custom shader – from here on in, being able to get cel shading and self shadowing working is kind of just details.

At present, there’s some boilerplate scripts that go along with this too – one that you’ll attach to the Create event of your dynamically lit objects and one that goes with the Draw event – which sets appropriate shader variables and that kind of stuff. My understanding is that Game Maker is actually usable without any custom scripts at all, so for the benefit of users who aren’t scripters, I’ll keep the interface with that script as small as possible and document thoroughly any interactions you have to do with it. They’ll almost certainly be very simple – stuff like setting light levels, etc.

That brings me to the next part of this. Unlike Unity, Game Maker doesn’t have a lighting engine built in – this means that there aren’t, for instance, native entity types for stuff like light sources. At this point, I don’t understand the engine well enough to know what the best solution to this is, but it should at least be possible to have a dummy object that acts as a light source, and then tell the scripts on the lit objects to update their shader variables based on the position of the dummy object.

Slightly more complicated is the question of multiple light sources. At the very least, it is simple to have an hemispheric ambient light and a single point light at any given time. I can also add a few more lights by just making the shader more complex (basically, have multiple shader variables for multiple light positions, and add the results together in the shader). However, having an arbitrary number of light sources becomes more complicated – this will involve building a system that automatically tracks lights and what objects they’re close enough to have an effect on. For the moment, my first release of the integration with Game Maker will be the shader and code snippets necessary to get basic lighting going. I’d rather get a basic implementation going for multiple engines, then return to more in depth work like that later if it’s in high demand.

Construct2

This is the least advanced and most recent investigation I’ve done, but I know a few important things so far, and I wanted to pass them on. I’ve also been in touch with one of the developers of the engine, which helped a lot.

Basically, the way Construct2 works doesn’t naturally lend itself to this kind of thing, because of course a 2D engine designed for ease of use isn’t designed with 3D lighting in mind. But, it can be done, with a few caveats.

The rendering engine of Construct2 works, simply enough, by drawing sprites. In addition to regular sprites, you can draw effect sprites – these have shaders attached to them, and are often used for things like colour shifts and distortion effects. They also have a texture attached to them, and can sample the colour of what they’re drawn over. We can leverage this to make some lighting effects.

My first attempt is based on a shader by a developer named Pode. Here goes:
ConstructLighting

So far so good. The folks from Scirra tell me there’s a built in bump map shader that would be a good basis for me to work from too, which works in a slightly different way (the above example multiplies the effect result with the underlying colour, but you can also directly sample the underlying colour and render the result over the top).

From here, given access to the shader, I can get a lot of Sprite Lamp’s lighting effects going. The things I said in the last two paragraphs on Game Maker apply to Construct2 as well, regarding multiple lights and dummy objects and whatnot.

However, there’s another issue with Construct2 that is a limiting factor – namely, an effect sprite can only have one texture attached to it. Now, the Sprite Lamp shader makes use of a lot of textures, some more necessary than others. The diffuse texture is taken care of – the diffuse channel will be contained in the layer underneath, rendered before the effect sprite. Obviously normal maps are required. If shadows are required, the depth map can probably be squished into the alpha channel of the normal map, except insofar as you will also need that alpha channel for opacity (since it’s a separate sprite) – possibly this can be worked around, however. Emissive maps can simply be rendered as another additive layer for objects that have them – you won’t even need a special shader for that. However, ambient occlusion and specularity/gloss maps might have to be sacrificed.

There is a possible workaround here that involves putting multiple textures into one (that is, literally just the diffuse and normal maps next to each other in one image, for instance) and then extracting them with uv maths in the shader. If this turns out to be worthwhile, I’ll add an option to Sprite Lamp to automate exporting maps in this form. However, I foresee visual issues if this is used on tiling textures if your game has a camera that zooms in and out (bilinear filtering and mipmaps will not be kind in this situation).

The other possible issue is performance. Shader complexity not withstanding, this system requires you to draw at least two sprites for every game entity that needs dynamic lighting. I don’t have much of a feel for how performance gets limited in Construct2, but as an educated guess, if your game already has (or almost has) performance issues relating to the number of sprites on screen at a time, they will probably get worse if you add dynamic lighting to them all – my guess is that this will be a load on the CPU and the GPU.

One last thing to note about Construct2 that the devs warned me about is the renderer. Everything I’ve said here is completely dependent on shader effects, which require a WebGL renderer. Unfortunately, this means that environments that use canvas2d instead won’t have access to any dynamic lighting effects (notably, Internet Explorer and Safari, apparently). Sorry about that – that’s out of my (and Scirra’s) hands.

Other engines

If you’re reading all this wondering when I’ll get to your engine – don’t worry, this isn’t a complete list. It just made the most sense to approach the most commonly requested engines first, but a minimal implementation doesn’t take a terribly long time (assuming it’s possible) for a given engine. Even if it’s an obscure engine that I’m not going to do an official engine for, I’m entirely happy to help you out with getting it up and running yourself. Feel free to give a shout regarding the engine you want looked at (besides these three) in the comments.

Graphics, Programming, Sprite Lamp

Extended depth map features – goal met!

Over at  Sprite Lamp’s Kickstarter campaign, we have just hit our next stretch goal at 20k! This is pretty awesome. This means extended depth map features, so it’s time to give a big rundown of how this will work. I mentioned two aspects of this – they are depth map editing from within Sprite Lamp, and exporting meshes that are based around the depth map.

Depth map editing

Currently, Sprite Lamp is able to generate a depth map from a normal map. This feature is most visible in the examples I’ve posted in the form of self-shadowing, particularly this image of a stone wall, which I’m going to post again because I love it so much:

Stonewall_TwoProfiles

If you look closely, Sprite Lamp has caught the depth quite accurately, right down to the little notches in the stone, and the bits where there are weird mortar shapes left on. Feel free to inspect the detail:

stonewall2_DepthNew

While it’s important to remember that there’s no such thing as a ‘correct’ depth map based on a drawing, I think the above image is pretty good. In general, I’m pretty pleased with Sprite Lamp’s algorithm/system for generating a depth map from a normal map. This is a pretty difficult problem, made worse by the fact that you get a physically imperfect normal map when it’s drawn by hand. Textures like the one above are, fortunately, a pretty good case for this application. Alas, not everything can be a best case scenario. I won’t go into too much detail here, but the worst case scenarios for Sprite Lamp’s depth generation are all about discontinuities in the depth map. They happen when what you draw has one object passing in front of another – it’s hard for Sprite Lamp to guess how far in front. This doesn’t happen much with textures (like the stone wall, above) but it does happen with character art. Sprite Lamp gets pretty good results here too – good enough for some nice self-shadowing effects, as I think is evidenced by the sample art on the main page and in the Kickstarter.

However! With algorithms like these that involve guessing what the artist intended, there are always times where you guess wrong. For that reason, I’d like to give the user the ability to mess with Sprite Lamp’s ‘interpretation’ manually a little bit. Now, naturally, I’m not going to offload all the dirty work onto the user – if I was going to do that you might as well just draw it by hand. The goal is small and intelligent tweaks.

With that in mind, I have a few features that I’m going to experiment with to make working with depth maps better. I’m not saying these will all make it into the final program – these are what I’ll try, but I don’t yet know what will be worth including and what will end up being a waste of time.

  • Edge detection: Sprite Lamp will go through the lighting profiles and try to figure out where depth discontinuities might be.
  • Silhouette maps: This is a plan I have to have the artist draw a simple map that will be deliberately easy for Sprite Lamp to detect the edges of. May or may not end up being necessary, but will mostly be trivial to create because it comes from mask layers that come from the drawing process anyway.
  • Edge-aware soft selection: The first two points here are ways of figuring out where depth discontinuities are – this is useful information to have, because it allows the user to easily select areas of the depth map without going ‘outside the lines’. If you have a foreground object and a background object, this will allow you to grab the depth values of the foreground object and move it back and forth without interfering with adjacent pixels (which are the depth values of another object entirely).
  • Spring systems representing depth values: This will allow the user to drag a single depth pixel forward and back, and have adjacent pixels come with to a greater or lesser extent. There won’t be springs connecting pixels along discontinuity edges.
  • Curve editor to do an image-wide readjustment of values: This is something similar to how colour readjustment works in programs like Photoshop, and it could indeed be done in those programs – however, I think it might be worth including it in Sprite Lamp because getting immediate feedback on how it looks will make things a lot easier to deal with.

So, that’s that. I’m looking forward to playing around with these things, and of course I’ll keep you posted as to how it all goes. As for the other part of this stretch goal…

Mesh Exporting

The other part of this stretch goal is to add the ability to export meshes. This doesn’t take quite as much explaining as the other feature, though it will take a bit of work on my part. The goal here is to export an intelligently-created mesh that takes the shape of the object according to the depth map. Naturally, the easy approach to this is to simply subdivide a quad into many quads, then displace the vertices according to the depth map. I’m hoping I can do quite a bit better than that, by placing vertices in important places along ridges and points, and making the geometry more sparse in flatter areas. The user will have a slider that allows them to adjust the target vert count of the generated mesh. This has a couple of concrete uses:

  • Stereographic rendering without relying on complex shaders: Naturally, I quite enjoy complex shaders, but there are times when it’s easier to just use geometry. This can save you from various annoying consequences of displacement shaders, such as correctly handling extreme angles.
  • Correct rendering into depth buffers for ‘true’ shadowing: I’m going to be writing a lot about the various sneaky tricks you can use to fake shadows using depth maps – however, there are certain shadowing algorithms where it’s just nicer to do things with geometry. Having geometry at your disposal gives you more options for plugging directly into existing ‘general’ shadowing options, such as shadow mapping and CSMs, which are well-documented already and amply implemented in existing engines.
  • As a base for further tessellation/displacement: The user will be able to determine how strong the vertex displacement will be, and that will include zero – that means that you can use the flat mesh as a basis for more effective GPU-based tessellation in the GPU, and displace generated verts based on the depth map in a shader.

In addition to these, though, I’m looking forward to see what else people come up with using this feature. I’m certainly planning on making some 3D prints of artwork once I get it done!

One last thing I’ll say on this subject is that I haven’t settled on formats for exporting of meshes. I’ll probably make use of the dubiously-named AssImp (short for ‘Asset Importer’, obviously), which can export to a handful of formats, in accordance with this list. However, if anyone knows of any straightforward mesh exporting libraries for C# you think would be better, I’m all ears.

Low-content rambling, Sprite Lamp

Sprite Lamp hits its goal!

Well, I was planning on posting an update when Colour Bind’s Kickstarter went live, but I ended up getting distracted by a bunch of social media stuff, and now that I am actually getting around to posting this, it has a different title.

Sprite Lamp on Kickstarter

That wasn’t quite what I was expecting. Sprite Lamp hit its goal of 6k around the seven hour mark, which I strongly suspect (but can’t prove) makes it the first official Australian kickstarter to be funded. Sweet! The obvious thing to say is THANK YOU to everyone who pledged, and everyone who helped boost Sprite Lamp by sharing the kickstarter on social media (and in general). Because of you, Sprite Lamp will not only happen, but it will happen in a timely fashion, and most likely it will get a bunch more features and be a higher quality product than it would have otherwise been. Also because of you, I get to buy a new pair of shoes (seriously, my old shoes have holes in them).

With that out of the way, it seems I have some work to do. I had put some vague stretch goals in the Kickstarter, but now I’ve got to make them specific much more quickly than I was expecting. I’ve already had some good suggestions. In accordance with one such suggestion, I’m going to set up some rudimentary forum here for talking about suggested features and the like.

I’m also going to make a series of blog posts here about a variety of graphics techniques that will be applicable to Sprite Lamp. I’m going to cover things like how the shaders work, issues with using normal maps that you might not see coming (like the fact that rotating a normal map in Photoshop results in incorrect normals), and a variety of shadow techniques that exist somewhere along the correct/fake spectrum that might work well in conjunction with Sprite Lamp.

Oh, and don’t forget that I need your precious votes on Steam Greenlight – any help there is much appreciated.

Thanks for all the support, everyone!

Graphics, Programming, Sprite Lamp

Returning from hiatus with Sprite Lamp

Okay so, I’ve been pretty absent around these parts. To some extent that’s because I’m a socially backwards hermit who hates the internet, but it’s also because I’ve been working on something that I haven’t gone public with yet.  Well, now I have, and that something is called Sprite Lamp.

Unlike everything else I’ve ever worked on or thought about working on, Sprite Lamp is not a game but a tool (for games). It’s all about combining dynamic lighting with 2D art. I think it’s kind of cool. You can read some details about Sprite Lamp or just enjoy this picture of the logo for it, at your leisure. I will be talking a bit more about this kind of stuff on social media, or at least trying to, in the near future, so that’s something to look forward to, too.

I’m going to be going all Kickstarter with this thing as soon as Kickstarter lets Australian projects go live, which is November the 13th. Wish me luck.

Sprite-Banner-big