So, now that I’m back from Christmas/New Year’s festivities, the first order of the day is to do a proper write up of the basic shader from Sprite Lamp’s preview window. I thought it would be good to get this out there early on, because people who are keen to get cracking with a version of it for an engine of their choice can do so. I’ll eventually be doing an implementation for major engines, but of course I can’t cover all engines myself, and I’m betting some people won’t want to wait for me anyway. With the exception of the self-shadowing, none of this is very complicated – if you’re advanced enough to get a normal mapping shader going, this should be well within reach.

Someone suggested that I make a page to collect links to people who have implemented these shaders in various game engines, and I think that’s a pretty great idea, so if you’re working on such an implementation, get in touch! I’ll be putting a page up here soon with that information, and eventually I’ll add my own integrations there when they get done.

First off, if you backed Sprite Lamp (which you can still do via PayPal, by the way), you already have the shaders in plain text. They might have some dodgy commented out code, and they’re not well documented, but they’re there in the ‘Shaders’ directory. The one I’m going to talk about for the moment is StandardPhong.fsd. As the name suggests, fundamentally this is based on the Phong illumination model (not to be confused with Phong shading).

I won’t talk too much about Phong illumination because it’s pretty common and well-documented elsewhere (including at that wikipedia link). Fundamentally, it consists of ambient lighting (pretty much just a constant value), diffuse lighting (which is calculated as the dot product of the light direction and the surface normal), and specular lighting (which is slightly too complicated to sum up in a sentence, but there’s more detail at the wikipedia page). I’m going to go over the modifications I’ve made in the Sprite Lamp version.

Cel shading

This is a fairly simple trick – the more complex palette-driven cel shading will be a subject of another blog, once the tech is finalised. Simply put, this applies a step pattern to the illumination level of the pixel. The relevant piece of code is this:

diffuseLevel *= cellShadingLevel;
diffuseLevel = floor(diffuseLevel);
diffuseLevel /= cellShadingLevel - 0.5;

By this point in the shader, ‘diffuseLevel’ should have been calculated to contain a value between zero and one representing the diffuse illumination level. ‘cellShadingLevel’ contains an integer value representing the number of levels of illumination (at least two). After this calculation, ‘diffuseLevel’ should be multiplied with the value in the diffuse map. Note that other things that affect the diffuse level, such as shadows (discussed below) or attenuation, should be calculated before this stage.

Wraparound lighting

I’m sure I’m not the first person to think of this feature, because it’s very simple, but I haven’t heard it given a name (perhaps for the same reason). Normal dot lighting is of this form:

 float diffuseLevel= clamp(dot(normal, lightVec), 0.0, 1.0);

In this scenario, the surface normal and the unit vector pointing in the direction of the light source are dotted together. This gives a result of 1.0 if the surface is facing directly at the light source, and a result of 0.0 if the surface normal is at right angles to the light rays. However, it also gives a result of -1.0 if the surface is facing directly away from the light source. Physically speaking, surfaces facing away from the light source should all be equally dark, and as they start facing the light source more and more, they become lighter – this is why the value is clamped to be between zero and one.

I don’t have any software handy for generating pretty graphs, but to visualise this, draw a graph of ‘y = cos (x)’ with x between 0 and 180 (degrees). If X is the angle between the surface normal and the light ray, Y is the light level. You’ll notice that from x=90 all the way up to x=180, the lighting value is negative (and thus gets clamped to zero).

However, if you don’t care about your lighting being a bit physically unsound, you can make use of the entire range of angles between 0 and 180 degrees. This makes the lighting wrap around the sides of the object, which can be useful for faking larger light sources (such as the sky while the sun is behind a cloud). For Sprite Lamp, I’ve created a special value called ‘lightWrap’ that varies between zero and one. Zero means normal diffuse lighting, and one means that every surface gets at least some light unless it is facing in the exact opposite direction to the light source. To visualise that second scenario, draw a graph of ‘y = (cos (x) + 1) / 2’. The implementation of this in the Sprite Lamp shader looks like this:

float diffuseLevel = 
     clamp(dot(normal, lightVec) + lightWrap, 0.0, lightWrap + 1.0)
            / (lightWrap + 1.0);

I find that light wrapping can look pretty weird if it gets higher than about 0.5, but of course the easiest way to play with the value is with the slider labelled ‘Wrap-around lighting’ in the Sprite Lamp preview window.

Hemispheric ambience

This is a ludicrously simple technique that can get you much better value out of your ambient lighting levels. Perhaps everyone out there in industry is doing this already, but for those unfamiliar, I’ll go through this quickly. Instead of specifying a single ambient light colour, with hemispheric ambience you specify two – an above light colour and a below light colour. Usually the above light colour would be a bit lighter, and perhaps a slightly different colour, but it depends on the environment. Then, you simply mix between them based on the y component of your world space normal. In Sprite Lamp’s shader code it looks like this:

float upFactor = normal.y * 0.5 + 0.5;
vec3 ambientResult = ambientLightColour * upFactor + 
                     ambientLightColour2 * (1.0 - upFactor);

In this scenario ‘ambientLightcolour’ is the above light, and ‘ambientLightColour2’ is the below light colour. ‘upFactor’ is basically the extent to which the normal is facing up. Note that if you aren’t rotating the object in question, ‘upFactor’ can simply be replaced with the green channel of your normal map, which makes things easier still.

Incidentally, if you’re also making use of ambient occlusion maps, you should get the colour of the AO map, mix it with some amount of white (depending on how intense you want the effect), and multiply the result with ‘ambientResult’ for the final ambient light colour.

Self-shadowing

This is the only effect here that is at all hard on the graphics card, because there’s a loop and a lot of texture lookups involved. It’s also mildly hacky in the way it softens the shadows, and in the current version of the shader included with Sprite Lamp, it includes a few magic numbers. Hopefully I can explain it in a way that makes sense.

float thisHeight = fragPos.z;
vec3 tapPos = vec3(centredTexCoords, fragPos.z + 0.01);
vec3 moveVec = lightVec.xyz * vec3(1.0, -1.0, 1.0) * 0.006;
moveVec.xy *= rotationMatrix;
moveVec.x *= textureResolution.y / textureResolution.x;
for (int i = 0; i < 20; i++)
{
   tapPos += moveVec;
   float tapDepth = 
             texture2D(depthMap, tapPos.xy).x * amplifyDepth;
   if (tapDepth > tapPos.z)
   {
      shadowMult -= 0.125;
   }
}
shadowMult = clamp(shadowMult, 0.0, 1.0);

Essentially what this is doing is tracing a dotted line from the fragment position towards the light source, and for each dot, checking if that point is inside the object (that is, beneath the depth map). Traditionally, if any point is inside the depth map that would mean the ray has hit an object and thus this fragment is in shadow. However, to fake soft shadows, I darken the fragment more as more points are inside the objects. This is completely nonphysical, but I found that it looks pretty alright. At the end, you have a value called ‘shadowMult’ that you multiply against other lighting values to darken them appropriately.

I’ll try to break this up so it makes a bit more sense. Note that fragPos has already been initialised to have value in it – it represents the world position of this fragment. The x and y values here are calculated from the actual position, but the z value is taken from the depth map.

We initialise the value ‘thisHeight’ to the z value of fragPos. ‘thisHeight’ is going to record the position of the current tap (dots along the dotted line) that we’re up to. ‘tapPos’ refers to the position in the texture that we’re looking at, and ‘moveVec’ is the vector that we increment ‘tapPos’ by to get to the next dot in the line. We need to rotate the x and y values of ‘moveVec’, and multiply the y value by the texel aspect ratio too. Finally, I’ll note that ‘moveVec’ is multiplied by a magic number, in this case 0.006, to determine the step size of the ray trace. Setting this to a higher value will make it possible to cast longer shadows, but at the cost of sometimes missing shadows cast by narrower obstacles.

We then go through the loop an arbitrary number of times (in this case, 20). The more iterations, the more expensive the shader, but also the longer the cast shadows can be.

Each step through the loop starts by moving ‘tapPos’ along the ray by adding ‘moveVec’ to it. We then obtain a value, ‘tapDepth’, by looking up into the depth texture at this position. Finally, we compare ‘tapDepth’ against the z component of ‘tapPos’ – this is checking whether this tap is inside the object or not. If it is, we subtract a value from ‘shadowMult’. The value that gets subtracted is another magic number – in this case 0.125. This number controls the softness of the shadows. At 0.125, it takes 8 or more taps inside the object to completely darken a fragment. If we set this value to 1.0, it would instead make the shadows completely hard (black or white). Lowering this value would make the shadows fuzzier still. At some point I’ll make these magic numbers tweakable from the Sprite Lamp UI.

Conclusion

Hopefully this will serve as a launching point for some of the more enthusiastic and experienced Sprite Lamp users who are wanting to implement the Sprite Lamp shaders in the engine they’re using. I’m not quite satisfied that the self-shadowing system here makes a lot of sense – revisiting it now has reminded me that it was somewhat hacked together when I first made it, and I never got around to cleaning it up. However, it’s hard to know which parts of an article like this need more detail and which parts don’t, so please feel free to get in touch and ask for clarification, and I’ll do my best to update the article.

Today I’m going to talk a bit about a tech demo I was working on recently. It’s Sprite Lamp-related (at time of writing, Sprite Lamp has three days left to reach its next stretch goal, by the way). This is part of a series of blog posts about more advanced techniques that fall into the category of “hand drawing and dynamic lighting” – some make heavy use of Sprite Lamp, others are merely suggestions for things to use in conjunction with Sprite Lamp. Just to be clear, the following examples make use of Sprite Lamp, but they are not possible using only Sprite Lamp. This is an example of something you can achieve with a dedicated graphics programmer on board. So yeah, basically, don’t support/buy Sprite Lamp because you want this effect in your game, unless you’re willing to do a lot of the programming tasks described here.

With that out of the way, the project was Project Lonsdale, and it was an attempt to create a full hand drawn scene with full dynamic lighting and shadowing. The plan was to put it to use in a point-and-click adventure game, and perhaps one day that’s what will happen. I’m showing this off now as a demonstration of some of the deeper uses of Sprite Lamp – this isn’t currently planned as a product launch, it was just a thing I played around with a few months ago.

[youtube=http://youtu.be/uV0wGBylkXs]

So there you have it. Explaining how this was all done will take ages and I don’t want to get too distracted from working on Sprite Lamp, so I’m going to do it in parts. Today I’m going to give an overview of what’s involved in creating the scene, both in terms of art and in terms of processing. Next time, I’ll give an overview of what goes into rendering a frame. Then over time, I’ll write more posts that flesh out bits of these overviews. Hopefully in the end, I’ll have a detailed technical write up.

Note that this tech is currently abandoned (though I may one day pick it up again), and that’s largely because the pipeline was just too involved for Halley and me to realistically create a whole game with. It’s really convoluted, seriously. Fully developing Sprite Lamp has made me realise a few ways in which it could be improved – what I’m describing here is the process, as far as we got.

Creation of a scene

I mentioned in passing that this involves not just Sprite Lamp, but also a bit of 3D modelling and quite a lot of programming. Here’s a quick rundown of what was involved.

1. Make a 3D model of the scene. This was actually a really easy step, because the 3D model didn’t have to be super sophisticated. In fact, for the first scene in the video, I (Finn, the programmer, and not a skilled artist at all) did the modelling in about a half hour. We really just couldn’t think of a good way to do proper shadow casting without some kind of mesh. Alas, the dream of not having to use modelling software at all dies. Here is what our mesh looked like at that point:

2. Load the mesh into the Lonsdale tool, and manually position the camera where you want it. This is for framing the image. Then, you export a bunch of stuff. Lonsdale at this point rendered out a 16-bit depth map, as well as a very basic preview render to be used as a guide for the artist.

3. Now it’s the artist’s turn. Halley would draw a whole bunch of stuff at this point. She’d draw the lighting images, as per Sprite Lamp’s requirements – the whole scene, approximately matched up to the render of the mesh, lit from all directions. Details that don’t need to cast shadows (such as textures on a surface, embossed lettering, etc) get added here. These would go into (the early version of the thing that eventually became) Sprite Lamp to create a normal map. Meanwhile, she would also paint a diffuse and a specularity map. Finally, she would draw something called a silhouette map. This is important, because it is simply a greyscale image where distinct edges define the outlines of the objects she’s drawn in the scene. There is always going to be a disparity between the rendered depth map and the hand-drawn everything else maps – the silhouette map helps resolve this disparity. More on that later.

4. And, back to the Lonsdale tool. This would do an edge detection on the silhouette map to figure out the shapes of Halley’s hand-drawn objects, and then jigger around with the rendered depth map to create a ‘fixed’ depth map. It would then combine the fixed depth map (red and green channels) with the silhouette map (blue channel), because the silhouette map is used in the shadowing algorithm. Now we have the depth/silhouette map, the normal map, the diffuse map, and the specularity/glossiness map. Not to mention the mesh. Everything we need to start rendering.

Coming up next time… rendering the scene

This is almost as involved as preparing the assets, but it does contain a cool screen-space shadow blur that I think might have some applications in more traditional rendering, too. Stay tuned.