But no, I'm too finicky to settle with the lowered resolution. It just looks so much better in 720p! So I cranked the res back up, and decided to see where I could squeeze out some extra performance. Naturally, the first place I went to was my show mapping shader. I already knew this particular shader was giving me trouble, since I'd discovered that decreasing the size of the buffer that I render the shadow occlusion to (since I do a deferred shadowing pass) resulted in significant performance gains. I'd already reduced thigns to 4 PCF samples on the 360 (did I mention I ditched VSM for the 360? Performance and precision ended up being so awful it wasn't worth it) so I couldn't squeeze that down anymore. At this point my eyes drifted down to little loop I had for determining which split of shadow map cascade to use, when I remembered an excellent presentation on shader performance that I'd read a long time ago. One of the things mentioned in there was that unrolling loops can have a significant impact on general purpose register usage, ALU usage, and shader compiler optimization. So I thought, "hey, let's try unrolling this loop and flattening this branch". I then changed my code from this:
for (int i = 1; i < NUM_SPLITS; i++)
{
if (vPositionVS.z <= g_vClipPlanes[i].x && vPositionVS.z > g_vClipPlanes[i].y)
{
matLightViewProj = g_matLightViewProj[i];
fOffset = i / (float)NUM_SPLITS;
}
}
to this:
[unroll(NUM_SPLITS)]
for (int i = 1; i < NUM_SPLITS; i++)
{
[flatten]
if (vPositionVS.z <= g_vClipPlanes[i].x && vPositionVS.z > g_vClipPlanes[i].y)
{
matLightViewProj = g_matLightViewProj[i];
fOffset = i / (float)NUM_SPLITS;
}
}
I ran the game again, and BAM: I shot up from 55fps to 65fps! Huge difference! I was very impressed with myself. Moral of the story: experiement with stuff, and make sure you profile it!
Today I decided to read through some other presentations to see what other useful bits I could find. This one from Gamefest 2007 pointed out that vfetch's should be aligned to 32-bytes on the 360. I did the math on the vertex declaration used for most of my models, and found out it's 48 bytes. Later tonight I'll have to see if I can squeeze it down to 32, and see if it makes performance any better.
I also came across this one from Gamefest 2008, which is all about how texture and surface formats are handled on the 360. This gave me some insight into some problems I'd come across already. For example, R32G32F (Vector2) isn't actually a format the GPU can render to! Apparently it renders to R16G16F, and then just expands it upon resolve from eDRAM. This explained why my VSM's with exponential warp were have such precision problems. Another thing pointed out in there is that the 360's texture units filter fp16 at 1/4 the rate of INT8! This would help explain why my VSM performance was so poor. Definitely good things to keep in mind.
-MJP]]>
A bit of an off-topic post for any Dutch readers that may come across our little blog. MS decided to revamp their Dutch .NET Magazine and change the subscription to Opt-In, meaning you'll explicitly need to tell them you want to keep receiving the magazine after the next issue of September 22nd. You can find more details and re-subscribe over at this page:
http://www.microsoft.nl/netjesgeregeld
It's quite a risky step for them to take, but they want to make sure they're reaching their reader base and get a clearer picture of the interests of their readers. The magazine remains free and they've drummed up a full-blown redactional team for the revamp, so if you've enjoyed the magazine so far make sure you re-subscribe!
]]>
-The 360 has 512MB of unified memory, which means it's shared by both the CPU and the GPU. You don't have all of that available to you, since some of it is taken up by the console's "OS" and some will also be taken up by the .NET Compact Framework. You'll also be working from the managed heap, rather than directly working with native memory.
-You can only execute pure managed code on the 360. You can't, for example, P/Invoke into a non-managed DLL.
-As far as GPU shaders go, you're pretty unrestricted. You can use SM3.0 HLSL, or you can also write portions of your shaders in the GPU's native microcode. This is really very nice...it lets you do things like un-normalized texture addressing, full texturing capabilities in the vertex shader, or directly fetching an element from a vertex stream. The microcode set is referred to as xvs_3_0 and xps_3_0.
-The 360's GPU is different from your average PC GPU in that it has an eDRAM framebuffer. The eDRAM is 10MB in size, and has tremendous bandwidth (256GB/s). What this means is that writing out to the framebuffer or reading it back for blending is very very quick. Multi-sampling is also very quick, since again you don't have the bandwidth problem. In fact MSAA would be "free" if it weren't for tiled rendering...you see the downside of eDRAM is that if your render-target + z-buffer is too big to fit in eDRAM, you have to render to it in tiles. This means you render one portion of the target, then another. This isn't so bad, except for the fact that any geometry that's on the edges of 2 tiles has to be drawn twice. If you're not doing scenes with hugely complex geometry you probably won't even notice tiling (it happens automatically). To figure out whether you're going to tile you need to count the amount of bytes per pixel and then multiply by resolution. So for example if you're rendering to the Color format which is 4 bytes per pixel and you're using Depth24Stencil8 which is also 4 bytes per pixel, you have 8 bytes per pixel total. When multi-sampling, you multiply this amount by the number of samples (so 4xMSAA would by 32 bytes per pixel). 1280 x 720 with 4xMSAA would be ~28MB, so you'd need 3 tiles.
-Be prepared to get CPU-bound really quick if you're doing anything non-trivial. DrawPrimitive calls are extremely expensive on the 360...I've seen my framerate go from about 70 to 30 just from going from 24 DP calls to 34. Instancing is a must if you need to draw a lot of meshes...there's a good sample on the CC website. By the same token if you're doing any really fancy logic on the CPU that's not graphics-related, you'll probably need to run it on another thread on a different core since your main thread can get bogged down pretty quick.
-Watch out for performance pitfalls with the .NET Compact Framework. Things like Garbage Collection compaction and virtual function calls are much more expensive than they are on the PC. I suggest reading this blog for tips. Just remember to keep your live object count as low as possible, and you should be okay.
-Floating-point performance is not so great on the 360 CPU. Most of the fp power is in the vector units, but you have no access to those through XNA.
-Avoid the surface formats that are larger than 32bpp. Mainly HalfVector4 and Vector2. Their performance is generally pretty terrible, and I've run into all kinds of driver bugs with them. This means you can't do HDR in straightforward way, but there are other options. There's an entry on my XNA blog where I talk about how I got around it.
-Watch your texture sampling bandwidth. Framebuffer access may be quick, but reads from textures are limited by the 22.1GB/s read bandwidth. This may be quite a bit less than what you're used to, if you're prototyping on a higher-end card like an 8800. This can be especially painful in scenarios where you want to take multiple samples per pixel, like PCF for shadow maps or SSAO.
-Prototyping and developing on the PC is a good idea since you have access to PIX, but make sure you test pretty often on the 360. You may need to optimize for quite few scenarios if you need to keep the framerate up.
- Standard HDR, fp16 buffer
+Very easy to integrate (no special work needed for the shaders)
+Good precision
+Support for blending on SM3.0+ PC GPU's
+Allows for HDR bloom effects
-Double the bandwidth and storage requirements of R8G8B8A8
-Weak support for multi-sampling on SM3.0 GPU's (Nvidia NV40 and G70/G71 can't do it)
-Hardware filtering not available on ATI SM2.0 and SM3.0 GPU's
-No blending on the Xbox 360
-Requires double space in framebuffer on the 360, which increases the number of tiles needed - HDR with tone-mapping applied directly in the pixel shader (Valve-style)
+Doesn't require output to an HDR format, no floating-point or encoding required
+Multi-sampling and blending is supported, even on old hardware
-Can't do HDR bloom, since only an LDR image is availble for post-processing
-Luminance can't be calculated directly, need to use fancy techniques to estimate it
-Increases shader complexity and combinations - HDR using an encoded format
+Allows for a standard tone-mapping chain
+Allows for HDR bloom effects
+Most formats offer a very wide dynamic range
+Same bandwidth and storage as LDR rendering
+Certain formats allow for multi-sampling and/or linear filtering with reasonable quality
-Alpha-blending usually isn't an option, since the alpha-channel is used by most formats
-Linear filtering and multisampling usually isn't mathmatically correct, although often the results are "good enough"
-Additional shader math needed for format conversions
-Adds complexity to shaders
My early prototyping used a standard tone-mapping chain and I didn't want to ditch that, nor did I want to move away from what I was comfortable with. This pretty much eliminated the second option for me off the bat...although I was unlikely to choose it anyway due its other drawbacks (having nice HDR bloom was something I felt was an important part of the look I wanted for my game, and in my opinion Valve's method doesn't do a great job of determining average luminance). When I tried out the first method I found that it worked as well as it always did on the PC (I've used it before), but on the 360 it was another story. I'm not sure why exactly, but for some reason it simply does not like the HalfVector4 format. Performance was terrible, I couldn't blend, I got all kinds of strange rendering artifacts (entire lines of pixels missing), and I'd get bizarre exceptions if I enabled multi-sampling. Loads of fun, let me tell you.
This left me with option #3. I wasn't a fan of this approach initially, as my original design plan called for things to be simple and straightforward whenever possible. I didn't really want to have two versions of my material shaders to support encoding, nor did I want to integrate decoding into the other parts of the pipeline that needed. But unfortunately, I wasn't really left with any other options after I found there were no plans to bring the support for the 360's special fp10 backbuffer format to XNA (which would have conveniently solved my problems on the 360). So, I started doing my research. Naturally the first place I looked was to actual released commercial game. Why? Because usually when a technique is used in a shipped game, it means it's gone trhough the paces and has been determined to actually be feasible and practical in game environment. Which of course naturally led me to consider NAO32.
NAO32 is a format that gained some fame in the dev community when ex-Ninja Theory programmer Marco Salvi shared some details on the technique over on the beyond3D forums. Used in the game Heavenly Sword, it allowed for multi-sampling to be used in conjuction with HDR on a platform (PS3) whose GPU didn't support multi-sampling of floating-point surfaces (The RSX is heavily based on Nvidia G70). In this technique, color is stored in the LogLuv format usinga standard R8G8B8A8 surface. Two components are used to store X and Y at 8-bit precision, and the other two are used to store the log of luminance at 16-bit precision. Having 16 bits for luminance allows for a wide dynamic range to be stored in this format, and storing the log of the luminance allows for linear filtering in multi-sampling or texture sampling. Since he first explained it other games have also used it, such as Naughty Dog's Uncharted. It's likely that it's been used in many other PS3 games, as well.
My actual shader implementation was helped along quite a bit by Christer Ericson's blog post, which described how to derive optimized shader code for encoding RGB into the LogLuv format. Using his code as a starting point, I came up with the following HLSL code for encoding and decoding:
// M matrix, for encoding
const static float3x3 M = float3x3(
0.2209, 0.3390, 0.4184,
0.1138, 0.6780, 0.7319,
0.0102, 0.1130, 0.2969);
// Inverse M matrix, for decoding
const static float3x3 InverseM = float3x3(
6.0013, -2.700, -1.7995,
-1.332, 3.1029, -5.7720,
.3007, -1.088, 5.6268);
float4 LogLuvEncode(in float3 vRGB)
{
float4 vResult;
float3 Xp_Y_XYZp = mul(vRGB, M);
Xp_Y_XYZp = max(Xp_Y_XYZp, float3(1e-6, 1e-6, 1e-6));
vResult.xy = Xp_Y_XYZp.xy / Xp_Y_XYZp.z;
float Le = 2 * log2(Xp_Y_XYZp.y) + 127;
vResult.w = frac(Le);
vResult.z = (Le - (floor(vResult.w*255.0f))/255.0f)/255.0f;
return vResult;
}
float3 LogLuvDecode(in float4 vLogLuv)
{
float Le = vLogLuv.z * 255 + vLogLuv.w;
float3 Xp_Y_XYZp;
Xp_Y_XYZp.y = exp2((Le - 127) / 2);
Xp_Y_XYZp.z = Xp_Y_XYZp.y / vLogLuv.y;
Xp_Y_XYZp.x = vLogLuv.x * Xp_Y_XYZp.z;
float3 vRGB = mul(Xp_Y_XYZp, InverseM);
return max(vRGB, 0);
}
Once I had this implemented and worked through a few small glitches, results were much improved in the 360 version. Performance was much much better, I could multi-sample again, and the results looked great. So once again things didn't exactly work out in an ideal way, but I'm pleased with the results.
One of the early goals I set for myself when starting the project was that I wanted to make some use of modern graphics techniques, and really try to squeeze some performance out of both the 360 and PC GPU's. This meant some decent shaders, HDR, 4xMSAA 1280 x 720 @ 60fps, and of a good shadowing implementation. Shadows are a topic that I keep near and dear to my heart, so I'm going to talk about them in my first entry.
When first planning out my renderer, I came up with the following choices for my shadow implementation:
- Stencil shadows
+Pixel-perfect shadows, no resolution issues
-Requires multi-pass for any lights needing shadows
-Hard edges are ugly!
-Softening is very expensive
-Tied to geometric complexity
-Not really any active research on the subject...even Carmack ditched them - Shadow maps, with standard PCF
+Very commonly used, lot's of known optimizations
+Can soften edges and give good filtering when enough samples are used
+Can be used with an R32F texture, which gives good precision
-No hardware filtering
-PCF samples are incredibly expensive on the 360, which has very limited bandwidth
-Biasing artifacts are very common - Variance Shadow Maps
+Can use hardware filtering
+Can be used with multi-sampling, which is "free" on the 360
+Can be pre-filtered, for example with seperable gaussian blur
-Very precision-hungry, in most cases R32G32F is needed (which doubles bandwidth per read)
-Light bleeding
One trick I've found to be quite effective on the 360 is deferred shadow maps. The technique is one I first saw mentioned by nAo over on the beyond3d forums, and it involves rendering the results of a shadow-map comparison to a texture in a seperate pass before doing the full lighting pass. It basically works like this:
-Have all geometry render linear depth to a screen-sized texture
-Render the shadow-map
-In a full-screen pass: sample depth from the depth texture, reconstruct view space, convert to light-space, and compare depth with the shadow-map
-Use the shadow occlusion texture to attenuate lighting in the actual main pass
The reason it's a win performance-wise, is because when you perform a full-screen pass you ensure that every quad of pixels is fully utilized. This isn't the case when you're rendering a bunch of 3D geometry, since the triangles often won't overlap all 4 pixels of a quad. It's all very nice from a pure design standpoint: your shadowing is completely decoupled from your lighting. Your fancy normal-mapping or sub-surface scattering don't need to be aware at all of what shadowing method you're using, it just needs to sample the final attenuation value from a texture. Plus you don't need to have two versions of those shaders: if shadows are disabled, you can just feed the lighting shaders a 1x1 white texture. Having a depth texture available also usually isn't a big deal, since pretty much every modern engine needs depth for just about every friggin' effect.
As of now with this shadowing system in place, I have everything running at a little over 60 fps with a 1024 x 1024 shadow map being rendering with 4xMSAA. In order to keep my framerate high enough to remain at a steady 60fps with vsync, I'm defintely going to need to find some more optimizations and the shadowing system is certainly somewhere I'm going to come back to. However at the moment, I'm quite satisfied with the results.
]]>Since the stuff I'm working on for XNAInfo is taking forever to finish, I thought I'd post a link to this little gem here in the meantime. On my forum rounds I find myself linking to Tom's Excellent DirectX Faq at least once a week. Obviously it's DirectX specific, but that easily translates to tons of useful information on XNA development on Windows. It's a great resource that should prove useful for just about anyone.
]]>
Some time ago a topic popped up on GameDev on how to stick a WebBrowser control on a D3D surface, or rather an XNA texture level. After some tinkering we got it to work (obviously Windows only), so it can be used to render any webpage to an XNA texture. The next step was to try and make it interactive, paving the way for HTML and Flash based GUIs. Unfortunately we ran into a strange bug here, which I haven't been able to solve. So I figured I'd post this out here, hoping anyone comes across this who can help out.
Here are the (messy) demo projects:
-
Original browser-to-texture code (useful if you're just looking to draw HTML in XNA)
The bug surfaces in the 2nd project. Basically it works fine until the user left-clicks anywhere on the control (doesn't have to be a link), after which WebBrowser.DrawToBitmap fails silently and only an empty white bitmap gets rendered. The strange thing is that the WebBrowser control does load the new webpage. By uncommenting line 241, mouse moves are posted to the control and the window title will show the HREF of links on the (loaded but invisible) page as you hover over them.
Anyway, perhaps someone better versed in window messages can check this out, see if the code for posting mouse presses/releases is correct. While researching this problem, I also came across this MSDN page which states DrawToBitmap isn't supported for the WebBrowser control anyway, so it seems a miracle it works in the first place. If anyone cares to comment on that, please let's hear it.
]]>In a recent discussion on RTS AI, Matias Goldberg pointed out the quite ancient text The Art of War by Sun Tzu. Since I've been fostering an RTS pet project for years now, I decided to get a copy and it turned out the best $4.95 I probably ever spent (ISBN10 0-486-42557-6).
Obviously it's not a cookbook for writing RTS strategies, but after a first read I can definitely see how various stratagems and axioms could be used to construct a clever AI. In fact, I'm amazed how many clear cut rules of thumb Sun Tzu puts forward that could be readily applied. These however may only serve to highlight the limited scale on which popular RTS games play out.
Food for thought at any rate.
]]>- Debugging shaders and profiling with PIX. I can't tell you how many times I've replied to a thread in the gamedev.net forums by saying "just step through your shader in PIX!". PIX is undoubtedly a huguely useful tool (even when it's occasionally crashing), and I think it's important that beginners get comfortable with it early on. There's documentation in the SDK, but I think a lot of beginner XNA programmers don't think to look through there and therefore don't understand how powerful PIX can be.
- HDR and tonemapping. It's probably pretty advanced for your average XNA project, but thesedays everybody wants to know how to do it. There's some great samples in the SDK but once again I'm not sure all XNA programmers want to go poking around in there, especially if they don't know C++. Plus there's always cool things I could throw in, like the LogLuv encoding scheme Rim and I worked out.
- Skyboxes. Another topic I see brought up all the time on the gamedev forums. It's a very very simple technique once you understand how it works, but unfortunately I don't know of any tutorials devoted to that topic. Plus there's always a few advanced things I could throw in there...like RGBE (or LogLuv) encoding for HDR, or comparing DXT compression vs. uncompressed.
- An article or maybe even a series about map editors. I've been working on a map editor for my own project, and I'm constantly amazed at all the cool stuff you can do with .NET when your engine is made with managed code.
- A follow-up article for my ModTool tutorial that discusses instancing. There's an excellent Instancing sample on the creator's club website, however integregating the the technique with the content authoring system I described can be tricky (it requires modifying the shaders quite a bit, and also requires making some changes to the content processor).
]]>Get your feet wet
When you're just starting out everything seems hard and no amount of reading is going to do you much good until you sit down and just take a shot at it. Don't worry if you don't create an AAA game on your first try or even when you hit a wall and don't know how to continue (I'll get back on that). The key is, you've at the very least got some basics down and tried out various things that did or did not work. Congratulations, you've now learned something and are no longer a n00b.
The best language
Please stop worrying about what would be the best programming language, please! If you even vaguely know how to code in some language, just stick with that unless you feel you absolutely need to move to another language since it offers something you really can't live without. Programming languages may all have their pros and cons (in no small part due to the libraries they'll allow you to use), but getting bogged down learning hard language X isn't going to help you create your game. Most coding skills you aquire carry over pretty well and should make it a lot easier to learn other languages anyway. And though it may be a fact that language X is the current industry standard, should you really care about that now?
Getting into the industry
Or rather, do you really want to be in the industry right now or in the near future? It's a nice dream to create games for a living and certainly worth hanging on to, but when you're just getting started you really shouldn't be worrying about the industry too much. Just have a go at creating some games or at least coding some fancy things to get a feel for it. Then if coding games really turns out to be the carreer what you want, you will still have plenty of time to brush up on industry standards and how to actually get in there. In the meantime, you'll have a bit of a portfolio layed down which just might show off your experience and passion better than stating you know language X. Here's a little anecdote.
Getting stuck
So you went and started coding your game, but you've hit a wall. That's ok, it means you have something new to learn. Research the problem, ask around on forums and try to get it working. Even if it turns out you've created a monolithic unwieldly behemoth of a program or that something's deeply flawed in your approach, you'll at least know to avoid it in the future and you can start fresh with your newly gained knowledge. But chances are things aren't this bad and you'll have learned how to solve the (perhaps common) issue.
Well, that about covers it for now. I'm sure more ramblings will be forthcoming 