Saturday, April 23, 2011

Give Blood

I started writing this post on my 21st birthday (this past Sunday), so it's going to be pretty frivolous and might not teach you anything-but hey, you've been warned. Because it's my birthday, I've been thinking about my successes and failures that I've had growing into the game developer that I am today. A bit of a postmortem of going from being a kid that loved video games and math to heading into my final year of college with more than a few games under my belt.

I decided I wanted to be a game developer when I was in 7th grade, which is now 8 years in the past (I'm a junior in college now). This decision was met with resistance from most people that I knew, but I was really serious about it. Most people thought I'd grow out of it and choose to do something that “made a difference”, especially those who knew that up until that point I was a kid that wanted to make use of my love of math and science at NASA working as a researcher.

Clearly, I didn't grow out of it. I suppose that it didn't help that I was already very aware that it was “nearly impossible” to get into the games industry. With this in mind, I managed to get a bunch of scholarship money at a private school on the other side of town for one reason: computer science classes. I didn't make games in high school (and I'm glad I didn't – I needed to be a kid), but getting serious about programming years before college was one of the best things to happen to me. Exposure to code before college helps so much farther down the line (by the way check out Brett Douville's awesome post about teaching his son about programming). But still, I knew that I had a great deal of work was ahead of me because I knew how hard it was to get a job.

Not that long ago, this video surfaced about game development:

To put my thoughts at the time into perspective, I acted a bit like I had that video running on repeat in my head (even though that particular video didn't exist yet). I made the decision to attend Michigan State University largely for financial reasons, but also because of job opportunities with the game development lab I currently work for. Attending a large university was a bit worrisome to me, given that I didn't think a diverse program could compete with a specialized school like Digipen or Full Sail. I felt like I had to work my ass off to make up to what I would probably be missing from class. I got involved with Spartasoft, MSU's game development club, and vigilantly attended every game jam that year. By the time the year closed, I had worked on no fewer than six games of varying sizes, and was starting to get interested in writing shader code. To say the least, I was working hard and was perhaps overreacting a bit to my worries that I might be getting a “lesser” education.

Freshman year was also the year that I watched many of the seniors that mentored me fail to get jobs. The industry was starting to feel the pains of the recession and things were rough, and it scared the shit out of me. No matter how hard I was working that year, I pushed myself into overdrive the next year. I was involved with a team in a game development competition that ended up winning a trip to GDC paid for by Ford Credit. Being an underclassmen on the team as well as the primary programmer, it would be the first project I seriously crunched for, but I'm glad I did because going to GDC that Spring changed my perspective about everything.

Up until that year, I suffered from something that I suspect afflicts many young programmers. I thought I could be both programmer and designer. I mean design gets all of the glory right? Every kid wants to be the next Miyamoto or the next Ken Levine, and I still reveled in the thought. But that year I would finally get enough experience to realize that design is really hard and you might kill yourself if you try to be both a good programmer and a good designer. Still, I thought maybe gameplay code was the place for me, or a scripting heavy design position. And then GDC hit me like a train hitting a chicken, blowing apart all my thoughts about game development as a career. The fact of the matter is that engine code is really cool, tool development is incredibly important, and hand optimizing assembly code makes you a badass. I had started to become interested in graphics and rendering, and John Hable's presentation about HD rendering in Uncharted 2 convinced me that I wanted to do graphics code professionally. If you get the chance, check out some of his presentations and his website, there's some great stuff there.

Going along with these newfound desires to work on lower level systems, I once again decided I wasn't working hard enough and hurled myself further into my work for the next year. I had the portfolio development class for MSU's game specialization that fall, and I began pulling tons of late nighters and all nighters for my games. I look back on college and realize having a laptop and being able to take my work everywhere with me was both a blessing and a curse. I got to the point where I got a feature implemented into a game while in the back seat of a car on a coffee run. I was out of control, wanting to learn and accomplish so much, so fast. So was it worth it? Now that you've read through several paragraphs of me admitting to crunching increasingly more throughout college, I'm going to finally get to my point.

For the first Spring in a long time, I don't feel like college is a time bomb with only so much time left. Three years of increasingly stepping up my dedication to learning the art of game development has finally lead me to become the programmer I want to be. Maybe I worked too hard at times, but it's no small task to become a programmer cut out for game development in just four years time. The way I viewed life, if you want to get a job in the games industry, you have to become a good game developer, and the only way that that will happen is if you love making games and learning how to make even better games. I went back to GDC this year, and I'll be back again next year, because I love learning about all the crazy techniques people are developing and trying. People talk about how you have to have connections to make it as a game dev (what are we, film?), but I honestly think that's bull shit. I don't want your card because I want a job, I want your card because I want to be friends with people riding the edge of what games can do (speaking of which, props to the Battlefield 3 team). If you want to make games throw yourself at it, because only you can make yourself a crack game developer.

When I'm walking to the lab and I feel like I'd rather go home and sleep through the afternoon, I listen to one song consistently. It's called “Ali vs. Frazier 1” by a Massachusetts hardcore band named Bane. I think it summarizes my view's on what it takes to become successful at anything worthwhile, including game development:

(rumble, young man, rumble)
how many more days will you sit
and talk about your ambitions
all that you can be
the person you are dying to be
the place you want to get to
but always out of reach
before that fury swells inside of you
grows so big that it forever quiets you
stand up to your demons
make a run at your goliath
find the best, find the worst
waiting in both of you
it's not the who or the what that is lasting
but how you fight
that is the fight
the only mark that will not leave you
and I will feel my heart drum its final beat
if it meant that I have given this my all
there's nothing left for me to believe in
if not your, if not this...
what else is there but death?
(it's your's all on you)
give more
give everything
give blood

Tuesday, April 5, 2011

Image Space for Beginners

I'm doing a relatively simple post today because I've had way too many milestones and sleepless nights in the past 7 days. This is why I'm going to talk about a very basic concept for graphics programming: doing image space calculations and effects. I realize that there are many seasoned game developers that frequent this blog, but this one is for the ones who are just getting started.

Tools and Motivation

As a developer of a real-time graphics technology (video games), you are almost certainly going to be making use of a GPU to accelerate your graphics processing. The GPU exploits high parallelization of rendering to speed up its work, and is responsible for transforming your 3D world into a 2D plane that is displayed to the player on their computer scene. However, not all effects are easy to simulate in a 3 dimensional space. Thats where leveraging further work on the GPU to perform additional image space calculations can come in handy. Some of these are quite obvious, such as depth of field calculations, being that depth of field is an artifact of lenses. Other popular image space effects include motion blur, color correction, and anti-aliasing. Image space calculations are also the foundations of deferred shading and lighting, rendering techniques that are becoming increasingly popular to handle a large number of lights in a scene.

The Actual Technique

Image space calculations can be performed on the CPU, but as with most things in graphics that would slow and booorrrriiinng (unless you're playing with SPUs, in which case carry on). The main point is that iterating over an image in the main thread while performing per-pixel calculations on the CPU would most likely be in bad taste. Instead you might consider the following GPU based solution:

  1. Render your scene normally, except to a texture if you are not already.
  2. Render a quad that covers the entire screen, with the texture from the previous step as a texture applied across the quad.
  3. Perform your calculations in the shader code used to render that full screen quad, modifying the how the texture is applied.

There are several catches with this that you have to keep in mind. First and foremost, each fragment's calculations cannot be too dependent on other locations on the screen. If you have to sample your frame many times, then those texture accesses will quickly add up, which is one of the big considerations that comes into play when performing screen space blurs. Secondly, while this might provide an effect for much cheaper than trying to model a similar effect in 3 dimensional space, keep in mind the actual performance is dependent on the resolution of the screen, which may be less than desirable. As resolution increases, so does the number of fragments being processed. In the end, it's all about picking when and where to perform different calculations in your game.

An Example

Here's a sample of a very simple post-processing fragment shader, written in CG. It does a simple screen space distortion based off of the x and y channels of a texture. The vertex shader doesn't do anything particularly special other than set up uvs that are interpolated from 0 to 1 across the quad. In general, most of the action happens in the fragment shader when doing post-processing.

uniform sampler2D _MainTex;

uniform sampler2D _DistortionMap;

uniform float _Distortion;

float4 frag (v2f i) : COLOR


float2 distortedOffset = (tex2D(_DistortionMap, i.uv).xy * 2 - 1);

distortedOffset *= _Distortion;

float2 distortedUV = i.uv + distortedOffset;

return tex2D(_MainTex, distortedUV);


The shader itself is really only a few lines of code! So easy! Here's what is happening a little more in depth. The float2 “distortedOffset” is simply the interpolated uv coordinate plus a lookup into the normal map which is then unpacked to fit into the range [-1,1] instead of the [0,1] range returned by tex2D(sampler2D, float2), which is then finally multiplied by _Distortion to control the strength of the distortion. Finally, a lookup into sampler2D MainTex is performed, where _MainTex is the previously rendered image. If there is no distortion, then the call would be equivalent to tex2D(_MainTex, i.uv), which would just copy the source image's color to the new target. Speaking of targets, you might consider rendering this post-processing pass into a texture as well, besides just your initial rendering of your 3D scene. This is so that you can pump the output of this post-processing into another post process you are implementing to be able to stack effects on top of each other.

Here is a sample of this particular distortion shader in action.
The original rendered scene:

Before Post-Processing

Texture that the distortion is calculated from in the shader:

Distortion Map

The final result:

After Post-Processing


Great success! The important question here should always be: how hard/expensive would it be to achieve the same effect in a different space? What do you gain/lose by doing it in image space? And also important is the question of whether or not this actually makes your game look any better. In the end, I personally think that post processing is great fun, especially used in terrible crazy ways on personal projects. You never know what you'll come up with when you play around with ideas in a different space, for example here's one paper exploring the possibility of moving skin rendering into screen space: Finally, fun fact of the day: I modeled and textured that fish in the sample images way back when I was a Freshman, which is why it's so shoddy and terrible.