I get asked fairly often what people need to know to be hireable as a graphics programmer. I figured it was time to make a page to link instead of re-typing it each time.
We are in a strange time with LLMs. I think ML as it is right now won’t live up to the hype, and the pendulum will swing away from ML a bit over the next couple years. I think the grifters will move onto quantum computing next or find some other thing to pump and dump. However, ML itself does have a place in the computer science tool box, so learning about the fitting and optimization techniques it offers is valuable IMO. I made a video you can watch to learn about the bare metal bits, but it’s up to you if you think it’s worth while to learn or not.
Besides that:
Modern rendering is sort of like two jobs in one.
- Learning the CPU side – Learning DirectX12, Vulkan, Metal, or similar modern “explicit” APIs and the engine programming to support loading assets and other supporting tasks.
- Learning the GPU side – the mathematics of modern lighting and shading, rendering techniques like shadows, ambient occlusion, and post processing effects. Also understanding what is fast and what is slow on the GPU, to know how to make things that run better in real time.
It’s very difficult to learn both things at once. If you want to focus on #2, then you could use a simpler thing for #1, such as opengl, webgl, DirectX11, an engine, or similar. If you want to focus on #1, you should work until you get a first triangle up on the screen, then get a mesh on screen, and so on, but don’t worry about it being very pretty.
Part of #2 is writing a path tracer. Path tracing is how movies do rendering, and it is what we try to approximate with modern real time rendering techniques. A great place to start with a path tracer is this free book online “Ray Tracing in One Weekend”. A lot of people have used it. It’s really approachable and shows you how to make photo realistic renderings.
Another part of #2 is learning “Physically based rendering” or PBR, which is a way of applying lighting (mainly specular, when it comes down to it). PBR is “principled,” meaning if you stick to the rules, you get good results. Before PBR, people wrote random equations for lighting with all sorts of random tweaks and hacks. It made it so you could make an asset that looked good in one situation, but changing the lighting would make it look too dark or would look like it was glowing. People had to make different versions of the assets for different lighting conditions, which was a lot of time and effort.
The PBR section (and subsections) on this page are a great intro to PBR:
Beyond that is the famous PBRT book “Physically Based Rendering: From Theory To Implementation” which is also free online:
Ideally you’ll end up with some source code you can share with prospective employers (like on github, linked to on your resume) to show as proof that you know these things. Something like:
- Something that looks vaguely like an engine in that it loads assets (models, textures) and renders them on the screen in real time, with lighting and a couple effects (shadows, depth of field, area lights, tone mapping, ray traced shadows, whatever). Preferably lit using PBR, with a user controllable camera, and is written using DX12, vulkan or similar, and in C++.
- A path tracer that generates a photo realistic image. Preferably in C++, but it could be a program without a window that just writes a png as output, it doesn’t have to be real time.
- Bonus points if the path tracer is just a separate mode of your “engine-like renderer” and you use it to help verify that your real time PBR rendering is correct, by showing that it matches path traced results. Extra bonus points if you point out where the two renderings don’t match, you can explain why they don’t match, and have thoughts on what you could implement in your real time renderer to make them match more closely while still staying real time.
You might wonder what math you need to know. If you do the above items, you will encounter the math you need to know but basically linear algebra (matrix multiplication, cross product, dot product), basic trigonometry, and a little bit of calculus is really all you NEED. The fun thing about graphics (and game development in general), is that while the amount of math you need is fairly minimal, the amount of math you can use is essentially unbounded.
The same is true algorithmically. You should know the basic abstract data types and algorithms such as linked lists, hash tables, sorting and searching. Often times, the fastest algorithms are the simplest. An array is far faster than a linked list. However, knowing more advanced algorithmic concepts can help you when you really do need something novel and custom.
In game development, C++ is the language to learn. Some people use rust, and it’s hard to tell if rust use is growing or not, but it does have a slice of the pie, while not being the standard language that people expect you to know. WebGPU has a lot of capabilities that WebGL did not have, and it’s becoming more of a serious platform, which lets you work in javascript to do the CPU side of the work. I haven’t seen a lot of WebGPU jobs posted though, and I don’t see a lot of WebGPU content on the web. Knowing C++ seems to be the thing to learn, by far, for CPU side programming.
For shader languages, hlsl seems most common, but some people work in glsl. The shaders are often transpiled to other shader languages in multi platform games.
I’ll keep this page updated as things change or as people ask questions not covered here.
Extended ML Commentary Regarding Agents: