LearnInto 3D
Lesson 19 of 30 · 55 min
Rays and cameras
Cast a ray per pixel, hit floors and spheres with math, and move a camera
In this lesson you will
- Understand the idea of shooting a ray from the eye through each pixel to decide its color
- Build a ray from
roandrd = normalize(vec3(p, 1.5)), and check its direction as a color - Solve for where a ray meets a floor (a plane) and a sphere, draw a checkered floor and a ball, and fade the distance with fog
- Relate focal length to field of view, and use
camera(ro, ta)to place and aim the camera anywhere - Orbit the camera with time or the mouse to show a scene in depth
01From 2D to 3D
Everything you have drawn in the last 18 lessons lived on a flat screen. This lesson takes us into 3D. The machinery of a shader does not change at all, though. Every pixel still answers "what color am I?" on its own. What changes is how it works out the answer.
Picture taking a photo. A little in front of your eye (the camera) stands the screen, like a pane of glass. The color of one point on that screen is decided by what the line of sight from your eye through that point hits first. If it hits a tree beyond the window, the point is the tree's color; if it hits nothing, it is the sky's color. That line of sight is called a ray.
A ray is described by two things: where it starts and which way it points. By convention we call the start ro (ray origin) and the direction rd (ray direction). Both are vec3s, groups of three numbers. The point you reach after traveling a distance t along the ray is ro + rd * t. With t at 0 you are at the eye; with t at 5 you are 5 units down the line.
1
2
3
vec3 ro = vec3(0.0, 1.0, -3.0); // where the eye is
vec3 rd = normalize(vec3(p, 1.5)); // the direction through this pixel
// points on the ray are ro + rd * t (t is the distance traveled)
In 3D we take x to point right, y up, and z into the scene (away from you, through the screen). If the eye sits at the origin and the screen stands 1.5 units in front of it, a screen point p is at vec3(p.x, p.y, 1.5) in 3D. The arrow from the eye to that point is exactly the ray's direction. vec3(p, 1.5) is the short way to write it.
Let's put in numbers. For the pixel at p = (0.5, 0.2), the arrow is (0.5, 0.2, 1.5). normalize keeps a vector's direction but scales it to length 1. This arrow's length is √(0.25 + 0.04 + 2.25) ≈ 1.59, so dividing all three numbers by it gives rd ≈ (0.31, 0.13, 0.94). With a length of 1, t really is the distance traveled, which makes the math that follows much easier.
02Seeing ray directions as color
When 3D gets confusing, start by turning intermediate values into color. The debugging habit from Knowing where you are pays off most here. Each component of rd runs from -1 to 1, so we map it to 0–1 with * 0.5 + 0.5 and use it directly as red, green and blue.
1
2
3
4
5
1.5 to 0.5 and the colors spread much more; at 5.0 the screen is nearly one color. Why?The center pixel has p = (0, 0), so rd = (0, 0, 1): straight ahead. As a color that is (0.5, 0.5, 1.0), a pale blue-violet. Moving right, rd.x grows and red comes in; moving up, rd.y grows and green comes in. The blue fades a little near the edges because a ray that tilts sideways has less left over for rd.z, its forward part.
Knowing the directions is enough to draw a sky. Rays that point up (large rd.y) get a deep blue, and rays close to horizontal get a lighter color. A ray with negative rd.y points down, so it gets the ground color. The boundary between the two, the horizon, appears where rd.y = 0: exactly halfway up the screen.
Let's add a sun. This calls for a new function, dot (the dot product). dot(a, b) computes a.x * b.x + a.y * b.y + a.z * b.z, and for two arrows of length 1 it tells you how closely their directions agree: 1.0 for the same direction, 0.0 for a right angle, -1.0 for opposite directions. Ten degrees apart gives about 0.985; four degrees gives about 0.998. The closer dot of the ray and the sun direction sunDir is to 1, the closer that pixel is to the sun.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
0.2 in sunDir to 0.02 to set the sun on the horizon. Change 24.0 to 4.0 and the glow spreads wideTip
pow(s, 24.0) keeps only the values of s that are close to 1 and pushes everything else quickly toward 0. At s = 0.99 you get about 0.79; at 0.9, about 0.08. The bigger the exponent, the tighter the glow. It is the same use of pow as in Shaping functions.
03Hitting the floor with algebra
Next, let's turn the plain ground color into a real floor. Say the floor is a flat surface (a plane) at height 0: the set of all points whose y is 0. We put the eye at height 1, ro = (0, 1, 0). What we want to know is how far the ray must travel before its height reaches 0.
The height of a point on the ray is ro.y + rd.y * t. Rearranging, the t that makes it 0 is t = -ro.y / rd.y. For rd.y = -0.25 (tilted slightly down), t = -1 / -0.25 = 4: the ray hits the floor 4 units out. For rd.y = -0.05 (almost level), t = 20, much farther. As rd.y approaches 0, t grows without limit, and the horizon is infinitely far away. A ray with positive rd.y points up and never meets the floor (its t would come out negative).
Once you have the hit point pos = ro + rd * t, its pos.x and pos.z are 2D coordinates on the floor. Draw the checkerboard from Repetition and tiles on them, just as before.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
ro to vec3(0.0, 3.0, 0.0) to raise the eye and watch the squares shrink. Set it to vec3(0.0, 1.0, time) to glide forward over the floorIt is a flat picture, yet it reads as a floor stretching away. Near squares are big, far ones small, squeezing toward the horizon. That is perspective. We never drew perspective; we only shot rays. Distant things look small because a square of the same width catches fewer and fewer rays the farther away it is.
04Fog in the distance
Look near the horizon in the last preview: the pattern is grainy and shimmery. Far-off squares are smaller than a pixel, so whether a ray lands on a light or dark square comes down to chance. This is called aliasing.
The simplest and prettiest fix is fog. In real air, too, distant mountains look pale and hazy. The larger the distance t to the hit, the closer we push the color toward the sky color. The exp you used for glow in Light and finishing does the job again. 1.0 - exp(-t * 0.08) is about 0.27 at t = 4, about 0.8 at 20 and about 0.98 at 50: the farther away, the closer to 1.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
0.08 to 0.3 for thick fog, or 0.02 for a clear day. Change time * 2.0 to time * 8.0 to race alongFor rays pointing down, clamp makes the sky color the horizon color (0.95, 0.8, 0.65). So the fog color matches the bottom of the sky exactly, and the floor seems to melt into it. Matching the fog to the sky is the cheapest trick there is for making a 3D scene look good.
05Hitting a sphere with algebra
After the floor, a sphere. A sphere's surface is the set of points whose distance from the center ce is exactly the radius ra. We look for the t that puts the ray point ro + rd * t on it.
Write oc = ro - ce (the arrow from the center to the eye). The squared distance from a ray point to the center is dot(oc + rd * t, oc + rd * t). Set it equal to ra * ra and expand, and because rd has length 1 you get t * t + 2.0 * b * t + c = 0, where b = dot(oc, rd) and c = dot(oc, oc) - ra * ra. That is a quadratic equation, and its solutions are t = -b ± sqrt(b * b - c).
1
2
3
4
5
6
7
8
float hitSphere(vec3 ro, vec3 rd, vec3 ce, float ra) {
vec3 oc = ro - ce;
float b = dot(oc, rd);
float c = dot(oc, oc) - ra * ra;
float h = b * b - c;
if (h < 0.0) return -1.0; // a miss
return -b - sqrt(h); // the near hit
}
t to the hit, or -1.0 for a miss- If
h = b * b - cis negative, there is no square root and no answer: the ray passes the sphere by - There are two answers: where the ray enters the sphere (near) and where it leaves (far). We see the near one, so we take the smaller,
-b - sqrt(h) - A negative
tmeans the sphere is behind the eye, so checkt > 0.0before using it
Let's check with numbers. Eye at ro = (0, 1, -3), sphere center ce = (0, 1, 0), radius 1. For the straight-ahead ray rd = (0, 0, 1): oc = (0, 0, -3), b = -3, c = 9 - 1 = 8, h = 9 - 8 = 1. So t = 3 - 1 = 2: two units out, at z = -1, the front of the sphere. Correct. The other answer, t = 3 + 1 = 4, is the back.
Tip
Don't worry if you can't follow every step of the algebra. If you know how to use it, "if h is negative it's a miss, otherwise -b - sqrt(h) is the distance", you can keep going. And next lesson you will learn a way to draw shapes with no equations like this at all.
First, just the sphere. Where a ray hits, we make it brighter the nearer the hit is. A ray hitting the middle has t = 2, and a ray grazing the rim has t of about 2.83, so 1.0 - (t - 2.0) gives 1 in the middle and about 0.17 at the rim. Distance alone is enough to suggest roundness.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
vec3(sin(time), 1.0, 0.0) and it slides side to side. What if the radius 1.0 becomes 0.5?Now put the floor and the sphere in the same scene. A single ray can hit both. When it does, the nearer one is what you see. That everyday fact, near things hiding far things, is decided by comparing the distances t. We keep the nearest hit found so far in t, and whenever something nearer turns up, we overwrite it along with the color. To handle rays that hit nothing, t starts out at a large value (100 here).
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
ce to vec3(0.0, 1.0 + abs(sin(time * 2.0)), 0.0) to make the ball bounce. Swapping the order of the floor and ball ifs does not change the picture. Why not?The final mix adds fog to the whole scene. Sky pixels still have t at 100, so the fog amount is almost 1 and they come out as the sky color itself. Keeping t in one variable means a single line handles floor, ball and sky correctly.
The ball's brightness uses pos.y - ce.y, how far above the center the hit point is (-1 to 1). It is a lazy shade that says "surfaces facing up are brighter". You will learn to find the surface direction properly and light it in Normals and lighting.
06Focal length and field of view
We have used the 1.5 in vec3(p, 1.5) all along. It is the distance from the eye to the screen, the camera's focal length. Changing it changes how wide a slice of the world you see: the field of view.
The top of the screen is at p.y = 1. With a focal length of 1.5, the ray through it rises 1 for every 1.5 it goes forward, which tilts it about 34 degrees above level. The bottom is the same, so the vertical field of view is about 67 degrees. A focal length of 1.0 gives 45 degrees each way, a 90-degree view (wide angle); 3.0 gives about 18 degrees each way, a 37-degree view (telephoto). Going the other way, if you want a 60-degree view, the focal length is 1 ÷ tan(30°) ≈ 1.73.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
focal at 0.6 for a near-fisheye wide angle, or at 4.0 and the ball overflows the screen. Then what happens if you change the -4.0 in ro to -10.0?Changing the focal length is like turning a zoom lens. The eye stays put; only the framing widens or narrows. Moving the eye closer or farther is like walking. Shoot wide from up close and you get strong perspective, with near things big and far things small; shoot telephoto from far away and depth gets compressed into a flat-looking picture. In the third exercise you will use that difference for a famous movie trick.
07Aiming the camera
So far the camera has always looked straight ahead, along +z. To look down from above or circle around to the side, we need to turn it. So we write a function, camera, that builds the camera's orientation from the eye position ro and the point we want to look at, ta (the target).
1
2
3
4
5
6
7
8
9
mat3 camera(vec3 ro, vec3 ta) {
vec3 f = normalize(ta - ro); // forward
vec3 r = normalize(cross(vec3(0.0, 1.0, 0.0), f)); // right
vec3 u = cross(f, r); // up
return mat3(r, u, f);
}
// how to use it; 1.5 is the focal length
vec3 rd = camera(ro, ta) * normalize(vec3(p, 1.5));
f(forward) is the direction from the eye to the target:ta - roscaled to length 1r(right) is at right angles to both straight up(0, 1, 0)and forward.cross(the cross product) returns an arrow perpendicular to both of its inputsu(up) is perpendicular to both forward and right: "up" as the camera sees itmat3(r, u, f)is a 3x3 table with these three as its columns, the 3D version of themat2from Moving space
Multiplying a vector by the matrix, mat3(r, u, f) * v, is the same as r * v.x + u * v.y + f * v.z. So camera(ro, ta) * vec3(p, 1.5) means "go p.x along the camera's right, p.y along the camera's up, and 1.5 along the camera's forward". It is the old "p.x along x, p.y along y, 1.5 along z" with the axes swapped for the camera's own three directions. When the camera looks straight ahead, r = (1, 0, 0), u = (0, 1, 0) and f = (0, 0, 1), and you get exactly what you had before.
In numbers: looking from ro = (0, 2.5, -4) at ta = (0, 0.8, 0) gives ta - ro = (0, -1.7, 4), which normalizes to f ≈ (0, -0.39, 0.92), pointing a little down. r stays (1, 0, 0), and u ≈ (0, 0.92, 0.39), so the camera's "up" leans slightly forward. All three are at right angles to each other and all have length 1.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
ro to vec3(0.0, 6.0, -3.0) for a bird's-eye view. Set ta to vec3(3.0, 0.0, 0.0) to look off to the right, past the ballWatch out
If ta sits directly above or below ro, forward lines up with (0, 1, 0) and cross returns an arrow of length 0. A zero-length arrow cannot be normalized, and the screen goes black or flickers. To look straight down, nudge ta slightly to one side.
08Orbiting the camera
With camera, you can move the camera and still keep it pointed at what you care about. Just as you moved a point around a circle in Time and motion, place the eye on a circle in the xz plane (the plane parallel to the floor) with sin and cos, and the camera orbits the ball. At angle a = 0, the eye is at (0, 1.5, -4) relative to the target, the same front view as before.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
time * 0.3 with mouse.x * 2.0 to orbit by moving the cursor left and right. Change both radii of 4.0 to 2.5 to move in closerThere is one ball and nothing in the scene moves. Yet the checkered floor sliding by makes it obvious that the camera is circling. Much of the fun of 3D is that you can choose freely where to view the same scene from. The camera(ro, ta) you built here is used in every remaining lesson of Part 3.
09Exercises
Put the cast-a-ray-and-find-the-hit routine together with your own hands. If you get stuck, open the hints one at a time.
A second ball
Add a second, smaller blue ball (center (1.6, 0.4, 0.6), radius 0.4) to the orbiting scene. Even at angles where it overlaps the big ball, the nearer ball must correctly hide the farther one. Give the small ball its own fake shadow on the floor, too.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
Sunset haze
Turn the scene of gliding over the floor into a sunset. Put the sun straight ahead, just above the horizon, and make the sky a gradient from orange to deep blue. Then make the fog color more orange toward the sun, so the distant floor glows as if lit by the setting sun.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
The dolly zoom
The camera is moving toward and away from the ball. Change the focal length along with it, so that the ball stays the same size on screen while the floor's depth stretches and squashes around it. This is the "dolly zoom", a trick films use for moments of shock.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
10Recap
- In 3D, each pixel shoots a ray from the eye and takes the color of the first thing it hits. Points on the ray are
ro + rd * t rd = normalize(vec3(p, 1.5)). The1.5is the focal length; bigger means a narrower view (telephoto)dotmeasures how closely two directions agree: 1 when they match, 0 at right angles- For a floor,
t = -ro.y / rd.y; for a sphere, a quadratic givest. When several things are hit, the smallesttwins - Blending toward the sky color by
1.0 - exp(-t * k)removes the shimmer and adds depth camera(ro, ta)builds the camera's orientation from the eye position and the target. Move the eye on a circle to orbit
We could draw the floor and the sphere because their hits can be solved with algebra. Doughnut shapes and smoothly blended forms have no such handy formula. Next, in Raymarching, you will extend the distance functions from Drawing with distance into 3D and learn one method that draws any shape the same way.
Finished lessons get a mark in the course contents