LearnMaking demos
Lesson 28 of 30 · 55 min
Directing with time
Structure a demo with scenes, transitions and beat-synced motion
In this lesson you will
- Split time into scenes with
floorandmod, and get the time elapsed inside each scene - Join scenes smoothly with fades and wipes
- Give motion anticipation and rest with easing and keyframes
- Compute beats from a BPM and make things flash or bounce on the beat
- Make perfect loops that work as GIFs and stream backgrounds
01Splitting time into scenes
A demo (a short piece of real-time visuals, usually set to music) doesn't show the same picture all the way through; it moves through several scenes in sequence. Shaders have no "go to the next scene" command, though. All you have is time, which keeps going up. So we slice time up and work out which scene we're in and how many seconds into it.
floor(time / len)… the scene number, going up by 1 everylensecondsmod(time, len)… the time inside the scene: runs from 0 up tolen, then back to 0mod(time, len) / len… how far through the scene we are, from 0 to 1
For example, with len = 3.0 and time = 7.5, the scene number is floor(2.5) = 2, the time inside the scene is mod(7.5, 3.0) = 1.5 seconds, and the progress is 0.5. Take the number mod 3.0 and you cycle 0, 1, 2, 0, 1, 2 … through three scenes.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
len to 1.0 for hectic switching. Change mod(n, 3.0) to mod(n, 2.0) and the blue scene never shows upTip
Scene numbers are floats, so rather than testing exact equality like id == 1.0, compare against the values in between, like id > 0.5 and id > 1.5. Even if rounding error turns a value into 0.9999, it still lands in the right scene.
02One function per scene
As scenes pile up, cramming them all into render becomes unreadable. Give each scene its own function, like vec3 sceneRings(vec2 p, float t), plus one drawScene that picks a scene by number. Adding or reordering scenes becomes easy.
Each scene receives its own local time through the t argument. Every scene starts at its own second zero, so you can design one scene's motion without worrying about the others.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
return lines in drawScene to change the order of the scenes. Change st in drawScene(mod(n, 3.0), p, st) to time and the motion no longer restarts with each sceneWatch out
Declaring a variable with the same name as a function makes that function uncallable. Name the scene number float scene, for example, and calling a scene(…) function gives a function name expected error. That's why this code calls the number n and the function drawScene.
Tip
Waiting six seconds every time you want to check the third scene gets old fast. Temporarily replace time with (time + 6.0) in floor(time / len) and mod(time, len) to start six seconds in, at the third scene. Change it back when you're done.
03Crossfades
A hard switch, a cut, can feel great, but calmer pieces suit a crossfade, where one scene slowly dissolves into the next. During the last fade seconds of a scene, compute the next scene as well and blend the two with mix.
Pass st - len as the next scene's time. With a 1-second fade, it runs from -1 to 0 while blending and flows straight into the next scene's real second zero at the switch. If you started it at 0 instead, the next scene's clock would jump back a second at the switch and its motion would visibly snap.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
fade to 0.2 for a quick transition. At 3.0, two scenes are blended almost all the timeAnother common choice is the fade through black: darken to black, then bring up the next scene. No blending needed; you just lower the brightness at the start and end of each scene.
1
2
3
// brighten over the first 0.5 seconds, darken over the last 0.5
float k = smoothstep(0.0, 0.5, st) * (1.0 - smoothstep(len - 0.5, len, st));
col *= k;
Tip
The if (m > 0.0) is there for speed: the next scene is only computed while blending, so most of the time you pay for one scene. The condition depends only on time, so every pixel takes the same branch, which costs the GPU next to nothing (more on speed in Fast and small).
04Wipes
A wipe replaces one scene with the next as a boundary line sweeps across the screen. Move the boundary with the transition progress, and show the next scene on one side of it. Unlike a fade, the blend depends on position, not just time.
Move the boundary with mix(-2.1, 2.1, m). Even on a wide 16:9 screen p.x only reaches about ±1.78, so going from -2.1 to 2.1 sweeps from fully off-screen to fully off-screen at any aspect ratio. Change the measured quantity s from p.x to length(p) and you get a circle that grows outward.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
// in front of float s = p.x; and remove the // from one of the two lines below it for a circular or diagonal wipe. Set wipe to 0.4 for a quick sweepA thin line of light rides along the boundary. exp(-abs(s - edge) * 30.0) is 1 on the boundary and drops quickly to 0 away from it. Multiplying by m * (1.0 - m) makes it brightest mid-wipe, so the line doesn't pop in or out at the start and end.
05Easing for expressive motion
In Time and motion we touched on easing: using smoothstep or pow to speed up and slow down. When directing, the easing you choose completely changes a movement's character. All it takes is passing the 0–1 progress k through a differently shaped 0–1 curve.
- Linear (
kas is): constant speed, which feels mechanical smoothstep(0.0, 1.0, k): starts gently and stops gentlyeaseInOutCubic: stronger acceleration and braking, lingering longer at both endseaseOutBack: overshoots a little, then settles back, full of energyeaseOutElastic: wobbles like a spring before coming to rest
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
smoothstep, easeInOutCubic, easeOutBack, easeOutElastic. Change 2.70158 and 1.70158 in easeOutBack to 2.0 and 1.0 for a smaller overshootEvery one of these returns 0 when k is 0 and 1 when k is 1. So whichever you swap in, the start and end positions stay the same; only how it gets there changes. easeOutBack goes slightly above 1 along the way, which is why its ball overshoots the right end and comes back.
Watch out
When writing easing formulas yourself, never pass a number that might be negative to pow, as in pow(x - 1.0, 3.0). GLSL leaves pow undefined for a negative first argument, and some GPUs return black (NaN). Write a cube as a product, like y * y * y.
06Choreographing with keyframes
Even within one scene you'll often want several moves in sequence: slide in, pause, rotate, pause, grow. Animation tools call these keyframes. In a shader, a tiny function that returns where t is within the interval from a to b, as a 0–1 value, makes this easy.
1
2
3
float seg(float t, float a, float b) {
return clamp((t - a) / (b - a), 0.0, 1.0);
}
seg(t, 1.2, 1.8) is 0 before 1.2 seconds, 1 after 1.8 seconds, and runs straight from 0 to 1 in between: at t = 1.5 it is 0.5. Pass it through an easing and use it for position, angle or size, and you decide exactly when each move starts and ends, in seconds. Leaving deliberate pauses between moves makes each one read clearly.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
seg(t, 1.2, 1.8) to seg(t, 0.4, 1.8) and it starts turning while still sliding in. Replace easeOutBack with easeInOutCubic for a calmer entrance07On the beat
For visuals that follow music, compute beats from the BPM (beats per minute). At 120 BPM there are 120 beats a minute, which is 2 per second. time * bpm / 60.0 tells you how many beats have gone by.
floor(beat)… which beat we're onfract(beat)… the position within the beat: 0 right on the beat, close to 1 just before the nextmod(floor(beat), 4.0)… which beat of the bar (4 beats) it is; 0 is the first beatfloor(beat / 4.0)… which bar we're in
A punchy on-beat hit comes from exp(-fract(beat) * 8.0). It's 1 right on the beat and drops off fast, much like a drum that's loud the instant it's struck and fades right away. A bigger 8.0 makes it shorter and sharper. Make the first beat of each bar stronger (an accent) and the four-beat groups start to come through.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
bpm at 90.0 or 174.0. Change the 8.0 in exp(-f * 8.0) to 3.0 for a soft, breathing pulseWatch out
Works on this site can't hear audio. Matching the BPM to a song gets the tempo right, but whether the beats land exactly on the song's beats depends on when the work started playing. For stream backgrounds and the like, a gentle pulse that doesn't mind being slightly off-beat is one good option.
08Moving the camera
In a 3D scene, camera movement is direction in itself. As in film, give each scene (each shot) its own camera setup, and within a shot make just one slow movement; viewers won't get lost. Three moves cover a lot:
- Orbit: circle around the subject to show its whole shape
- Dolly in: move toward the subject to draw attention to it
- Crane up: rise and look down to reveal the space around it
Each of these simply moves ro (the camera position) in camera(ro, ta) from Rays and cameras, using the shot's progress k and an easing. An if on the shot number picks the setup, and shots are joined with cuts.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
len to 6.0 for slower, calmer shots. In the second shot, change the 2.2 in mix(5.0, 2.2, …) to 1.4 to move in closerTip
Keep the camera slower than the subject and things look composed. If both the camera and the subject move fast, viewers can't tell what to look at. When you want speed, hold the camera still and move the subject fast, or the other way around.
09Seamless loops
If you'll export your work as a GIF or keep it running as a stream background, it helps to make it a loop that comes back to exactly the first frame after one cycle. A GIF jumps from its last frame back to its first, so anything that doesn't come back around visibly snaps at the seam.
The trick is not to use time directly. Make an angle ph = TAU * fract(time / LOOP) that goes from 0 to 2π every LOOP seconds, and build every motion from whole-number multiples of ph. Both sin(ph) and sin(ph * 3.0) come back exactly to where they started when ph advances by 2π.
- Swaying and pulsing:
sin(ph)orsin(ph * 2.0).sin(ph * 1.5)doesn't come back around - Rotation: one full turn per loop is
rot(ph). A shape with six-fold symmetry looks identical after a sixth of a turn, sorot(ph / 6.0)works too - Moving stripes and waves: shift by exactly one period (or a whole number of periods) per loop
- Noise: drifting in a straight line never returns, so travel through the noise in a circle (
p + vec2(cos(ph), sin(ph)) * radius)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
LOOP to 5.0 for a slower loop that fits a 5-second GIF perfectly. Change rot(ph / 6.0) to rot(ph / 5.0) and the petals end up at the wrong angle after one cycle, so the picture jumps at the seamBecause fract keeps ph between 0 and 2π, it drops from 2π back to 0 at the end of every cycle. The picture doesn't jump, because every motion was built to look the same at ph = 0 and at ph = 2π. Turn that around: if even one motion still uses time directly, that part won't come back around and the loop breaks.
The GIF export in /edit offers 3-second and 5-second lengths (at 20 frames per second). Pick a loop length that divides the GIF length evenly: 1, 1.5 or 3 seconds for a 3-second GIF, or 1, 2.5 or 5 seconds for a 5-second one. If you work with beats, one beat at 120 BPM is 0.5 seconds, so 6 beats make exactly 3 seconds and 10 beats exactly 5. The GIF starts from the current frame, but a true loop has no seam wherever it starts.
Tip
Loops also help when something runs for hours, like a stream background on /stage. time is a float, and as it grows over hours it loses fine detail, so fast motion like sin(time * 50.0) starts to stutter. It's worse when randomness or noise takes an input that grows with time: as you saw in Making randomness, patterns fall apart once the input to hash21 gets too large (think hash21(floor(time)), or noise on coordinates that drift with time). For works meant to run a long time, wrap time with fract(time / LOOP) or mod(time, period) so you always work with small values.
10Exercises
One exercise each for beats, transitions and loops. All three are techniques you'll use directly when finishing a demo.
A ball that bounces on the beat
Make the ball bounce once per beat at 128 BPM. When it lands it should squash a little sideways, and the floor should flash pink only on the first beat of each bar. The shadow code is already there: it shrinks and fades as the ball rises.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
A circular wipe
This piece cuts between a sunset and a starry night every 3 seconds. Change it so that during the last 0.8 seconds of each scene, the next scene grows out of the center of the screen as a circle. Give the circle a thin glowing rim, and ease how it grows.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
Make it loop in 3 seconds
This piece moves in several ways with time, but never returns to its first frame. Without changing the look or the speeds much, make it a loop that comes around exactly every 3 seconds, so that a 3-second GIF has no visible seam.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
11Recap
floor(time / len)gives the scene number andmod(time, len)the time inside it- Make each scene a function that takes local time; pass
st - lento the next scene so it joins up smoothly - Fades blend over time; wipes replace by position
- Easing sets a move's character;
seglays moves out second by second - Beats are
time * bpm / 60.0;fractgives the position within a beat andexp(-f * 8.0)the hit - Loops move only by whole multiples of
ph = TAU * fract(time / LOOP), with a length that divides the 3- or 5-second GIF
Next time we learn to make works fast and small: where the cost comes from, how to measure it in /edit, and how to fit inside the 8 ms posting check.
Finished lessons get a mark in the course contents