Devs share insight into Celeste's dialogue design process

Occasionally, we'll be intrigued by stories of how the gaming sauce is made. For Shacknews, we loved indie platformer Celeste. We'll still rave about it and any chance to learn something new about it, we'll take it. In this case, we recently had a chance to learn about the making of Celeste's dialogue design directly from one of the game's developers.

Power Up Audio Creative Director Kevin Regamey originally helped put Celeste's dialogue together. Over the weekend, he outlined how he went about this in an informative Twitter thread.

First, we explored some simple synth sounds to figure out a general tone for a given character's voice. Once we had a foundation timbre established, we moved on to setting up how that tone might change over time.

Essentially, using a Parametric EQ in FL Studio, we modelled what are called "formants" - that is, naturally occurring spectral peaks in human vowel sounds. These spectral peaks have specific frequency positions and relationships

We then automated the frequency positions of those peaks over time, to resemble the way the human voice might transition between vowel sounds.

Then, we broke down the emotional range of a given character, and figured out what the sonic characteristics of those emotions might be.

Is the emotion generally high or low in pitch? Does the pitch stay in one place, or move around a lot? Is the speech slow and careful, or is it rapid and pointed? What kind of pitch movement should we hear, and what kinds of sentences are these sounds representing?

This was basically a lot of me reading the script and "over acting" the dialogue.

We then pressed PLAY in FL Studio (to set the formant automation in motion), and then "performed" the emotions on a midi piano keyboard while messing with the pitch wheel to try to hit all those characteristics we'd laid out above.

We recorded the audio output of that performance on a separate computer, and then went through and picked out the "good takes". These good takes were sorted into 3 basic categories per emotion: Quick passing syllables (20), Emphasized syllables (10), Sentence endings (10)

Using @fmodstudio, we set up an emotion-driven (actually character portrait-driven) event system... We had one audio event per character, and each dialogue event plays non-stop for the duration of a given character's conversation in-game.

We loop on silence until it's that character's turn to speak. When that character speaks, the code-side sends the current character portrait info to FMOD, and FMOD responds by sending the playhead to the appropriate emotion on the event timeline.

When we arrive at that emotion, we progress through a random sequence of syllables via a transition system we built. FMOD doesn't have a great way of handling this degree of specificity when moving through a random sequence of audio files, so making this was pretty tedious.

It necessitated manually placing every syllable on the timeline, and setting up a "transition hub" that would rapidly send the playhead out to these various syllables. No syllable was allowed to play twice in a row, and emphasized syllables had a lower probability of playing.

The hub also made it trivial to fine-tune the spacing between syllables (as we could move entire hub's position rather than forty separate transition markers).

Anyway, once the text-draw ends in the game's UI, FMOD is notified that speech should conclude. The playhead then returns to silently waiting for the next portrait/emotion. (It either plays an end syllable before returning, or it returns immediately if on an emphasized syllable).

The Twitter thread itself offers much more, in terms of visual and audio aids. Those interested in checking out the audio side of game development may want to give that a look. Power Up Audio also has a YouTube channel filled with fascinating videos. For your convenience, we've embedded a video from January 2018, days before Celeste originally released.

Power Up Audio is currently working on the audio for the upcoming Darkest Dungeon 2, which is set to release on Steam Early Access later this year. While Extremely OK Games has moved on to its next project, the mysterious Earthblade, we'll always be on the lookout for more to say about Celeste, so keep it on Shacknews for more fun stories and anecdotes.