What the browser makes possible

The platform catches up

The PhD implementation exercised the architecture but fought its platform: JOAL’s positional audio was weak enough that three of my seven metaphors had to retreat from spatialization to musical or spoken encoding. Two decades later the platform has caught up. The Web Audio API offers PannerNode with head-related transfer function (HRTF) processing in every modern browser: true binaural positioning on ordinary headphones, inside a web page, with no installation at all. That changes what the sonic view can attempt, and it is the foundation of the demonstrator this case study now serves.

The design that follows assumes headphones and spatial audio, and it assumes users who retain stereo hearing across the musical registers. As always, the deeper principle is that the user decides how to experience the game. The Russian folk music can go and dance elsewhere: the functional audio is the soundtrack.

Three categories of information

Audio is sequential, so the design must ration it. Every piece of game information falls into one of three delivery categories, and keeping them distinct is what prevents replacing visual overload with auditory overload:

• Ambient/persistent: always present, continuously updated. The active piece’s position and identity, the urgency state.
• Event-driven: fires on change. Rotation, landing, line clear, level up.
• On-demand: player-queried when they have cognitive bandwidth. The silhouette scan, the next piece, the held piece.

Mapping the five in-fall channels: the current tile and its ghost relationship are ambient; the silhouette is on-demand plus an automatic replay after each lock; next and held are on-demand. The braided-audio insight survives intact. It has simply become a scheduling policy.

The sonic palette

Each tetromino gets a persistent tonal identity, not a sound effect but a timbre: the I-piece a pure sine, the O a warm pad, the T a plucked string, S and Z a detuned pair expressing their mirrored tension, J and L related but mirrored instruments. Horizontal position maps to stereo azimuth, the mapping that needs no learning at all. Descent maps to a falling pitch or closing filter: gravity as waterfall, rebuilt from oscillators. Rotation steps through a four-interval motif, so each orientation is heard as a chord position. The ghost is the piece’s own timbre processed into an echo, quiet, reverb-heavy, panned to the landing column, with the echo delay shrinking as the piece approaches it until the two sounds converge into an intuitive “about to land”. The musical sonar returns as the terrain scan: the ten columns played as a rapid arpeggio, pan giving where and pitch giving how high. Raggedness is heard as dissonance, a nearly-complete line as a smooth scale: musical harmony as board state. Urgency is a heartbeat whose tempo tracks fall speed; the lock timeout is a distinct accelerating tick; danger shifts the whole soundscape from major to minor pentatonic. Anchoring everything to a pentatonic scale keeps the emergent composition pleasant rather than cacophonous. Line completion resolves it: a horizontal sweep, harmonically stacked by line count (unison, octave, chord, and a full triumphant arpeggio for a Tetris), followed by a replay of the new terrain and a breath of silence before the next piece.

A skilled player keeping a clean board produces calm consonance; a struggling player produces dissonance and driving tempo. The game sounds like how well you are playing, an intuitive, emotional read on game state that costs the player no parsing at all.

Three first-person views

The polar-coordinate realization becomes explicit architecture. The game’s information space decomposes into three switchable first-person listening perspectives.

The Wall is the side view, and the default, because it is how a novice knows Tetris. The ten columns map onto an arc with the listener at its centre: every column equidistant (no volume bias), maximally separated in azimuth (the ear’s best dimension), with height carried literally in elevation. It excels at tracking the fall. It struggles with pattern-matching, which from the side is like judging whether a key fits a lock viewed edge-on.

The Well is the top-down view. The listener looks down into the well; the piece is near, the silhouette surface below, and a sonar ping’s return time gives the gap depth per column. The sonar metaphor is native to this view. The piece’s shape becomes a horizontal sound-pattern to align with a horizontal gap-pattern, which is a far more tractable auditory task, and rotation is heard as the footprint physically rearranging. This is the strategist’s view, at the cost of the fall’s kinetic urgency.

Mission Control is the dashboard: fixed spatial stations for held piece, next piece, level, lines, and score. It is a snapshot, not a stream, visited in the natural pause after a lock, just as a sighted player glances at the score panel.

Switching is by dedicated key, heralded by a short 3-D earcon, brief enough not to affect gameplay. The game does not pause on switch by default (knowing when to glance is part of the game), but pause-on-switch exists as an option: the game is complex enough, and the unadapted original deserves respect. Views can also be layered, a primary view at full level with another bled in quietly behind, which the mixing model below makes free.

Psychoacoustic ground rules

The design is constrained throughout by what ears actually resolve, and my old field notes agree with the textbook figures to a degree I did not expect:

• The X-axis is king. Horizontal discrimination is around 1–2°; ten columns across a frontal arc are comfortably discriminable. The most important game dimension gets the most precise perceptual dimension.
• Elevation is coarse. Vertical discrimination is perhaps 10–20°, in practice about five bands: very high, high, middle, low, very low. Twenty rows collapse to five perceivable zones, so the design works in zones (Sky, Upper, Middle, Lower, Ground, each with its own timbral character) and lets pitch carry the fine-grained height, with brightness and tempo as further redundant cues. No single perceptual channel carries critical information alone, and the mixing desk lets each player weight the cues that work for their ears.
• Avoid pure front/back. Sounds mirrored across the interaural axis produce nearly identical cues: the cone of confusion. Nothing critical sits at exactly 0° or 180°, and every Mission Control station is offset (±40° behind, ±30° front-below) so no two stations share a cone.
• The wobble. In life, the brain resolves front/back by micro head movements, which don’t happen wearing headphones at a screen. So the game space imperceptibly wobbles: a slow, irregular figure-eight oscillation of the audio listener’s orientation, the two axes at incommensurate frequencies (so the pattern never repeats and is never heard as rhythm), amplitude a subliminal 2–4°. This is the direct descendant of my dancing margins. In the PhD build I moved the sources to help the ears; now I move the listener, and nobody has to hear it happening.

The player’s mixing desk

User agency gets its own architecture. The soundscape is built as independently controllable layers (core piece, ghost, terrain, queue, scoring, tension, events), each with an on/off toggle, volume, density (full ten-column scan versus an abbreviated three-column one), and spatial spread. Presets bundle them: Minimal (piece and events only, to learn the basics), Standard (piece, ghost, urgency, events), Full (everything), Custom (the whole desk). A new player starts minimal and adds layers as their audio literacy grows; an expert runs the full soundscape. Player-defined information density is player-defined difficulty. It is also the manual-template adaptation of the architecture, with the player as the final authority on their own capability model. The old People/Capabilities/Preferences model becomes the settings store behind the desk.