Reference notebook

Download .ipynb Open in the browser

“Open in the browser” runs the notebook on your computer with JupyterLite: no installation, no account. Your changes stay in that browser: download the notebook to keep them.

Lesson 07 · Music as data

Fra Martino written once as data, then played in other scales and as a canon: clips, operators and scales, a small language of our own.

How to use this notebook

Run the cells in order. Each section starts from what the previous one built. The Your turn cells are for you: try before reading the answer below them.

Recall: classes, __add__, __eq__, Pitch and mix_sounds come from lesson 06. A name ending in _s is in seconds; without a suffix, time is in beats.

1. Setup

Last time's tools, with from dataclasses import dataclass, which we use today.

from IPython.display import Audio
from dataclasses import dataclass
import numpy as np
from matplotlib import pyplot as plt

SR = 48000

def play(wave):
    return Audio(wave, rate=SR, normalize=False, autoplay=False)

def pluck(f, dur_s=1.0, tau=0.3, amplitude=0.25):
    t = np.arange(round(dur_s * SR)) / SR
    env = np.exp(-t / tau)
    wave = np.sin(2 * np.pi * f * t)
    return amplitude * env * wave

def mix_sounds(sounds):                      # a sound is (onset_s, wave)
    dur_s = max([onset_s + len(wave) / SR for onset_s, wave in sounds])
    out = np.zeros(round(dur_s * SR))
    for onset_s, wave in sounds:
        onset = round(onset_s * SR)
        out[onset:onset + len(wave)] += wave
    return out

note_names = ["C", "C#", "D", "D#", "E", "F", "F#", "G", "G#", "A", "A#", "B"]

class Pitch:
    def __init__(self, value):
        if isinstance(value, str):           # a name like "C4": the octave last, the note before
            value = 12 * (int(value[-1]) + 1) + note_names.index(value[:-1])
        self.midi = value

    @property
    def f(self):                             # the frequency in Hz
        return 440 * 2 ** ((self.midi - 69) / 12)

    def __add__(self, semitones):
        if isinstance(semitones, Pitch):
            raise TypeError("Add semitones, not a pitch.")
        return Pitch(self.midi + semitones)

    def __sub__(self, other):
        if isinstance(other, Pitch):
            return self.midi - other.midi    # an interval
        return Pitch(self.midi - other)      # a lower pitch

    def __eq__(self, other):
        return isinstance(other, Pitch) and self.midi == other.midi

    def __repr__(self):
        return note_names[self.midi % 12] + str(self.midi // 12 - 1)

One change from lesson 06: freq() is now f, with @property on the line above. A line starting with @ is a decorator: it changes the function written right after it. @property lets us read the method like an attribute, p.f, without parentheses: Python calls the method for us.

It suits methods that only read the object and compute something from its state. A pitch has exactly one frequency, so p.f reads well. It is only a matter of notation: the frequency is still computed every time we ask for it. To compute it once, __init__ could store it.

p = Pitch("A4")
print(p.f)
print((p + 1).f)       # A#4, one semitone higher

2. From notes to sounds

The first bar of Fra Martino (Frère Jacques), from a root of A4. Each note is a pair: how many beats it lasts, and how many semitones it is above the root.

root = Pitch("A4")
call_notes = [(1, 0), (1, 2), (1, 4), (1, 0)]     # (beats, semitones)

From this natural description to something we can hear, one step at a time. First the semitones become pitches:

call_pitches = [(beats, root + semitones) for beats, semitones in call_notes]
call_pitches

Then the pitches become waves, with pluck:

call_sounds = [(beats, pluck(pitch.f)) for beats, pitch in call_pitches]
play(call_sounds[1][1])        # the second note

One step is missing. mix_sounds wants sounds (onset_s, wave): when each note starts, not how long it lasts. We would have to turn the durations into onsets, adding them up one by one: 0, 1, 2, 3. It can be done, but it is annoying, and it gets worse with every new thing we want to do: repeat a phrase, play two voices together, start a voice later.

3. What we would like to write

Instead of the steps, let us first write what we would like to write, and then work backwards to make it possible:

clip = note(0) >> note(2) >> note(4) >> note(0)
play_clip(clip)

note(0) is one note, the root, one beat long. >> reads as then: first this, then that. The result is a clip, as in a sequencer or a DAW: a group of notes we can play, join, repeat.

4. Events and clips

A sound is (onset_s, wave): when, in seconds, and a wave. An event is the same idea, one step more abstract: (onset, data). The onset is in beats, and the data is what happens. For now the data is a pitch:

(4.5, Pitch("A#4"))      # an A#4, on beat 4.5

A clip is a set of events and a length, in beats.

Dataclasses

Some classes only hold data. For them Python has @dataclass, another decorator: we list the fields with their types, and it writes __init__, __repr__ and __eq__ for us. A complex number, as an example:

@dataclass
class ComplexNumber:
    real: float
    imag: float

z = ComplexNumber(0, 1)
print(z, z.imag)

The type after each colon documents the field; Python does not check it. Our clip:

@dataclass
class Clip:
    events: list      # (onset, data): onset in beats
    length: float     # in beats: where the next clip starts
clip = Clip([(0, Pitch("A4")), (1, Pitch("B4"))], 2)
clip

Two events, A4 on beat 0 and B4 on beat 1. Why a length of 2? The events do not say how long the clip is: the last note could ring for one beat or for ten. The length is a choice, and it will say where the next clip starts.

Playing a clip

Each event becomes a sound: the onset stays, the pitch becomes a wave. Then mix_sounds puts the sounds together. beat_s is the length of a beat in seconds, the tempo: onsets in beats times seconds per beat give onsets in seconds.

def play_clip(clip, beat_s=0.5):
    sounds = [(onset * beat_s, pluck(pitch.f)) for onset, pitch in clip.events]
    return play(mix_sounds(sounds))

play_clip(clip)

A note is a clip

Now note from our wish is easy: a clip with a single event, at time 0. Its length is the length of the note, one beat unless we say otherwise.

def note(semitones, length=1, root=Pitch("A4")):
    return Clip([(0, root + semitones)], length)

note(0)
play_clip(note(0))

5. Then: >>

Python turns a >> b into a.__rshift__(b). For numbers, >> shifts the binary digits to the right; we do not care: what matters is that we can give it a meaning for clips. a >> b must be a new clip with the events of a and the events of b.

A first try: put the two lists of events together.

a = Clip([(0, Pitch("A4")), (1, Pitch("B4"))], 2)
b = Clip([(0, Pitch("C#5")), (1, Pitch("D5"))], 2)

together = Clip(a.events + b.events, a.length + b.length)
together.events

Both clips have events on beats 0 and 1, so their notes sound at the same time:

play_clip(together)

b must start where a ends: every onset of b moves later by the length of a.

events length
a (0, A4), (1, B4) 2
b (0, C#5), (1, D5) 2
a >> b (0, A4), (1, B4), (2, C#5), (3, D5) 4

The events of a stay where they are. The events of b keep their data, and their onsets grow by self.length, the length of a. The length of the result is the sum of the two lengths.

@dataclass
class Clip:
    events: list      # (onset, data): onset in beats
    length: float     # in beats: where the next clip starts

    def __rshift__(self, other):          # then
        moved = [(onset + self.length, data) for onset, data in other.events]
        return Clip(self.events + moved, self.length + other.length)

a and b were made with the old class, which has no >>: we make them again.

a = Clip([(0, Pitch("A4")), (1, Pitch("B4"))], 2)
b = Clip([(0, Pitch("C#5")), (1, Pitch("D5"))], 2)
(a >> b).events
play_clip(note(0) >> note(1))

One short method, and the wish of section 3 works:

call = note(0) >> note(2) >> note(4) >> note(0)
play_clip(call)
play_clip(call, beat_s=0.25)        # twice as fast

6. Again: *

Fra Martino sings every phrase twice. With lists, [1, 2] * 2 is [1, 2, 1, 2]. With clips?

try:
    call * 2
except TypeError as error:
    print("TypeError:", error)

* calls __mul__. We start from an empty clip, no events and no length, and add self after it as many times as asked:

@dataclass
class Clip:
    events: list      # (onset, data): onset in beats
    length: float     # in beats: where the next clip starts

    def __rshift__(self, other):          # then
        moved = [(onset + self.length, data) for onset, data in other.events]
        return Clip(self.events + moved, self.length + other.length)

    def __mul__(self, times):             # again
        out = Clip([], 0)
        for _ in range(times):
            out = out >> self
        return out

The old call was made with the old class, so we make it again:

call = note(0) >> note(2) >> note(4) >> note(0)
play_clip(call * 2)

Small methods, each a few lines, and a way of writing music emerges: a small language of our own, closer to the music than lists and loops.

7. Degrees of a scale

Fra Martino in minor? With semitones we would have to rewrite the notes: the third, 4 semitones above the root, would become 3. In his First Symphony, Mahler turned this tune into a funeral march: in minor, much slower.

A melody can be written one level higher, in degrees of a scale: the first, second, third note of the scale, counting from 0. note(0) >> note(1) >> note(2) >> note(0) is Fra Martino in a major scale and in a minor one: only the scale changes.

First, what we would like a scale to do. A scale is a root and its steps, the distances in semitones between neighbouring notes. The major scale goes up by 2, 2, 1, 2, 2, 2, 1:

scale = Scale([2, 2, 1, 2, 2, 2, 1], Pitch("A4"))
scale[0]      # A4
scale[1]      # B4
scale[2]      # C#5

To get there, the scale needs the distance of every degree from the root. Each distance is the previous one plus a step: we accumulate the steps.

class Scale:
    def __init__(self, steps, root):
        self.root = root
        self.distances = []
        distance = 0
        for step in steps:
            self.distances.append(distance)
            distance += step

s = Scale([2, 2, 1, 2, 2, 2, 1], Pitch("A4"))
s.distances

[0, 2, 4, 5, 7, 9, 11]: the distances of A, B, C#, D, E, F#, G# from A. Now s[2]:

try:
    s[2]
except TypeError as error:
    print("TypeError:", error)

Not subscriptable: the object does not know what square brackets mean. Python turns s[2] into s.__getitem__(2), and we can write that method too. The pitch of a degree is the root plus the distance of that degree, with the + of Pitch:

class Scale:
    def __init__(self, steps, root):
        self.root = root
        self.distances = []
        distance = 0
        for step in steps:
            self.distances.append(distance)
            distance += step

    def __getitem__(self, degree):
        return self.root + self.distances[degree]

a_major = Scale([2, 2, 1, 2, 2, 2, 1], Pitch("A4"))
assert a_major[0] == Pitch("A4")
assert a_major[1] == Pitch("B4")
assert a_major[2] == Pitch("C#5")
print(a_major[1], a_major[2])

8. Data, not pitches

Our note turns semitones into a pitch straight away: the clip holds pitches. Now we want it to hold degrees, and decide only when playing what they mean.

Clip already allows it: an event is (onset, data), and the clip never looks inside the data. Only note chose pitches. We take that choice away: the data is just stored. Deciding what it means becomes the job of whoever plays the clip.

It is what a sequencer does in a modular synthesizer: it sends out events in time with some values, and the module it is connected to decides what those values mean: a pitch, a volume, anything else.

def note(data, length=1):
    return Clip([(0, data)], length)

call = note(0) >> note(1) >> note(2) >> note(0)
call.events

The events now hold plain numbers. play_clip reads them as degrees of a scale: degree, then pitch, then frequency, then wave.

def play_clip(clip, scale, beat_s=0.5):
    sounds = [(onset * beat_s, pluck(scale[data].f)) for onset, data in clip.events]
    return play(mix_sounds(sounds))

play_clip(call * 2, a_major)

The minor scale goes up by 2, 1, 2, 2, 1, 2, 2. The clip does not change:

a_minor = Scale([2, 1, 2, 2, 1, 2, 2], Pitch("A4"))
play_clip(call * 2, a_minor)

The melody and the way it is played are now independent: the same clip, two scales.

Your turn · Another scale

Play call in A Mixolydian, a major scale with a lower seventh: steps 2, 2, 1, 2, 2, 1, 2. Then in a scale of your own: the steps must add up to 12, to come back to the root one octave higher.

a_mixolydian = Scale([2, 2, 1, 2, 2, 1, 2], Pitch("A4"))
play_clip(call * 2, a_mixolydian)

Fra Martino sounds the same in major and Mixolydian: it never reaches the seventh. The scale matters only for the degrees the melody uses.

9. The whole song

The other phrases, each written once: dormez-vous? climbs to the fifth, sonnez les matines rings like bells.

call = note(0) >> note(1) >> note(2) >> note(0)
climb = note(2) >> note(3) >> note(4, 2)
bells = note(4, 0.5) >> note(5, 0.5) >> note(4, 0.5) >> note(3, 0.5) >> note(2) >> note(0)

fm = call * 2 >> climb * 2 >> bells * 2
play_clip(fm, a_major)

The last phrase, ding, dang, dong, goes down to the fifth below the root: degree 4, one octave lower. Counting down from 0, that is degree −3.

cadence = note(0) >> note(-3) >> note(0, 2)
print(a_major[-3])
play_clip(cadence, a_major)

E5, not E4: the note goes up instead of down. For a list, a negative index counts from the end: distances[-3] is the third from the end, the distance of degree 4. Our scale does not know about octaves yet; we will teach it next time. For now the song ends with the bells.

10. Together: |

A canon: the same melody, sung again by a second voice that enters later. First we need two clips playing at the same time. We would like a | b, with two parallel lines.

Remember the first try at >>: putting the two lists of events together made the notes sound at the same time. That is exactly together. The length is the longer of the two.

@dataclass
class Clip:
    events: list      # (onset, data): onset in beats
    length: float     # in beats: where the next clip starts

    def __rshift__(self, other):          # then
        moved = [(onset + self.length, data) for onset, data in other.events]
        return Clip(self.events + moved, self.length + other.length)

    def __mul__(self, times):             # again
        out = Clip([], 0)
        for _ in range(times):
            out = out >> self
        return out

    def __or__(self, other):              # together
        return Clip(self.events + other.events, max(self.length, other.length))
call = note(0) >> note(1) >> note(2) >> note(0)
climb = note(2) >> note(3) >> note(4, 2)
bells = note(4, 0.5) >> note(5, 0.5) >> note(4, 0.5) >> note(3, 0.5) >> note(2) >> note(0)
fm = call * 2 >> climb * 2 >> bells * 2

play_clip(climb | bells, a_major)

A rest

For the canon the second voice must wait. A rest is a clip with no events, only a length:

Clip([], 8)
play_clip(fm | (Clip([], 8) >> fm), a_major)

The second voice enters 8 beats, two bars, after the first. Clip([], 8) works, but rest(8) says what we mean. Like note, a function that makes a clip:

def rest(length):
    return Clip([], length)

canon = fm | (rest(8) >> fm)
play_clip(canon, a_major)

The parentheses are for safety. Python applies >> before |, so fm | rest(8) >> fm means the same.

Your turn · Three voices, and Mahler

Add a third voice that enters after 16 beats. Then make it a funeral march: in minor, much slower.

canon3 = fm | rest(8) >> fm | rest(16) >> fm
play_clip(canon3, a_major)
play_clip(canon3, a_minor, beat_s=0.8)

11. What we built

expression meaning
note(d, length) a clip with one event
rest(length) a clip with no events
a >> b a, then b
a | b a and b together
a * 2 a, twice
scale[d] the pitch of degree d
  • @property reads a method like an attribute. @dataclass writes __init__, __repr__ and __eq__ for a class that holds data.
  • An event is (onset, data): when, in beats, and what. A clip is a set of events and a length; the length says where the next clip starts.
  • The clip does not look inside the data: what the data mean is decided when the clip is played.
  • A melody in degrees, the scale's steps accumulated into distances, and __getitem__ for square brackets: the same clip in major, minor, Mixolydian.
  • With a few small operators, a small language for music.

For next time · Higher

We would like to write fm + 7: every degree 7 higher, one octave up, for the second voice of the canon.

try:
    fm + 7
except TypeError as error:
    print("TypeError:", error)

Write __add__ for Clip: a new clip with the same onsets and the same length, and every data plus the number. Then try play_clip(fm + 7, a_major). What happens, and why? The cadence and its −3 are part of the same problem. Next time we fix both, and Fra Martino becomes an ambient piece.