Creating Music by Listeningby
Diplôme d’Ingénieur en Informatique et Télécommunications
M.S. Media Arts and Sciences
Author................................................................................... Certified by............................................................................ Accepted by........................................................................... |
Creating Music by Listening |
Thesis supervisor: Tod Machover, D.M.A.
Title: Professor of Music and Media
Thesis supervisor...................................................................... Thesis reader............................................................................. Thesis reader............................................................................. Thesis reader............................................................................. Thesis reader............................................................................. |
It goes without saying that this thesis is a collaborative piece of work. Much like the system presented here draws musical ideas and sounds from multiple song examples, I personally drew ideas, influences, and inspirations from many people to whom I am very thankful for:
My committee: Tod Machover, Peter Cariani, François Pachet, Julius O. Smith III, Barry Vercoe.
My collaborators and friends: Brian, Mary, Hugo, Carla, Cati, Ben, Ali, Anthony, Jean-Julien, Hedlena, Giordano, Stacie, Shelly, Victor, Bernd, Frédo, Joe, Peter, Marc, Sergio, Joe Paradiso, Glorianna Davenport, Sile O’Modhrain, Deb Roy, Alan Oppenheim.
My Media Lab group and friends: Adam, David, Rob, Gili, Mike, Jacqueline, Ariane, Laird.
My friends outside of the Media Lab: Jad, Vincent, Gaby, Erin, Brazilnut, the Wine and Cheese club, 24 Magazine St., 45 Banks St., 1369, Rustica, Anna’s Taqueria.
My family: Micheline, René, Cécile, François, and Co.
“A good composer does not imitate; he steals.”
– Igor Stravinsky
1-1 Example of paintings by computer program AARON
1-2 Life cycle of the music making paradigm
2-1 Sound analysis/resynthesis paradigm
2-2 Music analysis/resynthesis paradigm
2-3 Machine listening, transformation, and concatenative synthesis
2-4 Analysis framework
2-5 Example of a song decomposition in a tree structure
2-6 Multidimensional scaling perceptual space
3-1 Anatomy of the ear
3-2 Transfer function of the outer and middle ear
3-3 Cochlea and scales
3-4 Bark and ERB scales compared
3-5 Frequency warping examples: noise and pure tone
3-6 Frequency masking example: two pure tones
3-7 Temporal masking schematic
3-8 Temporal masking examples: four sounds
3-9 Perception of rhythm schematic
3-10 Auditory spectrogram: noise, pure tone, sounds, and music
3-11 Timbre and loudness representations on music
3-12 Segmentation of a music example
3-13 Tatum tracking
3-14 Beat tracking
3-15 Chromagram schematic
3-16 Chroma analysis example: four sounds
3-17 Chromagram of a piano scale
3-18 Pitch-content analysis of a chord progression
3-19 Musical metadata extraction
4-1 Similarities in the visual domain
4-2 3D representation of the hierarchical structure of timbre
4-3 Dynamic time warping schematic
4-4 Weight function for timbre similarity of sound segments
4-5 Chord progression score
4-6 Timbre vs. pitch analysis
4-7 Hierarchical self-similarity matrices of timbre
4-8 Pattern length analysis
4-9 Heuristic analysis of downbeat: simple example
4-10 Heuristic analysis of downbeat: real-world example
4-11 Pattern self-similarity matrices of rhythm and pitch
5-1 PCA schematic
5-2 Manifold examples: electronic, funk, jazz music
5-3 Rhythm prediction with CWM and SVM
5-4 SVM classification schematic
5-5 Time-lag embedding example
5-6 PCA reduced time-lag space
5-7 Supervised learning schematic
5-8 Intra-song downbeat prediction
5-9 Causal downbeat prediction schematic
5-10 Typical Maracatu rhythm score notation
5-11 Inter-song downbeat prediction
5-12 Segment distribution demonstration
5-13 Dendrogram and musical path
5-14 Compression example
6-1 Time-scaling schematic
6-2 Beat matching example
6-3 Beat matching schematic
6-4 Scrambled music source example
6-5 Scrambled music result
6-6 Reversed music result
6-7 Fragment-based image completion
6-8 Restoring music schematic
6-9 Segment-based music completion example
6-10 Video textures
6-11 Music texture schematic
6-12 Music texture example (1600%)
6-13 Cross-synthesis schematic
6-14 Photomosaic
6-15 Cross-synthesis example
A-1 Skeleton software screenshot
A-2 Skeleton software architecture