How to synthesize sounds?

Question

I'd like to produce sounds that would resemble audio from real instruments. The problem is that I have very little clue how to get that.

What I know this far from real instruments is that sounds they output are rarely clean. But how to produce such unclean sounds?

This far I've gotten to do this, it produces quite plain sound from which I'm not sure it's even using the alsa correctly.

import numpy
from numpy.fft import fft, ifft
from numpy.random import random_sample
from alsaaudio import PCM, PCM_NONBLOCK, PCM_FORMAT_FLOAT_LE

pcm = PCM()#mode=PCM_NONBLOCK)
pcm.setrate(44100)
pcm.setformat(PCM_FORMAT_FLOAT_LE)
pcm.setchannels(1)
pcm.setperiodsize(4096)

def sine_wave(x, freq=100):
    sample = numpy.arange(x*4096, (x+1)*4096, dtype=numpy.float32)
    sample *= numpy.pi * 2 / 44100
    sample *= freq
    return numpy.sin(sample)

for x in xrange(1000):
    sample = sine_wave(x, 100)
    pcm.write(sample.tostring())

Answer 1

Sound synthesis is a complex topic which requires many years of study to master.

It is also not an entirely solved problem, although relatively recent developments (such as physical modelling synthesis) have made progress in imitating real-world instruments.

There are a number of options open to you. If you are sure that you want to explore synthesis further, then I suggest you start by learning about FM synthesis. It is relatively easy to learn and implement in software, at least in basic forms, and produces a wide range of interesting sounds. Also, check out the book "The Computer Music Tutorial" by Curtis Roads. It's a bible for all things computer music, and although it's a few years old it is the book of choice for learning the fundamentals.

If you want a quicker way to produce life-like sound, consider using sampling techniques: that is, record the instruments you want to reproduce (or use a pre-existing sample bank), and just play back the samples. It's a much more straightforward (and often more effective) approach.

Answer 2

I agree that this is very non-trivial and there's no set "right way", but you should consider starting with a (or making your own) MIDI SoundFont.

Answer 3

Cheery, if you want to generate (from scratch) something that really sounds "organic", i.e. like a physical object, you're probably best off to learn a bit about how these sounds are generated. For a solid introduction, you could have a look at a book such as Fletcher and Rossings The Physics of Musical Instruments. There's lots of stuff on the web too, you might want to have a look at a the primer James Clark has here

Having at least a skim over this sort of stuff will give you an idea of what you are up against. Modeling physical instruments accurately is very difficult!

If what you want to do is have something that sounds physical, rather something that sounds like instrument X, your job is a bit easier. You can build up frequencies quite easily and stack them together, add a little noise, and you'll get something that at least doesn't sound anything like a pure tone.

Reading a bit about Fourier analysis in general will help, as will Frequency Modulation (FM) techniques.

Have fun!

Answer 4

As other people said, not a trivial topic at all. There are challenges both at the programming side of things (especially if you care about low-latency) and the synthesis part. A goldmine for sound synthesis is the page by Julius O. Smith. There is a lot of techniques for synthesis http://ccrma-www.stanford.edu/~jos/.