A Serpentine Path To Music

The answer is why?

I will attempt to answer all these questions and shed some light into very ill frequented corners of Java/JVM programming. My approach will not be to answer each in turn, but these questions will form the armature upon which what follows will be constructed.

Sonic Field and how to achieve complex audio synthesis on the JVM proved to be too large a subject for one post. I have am extremely lucky to have been given chance to make two. This post is the hard work; we will cover the difficult subjects like threading, memory management and some digital audio concepts. The next post is a lot more fun; it will look at why I chose Python and some thoughts on creativity.

Now let us look at how digital sounds is represented and manipulated. 

Believe it or not, there are two ways right from the start. However, everyone (including myself with Sonic Field) picks the same one. Sound is made of waves. Waves can be represented mathematically using trigonometrical functions like sin and cos. We can then apply non linear differential equations to those mathematical forms to produce more complex equations. Keep adding more and more transformations and eventually we end up with one equation which represents an entire piece of music. This never happens because it is far, far too hard! Such an approach is referred to as ‘analytic’ and is practical only for the simplest of sounds.

The spectrogram of the piece of music I rendered with Sonic Field for the Java Advent Calendar

More tractable are numerical approaches. Here we consider sounds as a series of samples. Just like the samples that make up a video, one frame at a time, sound is generated and manipulated one sample at a time. We must understand this is an compromise and it brings with it a whole series of computational challenges of its own.

A quick note on licensing: Sonic Field is free as freedom:

    Sonic Field Audio Processing And Synthesis Program
    Copyright (C) 2012-2014  Dr Alexander J Turner

    This program is free software: you can redistribute it and/or modify
    it under the terms of the GNU Affero General Public License as
    published by the Free Software Foundation, either version 3 of the
    License, or (at your option) any later version.

    This program is distributed in the hope that it will be useful,
    but WITHOUT ANY WARRANTY; without even the implied warranty of
    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
    GNU Affero General Public License for more details.

    You should have received a copy of the GNU Affero General Public License
    along with this program.  If not, see <http://www.gnu.org/licenses/>.

All the source code, including any Sonic Field patch scripts in this post are covered by the above licence. It only seems fair that anyone reading this should respect this as I am giving it away for free on Github!

Consider that we want audio to be modelled up to 20kHz, i.e. we want to generate sounds which have frequencies up to 20 000 cycles per second. To actually generate that all using a sampling, digital system we need 40 000 samples per second. This is intuitively easy to understand as a sound goes ‘up and down’ in a wave so we need an up sample and a down sample for each cycle of the wave.

Now let us consider making a sound. We can start with a sine wave. This is a pure sound, and it is very boring indeed. How about making a more interesting sound? Well, real sounds consist of many different sine waves mixed together. A note at concert pitch A is 440Hz, it this were played on a trumpet we might get a sequence of frequencies like this:

def trumpetBase(length,freq,z=1.0):
    voxA=[]
    hc=1.0
    freq=float(freq)
    while hc*freq<20000:
        hf=hc*freq
        voxA.append(sf.NumericVolume(sf.PhasedSineWave(length,hf,random.random()),(1.0/hc)**z))
        hc+=1
        
    vox=sf.Mix(voxA)
    vox=sf.Clean(vox)
    vox=polish(sf.FixSize(vox),freq)
    return sf.FixSize(vox)

sig=sf.Power(trumpetBase(440,10000),1.25)

Now I have created a new ‘signal’ (i.e. a sequence of samples) which is a distortion of the original. In so doing multiples of the original harmonics will have been introduced. 5×4400 is over 20000. Our 40000 samples per second will no longer be enough. What we need to do is sample at a higher rate. But eventually we are defeated by this approach as we add distortion on distortion. The solution is to sample at a higher rate and then filter out the high frequencies periodically. We perform synthesis by repeating two basic steps, process then filter. The process adds frequencies and the filter takes out everything above when we need.

OK, so what rate? It seems that for most of my work 96 000 samples per second is about right. More makes near no difference to the sound quality but as I go below this number a horrid harsh effect called ‘aliasing’ starts to be unavoidable. This is caused by trying to model a frequency higher than twice the sample rate.

Finally we have started to see why making music with Java is so hard. 96 000 samples per second is a lot! In my research anything other than double precision floating point mathematics adds noise and reduces the over all quality of the sound. So, 96 000 * 8 bytes for 1 second of music. Basically, a single note can take many megabytes of memory to store.

The simplest way would be to just use a Java array of doubles. Sadly this is not anything like enough. I use a 16Gig machine and it would run out of memory very quickly if I used simple arrays. The reason being that sounds are made of sounds (as we saw before). Each of those sine waves in the trumpet sound might be 10 seconds long (about 5 megabytes) and there could be 40 harmonics, so that is 400 mega bytes. Then we want to generate notes in multiple threads, for 8 threads that might be 8 notes at once which is 1.6 Gigabytes. That is just to great a 10 second note which is relatively simple. More complex and longer notes (especially in ambient music where notes can last a very long time) quickly exhaust the capacity of my machine. I could use a bigger machine and address say 128 Gig; that would do for most music, but again, such a machine is likely to have more cores and the parallel memory use issue appears again. Basically, sound processing is memory lop sided compared to most other forms of processing.

Sonic Field address this issue with two techniques: Memory mapped files and generators. Of these, generators are probably the most interesting programmatically because they are surprisingly hard to implement and do not work as well as we might hope.

package com.nerdscentral.audio;

import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.atomic.AtomicLong;

import com.nerdscentral.audio.pitch.CubicInterpolator;
import com.nerdscentral.sython.SFMaths;
import com.nerdscentral.sython.SFPL_RuntimeException;

public abstract class SFSignal implements AutoCloseable
{

    private final static AtomicLong uniqueId       = new AtomicLong(0);
    private final long              myId;
    protected final AtomicInteger   referenceCount = new AtomicInteger(1);

    public abstract boolean isKilled();

    public abstract SFSignal replicate();

    private static ThreadLocal<String> pythonStack = new ThreadLocal<>();

    public final SFData replicateEmpty()
    {
        return SFData.build(this.getLength());
    }

    public abstract double getSample(int index);

    protected SFSignal()
    {
        myId = uniqueId.incrementAndGet();
    }

    /**
     * Returns a linearly interpolated sample based on the samples either side of the the passed index. This is used for super
     * sampling or pitch effects.
     * 
     * @param index
     * @return
     */
    public final double getSampleLinear(double index)
    {
        double s = SFMaths.floor(index);
        double e = SFMaths.ceil(index);
        if (s < 0 || e >= getLength())
        {
            return 0;
        }
        if (s == e) return getSample((int) s);
        double a = getSample((int) s);
        double b = getSample((int) e);
        return ((index - s) * b + (e - index) * a);
    }

    /**
     * Returns a cubic interpolated sample based on the samples either side of the the passed index. This is used for super
     * sampling or pitch effects. Cubic interpolation uses two samples either side of the required point and so at the ends of
     * the sample this will fall back to linear interpolation.
     * 
     * @param index
     * @return
     */
    public final double getSampleCubic(double index)
    {
        int s = (int) SFMaths.floor(index);
        int e = (int) SFMaths.ceil(index);
        if (s < 0 || e >= getLength())
        {
            return 0;
        }
        if (s > getLength() - 3 || index < 1)
        {
            if (s == e) return getSample(s);
            double a = getSample(s);
            double b = getSample(e);
            return ((index - s) * b + (e - index) * a);
        }
        return CubicInterpolator.getValue(getSample(s - 1), getSample(s), getSample(s + 1), getSample(s + 2), index - s);
    }

    public abstract double setSample(int index, double value);

    public abstract int getLength();

    public abstract void setAt(int pos, SFSignal data) throws SFPL_RuntimeException;

    public abstract void setFrom(int pos, SFSignal dataIn) throws SFPL_RuntimeException;

    public abstract double[] getDataInternalOnly();

    /**
     * Get a globally unique identifier for this SFSingal
     * 
     * @return
     */
    public final long getUniqueId()
    {
        return myId;
    }

    public abstract void release();

    @Override
    public void close() throws RuntimeException
    {
        int c = referenceCount.decrementAndGet();
        if (c == 0) release();
        else if (c < 0) throw new RuntimeException(Messages.getString("SFSignal.1")); //$NON-NLS-1$
    }

    public void incrReference()
    {
        referenceCount.incrementAndGet();

    }

    public SFSignal __pos__()
    {
        incrReference();
        return this;
    }

    public SFSignal __neg__()
    {
        close();
        return this;
    }

    public int getReference()
    {
        return referenceCount.get();
    }

    public void decrReference()
    {
        referenceCount.decrementAndGet();
    }

    public static String getPythonStack()
    {
        return pythonStack.get();
    }

    public static void setPythonStack(String ps)
    {
        SFSignal.pythonStack.set(ps);
    }

    public void clear()
    {
        // NOOP on root class
    }
}

[This AGPL code is on Github]

This is the abstract root class for all signal data in Sonic Field. It is a bit of a mess to be honest, the perfect example of a one person generated class which has ended up a little unfocused. However, it works and concrete sub-classes of this make up generators and memory mapped file handling. We can also see a bunch of code for integrating with Python in a nice way and for explicit resource management; normal Java garbage collection is no use for memory mapped file based storage!

So what is a generator?

The basic architecture of synthesis ends up being (you can argue theoretically about this, but in reality this is what happens):

1. Make a signal
2. Manipulate that signal making a new signal
3. Repeat until the wanted sound appears

The obvious architecture is then to:

1. Signals are samples.
2. Read samples.
3. Apply and algorithm.
4. Write samples.

We represent samples as indexed sequences of double . Therefore, for the above model, each sound manipulator algorithm reads from a sequence and writes out to a sequence. However, this involves a huge amount of memory access and memory access is very slow. When we run out of memory the operating system starts flushing data to the memory mapped files to help cope; reading and writing then becomes even slower. Consider a manipulation like this:

1. Distort a signal using the ‘Saturate’ algorithm
2. Halve the resulting volume
3. Mix with another signal

We do not need to create intermediate data storage for these steps, they can be chained together. This is what generators do; they chain with other generators overriding the getSample method. Here is the saturate implementation:

 
public static class Translator extends SFSingleTranslator
{

    protected Translator(SFSignal input)
    {
        super(input);
    }

    @Override
    public double getSample(int index)
    {
        double x = getInputSample(index);
        double y = x >= 0 ? x / (x + 1) : x / (1 - x);
        return y;
    }
}

static class Translator extends SFMultipleTranslator
{

    protected Translator(List<SFSignal> input)
    {
        super(input);
    }

    @Override
    public double getSample(int index)
    {
        double d = 0;
        for (int mindex = 0; mindex < getNMembers(); ++mindex)
        {
            d += getInputSample(mindex, index);
        }
        return d;
    }
}

Why generators are not the solution to everything:

# Read the signal
signal=sf.ReadFile("mono.wav")
# Filter it
signal=sf.ButterworthLowPass(signal,1000,6)
# Split it
left=+signal
right=signal
# Add Haas effect
left  = sf.Concatenate(sf.Silence(30),left)
# Write it out
sf.WriteFile32((left,right,"stereo,wav")

How Java Is Terrible At Memory Management (and approaching a solution):

Now we have some background to why managing audio data is such a challenge we are ready to see why Java is a bit of a catastrophe for managing memory. Java has garbage collection which means we do not know how much memory is actually being used. There are some forms of instrumentation which are supposed to tell us the amount of memory being used, but they are not very ‘real time’. In a C++ based system (for example) we could intercept malloc and know, to the byte, the amount of memory being used from the heap. This luxury is not given to us in Java. I tried, oh I tried so hard, to get a reliable way of estimating when the JVM was about to throw an out of memory exception; but as far as I can tell, there is no fail-safe way of doing so.

However, all is not lost as there is a usable work around. Indeed, the work around is always evolving and I would not be shocked it it became almost as effective as a hand crafted memory management system in C++ but with the platform independence of Java. It is not there yet, but it is getting there.

Memory Mapped Signal Pages

Memory mapped files are somewhat supported in Java. It is frustrating as anything that we can only memory map byte buffers and not double arrays; nevertheless, we can use them with the appropriate accessor methods. So, what Sonic Field does is have 2 megabyte ‘chunks’ which are ByteBuffers created by memory mapping a file. Using explicit resource management (thanks to Java 7) we know when a chunk is being used and when it is free. When one is freed it is added to a ConcurrentLinkedDeque of free chunks. This means our chunks are recyclable. How about we take a look at some of the code which does this:

static
{
    tempDir = new File(System.getProperty(SONIC_FIELD_TEMP));
    String swapLimitraw = System.getProperty(SONIC_FIELD_SWAP_LIMIT);
    if (swapLimitraw == null)
    {
        swapLimit = MAX_IN_RAM; // 64 megabytes or 87 seconds at 96000
    }
    else
    {
        swapLimit = Long.parseLong(swapLimitraw);
    }
    chunkLen = swapLimit / 2;

    String pid = ManagementFactory.getRuntimeMXBean().getName();
    try
    {
        if (tempDir != null)
        {
            coreFile = File.createTempFile("SonicFieldSwap" + pid, ".mem", tempDir); //$NON-NLS-1$ //$NON-NLS-2$
        }
        else
        {
            coreFile = File.createTempFile("SonicFieldSwap" + pid, ".mem");  //$NON-NLS-1$//$NON-NLS-2$            
        }
        coreFile.deleteOnExit();
        // Now create the actual file
        coreFileAccessor = new RandomAccessFile(coreFile, "rw"); //$NON-NLS-1$
    }
    catch (IOException e)
    {
        throw new RuntimeException(e);
    }
    channelMapper = coreFileAccessor.getChannel();
}

[This AGPL code is on Github]

Here we can see the creation of the swap file (the name I use for the memory mapped file). The most important thing is that we get the channel for the swap file. This is the object we can then use for memory mapping. Now we can see the code which creates the mapping for a new SFData object:

private void makeMap(long size) throws IOException
{
    long countDown = size;
    int chunkCount = 0;
    while (countDown > 0)
    {
        ++chunkCount;
        countDown -= chunkLen;
    }

    countDown = size;
    chunks = new ByteBuffer[chunkCount];
    chunkCount = 0;
    while (countDown > 0)
    {
        ByteBuffer chunk = freeChunks.poll();
        if (chunk == null) break;
        chunks[chunkCount] = chunk;
        ++chunkCount;
        countDown -= chunkLen;
    }
    if (countDown > 0)
    {
        synchronized (coreFile)
        {
            long from = coreFile.length();
            while (countDown > 0)
            {
                ByteBuffer chunk = channelMapper.map(MapMode.READ_WRITE, from, chunkLen);
                chunk.order(ByteOrder.nativeOrder());
                chunks[chunkCount] = chunk;
                ++chunkCount;
                from += chunkLen;
                countDown -= chunkLen;
            }
        }
    }
}

[This AGPL code is on Github]

It is lovely and simple! We just use up chunks on the free ‘stack’ and if there are not enough to represent all the signal we create a few more until we have enough.  I use a deque as a stack because the most recently added chunk is also most likely not to be ‘swapped out’ by the operating system; optimum performance is achieve by reusing the same chunks over and over. There is no concurrent stack in the standard JDK, so the deque made a good choice.

It is worth noting that recycling the last used chunk rather and using a first in first out approach gave a huge performance improvement on my machine. When I tried a first in first out approach the system would run fast until swapping started and then slow right down as it constantly read back in swapped out chunks. This was a complete waste of time because the swapped out chunks contained data that was not going to be used again. By recycling last in first out, this performance issue disappeared and Sonic Field achieved very efficient use of the operating system file cache and virtual memory systems.

Using explicit resource management, we can ensure that chunks which are no longer required get reused:

@Override
public void close() throws RuntimeException
{
    int c = referenceCount.decrementAndGet();
    if (c == 0) release();
    else if (c < 0) throw new RuntimeException(Messages.getString("SFSignal.1")); //$NON-NLS-1$
}

Yes, Sonic Field uses reference counting. This is tide into the explicit resource management in that SFSingal implements AutoCloseable. I will discuss the mechanism behind this a little more when we discuss the way the Java interacts with Python. However, what matters for now is that when the reference count hits zero ‘release()’ is called:

@Override
public void release()
{
    for (ByteBuffer chunk : chunks)
    {
        freeChunks.push(chunk);
    }
    chunks = null;
    resourceTracker.remove(this);
}

Release is responsible for putting the now unused chunks back onto the free deque. As the free deque is concurrent, we do not have to use synchronization blocks anywhere other than when creating new chunks.

Heavy disk IO as the OS starts serialising blocks of the memory mapped file to disk

from java.lang import System
sig=sf.WhiteNoise(10000)
t0=System.nanoTime()
for x in range(1,10):
    sig=sf.Swap(sig)
    for z in range(1,5):
        sig1=sf.Realise(sf.Pcnt10(+sig))
    sig=sig1
        
t1=System.nanoTime()
for y in range(1,5):
    t0=System.nanoTime()
    for x in range(1,10):
        sig=sf.Swap(sig)
        for z in range(1,5):
            sig1=sf.Realise(sf.Pcnt10(+sig))
        sig=sig1
    t1=System.nanoTime()
    print t1-t0

Times using a double buffer:

1497275000

1474564000

1410076000

1474493000

Sonic Field using 48 Gig of swap file on a machine with 16 Gig of RAM

Times using a byte buffer only:

1375515000

1376532000

1478389000

1438900000

On my machine the memory mapped files would not have had to actually resort to disk (SSD actually) as there was plenty of RAM available. Making sure the byte order is correct for the current platform (in my case LITTLE_ENDIAN, which one can get from ByteOrder.nativeOrder()) does help a little, which is interesting as it shows some form of optimisation going on.

A Quick Note On Caching

A quick lesson in synthesis. The pioneers of electronic music made notes and sounds. Later came the work of the legendary Dr Robert Moog. He created the concept of what we now call a synthesiser. As always, we could include many others in this creation, some working with him and some in parallel work. But let us just think about his work. He realised that very flexible sound creation could be achieved by oscillators, which create tones, and filters, which remove unwanted tones. These were ‘analogue’ electronic circuits which produced a limited number of tones and could filter them in a limited number of ways to produce sounds in ‘real time’.  The complexity of the sound was controlled by the complexity of the synthesiser. However, limitations in analogue technology set an upper limit to complexity; every circuit added some distortion and some noise, eventually these artefacts would overwhelm the desired sound and so the upper complexity limit was set.

These early pioneers created what we now think of as the ‘principles of synthesis’ and by so doing formed a wonderful working model for people to learn but also restricted creativity. Computational and digital synthesis tends to continue to follow the same path as the original analogue work. It is real time (press a key and out comes sound) and restricted (there are a fixed number of digital sound sources and filters etc). “Arbitrary complexity” simply means that there is no upper limit to the number of filters, oscillators and such that can be used. One key concept behind Sonic Field (which I have not yet quite gotten around to implementing) is that it can function in a cloud environment an coordinate an arbitrarily large number of nodes to create an arbitrarily complex piece of music or soundscape. Whilst the implementation is not quite there yet, the current architecture is very much suited to this approach.

There exists a completely different sort of digital sound creation which some people call synthesis (but I do not). This is sample looped based. Here sounds are recorded or created digital and stored as a ‘sample’ of that sound. These samples are then played back in a loop to create the notes heard. This is not ‘from first principles’ because the sounds already existed. Creating sounds by the direct manipulation of mathematical equations or the direct application of electronic circuits creates the sounds from first principles. This is where ‘ab-initio’ comes from.

I get asked this quite a bit when I discuss Sonic Field with people. The first question often is why not do this in C++? There are many answers and as this post is in a Java advent calendar I do not intend to defend Java. I think the question comes from an incorrect view that Java is somehow non performant. I often hear people saying Java is interpreted, even now! My view is that Java has some huge specific benefits for this work. They centre around multi-threading, clustering and portability:

On personal reflection, I do not think the project would have survived this long if it had been in C++. The effort of maintaining the build and updating from one version of gcc to the next and across to Visual Studio would have caused me to give up. Seriously, the memory management issues might one day start me porting off the JVM but I still hold hope I will keep coming up with a fixes for them rather than give up on this amazing platform; current evidence is that the memory mapped files approach is ‘good enough’ and I live in hope of coming up with either more tweaks to that or an even more effective approach.

An Aside On Threading

I said Java was good for threading. If you know anything about Python in C you will know it is single threaded (well, the interpreter is through a thing called the global interpreter lock). Jython is very multi-threaded. One of the driving concerns in Sonic Field has always been parallel execution be that via clustering or local threading. So I think it is time we took a look at the current threading model in Sonic Field (which changes almost as often as the memory management).

Now, this is a bit complex for a single post like this, so I will skim over a lot of the details. Nevertheless, I think this mechanism is a good demonstration of how Jython and Java can work well together.

import threading
import time
from java.util.concurrent import Executors, TimeUnit
from java.util.concurrent import Callable, Future
from java.lang import System
from java.lang import ThreadLocal
from java.lang import Thread
from java.util.concurrent import TimeUnit
from java.util.concurrent.locks import ReentrantLock

SF_MAX_CONCURRENT = int(System.getProperty("synthon.threads"))
SF_MAX_CONQUEUE   = SF_MAX_CONCURRENT

print "Concurrent Threads: " + SF_MAX_CONCURRENT.__str__()
SF_POOL = Executors.newCachedThreadPool()

class sf_callable(Callable):
    def __init__(self,toDo):
        self.toDo=toDo
        
    def call(self):
        return self.toDo()

class sf_futureWrapper(Future):
    def __init__(self,toDo):
        self.toDo=toDo
        self.gotten=False

    def __iter__(self):
        return iter(self.get())
        
    def get(self):
        if self.gotten:
            return self.result
        else:
            self.result=self.toDo.get()
            self.gotten=True
            return self.result
    
    def __pos__(self):
        obj=self.get()
        return +obj

    def __neg__(self):
        obj=self.get()
        return -obj

class sf_getter(Future):
    def __init__(self,toDo):
        self.toDo=toDo
        self.result=self.toDo()

    def get(self):
        return self.result

    def __iter__(self):
        return iter(self.get())
        
    def __pos__(self):
        obj=self.get()
        return +obj

    def __neg__(self):
        obj=self.get()
        return -obj

class sf_taskQueue(ThreadLocal):
    def initialValue(self):
        return []

SF_TASK_QUEUE=sf_taskQueue()

class sf_superFuture(Future):

    def __init__(self,toDo):
        self.toDo=toDo
        queue=SF_TASK_QUEUE.get()
        queue.append(self)
        if len(queue)>SF_MAX_CONQUEUE:
            self.submitAll()

    def submit(self):
        count=SF_POOL.getActiveCount()
        if count<SF_MAX_CONCURRENT:
            task=sf_callable(self.toDo)
            self.future=sf_futureWrapper(SF_POOL.submit(task))
        else:
            self.future=sf_getter(self.toDo)

    def submitAll(self):
        queue=SF_TASK_QUEUE.get()
        while(len(queue)):
            queue.pop().submit()

    def get(self):
        self.submitAll()
        while not hasattr(self,'future'):
            Thread.yield()
        r = self.future.get()
        return r

    def __iter__(self):
        return iter(self.get())
            
    def __pos__(self):
        obj=self.get()
        return +obj


    def __neg__(self):
        obj=self.get()
        return -obj

def sf_do(toDo):
    return sf_superFuture(toDo)

def shutdown_and_await_termination(pool, timeout):
    pool.shutdown()
    try:
        if not pool.awaitTermination(timeout, TimeUnit.SECONDS):
            pool.shutdownNow()
            if not pool.awaitTermination(timeout, TimeUnit.SECONDS):
                print >> sys.stderr, "Pool did not terminate"
    except InterruptedException, ex:
        pool.shutdownNow()
        Thread.currentThread().interrupt()
        
def shutdownConcurrnt():
    shutdown_and_await_termination(SF_POOL, 5)

[This AGPL code is on Github]

This mechanism launches futures to process threads. It does this using closures which are really easy to do in Python. It gets much more complex than one might think though because of the issue of recursive launching (which I will touch on here in a bit but is covered in more detail in my post ‘The Embrace Of Meh‘). Sonic Field aims to have a simple approach to threading. The very idea that such a thing exists is a little naive! However, it does get some way towards this goal. Here is an example. Consider that we have function sf.FrequencyDomain() which is very expensive and a function xmulti() which cross multiplies the output of this expensive processor.

# Form A
def mixer(siga,sigb):
    return sf.CrossMultiply(
        sf.FrequencyDomain(siga),
        sf.FrequencyDomain(sigb)
    )

We can convert that into running in parallel simply by making the bulk of the work happen in a closure:

# Form B
def mixer(siga,sigb):
    def mixerInner():
        return sf.CrossMultiply(
            sf.FrequencyDomain(siga),
            sf.FrequencyDomain(sigb)
       )
    return sf_do(mixerInner)

I love this because it requires so little syntax and just works. sf_do creates a future for the execution of the closure. The snag comes when we want to use that future. Because in the Java world we need the result (a signal) not a future. How can we  fix this type safety issue? Despite the parallel submission code being written in Python it uses java.util.concurrent.Future from the JDK. This means we can dynamically dispatch of the instance type of arguments. All Sonic Field processors convert their argument dynamically from Object to what ever they require. They do this via the static methods on the com.nerdscentral.synthon.Caster class. The key one of those methods being this:

public static Object checkAutoTranslation(Object o) throws SFPL_RuntimeException
{

    if (o instanceof Future)
    {
        Future doer = (Future) o;
        try
        {
            Object d = doer.get();
            return checkAutoTranslation(d);
        }
        catch (Throwable t)
        {
            throw new SFPL_RuntimeException(t);
        }
    }

    if (o == null)
    {
        throw new SFPL_RuntimeException(Messages.getString("Caster.12"));             //$NON-NLS-1$
    }

    return o;
}

There is so much I wanted to discuss and did not! For example, I wanted to discuss explicit resource management in much more detail. I would love to have discussed caching in FFT. How about sub-normal floating point performance? Immutable data structures and threading for large scale mathematics?

What I hope I have shown is how some of the ‘heavy engineering’ in Sonic Field has allowed something one would often not associated with Java – huge scale mathematics – to work rather cleanly in this language. Sonic Field has taught me so much about practical threading issues, real memory management challenges and effective flexible processing architecture. I hope that at least some of these concepts will be of use to you the reader.