To preface, if you are unfamiliar with frequency modulation synthesis, have a quick skim of this Wikipedia article: https://en.wikipedia.org/wiki/Frequency_modulation_synthesis
I have made this thread because I am trying to solve a difficult programming problem. Forgive me for not hastily getting to the point/my question; there is quite a lot of assumed knowledge of both C++ and audio synthesis in general that will be required to help me with my problem, so I have written a detailed post covering the basics of frequency modulation before delving into the autistic guts of my problem. Even if you are unable to provide me with any assistance, I hope you enjoy reading the thread (because it was fun to write).
Finally and before I begin, if anyone is interested in audio synthesis and video game console sound chips in general, have a look at this fun post I made some time ago on the TIGSource forums: http://forums.tigsource.com/index.php?topic=28653.msg795584#msg795584
Anyway, I am working on a real-time audio synthesis library in C++ styled upon sound chips of the 16-bit video game console era (Sega Genesis/Mega Drive, Neo Geo, etc.). I have a preference for the audio synthesis philosophy and methodology of Yamaha, who are a major producer of sound chips and electronic keyboards, most of which make use of frequency modulation synthesis.
The primary key to understanding frequency modulation synthesis is the difference between carrier operators and modulation operators. An FM operator is in its simplest form an oscillator which in most cases generate a sine wave, although my synthesizer defines 19 different types of wave forms (which is more or less irrelevant in the present context) and implements the ability to receive prerecorded wave data as well. An FM operator can be either a modulator, carrier or both. A modulator is fed into other operators and adjusts their frequency regardless of whether or not the latter is a carrier or another modulator. Carrier operators are not fed into other operators, but output the resulting wave data. Multiple carriers may be blended together to create the final output, usually by simply averaging them together.
This diagram conveys the difference between carrier operators and modulation operators:
FM synthesizers require at least 2 operators, one of which must be a carrier. A single operator (which is inevitably a carrier) cannot realistically be considered as frequency modulated synthesis because it is not modulated by anything (unless it is being modulated by prerecorded wave data instead of another operator, although that is besides the point).
The secondary key to understanding frequency modulation synthesis the concept of 'algorithms'. An FM algorithm defines the way that multiple operators (carriers and modulators) interact with one another. The number of operators that a particular FM synthesizer makes use of (which has in the past been constantly defined depending on the design of the synthesizer or limitations of the hardware) sets an upper limit to the number of possible algorithms. Similarly and historically, the number of algorithms and their respective configurations have been constantly defined.
The audio synthesis library I am developing differs from previous generations of FM synthesizers in that the number of operators, number of algorithms and algorithm configurations are not constantly defined or restricted by hardware limitations (i.e. they are defined by the user).
The majority of video game console FM sound chips make use of 4 operators. Notably, the Yamaha YM2612 sound chip (famously incorporated into the Sega Genesis/Mega Drive) defines 8 algorithms for its 4 operators, which are displayed here:
Although the YM2612 defines only 8 algorithms, many more are possible. There is an upper limit to the number of possible algorithms based on 4 operators, but I have not determined what it is, and that is part of the problem I am trying to solve.
More advanced FM synthesizers (such as electronic keyboards and virtual instruments) often make use of more than 4 operators. The number of possible algorithms to an FM synthesizer increases exponentially with each additional operator. Typically, the designers of these FM synthesizers define a constant number of possible algorithms and fixed algorithm configurations based on a trial and error test case methodology to determine which algorithms output the most interesting and useful audio. That being said, the upper limit of possible algorithm configurations in the case of higher numbers of operators can defy imagination.
To convey exactly how mind-boggling the possibilities are, here is a small subset of predefined algorithms to an electronic keyboard utilizing 6 operators:
Now that I have established the fundamentals of frequency modulation synthesis, we can move on to the specifics of my code.
I have yet to determine a method of mapping the possible algorithms based on the user defined number of operators in my synthesizer. I have already attempted to manually define all possible algorithms in the range of 1-5 operators, yet being a mere human being makes me prone to error. There are far too many variables to take into consideration, and with every additional operator I attempt to solve, another dimension is opened that I am not able to comprehend.
In the meantime, I have declared a function pointer towards a function that calculates the output of a single frame iteration as so:
f32 (voice::*iterate_func)(f32**,s32,f32);
When the user attempts to change the algorithm number (which I have attempted to predefine, yet missed the mark), this function is executed:
void voice::set_algorithm( s32 v )
{
BDAPI_MUTEX( mutex );
switch( operator_amount )
{
case 1:
{
algorithm = 0;
iterate_func = &voice::iterate_1_0;
return;
}
case 2:
{
switch( algorithm = v % 2 )
{
case 0: iterate_func = &voice::iterate_2_0; return;
case 1: iterate_func = &voice::iterate_2_1; return;
}
}
// etc
These iteration functions invoke the operators as modulators and carriers based upon their function parameters; an operator iteration that receives the output from another operator is modulated:
f32 voice::iterate_1_0( f32** o, s32 i, f32 p_0 )
{
return o[0][i] = operators[0]->iterate( 0.f, p_0 );
}
f32 voice::iterate_2_0( f32** o, s32 i, f32 p_0 )
{
o[0][i] = operators[0]->iterate( );
o[1][i] = operators[1]->iterate( 0.f, p_0 );
return ( o[0][i] + o[1][i] ) / 2.f;
}
f32 voice::iterate_2_1( f32** o, s32 i, f32 p_0 )
{
o[0][i] = operators[0]->iterate();
return o[1][i] = operators[1]->iterate( o[0][i], p_0 );
}
// etc
Now, when I crank the number of operators into overdrive (5 in this particular case) things quickly become hectic and confusing:
f32 voice::iterate_04_00( f32** o, s32 i, f32 p_0 )
{
o[0][i] = operators[0]->iterate( );
o[1][i] = operators[1]->iterate( o[0][i] );
o[2][i] = operators[2]->iterate( o[1][i] );
o[3][i] = operators[3]->iterate( o[2][i] );
return o[4][i] = operators[4]->iterate( o[3][i], p_0 );
}
f32 voice::iterate_04_01( f32** o, s32 i, f32 p_0 )
{
o[0][i] = operators[0]->iterate( );
o[1][i] = operators[1]->iterate( o[0][i] );
o[2][i] = operators[2]->iterate( o[1][i] );
o[3][i] = operators[3]->iterate( );
return o[4][i] = operators[4]->iterate( ( o[2][i] + o[3][i] ) / 2.f, p_0 );
}
f32 voice::iterate_04_02( f32** o, s32 i, f32 p_0 )
{
o[0][i] = operators[0]->iterate( );
o[1][i] = operators[1]->iterate( o[0][i] );
o[2][i] = operators[2]->iterate( );
o[3][i] = operators[3]->iterate( o[2][i] );
return o[4][i] = operators[4]->iterate( ( o[1][i] + o[3][i] ) / 2.f, p_0 );
}
f32 voice::iterate_04_03( f32** o, s32 i, f32 p_0 )
{
o[0][i] = operators[0]->iterate( );
o[1][i] = operators[1]->iterate( o[0][i] );
o[2][i] = operators[2]->iterate( );
o[3][i] = operators[3]->iterate( );
return o[4][i] = operators[4]->iterate( ( o[1][i] + o[2][i] + o[3][i] ) / 3.f, p_0 );
}
f32 voice::iterate_04_04( f32** o, s32 i, f32 p_0 )
{
o[0][i] = operators[0]->iterate();
o[1][i] = operators[1]->iterate();
o[2][i] = operators[2]->iterate();
o[3][i] = operators[3]->iterate();
return o[4][i] = operators[4]->iterate( ( o[0][i] + o[1][i] + o[2][i] + o[3][i] ) / 4.f, p_0 );
}
f32 voice::iterate_04_05( f32** o, s32 i, f32 p_0 )
{
o[0][i] = operators[0]->iterate();
o[1][i] = operators[1]->iterate();
o[2][i] = operators[2]->iterate();
o[3][i] = operators[3]->iterate( ( o[0][i] + o[1][i] + o[2][i] ) / 3.f );
return o[4][i] = operators[4]->iterate( o[3][i], p_0 );
}
f32 voice::iterate_04_06( f32** o, s32 i, f32 p_0 )
{
o[0][i] = operators[0]->iterate( );
o[1][i] = operators[1]->iterate( );
o[2][i] = operators[2]->iterate( ( o[0][i] + o[1][i] ) / 2.f );
o[3][i] = operators[3]->iterate( o[2][i] );
return o[4][i] = operators[4]->iterate( o[3][i], p_0 );
}
// etc
My failure is that I am unable to account for all possible algorithm configurations when the number of operators is variable. Manually coding each and every algorithm I can think of up to 5 operators seems impossible, and I don't even want to begin thinking about the implications of 6 operators. However, many advanced FM synthesizers that have existed for decades (electronic keyboards, etc.) have made use of 6 operators and beyond. My aim is to create the most thorough and flexible frequency modulated synthesis system thus far and I feel that there is a way to harness the power of C++ to do so (which has so far eluded me).
The code I have presented to you above will most likely be scrapped, but I think it does a sufficient job of conveying the need for some sort of C++ template metaprogramming or perhaps a simpler and more straight forward method. Performance is critical, so I want to minimize overhead as much as possible.
For reference, here is the code for a few of the functions specificed above:
// iterate
f32 operate::iterate( f32 modulation, f32 pulse )
{
BDAPI_MUTEX( mutex );
operator_oscillator->set_frequency( interpolate_frequency() );
return last_frame = operator_oscillator->iterate( powf( 2.f, pulse ) * powf( 2.f, modulation )
* ( 1.f + multiplier ) ) * envelope->iterate() * velocity;
}
// private interpolate frequency
f32 operate::interpolate_frequency()
{
if( envelope->get_state() != adsr::attack )
{
return target_frequency;
}
else
{
return math::mix<f32>( previous_frequency, math::clamp<f32>( math::inv_lerp<f32>( last_adsr,
envelope->get_last_frame(), envelope->get_attack_level() ), 1.f ), target_frequency );
}
}
// run
raw& voice::run()
{
BDAPI_MUTEX( mutex );
if( voice_state == idle )
{
return *output;
}
fori( samples )
{
if( using_amp )
{
if( using_pulse )
{
output_data[i] = ( this->*iterate_func )( operator_data, i, lfo_pulse->iterate() )
* 1.f - ( lfo_amp->iterate() + 1.f ) * 0.5f;
}
else
{
lfo_pulse->skip();
output_data[i] = ( this->*iterate_func )( operator_data, i, 0.f )
* 1.f - ( lfo_amp->iterate() + 1.f ) * 0.5f;
}
}
else
{
lfo_amp->skip();
if( using_pulse )
{
output_data[i] = ( this->*iterate_func )( operator_data, i, lfo_pulse->iterate() );
}
else
{
lfo_pulse->skip();
output_data[i] = ( this->*iterate_func )( operator_data, i, 0.f );
}
}
}
if( voice_state == state::key_on && duration > 0.f )
{
duration -= samples;
if( duration <= 0.f )
{
fori( operator_amount )
{
operators[i]->key_off();
}
voice_state = state::key_off;
}
}
if( voice_state == state::key_off )
{
bl set_idle = true;
fori( operator_amount )
{
operators[i]->update_state();
if( operators[i]->get_state() == idle )
{
operators[i]->clear();
}
else
{
set_idle = false;
}
}
if( set_idle )
{
voice_state = idle;
output->clear();
}
}
return *output;
}
Any ideas whatsoever would be greatly appreciated.
Thank you for your patience.