I've been working on an RPG that is procedurally generated (world, NPCs, story, questlines, items etc.) in my spare time and i have either working portions or conceptual models for each of them. One thing i have yet to completely figure out is what to do about music. First off, i know it can be done as there are great historical examples: the Emily Howell program being one of the best:
but exactly where to start from is a good quandry.
My best guess right now would be to feed a markov chain algorithm music samples of a style of music i want it to attempt to reproduce and tweak things from there, but even if i did get a bunch of notes, i need a decent way to go about either running them real-time (if i can do it fast enough) or creating a usable music file (if i cant do it fast enough) Ala .wave, .midi, etc.
So i guess to sum it up:
Has anyone had any experience in this area, what did you do?
What is a good way to write/play music procedurally (from c# if possible) [Libraries, APIs, etc]
Are there any open forums on this topic (aigamedev has this info paywalled. and im not sure i want to sub if i don't know if its going to help)
Has anyone had any experience in this area, what did you do?
What is a good way to write/play music procedurally (from c# if possible) [Libraries, APIs, etc]
Are there any open forums on this topic (aigamedev has this info paywalled. and im not sure i want to sub if i don't know if its going to help)
1. Not much experience, but I've read a couple of books on the subject. Of course, the opinions of anyone with actual experience doing this trump mine.
2. I've played around with a Python library called Mingus and I really liked the way it's structured. It has notions of note, interval, chord, bar... and it can also read and write MIDI files.
3. That interview at aigamedev ("Procedural Music Generation for Games using Statistical Models") is actually not bad. I can tell you the basic ideas in it. The discussion concentrated on how to generate a single melodic line. There were a couple of comments about coordinating instruments, but it wasn't detailed enough to reproduce. They take a MIDI file as input (or several MIDI files, but their simple approach only works well if they are extremely similar styles) and consider a sequence of notes as being a sequence of symbols (they call them "states", but I think "symbols" is more clear, because they play the role of letters in a name generator). A symbol contains the following information: pitch, duration, and part of the bar in which the note starts (including this is a key idea). They accumulate statistics about which symbols follow which other symbols, and then proceed to generate music following the same distributions (a.k.a., they use a Markov model). They can also consider that the state consists of the last several consecutive symbols (e.g., 2 or 3 of them), like you probably would do in a name generator (the number of symbols that constitute the state is often referred to as the "order").
I feel that the results are too similar to the input and lack overall structure. If I were to do something like this, I would probably take multiple MIDI files, transpose them so they share a key, enrich the input with chord notations and then have two Markov chains: A high-level one that generates chord changes and a low-level one that generates notes in the manner described above, but I would make the chord part of the symbol, and perhaps the next chord as well. I can see something like that being able to generate very believable jazz, for instance. You can also generate the chord changes in a more prescribed fashion (12-bar blues, boogie, "rhythm changes", etc.).
Thanks for the info, I'll have to check Mingus out. Also good to know i was somewhat on target with the Markov based approach, but how you described it makes a lot of sense in setting up the contributing markov models. I'll give it a shot. If / When i get something working, I'll post a journal about it.
Thanks for the info, I'll have to check Mingus out. Also good to know i was somewhat on target with the Markov based approach, but how you described it makes a lot of sense in setting up the contributing markov models. I'll give it a shot. If / When i get something working, I'll post a journal about it.
Please give a shout if you get something running, it'd be interesting to see how you did it, I'll add a watch to this thread
Well, I've got phase1 of the Markov Token Engine working: a basic markov chain text parser (hey, ya gatta walk before you run).
Anyway, I fed it "A Midsummer Night's Dream" as the input text doing a 1-4 Token Association and here is an excerpt:
"he stay, whom love doth the moon shine that you should think, we shadows have offended, Think no more of this discord? PHILOSTRATE A play the lion too: I have come, great clerks have purposed To greet me with premeditated welcomes; Where I have seen through the lion's neck: and he himself must have a wall in wax By him imprinted and within his power I am made bold, Nor how it may extenuate-- To death, or to a vow of maiden's patience."
Not bad.
So, this guy is pretty useful already, and He'll end up in my game as a name generator for sure for my NPC generator. For those of you whom are curious, I offer you the code to play with:
"NOTE: if your text isnt large enough, you increase the chance of circular loops in your distribution, if you see lots of repeating words, that is why"
"NOTE2: the last 'n' tokens in the input text are non-associated with any tokens, but the Length-n-1 token is, and the last 'n' tokens are added to its association... if the final token in the chain is a unique word, you may get issues if the Length-n-1 token is ever chosen"
"NOTE3: the code is a bit dirty, so I'm sure it can be improved upon, and there may even be a bug or two.
public class MarkovChain
{
/// <summary>
/// dictionary of token associations
/// </summary>
private Dictionary<String, List<String>> mc_MarkovTokens = new Dictionary<String, List<String>>();
/// <summary>
/// basic random number generator
/// </summary>
private Random mc_Rand = new Random();
//up to how many follow-on tokens we want to associate per token
//NOTE - doesnt always guarantee 4, i've seen 3 and 2 (likely a whitespace cleanup error somewhere)
private int n = 4;
/// <summary>
/// basic constructor
/// </summary>
public MarkovChain(){}
/// <summary>
/// parses a file into a markov token dictionary
/// </summary>
/// <param name="fileName"></param>
public void parse(String fileName)
{
//get access to the input file
FileStream fs = new FileStream(fileName, FileMode.Open);
//setup the stream reader
StreamReader sr = new StreamReader(fs);
//holds tokens in work
String[] tokenArray;
//the text following the token
String followingText = "";
//create a word token placeholder for carryovers
String tokenCarry = "";
//keep reading the file until you reach the end
while (!sr.EndOfStream)
{
//check to see if a token was carried over from last loop
if (tokenCarry == "")
{
//replace token carry with the line
tokenCarry = sr.ReadLine();
}
else
{
//combine with tokenCarry
tokenCarry += " " + sr.ReadLine();
}
//split tokenCarry into tokens
tokenArray = tokenCarry.Trim().Split(' ');
//create tokens and store them into the markov Token Dictionary
for (int i = 0; i < tokenArray.Length; i++)
{
//ensure you have room to define a new token entry
//[t1][t2 t3 - tn]<[EOL]
if ((i + n) < (tokenArray.Length - 1))
{
//calculate the following text
for (int j = 1; j <= n; j++)
{
followingText += tokenArray[i + j].Trim() + " ";
}
//check to see if the token already exists
if (mc_MarkovTokens.ContainsKey(tokenArray.ToLower()))
{
//it does exist, so add the new token association
mc_MarkovTokens[tokenArray.Trim().ToLower()].Add(followingText.Trim());
}
else
{
//it doesnt exist, so create it and its token association
mc_MarkovTokens.Add(tokenArray.Trim().ToLower(), new List<String>());
mc_MarkovTokens[tokenArray.Trim().ToLower()].Add(followingText.Trim());
}
//reset following text
followingText = "";
}
else
{
//calculate the following text
for (int j = i; j < tokenArray.Length; j++)
{
followingText += tokenArray[j].Trim() + " ";
}
//carry over the training text to the next iteration
tokenCarry = followingText.Trim();
followingText = "";
break;
}
}
}
//it is likely we missed tokens near the end, do the same thing as we did above, but for them
if (tokenCarry.Length > 0)
{
tokenArray = tokenCarry.Trim().Split(' ');
//calculate the following text
for (int j = 1; j < tokenArray.Length; j++)
{
followingText += tokenArray[j].Trim() + " ";
}
if (mc_MarkovTokens.ContainsKey(tokenArray[0].ToLower()))
{
mc_MarkovTokens[tokenArray[0].Trim().ToLower()].Add(followingText.Trim());
}
else
{
mc_MarkovTokens.Add(tokenArray[0].Trim().ToLower(), new List<String>());
mc_MarkovTokens[tokenArray[0].Trim().ToLower()].Add(followingText.Trim());
}
}
//close the streams
sr.Close();
fs.Close();
}
/// <summary>
/// outputs a file with generated text based on the current token association dictionary
/// </summary>
/// <param name="fileName"></param>
public void output(String fileName)
{
//start with "he" as 1st token (its common in use)
String token = "he";
List<String> tokens;
String output = token;
int index = 0;
//get 1000 token pairs
for (int i = 0; i < 1000; i++)
{
//find the first token
tokens = mc_MarkovTokens[token.ToLower()];
//select a random index within its associated list
//(this also keeps us within distribution parameters since i allow duplicate entries)
index = mc_Rand.Next(0, tokens.Count - 1);
//get the last word from the associated list, make it the next token
token = tokens[index].Split(' ')[tokens[index].Split(' ').Length - 1];
//append the chosen token list to the output string
output += " " + tokens[index];
}
//create a file access stream
FileStream fs = new FileStream(fileName, FileMode.Create);
//create a stream writier to write the string
StreamWriter sw = new StreamWriter(fs);
//write the file
sw.Write(output);
//close the streams
sw.Close();
fs.Close();
}
}
I guess phase 2 will involve learning as much about various music formats and how to turn them into tokenizable symbols to feed to a similar (but not the same) Tokenizer like i have above. From there i can goto phase 3 - initial crappy music
Ok, for part 1 of Phase 2, I've decided to go with ABC Music notation as the input standard. Why? The format is already symbol based, has defined musical notes, measures, timings, etc in a human readable and token parsable format. Could I use midi, perhaps, but i would still need to translate the bitstreams into symbols to be tokenized, thus making ABC the better choice for experimentation, as well as the fact i have TONS of these already for use in LoTRO's built-in music system.
For more on the ABC format here: http://en.wikipedia....ki/ABC_notation and here http://abcnotation.com/blog/2009/12/23/how-to-get-started-with-abc-notation/ and here http://abcnotation.com/blog/2010/01/31/how-to-understand-abc-the-basics/
Next Steps: Take the ABC format and build a parser from textual symbols to logical symbols as notes have more meaning than just abcdefg
Hah! Moderate Victory! The choice of ABC format was a success! With almost no tweaks, I've gotten my token parser to read ABC files and generate output. I then confirmed quality by loading up LoTRO (Lord of The Rings Online) and having my character play them. And it Worked! The files resemble the source material and flow together (for the most part). While not concert quality, it produced ABC readible & playable music that did not sound entirely random. So i guess i skipped Phase 2 pt 2 straight to Phase 3: crappy music. Now all is needed is to get the generator to learn multiple similar music scores and attempt to build a song out of them, additionally, I'll need to create 3-4 different parsings and dictionaries based on different token lengths, and then assemble them together based on a rule set to create a more harmonious music stream output.
I Then used this, cut off the header portion and fed it to the Markov code i created and the re-attached the header after trimming some spurrious repeats at the end. I got this:
public class MarkovMusic
{
/// <summary>
/// dictionary of token associations
/// </summary>
private Dictionary<String, List<String>> mc_MarkovTokens = new Dictionary<String, List<String>>();
/// <summary>
/// basic random number generator
/// </summary>
private Random mc_Rand = new Random();
//up to how many follow-on tokens we want to associate per token
//NOTE - doesnt always guarantee 4, i've seen 3 and 2 (likely a whitespace cleanup error somewhere)
private int n = 3;
/// <summary>
/// parses a file into a markov token dictionary
/// </summary>
/// <param name="fileName"></param>
public void parse(String fileName)
{
//get access to the input file
FileStream fs = new FileStream(fileName, FileMode.Open);
//setup the stream reader
StreamReader sr = new StreamReader(fs);
//holds tokens in work
String[] tokenArray;
//the text following the token
String followingText = "";
//create a word token placeholder for carryovers
String tokenCarry = "";
//keep reading the file until you reach the end
while (!sr.EndOfStream)
{
//check to see if a token was carried over from last loop
if (tokenCarry == "")
{
//replace token carry with the line
tokenCarry = sr.ReadLine();
}
else
{
//combine with tokenCarry
tokenCarry += " " + sr.ReadLine();
}
//split tokenCarry into tokens
tokenArray = tokenCarry.Trim().Split(' ');
//create tokens and store them into the markov Token Dictionary
for (int i = 0; i < tokenArray.Length; i++)
{
//ensure you have room to define a new token entry
//[t1][t2 t3 - tn]<[EOL]
if ((i + n) < (tokenArray.Length - 1))
{
//calculate the following text
for (int j = 1; j <= n; j++)
{
followingText += tokenArray[i + j].Trim() + " ";
}
//check to see if the token already exists
if (mc_MarkovTokens.ContainsKey(tokenArray.ToLower()))
{
//it does exist, so add the new token association
mc_MarkovTokens[tokenArray.Trim().ToLower()].Add(followingText.Trim());
}
else
{
//it doesnt exist, so create it and its token association
mc_MarkovTokens.Add(tokenArray.Trim().ToLower(), new List<String>());
mc_MarkovTokens[tokenArray.Trim().ToLower()].Add(followingText.Trim());
}
//reset following text
followingText = "";
}
else
{
//calculate the following text
for (int j = i; j < tokenArray.Length; j++)
{
followingText += tokenArray[j].Trim() + " ";
}
//carry over the training text to the next iteration
tokenCarry = followingText.Trim();
followingText = "";
break;
}
}
}
//it is likely we missed tokens near the end, do the same thing as we did above, but for them
if (tokenCarry.Length > 0)
{
tokenArray = tokenCarry.Trim().Split(' ');
//calculate the following text
for (int j = 1; j < tokenArray.Length; j++)
{
followingText += tokenArray[j].Trim() + " ";
}
if (mc_MarkovTokens.ContainsKey(tokenArray[0].ToLower()))
{
mc_MarkovTokens[tokenArray[0].Trim().ToLower()].Add(followingText.Trim());
}
else
{
mc_MarkovTokens.Add(tokenArray[0].Trim().ToLower(), new List<String>());
mc_MarkovTokens[tokenArray[0].Trim().ToLower()].Add(followingText.Trim());
}
}
//close the streams
sr.Close();
fs.Close();
}
/// <summary>
/// outputs a file with generated text based on the current token association dictionary
/// </summary>
/// <param name="fileName"></param>
public void output(String fileName)
{
//get the first key
String token = mc_MarkovTokens.Keys.ToList<String>()[0];
List<String> tokens;
String output = token;
int index = 0;
//get 1000 token pairs
for (int i = 0; i < 1000; i++)
{
if (mc_MarkovTokens.ContainsKey(token.ToLower()))
{
//find the first token
tokens = mc_MarkovTokens[token.ToLower()];
}
else
{
break;
}
//select a random index within its associated list
//(this also keeps us within distribution parameters since i allow duplicate entries)
index = mc_Rand.Next(0, tokens.Count - 1);
//get the last word from the associated list, make it the next token
token = tokens[index].Split(' ')[tokens[index].Split(' ').Length - 1];
//append the chosen token list to the output string
output += " " + tokens[index];
}
//create a file access stream
FileStream fs = new FileStream(fileName, FileMode.Create);
//create a stream writier to write the string
StreamWriter sw = new StreamWriter(fs);
//write the file
sw.Write(output);
//close the streams
sw.Close();
fs.Close();
}
}
As you can see, it bears almost exact resemblance to the other code, just a bit of tweaking on the output to watch for unknown tokens and to ensure the first token chosen always exists within the dictionary keys
I've been wondering how to implement something like this, and your code pretty much says it all.
I tried to convert your ABC score to a MIDI file using abc2midi, but it fails to read your ABC syntax. How do you convert your ABC to MIDI to listen to it or do you have some ABC music player?
Best thing I can recommend (there is rather a derth of good ABC players) is to goto: http://sites.google.com/site/lotroabc/ and get the ABC Navigator- Editor/Player
you may need to reduce the number of notes per line in order to get my example to work, here is a shortened version i truncated a bit to make it work.
) )
