37 replies to this topic

[leopardpm]

First off, I am an AI novice. I have researched FSMs, Hierarchical behaviors, and had my mind melted while trying to understand neural-nets. I have implemented FSMs (not hard) and am in the process of adding a few twists.

Here is my thought: I do believe that emergent patterns that mimic life-like behaviors can and will be produced from a basic simple ruleset. So, along these lines, I am trying to program an AI which, based on a basic Maslov Hierachy filtered through a set of variables which are particular for each agent (lazy-thru-productive, weak-thru-strong, stupid-thru-smart, cowadly -thru-courageous, follower-thru-leader, homebody-thru-adventuresome, favorites: color blue, forests> mountains, fish>meat>fruits>bread, etc,), and watch the agent 'live' in its virtual world, scratching out a meager existence and interacting with other agents. My biggest problem that I have encountered so far is in trying to determine which of multiple possible solutions to pick. I have some background in micro-economics and am trying to incorporate some of that into the decision process, so, one thought is to rate each possible solution based on an estimate of time it would take to complete it, weighted by the different categories of labor involved, and come up with a value which could then compare to other possibilities.

It would work like this: Agent A determines that it is hungry, and knows that it can fish in the nearby stream, or, go to a far tree to pick some fruit, and either method would satisfy its 'hunger' issue. It calculates the distance to the stream, an estimate of amount of time spent fishing modified by how much it 'likes' to fish, and then compares it to the same thing but the steps involved in picking fruit. OK, this works fine.

The problem I have is when economics and interaction with other agents occurs. Suppose all agents are willing to sell any object they possess if the price is higher than their cost to produce. And any agent is willing to buy if the price is less than their costs to produce. I am hoping that market prices will naturally arise from such a system, where the 'seller' initially offers cost + 100%, and the buyers initially offer cost - 50%.... and perhaps a short series of haggling in between until either both agree or decide not to trade. NOW, assuming that this 'price system' produces some sort of market prices, how in the world do I get Agents to take into account the possibility of purchase when making its decision on how to satisfy its hunger issue? I mean, its easy enough to compare to 'like' variables which describe the same 'units' like time and preference, if fishing takes 10 min and berry picking takes 5 minutes then perhaps the choice to fish only occurs if the agent 'likes fishing' sufficiently enough (twice as much?) as he likes picking berries... but what about going to market and buying the Fish? I can total up the distance to market, use the current market price for fish, even make sure that 'fish' are currently available in the market, but how do I equivocate in a money price into the equation?

Besides the interesting nature of the problem itself, the reason I am pursuing this line of simulation is that I feel that the future of MMO's will be entirely dependent on 3 main things: an AI living world that players can participate in (the Sims, Dwarf Fort, etc), a dynamic world which can be affected physically by players (Dwarf Fort, Minecraft, etc) and advanced user interface design (the Wii, virtual eyewear, Xbox Connix, etc). Ultima Online at its release WAS going to go down this path, but they got overwhelmed by a myriad of other unforeseen issues with player interaction which put the kabbosh on doing things like: the players decimate the local deer population (the current food source of the bears), and so the hungry bears start rampaging closer to town looking to players as a replacement food source.... I always was awe-struck by that sort of living, breathing environment.

Well, any comments or help would be greatly appreciated. Thanks for reading my ramblings!

[IADaveMark]

Hooo boy, does someone need a copy of my book. ;-)

Be that as it may, you hit on one of the major problems of economic utility... that of comparing unlike units. For example, time vs. hunger vs. money vs. security vs. physical effort, etc. The short answer is that there is no correct answer. Much of it is a balance thing. To make matters worse, much of the "exchange rate" is based on the units you define for concepts not typically quantified -- such as "security". The best bet in that case is to make sure everything you are using is normalized (i.e. 0..1). Then, by using your personality-based modifiers, you can come up with coefficients that are used in the decision equations. For example, for PersonA, Security = 2x money. For PersonB, Security = 2.5x money.

Anyway, too much football and beer for right now... I'm digging your project, though. Hang in there!

[leopardpm]

Hooo boy, does someone need a copy of my book. ;-)

Be that as it may, you hit on one of the major problems of economic utility... that of comparing unlike units. For example, time vs. hunger vs. money vs. security vs. physical effort, etc. The short answer is that there is no correct answer. Much of it is a balance thing. To make matters worse, much of the "exchange rate" is based on the units you define for concepts not typically quantified -- such as "security". The best bet in that case is to make sure everything you are using is normalized (i.e. 0..1). Then, by using your personality-based modifiers, you can come up with coefficients that are used in the decision equations. For example, for PersonA, Security = 2x money. For PersonB, Security = 2.5x money.

Anyway, too much football and beer for right now... I'm digging your project, though. Hang in there!

thanks! I found your site (IA) a few weeks ago in my searching around and really appreciated your insights and abilities. I am considering buying your book actually, but I fear it may be above me in alot of areas... but, I guess it can't hurt me any though, huh?! You are correct, obviously, in the problem of comparing economic utility - its a wholly subjective thing which is why it is and will always be impossible to model human economic systems to any good degree. I guess that I will have to just trial and error a bit with the variables (normalized, as you said) until I get what I happen to consider 'human-like' behavior to certain situations... it will be a long drawn out process I fear though.

I have started out with a few simple world objects for the agents to utilize (tree has: wood, fruit, branches, stream has:water, fish) and a very basic maslov hierarchy (shelter/food so far), and just getting into all the sub-tasks that even these few things require..... its gonna take alot of typing to get all these various FSMs in... thinking of building a graphical tool to help do alot of this.

[IADaveMark]

The book will definitely not be over your head. It's a pretty straight-forward read.

Considering multi-component multi-stage actions, you might want to look into a planner architecture rather than a ton of FSMs. The economics of the situation now modify the edge weights of your planner topology in order to determine the "best route" to get what you want.

[leopardpm]

The book will definitely not be over your head. It's a pretty straight-forward read.

Considering multi-component multi-stage actions, you might want to look into a planner architecture rather than a ton of FSMs. The economics of the situation now modify the edge weights of your planner topology in order to determine the "best route" to get what you want.

ok ok you got yourself another customer... I will buy the book later today. I hope it has stuff that covers 'memory' as I have determined that each agent will need to remember various things in order to make decisions faster and better - already each agent has, what I call, a 'Task Stack' which allows tasks to be interrupted by more pressing needs (something dangerous comes into the area while trying to fish so run away for a bit until safe, then go back to fishing.... wood axe breaks while chopping wood so go get another from storage then go back to chopping for whatever reason, etc) and I dread having a more space hogging memory for these agents because I want to keep storage down to a minimum to allow for lots of agents. Plus I don't know how to do a memory yet.

One of the goals I have is to have one large AI routine which is sophisticated enough to, just by having a large set of agent definer variables (Lazy, leader, strong, etc) can be used for every type of agent in the world from animal to more intelligent, more abilities human-like agents. I notice that there is some focus on re-using FSMs for different agents in literature, but usually grand structures have separate routines. This brings up yet another problem that I haven't tackled yet: can an individual agent transcend into a 'master planner' agent? Meaning, once a lowly agent has fulfilled its Maslov tree pretty good, and has the proper gumption (Leader, etc), can it start exerting influence over other agents - think town planning/layout, etc.

Anyways, I appreciate your feedback and look forward to being overwhelmed by something with such a scary title as 'Behavioral Mathematics'!

[LorenzoGatti]

Since you seem to already have a good grip of comparing and evaluating plans of concrete actions, you might treat money as worth the expected amount of work needed to earn it back, based on possibly wrong projections of market prices.
For example if someone buys wild rabbits at the farmer's market today for $5 and you expect to catch one per hour of hunting, a $20 restaurant meal is worth 4 hours of rabbit hunting: you might prefer to eat one rabbit at home (only 1 hour of hunting, plus cooking) or not. Whether you need to go hunting before eating or you already have $20 in your pocket isn't relevant, unless the unpleasantness of hunting depends on when you do it.
Of course there might be different available jobs; hunting and cooking one rabbit might be less unpleasant than 4 hours of rabbit hunting, but more unpleasant than a 2% quota of a quick $1000 armed robbery.

The market system shouldn't try to find "true" prices, because they don't exist.
Every agent can decide ask and bid prices, based on the other options it has; for example hunting x rabbits (sold for the needed $20) = hunting 1 rabbit (H) + cooking it (K)+ the pleasure of eating better at the restaurant (F): assuming you have numbers for H,K and F you have (x-1)H=F+K, x=1+(F+K)/H, a rabbit can be sold acceptably for $20/x and above (presumably with some greed and haggling heuristic, e.g. asking twice the reserve price at first and trading at the average between the asking and bidding prices).

[willh]

It would work like this: Agent A determines that it is hungry, and knows that it can fish in the nearby stream, or, go to a far tree to pick some fruit, and either method would satisfy its 'hunger' issue. It calculates the distance to the stream, an estimate of amount of time spent fishing modified by how much it 'likes' to fish, and then compares it to the same thing but the steps involved in picking fruit. OK, this works fine.

The problem I have is when economics and interaction with other agents occurs. Suppose all agents are willing to sell any object they possess if the price is higher than their cost to produce. And any agent is willing to buy if the price is less than their costs to produce. I am hoping that market prices will naturally arise from such a system, where the 'seller' initially offers cost + 100%, and the buyers initially offer cost - 50%.... and perhaps a short series of haggling in between until either both agree or decide not to trade.

I would assume Dave has some good ideas around how to implement this in a video game.

However, since you're interested in simulating a complex financial environement here are some ideas you might find interesting.

In the real world price dynamics emerge because people are agreeing on prices before anything actually changes hands. In other words, oil is sold before it's even pumped from the ground. When you are told that 'the price of oil is $xxx a barrel' what you're getting is an estimate based on contractual agreements for a future delivery. Think of it as an auction for something that might not actually exist.

The general (and over simplified) idea is this:
- Assume a commodity, R, (oil, gas, fish, fruit) has some fixed cost, Xf, to collect at time T.
- Buyers offer a price, Xb, for commodity R based on what they think the value will be when the commodity is actually available for pickup at T+i.
- Sellers offer a price, Xs, for commodity R based on what they think the value will be when the commodity is available for delivery at T+i.

Here comes the fun bit:
- Xb and Xs are affected by what the buyer and seller preceive as the supply and demand of the commodity. This perception is based on what both buyer and seller think future demand will be.

- The perception of supply and demand is based on what has already been agreed on for prices (a contract).
-- If Company A agrees to sell 10 tonnes of ore in 2 days at $4 /KG, the supply in 2 days set at 10 tonnes, and the demand at $4/KG. The contract is set, and someone is paying.
-- If Company B agrees to sell 10 tonnes of ore in 2 days at $3.50 / KG, the supply in 2 days is now set at 20KG, with a demand of $3.75.
-- The flip-side could be true. Someone might REALLY want ore, and offer Company B $4.50/KG, then the supply is now 20KG, with a demand of $4.25/KG
-- Note that the list demand is in no way a reflection of the cost of a new contract; it's only a reference.

This perception can be rigged. Sometimes a fish seller could buy his own fish (or his competitors fish) to trick the market in to thinking there is a surge in demand. This happens in World of Warcraft sometimes, where someone selling a potion will buy up all of the same potion from the auction house only to relist at a higher price.

The important thing to remember is that all sales are final REGARDLESS of what may become available on the date of delivery, and that buyers and sellers both have an incentive to make deals as far ahead in the future as possible.

So, Xf and Xs will determine production, Xb is a function of buyers need and their own sense of (Xf,Xs), and Xs is typically based on what the seller knowns about the compeition and the dynamics of their given market. Even Xf is unknown because anything can happen before the commodity is delivered. (Edited): This is also what drives cyclical price fluctuations-- producers continue to produce more until this is a glut, driving prices down, reducing production, which in turn causes prices to rise again, and the cycle continue

If done properly you should see wars between fish producers errupt as a way to inflate fish prices.. which should in turn force ore prices up do to increased demand of weapon sales, which should drive wheat prices up do to the increased cost of building tractors (ore is being diverted in to tanks), which should then cause the creation of government and regulatory boards as a way to bring down wheat prices, which in turn will slowly inflate the cost of wheat (due to price fixing) at the expense of fish prices. You might even see the invention of the corporation, cartels, international trade agreements, and unions! Maybe add a 'lawyer' character class so you can resolve trade disputes and mediated resolutions. (I'm joking of course)

In terms of a video game: If you have shop keepers, you might want to keep a global commodities index running, and have the shop keeper essentially act as a broker that gets him/her a premium (a function of the index price * volatility/risk * profit expectation). If you assigned production of certain regions of the world, and adjust output/cost based on how much combat (or other type of events) occures in those areas then you provide a way for player dynamics to indirectly affect global prices.

[leopardpm]

yes yes yes - those are my thoughts as well! So called 'market prices' would be forever changing depending upon the environmental situation as well as other effects (like: more agents have came into the area and all of them like rabbit stew so the demand for rabbits has increased, outstripping supply which makes the labor of hunting rabbits more valuable thereby increasing the number of agents that might decide to hunt rabbits and this all brings the market price back down to a new equilibrium (which is never actually reached because the environment is forever changing in different subtle ways).

I see you have at least some economics background (using the term 'reserve price', which most un-economic folks don't know), and you are looking at the problem through economic lenses, which is how I am as well. Instead of some 'master AI' trying to central plan the economy, I would like the individual agents to have natural market incentives/disincentives to determine how to 'grow' or otherwise increase their well-being.

IF such a system could be developed, then some very interesting game situations could arise AND content would be dynamically created instead of hardcoded by developers. for instance, a village which has easy access to fishing would naturally become a hub of fishing, trading with the mining village with its particular comparative advantages. This fluctuating market prices also end up in creating 'quests' for players who happen to be participating in the world. Like, being hired as a guard to protect the trade caravan which keeps getting robbed on the way to trading with the fishing village.... although that seems very specific and 'high-level', my feelings are that once I can get agents to evaluate possible known solutions to problems, this can be applied to a variety of different situations, from determining whether to buy a fish or go fishing, to determining how much to offer to pay for a 'guard' position of a caravan. Not too mention the decision to enter into arbitrage and take fish from one village worth $5 and sell them in another village for $20 (with all associated costs with such travel included).

lofty goal, just starting out with the basics at first and seeing if I can uncover 'patterns' and other methods to tackle the larger problems - plus its fun watching a bunch of agents on screen doing their own thing (even if it is only chopping wood or fishing at the moment!).

[IADaveMark]

I hope it has stuff that covers 'memory' as I have determined that each agent will need to remember various things in order to make decisions faster and better...

Not directly, no. However I do deal with "decision momentum" -- which is not the same. It is simply to prevent "strobing" of behaviors from one to the other and not accomplishing anything. This, of course, will be important to what you are doing anyway.

- already each agent has, what I call, a 'Task Stack' which allows tasks to be interrupted by more pressing needs (something dangerous comes into the area while trying to fish so run away for a bit until safe, then go back to fishing.... wood axe breaks while chopping wood so go get another from storage then go back to chopping for whatever reason, etc)

That is a common approach. However, you have to be aware of the possibility that a remembered task is now no longer valid by the time you return to it. Make sure you re-check upon returning to each level.

...and I dread having a more space hogging memory for these agents because I want to keep storage down to a minimum to allow for lots of agents. Plus I don't know how to do a memory yet.

Some of the shared information can be held in blackboards not only for memory purposes but for speed in calculation. For example, don't make each agent calculate the "current market price of rabbit meat" every time in may need that information.

One of the goals I have is to have one large AI routine which is sophisticated enough to, just by having a large set of agent definer variables (Lazy, leader, strong, etc) can be used for every type of agent in the world from animal to more intelligent, more abilities human-like agents.

Again, a powerful technique. However, be aware that many of your higher level thought processes are simply going to be NULL for the lower-level agents. Not everything scales!

I notice that there is some focus on re-using FSMs for different agents in literature, but usually grand structures have separate routines. This brings up yet another problem that I haven't tackled yet: can an individual agent transcend into a 'master planner' agent? Meaning, once a lowly agent has fulfilled its Maslov tree pretty good, and has the proper gumption (Leader, etc), can it start exerting influence over other agents - think town planning/layout, etc.

This is similar to above. Don't think of it has "transcending" but rather switching certain aspects on and off. Through communication of information and biasing influence, you can quite easily have one agent affect others by subtly "turning their knobs."

Anyways, I appreciate your feedback and look forward to being overwhelmed by something with such a scary title as 'Behavioral Mathematics'!

Meh... I just chose the title to impress chicks.

[IADaveMark]

Since you seem to already have a good grip of comparing and evaluating plans of concrete actions, you might treat money as worth the expected amount of work needed to earn it back, based on possibly wrong projections of market prices.

Excellent point... all of them can be abstracted into a generic "unit". Labor, desire, money, etc. Jeremy Bentham did something similar with his "hedonic calculus" -- something which I touch on in the book.

Of course there might be different available jobs; hunting and cooking one rabbit might be less unpleasant than 4 hours of rabbit hunting, but more unpleasant than a 2% quota of a quick $1000 armed robbery.

Remember to include the potential for success and failure -- and the ramifications of each. For example, what is the risk-reward ratio of the armed robbery? If the penalty is negligible, then give it a shot. If the penalty far outweighs the gain, then pause. But what is the potential for getting caught? If the penalty is death, but you have a 0.001% of getting caught, it doesn't matter. Fun with math, folks!

The market system shouldn't try to find "true" prices, because they don't exist.

Another excellent point. This is how you can build true market systems. Everything is relative. A well-designed system will automatically self-balance based on the environmental supply and demand, the numbers you used for time costs of production, the satisfaction rates of the goods, etc.

Interestingly, while it isn't hard to create a system that self-balances based on these factors, it is actually hard to make it balance in such a fashion that things make sense to us when we compare them to reality. For example, if I told you the going price for AbstractObjectA was $100, you would have no frame of reference and just accept it. If I told you that AbstractObjectA was rabbit meat per pound, you might be mildly alarmed. We have just broken the suspension of disbelief because it doesn't match up with our pre-conceived notion of what rabbit meat should cost.

[IADaveMark]

I would assume Dave has some good ideas around how to implement this in a video game.

Certainly... but I read more books on economics, psychology, and math than I do on games. *shrug*

In the real world price dynamics emerge because people are agreeing on prices before anything actually changes hands. In other words, oil is sold before it's even pumped from the ground. When you are told that 'the price of oil is $xxx a barrel' what you're getting is an estimate based on contractual agreements for a future delivery. Think of it as an auction for something that might not actually exist.


In terms of a video game: If you have shop keepers, you might want to keep a global commodities index running, and have the shop keeper essentially act as a broker that gets him/her a premium (a function of the index price * volatility/risk * profit expectation). If you assigned production of certain regions of the world, and adjust output/cost based on how much combat (or other type of events) occures in those areas then you provide a way for player dynamics to indirectly affect global prices.

That's a great point. People in the real world bitch about how gas prices at the pump increase immediately when the price of oil increases. "But you didn't pay that much for this gasoline! How come you won't sell it at the price you paid for it?" The simple answer is, "because I need to be able to afford to buy more to replace it."

Therefore, in the shopkeeper example, you should not use of what the shopkeeper paid for it. You need to calculate what he would have to pay for it if he wanted to replace his supply. Doing it the other way punishes him for buying a huge inventory when the price is low... which is economics rule #1 (#2 being "sell high").

However, there is a reason to keep track of what the shopkeeper paid... and that is that he may not want to sell at a certain price. For example, if he has an inventory of A, and the price of A drops below what he paid (in cost, effort, etc.) he may find it beneficial to sit on his inventory and hope the price goes back up. Why sell at a loss? (Of course, then you get into a classic situation where the price continues to drop and he must decide to cut his losses by selling... but that's why economics is so much fun!)

[leopardpm]

yes, it is a common misconception of the average person to assume that prices are 'determined' by costs, where actually it is backwards: prices determine costs. The reason a particular apple is priced at $1 is NOT because it costs $0.50 to pay the laborer to pick it and the owner wants to make a profit, but rather the laborer is paid $0.50 BECAUSE the apple is only worth $1 on the market. And, really, its not 'prices' as if they are set, its 'valuations'.

Which brings us back around to, what I see as a major flaw in my method for choosing which task to attempt in order to satisfy a need: If I base the calculations on 'labor type and time', then I have set into the system an economically fraudulent assumption - Marx's Labor Theory of Value. and it would definitely be the ruin of all the world over, or at least have a world of 'ants' instead of simulating people to some capacity....

I am also toying with the idea of not forcing a monetary system and seeing if one springs forth - but the ability to 'spring forth' involves an awareness that I don't think I can code into these agents. But, maybe if I focus on just commodity valuations: ie 10 fish = 2 ore, 20 wheat = 10 fish, 10 wheat = 1 ore, etc then perhaps a certain commodity will spring up as being 'the easiest' in each locality as a money base (fish=money in fish town, ore=money in ore town... or perhaps it ends up being reversed where ore=money in fish town and fish=money in ore town because the units are more valuable respectively in those particular locals. I think I will push such dreaming off until I get to that bridge, and keep focusing on the individual details, the small tasks, and see what happens...

I have been researching this idea for over a year, and my travels have taken me to economic simulators that economists have tried to create... but they almost always use a top-down approach, central planned, without even thinking about it.... they are trying to derive 'formulas' which model human behavior instead of focusing on the individual differences between human actors in an economic setting. I think their approach is a fool's errand. You can't calculate a person's ever-changing desire. Though I guess that is exactly what I am trying to do too!!! LOL!

we shall see.

[leopardpm]

LOL! I just experienced my first 'emergent' behavior - not really, but it was 'fun':

I had been playing around with my sim (as it were) and started adding the rudimentary bits for 'digging' (I am thinking that to get 'ore' one needs to do a bit of actual excavation to get to it), but figured that since I hadn't put in the hooks yet to garner anything useful to the agents (they don't know what ore is yet so have no reason in their thinking to dig for it) then I wouldn't see any effect. Well, after playing the sim a few times (its fun to watch), I noticed every once and awhile a dude would start digging like crazy! I was like, "WTF!" . After clicking on him and trying to figure out the 'error' of my ways... it turns out that I had forgotten that I put 'gardening' in as one of the things that agents can 'like to do' when they are 'relaxing' - the clue I had was that all the 'diggers' were on the LAZY side of their personality traits - (I am trying to incorporate the Disutility of Labor into the agents so that they will, if nothing too pressing is needed, choose to do something they each consider 'fun' - reading, gardening, napping, playing games (hopefully they will go to the local tavern and play some 'virtual chess' or gambling games at some point)), etc. Well, one of the things about 'gardening' was they would 'like' dirt... and the action of digging produces... dirt! Very fun, the thought of these little dudes digging around their world for pleasure because they liked 'dirt'!

oh well, the episode points to a few different things I learned - as more and more game objects and possible actions are introduced, the complexity becomes exponential... alot more probability for interesting behavior... and alot more possibility for 'strange' unnatural behavior (judged by standard human perspective, not from the perspective of the agents themselves!).

Yeah! AI is Fun!

[leopardpm]

BTW - has anyone else been amazed at the AI used in the game Dwarf Fortress? That is one of my 'examples' as far as trying to get agents to do interesting and yet 'natural' things that kinda make sense from an outsiders perspective.

[leopardpm]

sorry about the multiple posts...

DaveMark, I just ordered your book (mainly because I went ahead and just finished reading the first chapter available on Amazon, and secondly, I read an interaction between you and another somewhere else in this forum and I appreciated the advice you gave (understanding their particular programming/AI ability: novice) and how you dealt with an adversarial poster that considered your advice 'wrong' - which it wasn't.

Now I gotta wait 5 days for the book to get here...

meanwhile, are there any other books you might recommend? What about Steve Rabin "AI Wisdom" et al?

[IADaveMark]

sorry about the multiple posts...

DaveMark, I just ordered your book (mainly because I went ahead and just finished reading the first chapter available on Amazon, and secondly, I read an interaction between you and another somewhere else in this forum and I appreciated the advice you gave (understanding their particular programming/AI ability: novice) and how you dealt with an adversarial poster that considered your advice 'wrong' - which it wasn't.

Thank you for the order and the compliment. I hope the book helps.

Now I gotta wait 5 days for the book to get here...

That part I can't help with. :-(

meanwhile, are there any other books you might recommend? What about Steve Rabin "AI Wisdom" et al?

See the pinned topic on this forum. Note that the AI Wisdom books are a collection of articles between 10-18 pages from various authors. (Full disclosure, I wrote for #4.) Therefore, it isn't as good a foundation but more tips and tricks. Same with the Gems books (I wrote for #8).

[LorenzoGatti]

Of course there might be different available jobs; hunting and cooking one rabbit might be less unpleasant than 4 hours of rabbit hunting, but more unpleasant than a 2% quota of a quick $1000 armed robbery.

Remember to include the potential for success and failure -- and the ramifications of each. For example, what is the risk-reward ratio of the armed robbery? If the penalty is negligible, then give it a shot. If the penalty far outweighs the gain, then pause. But what is the potential for getting caught? If the penalty is death, but you have a 0.001% of getting caught, it doesn't matter. Fun with math, folks!

I'm of course referring to expected value, which in the case of rabbit hunting can be easily modeled as a random variable (time needed to catch one rabbit) or two (time needed to find a rabbit, and time needed to kill it) but in the case of a robbery requires a rather complex weighted tree of outcomes and random variables (successful completion, losing the loot to run away, being caught, kinds of penalty if caught, loot obtained if not caught, amount of fine if fined, length of vacation if jailed, choice of prison if jailed...).

[TechnoGoth]

It’s an interesting project and one I’ve toyed with myself a few times.

One factor you can include in your decision making is the value people put on their time. This is normally affected by a person’s earning potential, circumstances, and other values but generally we all have an internal figure which says how much our time is worth.

Take the following scenario.

An agent has 3 ways to get to work. 1) Walking costs nothing but takes 2 hours. 2) Taking the bus costs $2 takes 40 minutes and only leaves at set times. Or 3) take a taxi which costs $12 and takes 10 minutes.

The option to take in the case would be based on the agent’s current funds and the value they put on their time.

An agent who has a value of $10 an hour will pick option 2 but if the agent has a value of $100 an hour they will go for option 3.

The same would apply to your scenario of fishing, picking berries, and buying from the market, but in that case you’d also want to adjust the result by the agent’s likes. I’d probably make likes and dislikes magnifiers in this case. If an agent likes fish(x2) and hates berries (x4) then he more or less has to be standing next to the berry bush and starving before they would choose them over going fishing.

Then of course there is whole other level magnitude you can look at once you everything working and that’s the interplay between short term goals and long term plans. For instance in my getting to work example how would the short term goal change if you wanted to consider the long term plan of thinking of buying a car?

[leopardpm]

Of course there might be different available jobs; hunting and cooking one rabbit might be less unpleasant than 4 hours of rabbit hunting, but more unpleasant than a 2% quota of a quick $1000 armed robbery.

Remember to include the potential for success and failure -- and the ramifications of each. For example, what is the risk-reward ratio of the armed robbery? If the penalty is negligible, then give it a shot. If the penalty far outweighs the gain, then pause. But what is the potential for getting caught? If the penalty is death, but you have a 0.001% of getting caught, it doesn't matter. Fun with math, folks!

I'm of course referring to expected value, which in the case of rabbit hunting can be easily modeled as a random variable (time needed to catch one rabbit) or two (time needed to find a rabbit, and time needed to kill it) but in the case of a robbery requires a rather complex weighted tree of outcomes and random variables (successful completion, losing the loot to run away, being caught, kinds of penalty if caught, loot obtained if not caught, amount of fine if fined, length of vacation if jailed, choice of prison if jailed...).

When I have looked at this aspect with my limited knowledge of AI, my first instinct was apply the following formula:

(1) quick estimate based on known factors (distance to goal, amount of time to work it, distance back, etc). Remember, the 'goal' might be 'Build a shelter', of which one part will be gather the necessary resources - BUT not just gather them, also transport them to the desired location of where the shelter is to be built. So need to include the 'distance back' figure.

(2) Check a 'memory' table (don't know how to do this efficiently yet - need to learn) of previous attempts of achieving each part of the goal.

the #2 (check memory) was an attempt to estimate tasks which had a random time component and random 'danger' component - for instance: hunting rabbits! If agent goes out hunting rabbits, and spends x time looking (its variable), then y time actually trying to kill the thing, then another z time traveling back home... then next time he can use the previous total time as a weighted average for a better guestimate. This method will also allow for agents to factor into a decision if a task is too dangerous or not. In my thoughts, I wanted to accomplish two things here: (1) be able to predict an outcome roughly if never attempted, (2) to better predict an outcome if done before, and have this prediction change when outside influences affect the task (for instance, the first 10 times he goes out to hunt rabbits, it takes a running average of 100 turns, but, he (and possibly others) have now decimated the local rabbit population and it takes longer and longer to find rabbits to hunt - or, dangerous entities have moved into the area and rabbit hunting becomes more risky as time wears on): so each agent needs to have a way to constantly adjust their predictions to reflect a dynamic environment and the actions of other agents (and non-agents: players!)

An agent who has a value of $10 an hour will pick option 2 but if the agent has a value of $100 an hour they will go for option 3.

The same would apply to your scenario of fishing, picking berries, and buying from the market, but in that case you’d also want to adjust the result by the agent’s likes. I’d probably make likes and dislikes magnifiers in this case. If an agent likes fish(x2) and hates berries (x4) then he more or less has to be standing next to the berry bush and starving before they would choose them over going fishing.

the 'time value' thing is relatiely easy to calculate EXCEPT for determining the 'unit' of value to use (this is, in essence, the money unit) - I can randomly choose a unit of any commodity (berries, ore, wood, chairs, sandals, etc) but need to figure out a way to actually make a choice between using one type of unit vs another based on logical agent rational - the same determination for what commodity to use for money: (1) needs not to depreciate over time (2) easily divisible (3) easy to transport and carry (4) recognized as valuable by others (5) and I forgot the 5th quality of money that humans have looked for in the past.... How do I program an agent to weigh these abstract qualities of each type of commodity (potential money)? Don't know yet - its a problem that may just need to be solved by brute force (ie: introduce GOLD and force all agents to calculate in those terrms)

[TechnoGoth]

That is true and it’ll depend on what you are trying to achieve whether you want money to exist as entity in of itself or design your sim to have an emergent monetary system.



Then again rather then a commodity based economic system you could always have a labour based one. An agent only has 1440 minutes of “work” available each day how much a unit of work is worthy would very from agent to agent, and would influence their decisions. But again you’d need mechanism of evaluating the results of a task against the cost of doing that task in terms of that agents own work value. This would require some kind of community knowledge and an assessment system.



So that you can evaluate cases such as you have a fisherman and lumberjack. If the fisherman needs wood is it better for him to spend an hour gathering the wood himself or spend and hour fishing and then trade some fish to the lumberjack for the wood.