Bayesian Decision Theory

[Chaerophon]

I'm revising a paper that I've written on John Rawls and I've come across some articles by John C. Harsanyi that have piqued my interest, and require an understanding of Bayesian theory that goes beyond my very limited capabilities. If anyone here (I'm suspecting that some do...) has even basic knowledge of this stuff, could you post in this thread or drop me a PM? The input can be derived from any context at all; I understand it's a popular topic among AI buffs. All that I need is a good, basic link (which I've yet to find) or a basic description of the principles behind the theory. Math's ok, but KISS please! (if that's possible). :) Thanks!

[Guest]

The math is trivial. In fact, despite all the noise about Baysian filtering lately, the whole concept is pretty simple. The interesting thing is that it can be done completely automatically, and yields better results than having a person try to do the same thing by hand.

In plain english, one feeds the analyzer two sets of inputs--a 'Good' set and a 'Bad' set, each containing many objects. Each object has properties that can be analyzed statistically. One typically feeds in several thousand objects of each class.

For instance, the objects might be email messages, with 'Good' ones being real mails and 'Bad' ones being advertizements. In this example, the words that appear in the messages would be the properties that the analyzer uses.

Continuing this example, the analyzer might find that the word 'Enlargement' appears in 10% of the advertizements, but in only 0.1% of the real mails. Thus, when presented with a new email the presence of the word 'Enlargement' is an indication that the mail is an advertizement. In fact, using just that one word as an indicator, the odds are 10% to 0.1%, or 100-to-1 after moving the decimal places.

Now imagine that the unknown email contains several words that are known to be indicators. The Baysian step is to multiply together the odds for each of the words to get an aggregate estimate for the odds of the unknown email being good or bad.

Hope this helps. Try googling for 'Baysian email filtering' if you need to know more. There's also lots of free source-code out there for this.

[yangman]

Woo, I just learned the basics of this in my discreet mathematics course. :P

Basically, Baye's Theorem relates P(A|B) to P(B|A), that is the probability of A given that B has happened to the probability that B happens when it is known that A happened.

So, say you build a database of commonly used words in spam emails, and how often they occur, which naturally gives you a set of probabilities of words to occur in those emails. Apply Baye's theorem, and you can derive the probability that an email is spam, given that it contained those words.

I could be wrong, of course, in regards to the actual application, since I'm just making this up as I type, but the parts about how Baye's theorem works is true, afaik. ;)

[Chaerophon]

Ok, I've got the theorem now, I think. (I've had to take the requisite stats course for my Poli Sci major, so I see what's going on there, at least mathematically I do...) Thanks to both of you for that. Now, the question is, how to apply it to decisions made under uncertainty. In other words, if an individual A must choose from one of several potential societal models from behind a "veil of ignorance", a position wherein A has no knowledge of their social status, wealth, relative intelligence, etc., how could "Beyesian utility" be a part of their decision process?

What I've got so far from Harsanyi is that the individuals know that their likelihood of being the best or the worst off in society when the "veil drops" is equal to 1/n. (n being = to their relative position). What they don't know is what the relative distribution of wealth will be, i.e. what percent of the people are terribly poor, extremely wealthy, etc... However, since each has an equal probability of obtaining any given relative position in society (1/n), he argues that they will select the societal form that they presume will have the highest "average utility", through an analysis on the basis of "subjective probability". I assume that this means that, in the absence of "objective" knowledge, and the probabilities that go along with it, they "go on a hunch". They assign values to the societal attributes that they subjectively value, and add up the estimated totals for each societal model. The one that scores highest is the one that they go with. The example that he provides is that, since under all societal models, one's probability of returning at a given position is the same, 1/n, the chooser will choose the model with the highest potential reward in terms of average utility. So... Bayes' theorem applies how? That looks like straight up multi-attribute utility analysis... Is the theorem in the motivation for the choosing strategy, itself? I.e., since probability of being, say, worst off, is the same in all models, regardless of one's choice, (Prob 1/n given choice A is the same as prob 1/n given choice B?), one's choice will not be maximally risk-averse, but will be made through a subjective estimation of average utility...?

Hmmm...


Help!!!!

[kandrathe]

Consequentialism and Bayesian Rationality in Normal Form Games -- Peter J. Hammond

This reference from the paper seemed to be along the lines you are exploring.

Pierpaolo Battigalli (1996) “Comment [on Mariotti (1996)]”, in: Kenneth J.
Arrow, Enrico Colombatto, Mark Perlman, and Christian Schmidt
(Eds.) The Rational Foundations of Economic Behaviour (London:
Macmillan), pp. 149–154. Published by: Palgrave

Deep topic and hard to KISS. :)

This lengthy treatment of the topic also seems to go into depth into the treatment of Bayesian decision making within economic models.
Behavioral Economics: Past, Present, Future -- Colin F. Camerer

[Jester]

"In other words, if an individual A must choose from one of several potential societal models from behind a "veil of ignorance", a position wherein A has no knowledge of their social status, wealth, relative intelligence, etc., how could "Beyesian utility" be a part of their decision process?"

(again, here's me stepping in way over my head...)

If I recall correctly from my Political Philosophy course, the objective Rawls is trying to get at with the "veil of ignorance" is a kind of pareto-optimal society. Not everyone has to be equal, but the distribution must be such that any way to improve the lot of anyone improves the lot of the least well off person.

Beyesian utility, then, would probably be a statistical method of determining how near or far some situation is from pareto-optimality, and therefore from the kind of liberal equality Rawls is promoting. It doesn't seem to fit very well with his philosophy as I understand it, but that might just be my poor understanding.

"What I've got so far from Harsanyi is that the individuals know that their likelihood of being the best or the worst off in society when the "veil drops" is equal to 1/n. (n being = to their relative position). What they don't know is what the relative distribution of wealth will be, i.e. what percent of the people are terribly poor, extremely wealthy, etc... However, since each has an equal probability of obtaining any given relative position in society (1/n), he argues that they will select the societal form that they presume will have the highest "average utility", through an analysis on the basis of "subjective probability"."

This is distinctly *not* what I remember Rawls promoting. A society where the average (mean) is high, but the lowest are significantly worse off than the highest is not a society he wants to emerge from the "veil of ignorance". His notion is that nobody would want to chance being a peon just for the 1/1000000000 shot at being emperor, even if that emperor was a trillion times better off than the peons. They would pick a more egalitarian model, one which is not utility maximized (unless the equal model happened to be the most useful). Since the "veil" proposes a game in which nobody even knows what player they are yet, a very egalitarian society would make the most sense for everyone. Rawls' philosophy was very much a rejection of accepted utilitarian notions for that reason.

Jester

(other thoughts: I seem to remember Rawls being a bit of a ninny at game theory... I also think that the Beyes model was probably introduced as a theoretical framework accounting for the different preferences of different people, although it seems to me that this is a problem which is largely outside the scope of the "veil of ignorance" thought experiment.)

[Chaerophon]

Jester,

You've got it all right. Harsanyi was a critic of Rawls. He endorsed Rawls's use of the original position, but felt that his conclusions were off (Entirely missing the point in the process, if you ask me, but that's beside the point). Unlike Rawls, he believed that individuals behind the veil of ignorance would choose according to a decision rule based on subjective calculations of expected average utility rather than maximin (best for the least well off). Apparently, this is on the basis of Bayes' Theorem... :) Hence my problem.

[Chaerophon]

Thanks for the links Kandrathe, I'll have to look at them when I get to school. I haven't been able to view .pdf's on my comp for the last couple of days. Great time for it to get uppity <_<

[Jester]

The "veil of ignorance" is a somewhat flawed model for support of Rawls' position, in that (as Harsanyi apparently points out) if you were impartial, you very well might choose a social system that maximizes utility, rather than maxmin. Assuming the total utility for society was maximized (not just one kind of utility, but all kinds), then perhaps a "veiled" being would see the greater good of all being more important than the possibility that he (or someone else) would get stuck with a bad position. I think the Bayes Theorem is quite trivial to the problem, since it is nothing but a statistical model, while the actual problem is philosophical (or psychological, in a sense).

However, it has always seemed somewhat academic to me, since I'm with Adam Smith on this one: a well run system tends towards equality, and general equality tends to make a system run better. So a society which is utility-maximized with also be fairly close to pareto-optimal (though very unlikely to be Nash-optimal, since opportunism will always provide some way for a player to increase their share), with a fairly high level of equality. I think Rawls would have agreed with this position, and his critics are mostly people who believe either that this is not true, or just not necessarily true.

Jester

[Vornzog]

Chaerophon

Don't know how KISS this will be, but it should help a little on the math side of things. Afraid I don't know much about it in the context you are really looking for

Monty Hall Door Problem

(For the purists out there, this is not exactly the game played on Let's Make a Deal, but it is better for describing the math.)

There are three doors. Behind two of them are Greased Rabid Weasels (GRWs). You don't want these. Behind the other door is the cash/car/vacation package/etc. You pick a door. Monty Hall opens on of the other two doors, showing you a GRW. You have the choice to stick with your original door, or switch. You get to keep what is behind the door you choose. Should you stay with your original door, or switch to the other door to maximize your chances of winning the good prize.

Remember that Bayes theorem is something like

P(O|M) = P(M|O) * P(O) / SUM(P(M|Oi) * P(Oi))

Here, O is an observable - this is what you see behind a door. M is the model. You could switch doors, or stay with your first choice.

Let's start filling in some numbers. P(M|O) is the probability you should switch, if you know where the money is. This is going to be 2/3 in this case. Think of it this way - 1/3 of the time you picked the right door to start with. So 2/3 of the time, you pick the wrong door, see a GRW behind a second door, and the money will be behind the third door - the one you can switch to.

P(O) is the probability of picking the money. This is 1/3, even after Monty Hall has opened a door for you. Remember that you made the choice without knowing which door he would open, so your probability of picking the correct door the first time is only 1/3, not 1/2 as many people will argue. Draw yourself pictures if this hurts your brain.

The bottom is the sum over the probabilities for all the models. Written out more explicitly,

P(switch|$) * P($) + P(stay|$) * P($)

Remember that the probability of you choosing the money correctly is only 1/3 - this is still P($). We already said that the probability you should switch if you knew where the money was was 2/3, so the probability you should stay is 1/3.

If you plug everything in, you'll get that switching gets you the prize 2/3 of the time.

Disease

Here is another concrete example for you to try. A genetic disease is found in 1 of every 1000 people. A new test has been developed for it. If you have the disease, the test will give a positive result. It also has a 5% false positive rate. If you get the test and get a positive result, what is the probability that you have the disease?

Try setting it up. Remember that the denominator is the sum over the probabilities of all the ways of getting a postive result. If you get everything set up right, you should find a positive result gives less than a 2% chance of having the disease.

Now try to apply the idea to your 'highest average utility' situation. Each person will consider all the different models, asking which has the highest probability of being good for them. It is Bayesian because they don't know which model will be the absolute best for them, but they will try to choose the model that has the highest probability of being the best. In order to figure that out, you'll need to know something about the different models and the probability of getting a favorable result for those models. This is just like the door problem. You have two models for how to behave (switch, stay). There is one favorable outcome and two unfavorable GRWs. You know something about your chances of picking the correct door initially. Pick the model that benefits you the majority of the time.

That whole concept of the veil really throws me - Bayesian statistics is all about knowing something in advance, and the veil of ignorance sounds like a bad item to get in a role playing game or when trying to choose an appropiate society.

Hope that helped in some way,

-V-

[Nystul]

If you plug everything in, you'll get that switching gets you the prize 2/3 of the time.

Isn't this the wrong conclusion? Or at least poorly worded? 2/3 seems to be your total probability of getting what you want by making the logical choices. Whether to switch or not depends on what you see the first time. If door number 2 is "bad", the choice of whether to stay with door number 1 or switch to door number 3 is random: 50% chance to win if you stay, 50% chance to win if you switch. If door number 2 is "good", the chance to win is 100% if you switch to door number 2, 0% if you stay on door number 1, and 0% if you switch to door number 3.

So, there is a 1/3 chance that the first door revealed to you is good, in which case you will take it and guarantee a win. There is a 2/3 chance that the first door revealed to you is bad, in which case you will randomly take one of the other two doors which will then each have a 1/2 chance of being good. Thus your total chance of getting the prize by this method is (2/3)*(1/2) + (1/3)*(1) = 2/3.

I'm not quite sure how this concept applies to the society discussion, and I have absolutely no background knowledge on any of this. It seems to me that a person would only be guaranteed to pick the more egalitarian system if they would consider the enhanced minimum place in that society to be a "good" result. Otherwise, the person would consider risking a system in which he could end up far worse off, if it gives a greater probability of him reaching the lifestyle he would be happy with. For an example with maybe less variables than an entire society, if a person was given the choice: either live in a slum or have a 50/50 chance between a ranch house or a cardboard box, the choice is not an automatic one. It is only an automatic choice under the assumption that living in the slum is considered a good outcome. If we improve the first case to 80% chance of living in a slum and 20% chance of living in a ranch house, it's still not an automatic choice.

[--Pete]

Hi,

door number 2 is "good", the chance to win is 100% if you switch to door number 2, 0% if you stay on door number 1, and 0% if you switch to door number 3.

That can never happen. You are always shown what is behind a "losing" door. If you had picked the "winning" door, it could be either of the "losing" ones. If you had picked one of the losing ones, you would be shown the other losing one.

Simple analysis:

On the first guess:
1/3 of the time you will pick the winning door.
2/3 of the time you will pick one of the two losing doors.

If you do not change after being shown that a door you did not chose was a losing door, you will win 1/3 of the time you play.

If you do change, then:
If you had originally picked the winning door, you would change to the remaining losing door (1/3 of the time).
If you had originally picked a losing door, you would then change to the winning door (2/3 of the time).

Bayes' Theorem can be used to derive this result, but the derivation is only "clear" in a mathematical sense. However, Bayesian analysis is more than just the theorem. The fundamental concept of Bayesian analysis is that probabilities can be assigned by means other than the strict frequency interpretation. While that may be true (and, indeed *is* true, since not even the most rabid "frequency" believer has ever spent infinity flipping a coin -- nor probably ever will), the "gut feel" interpretations sometimes lead to strange -- and even foolish -- results. Of course, once an a priori estimate of the probabilities is made (however it is made), using Bayesian methods (again, mostly the theorem) to refine the estimates is both a valid and a common technique, used by everyone in methods both casual and formal.

The "application" of Bayesian analysis to the socio-political arena is something that I will not get involved in. Frankly, I think that it is mostly a crock, much like the application of Darwinism or of relativity. Applying poorly understood mathematics to a subject that is not (yet, anyway, since Hari Seldon is still in the far future) at all quantitative is fun at a cocktail party (assuming that cocktail parties ever were or will be fun). When supposed serious "intellectuals" do so, then I am reminded of one of my colleague's favorite expression: "metal masturbation". ;)

--Pete

EDIT Must *not* hit ENTER after spell checking without restoring the focus to the input box :(

[Chaerophon]

Wow, very helpful. At least I can see from that what the Bayesian calculation is intended to do! I'll need to refocus on it a bit later to get the math done. (Exam time, and I'm trying to learn something new...stupid) Anyways, Harsanyi suggests that the choices made under uncertainty will be on the basis of some sort of internalized subjective multi-attribute utility analysis (von Neumann-Morgenstern???) that is based on what value each person will subjectively put on certain societal factors... i.e., I suppose, various freedoms, etc.

So... I suppose that the role of Bayesian theory in this process is to delineate the fact that individuals faced with such a choice will always choose the outcome on the basis of a maximal score under this von Neumann - Morgenstern test, whether they are assured of its veracity or not, whereas under Rawls's scheme, they always interpret their chances at failure as n=1. Sorry, I'm a game theory newb. :) I think that sounds reasonable though.

[Nystul]

That can never happen. You are always shown what is behind a "losing" door.

Ah, so that is a rule of the game rather than just being what happened in this instance of the game. That does make quite a difference. Although by either set of rules, you make a random choice followed by an informed choice and end up looking like a fool 1/3 of the time. :)

[kandrathe]

I've heard of "mental masturbation" or perhaps your colleague was a smith? :D Of course, my mind is a flurry of possibilities.

My trouble with the topic is that few economists and sociologists apply the mathematical and statistical theory with rigor. Often their conclusions sound correct or reasonable, but upon scrutiny are only half baked.

[--Pete]

Hi,

And hear eye though eye prof red hat goo :)

My trouble with the topic is that few economists and sociologists apply the mathematical and statistical theory with rigor.

I don't see how any can. After all, it is a numerical theory, requiring some good data without which it is subject to the GIGO law. And even with the best of data, it appears that social interactions are chaotic, both in the general and the technical sense of the word. So, applying a rigorous mathematical analysis on a model that is insufficient to capture the essence of the situation and starting with data that doesn't necessarily have any validity seems nothing more than a way to generate the assumptions which will then be used to generate the conclusions which were desired in the first place. Rationalizing rather than rational.

Often their conclusions sound correct or reasonable, but upon scrutiny are only half baked.

"Half baked"? I'd say hardly kneaded. Or perhaps not needed at all. ;)

--Pete

[Chaerophon]

I tend to agree, it does seem a bit, hmmm, perhaps we could say "constructed to meet his purposes"; however, he did win a Nobel Prize for it, at the same time as John Nash. :) That being said: fair enough!

[kandrathe]

My experiences have only been through osmosis from my spouse who's undergraduate degree is in applied statistics. While working on her masters degree she worked as a demographer for a retailer. She seemed to be able to make predictive models for human behavior based on a myriad of economic variables, but only for short periods of time. I remember many variaties of segmentation models and confidence intervals. So from her work, I can see that statistics can be beneficial predictive tool if applied correctly. But, she was a mathematician who was constantly struggling with co-workers who wanted to apply voodoo.

The other area I've worked in is in applying neural networks to predict specific commodity prices. The difficulty is in capturing all the real world factors that influence the price.

[Vornzog]

Hey Nystul,

Glad Pete helped clear that up for me a bit. When you were shown a losing door, you got some extra information. Indeed, you will then try to make the informed choice, picking the door that has the higher probability of giving you a good prize. Bayes is a little counter intuitive...

Two points of clarification on the door problem:

1) If you ignore the extra information and choose randomly between the two remaining doors, you'll get a good prize 50% of the time, just like you'd expect.

Another way to see the extra information is to consider a much more extreme case. What if there were 1000 doors. You would pick one - your chances that it was the one with the prize is 1/1000. Monty Hall opens 998 doors. You can either stay with this door, or switch. Which would you choose now? You still don't know for sure where the good prize is, but you have a much better idea.

2) In the original game, I think Monty Hall might have opened one of the doors at random. If he opened the door with the good prize, I think you could just switch to that one and win 100% of the time. The way I set it up is the way it is usually discussed for the purposes of Bayesian statistics - it makes it a bit cleaner problem.

With that said, I still don't see how anyone could ever apply this to society in a rational manner. It is fine to speculate about the theory, but lacking hard data, it would seem to remain strictly in the theoretical domain.

My (extremely limited) understanding of game theory suggests that people often don't behave in what would seem to be the 'most logical' way when presented with a choice or decision, and especially not when more than one person is involved. Staticital decision analysis in a non-quantative setting would seem to cause more headaches than it is worth...

-V-

[Vandiablo]

2) In the original game, I think Monty Hall might have opened one of the doors at random. If he opened the door with the good prize, I think you could just switch to that one and win 100% of the time.

Maybe, the first time it was played. But when I watched Let's Make A Deal back in the 70's, it was always always always a "bad" door that was opened up before the choice of switching was made.

I don't think this door switch game, tho, is the best example to illustrate Bayes Theorem, at least not for the way I learned Bayes.

IIRC the best examples are the ones using medical tests -- ... hmmm, say the odds of you (as an untested person) having Gödel's Syndrome are 1 in 10,000, the odds of getting a false positive are .01 and the odds of getting a false negative is .005, and you get a "positive" test result, what is the (new) probability that you actually have the disease??

Hmm, test 10000 people, get 99.99 false positives, and 0.995 real positives. So the odds of you actually the disease are (0.995)/(99.99+0.995) or 0.009853. (That's the Bayes part, the division.

So, even though the test is 99% accurate, your actual odds of having the disease are still less than one percent. Sure, that's much higher than 1/10000, but it's not anything near 99/100.

-V

edit: added quote to quote