華遠(yuǎn)地產(chǎn)股份有限公司ppt課件

1、Learning with Bayesian NetworksDavid HeckermanPresented by Colin Rickert太原房產(chǎn)網(wǎng) 52youju.Introduction to Bayesian NetworksBayesian networks represent an advanced form of general Bayesian probabilityA Bayesian network is a graphical model that encodes probabilistic relationships among variables of inter

2、est1The model has several advantages for data analysis over rule based decision trees1.OutlineBayesian vs. classical probability methodsAdvantages of Bayesian techniquesThe coin toss prediction model from a Bayesian perspectiveConstructing a Bayesian network with prior knowledgeOptimizing a Bayesian

3、 network with observed knowledge (data)Exam questions.Bayesian vs. the Classical ApproachThe Bayesian probability of an event x, represents the persons degree of belief or confidence in that events occurrence based on prior and observed facts.Classical probability refers to the true or actual probab

4、ility of the event and is not concerned with observed behavior.Bayesian vs. the Classical ApproachBayesian approach restricts its prediction to the next (N+1) occurrence of an event given the observed previous (N) events.Classical approach is to predict likelihood of any given event regardless of th

5、e number of occurrences.ExampleImagine a coin with irregular surfaces such that the probability of landing heads or tails is not equal.Classical approach would be to analyze the surfaces to create a physical model of how the coin is likely to land on any given throw.Bayesian approach simply restrict

6、s attention to predicting the next toss based on previous tosses.Advantages of Bayesian TechniquesHow do Bayesian techniques compare to other learning models? Bayesian networks can readily handle incomplete data sets. Bayesian networks allow one to learn about causal relationships Bayesian networks

7、readily facilitate use of prior knowledge Bayesian methods provide an efficient method for preventing the over fitting of data (there is no need for pre-processing).Handling of Incomplete DataImagine a data sample where two attribute values are strongly anti-correlatedWith decision trees both values

8、 must be present to avoid confusing the learning modelBayesian networks need only one of the values to be present and can infer the absence of the other:Imagine two variables, one for gun-owner and the other for peace activist. Data should indicate that you do not need to check both values.Learning

9、about Causal RelationshipsWe can use observed knowledge to determine the validity of the acyclic graph that represents the Bayesian network.For instance is running a cause of knee damage?Prior knowledge may indicate that this is the case.Observed knowledge may strengthen or weaken this argument.Use

10、of Prior Knowledge and Observed BehaviorConstruction of prior knowledge is relatively straightforward by constructing “causal edges between any two factors that are believed to be correlated.Causal networks represent prior knowledge where as the weight of the directed edges can be updated in a poste

11、rior manner based on new data.Avoidance of Over Fitting DataContradictions do not need to be removed from the data.Data can be “smoothed such that all available data can be used.The “Irregular Coin Toss from a Bayesian PerspectiveStart with the set of probabilities = 1,n for our hypothesis.For coin

12、toss we have only one representing our belief that we will toss a “heads, 1- for tails.Predict the outcome of the next (N+1) flip based on the previous N flips: for 1, ,ND = X1=x1, Xn=xnWant to know probability that Xn+1=xn+1 = heads represents information we have observed thus far (i.e. = D.Bayesia

13、n ProbabilitiesPosterior Probability, p( |D,): Probability of a particular value of given that D has been observed (our final value of ) . In this case = D.Prior Probability, p( |): Prior Probability of a particular value of given no observed data (our previous “belief)Observed Probability or “Likel

14、ihood, p(D|,): Likelihood of sequence of coin tosses D being observed given that is a particular value. In this case = .p(D|): Raw probability of D.Bayesian Formulas for Weighted Coin Toss (Irregular Coin)where*Only need to calculate p( |D,) and p(|), the rest can be derived .IntegrationTo find the

15、probability that Xn+1=heads, we must integrate over all possible values of to find the average value of which yields: .Expansion of Terms 1. Expand observed probability p(|D,):2. Expand prior probability p(|): *“Beta function yields a bell curve upon integration which is a typical probability distri

16、bution. Can be viewed as our expectation of the shape of the curve.Beta Function and IntegrationIntegrating gives the desired result:Combine product of both functions to yield.Key PointsMultiply the results of the beta function (prior probability) with results of the coin toss function for (observed

17、 probability). Result is our confidence for this value of .Integrating the product of the two with respect to over all values of 0BFor all nodes that do not have a causal link we can check for conditional independence between those nodes.ExampleUsing the above graph of expected causes, we can check

18、for conditional independence of the following probabilities given initial sample datap(a|f) = p(a)p(s|f,a) = p(s)p(g|f,a, s) = p(g|f)p(j|f,a,s,g) = p(j|f,a,s).Construction of “Posterior knowledge based on observed data:For every node i, we construct the vector of probabilities ij = ij1, ijn where ij

19、 is represented as row entry in a table of all possible combinations j of the parent nodes 1,n The entries in this table are the weights that represent the degree of confidence that nodes 1,n influence node i (though we dont know these values yet).Determining Table Values for i How do we determine t

20、he values for ij?Perform multivariate integration to find the average ij for all i and j in a similar manner to the coin toss integration:Count all instances “m that satisfy a configuration ijk then observed probability for ijk becomes ijkm(1- ijk )n-mIntegrate over all vectors ijk to find the average value of each ijk .Question1: What is Bayesian Probability?A persons degree of belief