Quantcast
Channel: Machine Learning
Viewing all articles
Browse latest Browse all 62797

How Many Lines (how do I fit a mixture of linear models)?

$
0
0

I have recently started reading about Bayesian methods, and saw a few examples of people fitting mixture models by MCMC: create a graphical model where the observed variables are drawn from K Gaussian distributions. K is fixed, but the means and standard deviations aren't. This is pretty straightforward, and works great.

Now, I want to expand this example and try to solve something a little harder. Let's say I have data generated through the following process: 1. Pick a uniformly random K between 1 and M (M is unknown to the observer, but small, ~20 or so). 2. For each line L_i, draw a_i, b_i from some random distribution (say Gaussian). L_i is the line xa_i + b_i. 3. For each generated point (x, y), pick a random i (uniformly from 1..K), and a random x (from some fixed distribution - could be normal), and set y=a_ix + b_i + epsilon (where epsilon is random Gaussian noise, with mean 0 and std < a_i).

The description above is a bit long, but the idea is very simple: I am observing a mixture of an unknown number of signals, and what to simultaneously infer the number of signals and their parameters.

How would you go about it?

EDIT: After thinking about it for a bit, I know how I'm thinking to go about it: Fit one model for every possible value of K (#lines), and then use Bayesian model selection to pick my favorite one. I would be happy to hear about other alternatives, in any case.

submitted by orangecat99
[link][2 comments]

Viewing all articles
Browse latest Browse all 62797

Trending Articles