Best ML technique for my problem...

Hi everyone,

I was thinking about the best technique to solve a problem I have and would appreciate any input.

Here's the problem: I have a set of features X which get put into a unknown algorithm with parameters A. The algorithm uses parameters A and features X to classify (binary classification) X. I have a bunch of truth data, each row of which has my features X and parameters A and output Y (y=1 for correct classification, y=0 for incorrect classification). The goal is to choose a set of parameters A for new incoming features X which will maximize my chances of correct classification.

It's interesting because while some set of parameters A might work optimally for some set of features X, that same set of parameters A could completely fail given radically different inputs X.

The method I am currently considering using is some type of collaborative filtering (similar to netflix problem), where each "movie" is one of parameters or features from the combined set of features and parameters X,A. And each entry ("user") is the set of values for the features and parameters that correctly classified the features. I would then have a matrix of correct classifications and a new entry would consist of only the features X, and i estimate the values of A.

Thoughts? I would appreciate any opinions on this.

Thanks!