I've been thinking of the problem of how to supply information to a neural network. Given that I have a defined amount of information (e.g x + y + z + a = 100), how can I maximize the volume of the neural net. In a simple case, this optimization problem can be formed as:
(x1) x (y2) x (z3) x (a4) where x+y+z+a=100.
The trick is, it's not to put them all evenly, or even put them so that the weight sights toward the a4. It's been bothering me for about a week now, and I don't know any methods that solve this kind of optimization.
Here's a start to see why:
(251) x (252) x (253) x (254) = 9.53x1013, which is essentially 2510. So we take 1 from the (254) and distribute to each and make it (283) x (247).. which if you break it down is just a simplification of (241) x (242) x (283) x (244) = (247) x (283)
and that makes (283) x (247)> 2510
This problem may have many local maxima and many local minima, could I get some insight on this?
Thanks!
[link][5 comments]