I have a large set of images decomposed into 5 spatial frequency bands, with each band downsampled appropriately. So for each image I have 64x64, 32x32, 16x16, 8x8 and 4x4 [5 spatial frequencies].
Now I want to do NMF on the combined image set, considering all spatial frequencies together. The first option would be to upsample all spatial frequencies to 64x64, but then I have ~20,000 features and I do not have a machine with enough RAM to do that (I have >200k items). Doing NMF on the 5456 downsampled vectors works, but obviously it's biased very much towards to the high spatial frequency.
I saw NMF uses alternating least squares algorithm, so I thought if I could weight the components of the different spatial frequencies appropriately, this could correct the imbalance in number of pixels and result in equal overall weight to each SF component.
The key part of the ALS algorithm consists of the following lines:
h = max(0, w0\a); w = max(0, a/h); So I modified this to:
h = max(0, lscov(w0, a, weight)); w = max(0, a/h); I also updated the error term norm calcualtion to be a weighted Frobenius. Based on my first thought about how to weight to normalise the number of elements I thought I would weight each SF (high to low) by [1 4 16 64 256] respectively (because each lower SF has 4x less elements). But this produced solutions weighted far too much to the low SF. Instead weights of [1 2 4 8 16] give solutions that look great - but obviously I have to come to this in a completely ad hoc way so I am a bit worried if it is justifiable.
So the questions:
- Is my modification to the NMF ALS algorithm valid? I couldn't find a way to add weights to the second iteration for w - because it is along the other axis so needs weights along the number of different images direction.
- Why does 2x weights seem to be correct rather than 4x weights (is there a square somewhere that I am missing - I thought weights were applied like XWX so would not be squared)?
[link][1 comment]