Active learning to reduce the cost and number of stored instances for "instance-based" (memory-based) methods? (x-post with metaoptimize)

I have been working on an "active learning" approach for jointly reducing the cost and number of labeled samples (instances) in "instance-based" methods and ensembles (specifically General Regression Neural Networks, Probabilistic Neural Networks, k-Nearest Neighbor, and ensembles based on these methods).

The basic idea is that active learning will be used to first select from a pool of unlabeled samples, and then once labeled, determine whether said sample should be "stored" as an instance. Alternatively, this could be done for a stream of samples, where each new sample is chosen randomly and then requires a decision to label it, and a decision to keep it or not.

Does this seem like it would make sense as an approach, or would it be better just to perform some kind of direct sample reduction technique (a la "Reduction Techniques for Instance-Based Learning Algorithms" by Wilson and Martizon, 2000)? Has something like this been attempted, or would it not really provide any value anyways?