- I have a task where I have to mark ever instance as "X" or "Y". For evaluation, I have to calculate my precision over every part (A,B,C etc) of the dataset. If I mark 0 as X, from part A. What is my precision over A? Basically, what is the scientific view on 0/0 in this case for precision. (I need this because I want to average precision over A,B,C etc). Can you cite a source (published) for your view?
- In same problem as above, lets say the data set, in every part has more Y than X's. So if during evaluation, i am able to label X's correctly(i have gold standard labels), Can I argue that am actually doing a good job?--if training data also had same distribution or distribution skewed towards Y. If I am doing a "good job"--how can I measure how good job am doing? (I am feeling "recall" is not the right parameter)
Thanks for answering my questions. Sorry, if they come as naive, i am new to this and trying to learn.
[link] [1 comment]