Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create a confusion matrix based on tumors not per pixel for segmentation models #8197

Open
AHarouni opened this issue Nov 7, 2024 · 0 comments

Comments

@AHarouni
Copy link

AHarouni commented Nov 7, 2024

Is your feature request related to a problem? Please describe.
Confusion matrix as true positive (TP), false positive (FP), true negative (TN) and false negative (FN) is a powerful accuracy metric that we have for classification.
We also have it segmentation tasks based on pixel counts. However, this is not meaningful for radiologist or data scientist since they care about number of tumors missed not number of pixel missed.

Describe the solution you'd like
It would be great if we have an accuracy metric that would do: for each label calculate connected component and compare it with the ground truth. If there is more than x % overlap then it is calculated as TP. if there is no ground truth then it is FP. Then we should check on the ground truth that doesn't have any inference should be counted as FN
We can ignore TN as it is about the background.

Describe alternatives you've considered
Manually have radiologist count the tumors and write values in excel files. This doesn't scale at all

Additional context

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant