Finding the diagnosis for the images using the "normalize sums" method.
Problem:- We are dealing with a medical diagnostic system. We are seeking to automate the diagnosis of the retinal images. The system is configured to deal with 39 manifestations and 14 diagnoses. Each of these 39 manifestations can be present in different states. The number of possible states varies from 2 to7. For eg.Manifestation 1 i.e. RPED can be present in 2 states, either in the absent state or in the present state.

We are given a set of beliefs by the ophthalmologist/expert. This set of beliefs is a probability table which gives the probabilities of a manifestation's presence in a particular state due to the presence of the 14 different diagnoses (P(M/D)). For eg. RPED in the absent state has the following belief values.

Diag0 Diag1 Diag2 Diag3 Diag4 Diag5 Diag6 Diag7 Diag8 Diag9 Diag10 Diag11 Diag12 Diag13
1 1 1 11 1 11 1 11 1 1 0.66667
Also each diagnosis is linked to a subset of the set containing the 39 manifestations. From the example above we can see that manifestation 1 has a probability value not equal to 1 only when Diagnosis 13 is present. This implies that manifestation 1 is linked only to Diagnosis 13.

Each image also has an annotation file. In the annotation file the state in which each manifestation is present in this particular image is stored. For eg. If manifestation 3 is annotated as 2 in the annotation file it implies that manifestation 3 is present in state 2 in the image under consideration.

The system is required to simulate the physician's actions and form a diagnosis. For the system to make a diagnosis for a given image, it uses an annotation file and the set of beliefs given by the physician. The main issue here is the interpretation of the beliefs and the decision regarding a methodology for the manipulation of this data.

We have used the Bayes theorem to make the diagnosis for each of these images. The Bayes theorem uses the formulation

P(D/M=Sk)=P(M=Sk/D)*P(D)/ ån=014P(Mi=Sk/Dn).

Using this method the results have been very poor. The system was able to diagnose correctly 10% of the time.

As an alternative we propose to examine an alternative formulation to belief combination. The Bayesian techniques use multiplicative combinations of the given beliefs. This implies a relative relationship that is globally consistent between all manifestations. We propose that the ophthalmologist-given beliefs do not actually possess this global relativity. We propose that for a given manifestation, the relative beliefs between states be interpreted as given, but that each manifestation be considered independently in scale. We call this formulation the normalize-sums method.

Method used:-In the method used here the manifestations which are annotated as being present and are linked to a diagnosis are the only ones considered for that particular diagnosis. For an annotated and linked manifestation Mi a comparison of the values of P(Mi/D) for the various linked diagnoses is made and the highest value of the P(Mi/D)'s is found. All the P(Mi/D) values are then normalized by this value. This procedure is repeated for every manifestation and the normalized values associated with the corresponding diagnoses are added. These sums are called the normalized sums. Consider for example that the belief table for manifestation 3 in state 2 looks as follows:

Diag1 Diag2 Diag3 Diag4 Diag5 Diag6 Diag7 Diag8 Diag9 Diag10 Diag11 Diag12 Diag13 Diag14
0.2 0.1 0 0 0 0 0 0 0.5 0.30 0.1 0.1 0.4
then by inspection we see that the highest probability value is 0.5. Hence all the values are divided by this value and the resulting table is as follows:

Diag1 Diag2 Diag3 Diag4 Diag5 Diag6 Diag7 Diag8 Diag9 Diag10 Diag11 Diag12 Diag13 Diag14
0.4 0.2 0 0 0 0 0 0 1 0.60 0.2 0.2 0.8
These values in the above table are the normalized values of P(M/D). When these normalized values for each manifestation are added they yield the normalized sum values for the 14 diagnoses. Once the normalized sums are found these values are run through the Fisher's Linear Discriminant test to find the diagnosis that is the major cause for the presence of the manifestations.

Results:- We applied 2 criteria-

1. Any match- The system diagnosis has to match at least one of the ground truth diagnoses. The results with this approach - 187/246 i.e 187 of the 246 images were diagnosed correctly making it 76% accurate.

2. Perfect match- The system and the ground truth diagnoses have to be an exact match. The results with this approach - 79/246 i.e 79 of the 246 images were diagnosed correctly, therefore making it 32%accurate.

Diagnosis based results:- This gives us information about how many images with a particular diagnosis were considered and how many of them were diagnosed correctly.

diagnosis no. of images considered no.of any matches
1 13 1
2 7 5
3 11 4
4 10 1
5 26 23
6 12 9
7 49 36
8 21 16
9 25 7
10 23 5
11 13 10
12 8 7
13 41 40
14 34 23

Conclusions:-

1. The results of this test are extremely encouraging for most of the diagnoses. But the failure of this test to yield 100% results has raised quite a few questions and has lead to an enquiry into "the ideal annotation for each diagnosis and the safe variation from each ideal".

2. We are also investigating the relative performance of the Bayes formulations and the Normalize-Sums methods. We are investigating the possibility of a correlation between the diagnosis made by the 2 methods. We are also investigating the causes for a wrong diagnosis. For eg.The system is configured presently to treat only those images as normal which have all manifestations annotated as normal. But sometimes although a manifestation is not in the normal state its variation from the normal is too small or within an acceptable range, leading to a normal diagnosis by the physician. For the system to make this distinction we need to address the issue of a safe variation from the ideal.

Appendix:-

Fisher's Linear Discriminant Test:-

The Fishers test is as follows: The 14 values(normalized sums) are sorted such that Ni<=Nj for all i,j. These sorted values are partitioned into 2 sets A and B with the first 13 elements into the first set and the last element into the second set. Then the mean and the standard deviation for both the sets are calculated and the Fishers criterion is found as ( m a - m b)*( m a- m b)/( s a* s a+ s b* s b)=Fp1 , where m a, m b are the means of the partitions and s a and s b are the standard deviations of the 2 partitions.

In succeeding iterations the size of set A is decreased and that of set B is increased. The largest Fishers criterion so found is considered as the breakpoint and all the diagnoses in set B are considered as the system diagnosis. Sometimes more than one diagnosis can emerge as the cause for the presence of the manifestations. There is a maximum limit of 3 diagnoses that can be so indicated. This is to ensure that the system makes a definitive diagnosis rather than one which indicates all the diagnoses in the database as the cause. The results of the Fishers test are the resultant system diagnosis.