next up previous
Next: Combining SignalNet with CleavageNet Up: SPEP analysis Previous: SPEP analysis

SignalNet and CleavageNet performance

After a thorough search in the neural network parameter space, we ended up with six best performing different predictors (two networks for each organism type). All neural networks have a hidden layer and one output neuron, and differ for the number of neurons in the input and hidden layers. In order to test the predictor, we examined only the first 65 residue of each protein chain, since if present, the signal peptide falls inside this N-terminal region. In Table 3 we report the accuracy obtained with the best three network systems for each organism type. As it can be seen SignalNet performs very well when the window is symmetric (left=right). The level of accuracy measured per residue is also very high, reaching 95% for Eukaryotes and 92% and 91% for gram positive and gram negative bacteria, respectively. This values together with those computed for the correlation coefficient ($C$) show the very high level of precision achieved by our networks.
Table 3: SignalNet accuracy

Organism
Window C Q2
  Length    

Eukaryotes
11-1-11 0.75 0.92
- 13-1-13 0.82 0.94
- 15-1-15 0.83 0.95
Gram positive 11-1-11 0.76 0.90
- 13-1-13 0.78 0.91
- 15-1-15 0.79 0.92
Gram negative 9-1-9 0.72 0.88
- 11-1-11 0.78 0.92
- 13-1-13 0.77 0.91

When the task of the exact cleavage site is tackled, the accuracy tends to be apparently higher, due to the fact that the non-cleavage positions are far more abundant than the single positions present into the proteins containing the signal peptide. However, this is reflected into a lower probability of exactly locating the cleavage position, which consequently can be visible as a drop of the correlation coefficient. This claim is confirmed from the results reported in Table 4, where we show the accuracy obtained with the best three CleavageNet systems for each organism type. Nevertheless, the values achieved by the correlation coefficient are noteworthy. It is also interesting to note that the best performing architectures in the case of CleavageNet are asymmetric, indicating that in this case the information present into the left part of the window dominates.
Table 4: CleavageNet accuracy

Organism
Window C Q2
  Length    

Eukaryotes
15-1-2 0.61 0.97
- 17-1-2 0.58 0.96
- 19-1-2 0.59 0.96
Gram positive 19-1-2 0.55 0.96
- 20-1-3 0.56 0.96
- 21-1-2 0.56 0.95
Gram negative 9-1-2 0.61 0.95
- 11-1-2 0.62 0.96
- 13-1-2 0.61 0.95
The final best architectures are listed here


next up previous
Next: Combining SignalNet with CleavageNet Up: SPEP analysis Previous: SPEP analysis
2003-06-12