EVA home   EVA e-mail   EVA mirrors   -   Secondary structure   Comparative modelling   Threading   Contacts
 

Statistical analysis of Eva-CM results.

The Student t distribution statistics are used to assess the statistical significance of ranking of protein structure modeling methods (details).

DISCLAIMER: Any final ranking list will be frustrated in the sense that method i assessed to be better than method j in the direct pairwise comparison may be positioned worse than method j in the final ranking because of the impact of the pairwise comparisons with other methods, based on different sets of common models.


Analysis of Fold accuracy (% Equivalent Positions):
    Ranking of the methods:

      1. sdsc1
      2. SwissModel
      3. 3djigsaw
      4. cphmodels
      5. esypred

    t-Student statistical analysis of the comparisons:
    sdsc1SwissModel3djigsawcphmodelsesypred
    1. sdsc1 1.73 ± 16.72
    [ 1866] ( 1866)    		 1.42 ± 2.02
    [ 10] ( 2)    		 2.24 ± 13.66
    [ 374] ( 374)    		< 2 models
    2. SwissModel -1.73 ± 16.72
    [ 1866] ( 1866)    		 -0.03 ± 11.54
    [] ( 10529)    		 1.86 ± 15.81
    [ 1082] ( 1082)    		 1.28 ± 13.33
    [ 10909] ( 10909)
    3. 3djigsaw -1.42 ± 2.02
    [] ( 2)    		 0.03 ± 11.54
    [] ( 10529)    		 -0.80 ± 14.48
    [] ( 114)    		 1.16 ± 13.12
    [ 12094] ( 12094)
    4. cphmodels -2.24 ± 13.66
    [ 374] ( 374)    		 -1.86 ± 15.81
    [ 1082] ( 1082)    		 0.80 ± 14.48
    [] ( 114)    		 -1.61 ± 13.55
    [ 373] ( 373)
    5. esypred< 2 models -1.28 ± 13.33
    [ 10909] ( 10909)    		 -1.16 ± 13.12
    [ 12094] ( 12094)    		 1.61 ± 13.55
    [ 373] ( 373)

Analysis of Alignment accuracy (% of correct aligned positions):
    Ranking of the methods:

      1. SwissModel
      2. sdsc1
      3. cphmodels
      4. 3djigsaw
      5. esypred

    t-Student statistical analysis of the comparisons:
    SwissModelsdsc1cphmodels3djigsawesypred
    1. SwissModel 2.79 ± 20.67
    [ 1866] ( 1866)    		 4.03 ± 25.00
    [ 1082] ( 1082)    		 1.75 ± 18.32
    [ 10529] ( 10529)    		 6.03 ± 24.96
    [ 10909] ( 10909)
    2. sdsc1 -2.79 ± 20.67
    [ 1866] ( 1866)    		 0.08 ± 17.19
    [] ( 374)    		 1.67 ± 2.35
    [ 10] ( 2)    		< 2 models
    3. cphmodels -4.03 ± 25.00
    [ 1082] ( 1082)    		 -0.08 ± 17.19
    [] ( 374)    		 -2.57 ± 27.86
    [] ( 114)    		 -2.50 ± 28.18
    [] ( 373)
    4. 3djigsaw -1.75 ± 18.32
    [ 10529] ( 10529)    		 -1.67 ± 2.35
    [] ( 2)    		 2.57 ± 27.86
    [] ( 114)    		 4.06 ± 25.27
    [ 12094] ( 12094)
    5. esypred -6.03 ± 24.96
    [ 10909] ( 10909)    		< 2 models 2.50 ± 28.18
    [] ( 373)    		 -4.06 ± 25.27
    [ 12094] ( 12094)

Analysis of RMSD (Ca global rmsd):
    Ranking of the methods:

      1. sdsc1
      2. SwissModel
      3. cphmodels
      4. 3djigsaw
      5. esypred

    t-Student statistical analysis of the comparisons:
    sdsc1SwissModelcphmodels3djigsawesypred
    1. sdsc1 0.32 ± 2.05
    [ 1866] ( 1866)    		 -0.02 ± 1.02
    [] ( 374)    		 -0.35 ± 0.49
    [] ( 2)    		< 2 models
    2. SwissModel -0.32 ± 2.05
    [ 1866] ( 1866)    		 -0.23 ± 2.10
    [ 1082] ( 1082)    		 -0.11 ± 1.87
    [ 10529] ( 10529)    		 -0.40 ± 2.20
    [ 10909] ( 10909)
    3. cphmodels 0.02 ± 1.02
    [] ( 374)    		 0.23 ± 2.10
    [ 1082] ( 1082)    		 0.11 ± 1.82
    [] ( 114)    		 0.23 ± 2.22
    [ 373] ( 373)
    4. 3djigsaw 0.35 ± 0.49
    [ 10] ( 2)    		 0.11 ± 1.87
    [ 10529] ( 10529)    		 -0.11 ± 1.82
    [] ( 114)    		 -0.28 ± 2.32
    [ 12094] ( 12094)
    5. esypred< 2 models 0.40 ± 2.20
    [ 10909] ( 10909)    		 -0.23 ± 2.22
    [ 373] ( 373)    		 0.28 ± 2.32
    [ 12094] ( 12094)

Analysis of % of modelled residues (% of Coverage):
    Ranking of the methods:

      1. sdsc1
      2. SwissModel
      3. 3djigsaw
      4. esypred
      5. cphmodels

    t-Student statistical analysis of the comparisons:
    sdsc1SwissModel3djigsawesypredcphmodels
    1. sdsc1 3.67 ± 16.05
    [ 1866] ( 1866)    		 0.00 ± 0.00
    [] ( 2)    		< 2 models 2.68 ± 12.92
    [ 374] ( 374)
    2. SwissModel -3.67 ± 16.05
    [ 1866] ( 1866)    		 -1.15 ± 10.92
    [ 10529] ( 10529)    		 -1.28 ± 11.07
    [ 10909] ( 10909)    		 0.66 ± 13.59
    [] ( 1082)
    3. 3djigsaw 0.00 ± 0.00
    [] ( 2)    		 1.15 ± 10.92
    [ 10529] ( 10529)    		 -0.29 ± 8.72
    [ 12094] ( 12094)    		 -0.82 ± 5.64
    [] ( 114)
    4. esypred< 2 models 1.28 ± 11.07
    [ 10909] ( 10909)    		 0.29 ± 8.72
    [ 12094] ( 12094)    		 0.66 ± 8.95
    [] ( 373)
    5. cphmodels -2.68 ± 12.92
    [ 374] ( 374)    		 -0.66 ± 13.59
    [] ( 1082)    		 0.82 ± 5.64
    [] ( 114)    		 -0.66 ± 8.95
    [] ( 373)
 

  EVA home   EVA e-mail   EVA mirrors   -   Secondary structure   Comparative modelling   Threading   Contacts