The advantages of our method are we don’t need strong statistical assumption before using and compute rapidly. Especially when the sample size is small, the resulting graph size is still similar to the graph size of larger sample size. We give the following conclusions:
1. The method proposed by Cheng (2003) is not always proper for small sample sizes. We find that the threshold of independence need to be adjusted adapt to the sample size when sample size is smaller than 100. If we all use 0.9 for the threshold score and ignore the effect of the sample size, the graph size will grow quickly as the sample size decreases. We suggest that we should find the threshold score by simulation before starting the real data analysis. Table 4 gives the result of the threshold score from our simulation study.
2. If we only consider two variables, the method proposed by Cheng will be better than our method. If we need to identify the relevant connections between sets of genes, our method will be better. When the threshold point is not adjusted, the graph size of the result using the method proposed by Cheng will be a serious problem.
3. From simulation results, we can see that the resulting graph sizes of our method do not change acutely when the sample sizes are different.
4. There are some particular functional structures that can not be easily detected by our methods. For example, f x( )=x2. The correlation test of the first step cannot detect some symmetrical functions. But we can adjust the result by the second step. If the functional structure between two variables is symmetrical and we cannot detect from the first step, the p-value of the chi-square test for
one direction at the second step will be very close to zero (<0.0001). The detail is showed in the Table 11.
5. Cheng (2003) has mentioned that if the lambda is too small, then the fitting curve will be too rough. If a lambda is smaller than10−6, it is adjusted to a new lambda such as 1 2
new 2
λ = λ λ+ or λnew = λ λ1 2 . Our methods (RANK and
RANGE) also use the nonparametric regression to estimate the curve. But we find that the magnitude of lambda has no big influence for our methods. If we ignore the small value of lambda and do not adjust the lambda, we still can discover the relations. Also, without adjusting lambda, we may get a better direction.
We consider the following problems as future works:
1. How to discretize the continuous data and not miss much of the information.
2. Use other correlation tests which are sensitive to the unusual points. We use two nonparametric rank correlation tests that are both not sensitive to the unusual points. That may ignore the effect of influential points.
3. Our method is not sensitive with symmetric and linear functional structure.
How to cope with this situation?
References
[1] Akutsu T, Kuhara S, Maruyama O, and Miyano S. (2003). “Identification of gene regulatory networks by strategic gene disruptions and gene overexpressions.”
Theor. Comput. Sci., 298, 235-251. (Preliminary version has appeared in Proc. 9th ACM-SIAM Symp. “Discrete Algorithms.” (1998), 56, 695-702.)
[2] Akutsu T, Miyano S, and Kuhara S. (1999). “Identification of genetic networks from a small number of gene expression patterns under the Boolean network model.” Pac. Symp. Biocomput., 4, 17-28.
[3] Akutsu T, Miyano S, Kuhara S. (2000). “Algorithms for identifying Boolean networks and related biological networks based on matrix multiplication and fingerprint function.” J Comput Biol., 7, 331-343.
[4] Akutsu T, Miyano S, Kuhara S. (2000). “Inferring qualitative relations in genetic networks and metabolic pathways.” Bioinformatics, 16(8), 727-734.
[5] Chakravarti, Laha, and Roy. (1967) Handbook of Methods of Applied Statistics.
Volume I, John Wiley and Sons.
[6] Chen T, He HL, Church GM. (1999). “Modeling gene expression with differential equations.” Pac Symp Biocomput., 29-40.
[7] Cheng C Y. (2003). “Determining the Cause-Effect Relationship between Two Variables by Nonparametric Regression.” Master Thesis. Institute of Statistics, National Chiao-Tung University.
[8] Lu C H. (2003). “Determine the causal relationship between two variables.” Master Thesis. Instituteof Statistics, National Chiao-Tung University.
[9] De Hoon M J, Imoto S, Kobayashi K, Ogasawara N, Miyano S. (2003). “Inferring gene regulatory networks from time-ordered gene expression data of Bacillus subtilis using differential equations.” Pac Symp Biocomput., 17-28.
[10] Friedman N. and Goldszmidt M. (1998). Learning Bayesian Networks with local Structure. Jordan, M.I. (ed.), Kluwer Academic Publisher.
[11] Friedman N, Linial M, Nachman I, Pe'er D. (2000). “Using Bayesian networks to analyze expression data.” J Comput Biol. 7(3-4), 601-620.
[12] Helsel D R and Hirsch R M. (1995). Statistical Methods in Water Resources, Studies in Environmental Sciences 49. Elsevier, Amsterdam.
[13] Hogg R V and Tanis E A. (1997). Probability and Statistical Inference. MacMillan
Publishing, Co., New York.
[14] Imoto S, Goto T and Miyano T. (2002). “Estimating of genetic networks and functional structures between genes by using Bayesian networks and nonparametric regression.” Proc. Pacific Symposium on Biocomputing, 7, 175-186.
[15] Imoto S, Higuchi T, Goto T, Tashiro K, Kuhara S and Miyano S. (2003).
“Combining Microarrays and Biological Knowledge for Estimating Gene Networks via Bayesian Networks.” Technical Report, Human Genome Center, Institute of Medical Science, University of Tokyo, Japan.
[16] Imoto S, SunYong K, Goto T, Aburatani A, Tashiro K, Kuhara S, and Miyano S.
“Bayesian network and nonparametric heteroscedastic regression for nonlinear modeling of genetic network.” Journal of Bioinformatics and Computational Biology, in press. (Preliminary version has appeared in Proc. IEEE Computer Society Bioinformatics Conference.2002, 219-227.)
[17] Jeffrey S Simonoff. (1996). Smoothing Methods in Statistics. Springer series in statstics.
[18] Maki Y, Tominaga D, Okamoto M, Watanabe S, Eguchi Y. (2001) “Development of a system for the inference of large scale genetic networks.” Pac Symp Biocomput., 446-458.
[19] Pearl J. (2000). Causality: models, reasoning, and inference. Cambridge University Press, Cambridge, New York.
[20] Shmulevich I, Dougherty ER, Kim S, Zhang W. (2002). “Probabilistic Boolean Networks: a rule-based uncertainty model for gene regulatory networks.”
Bioinformatics, 18(2), 261-74.
[21] Sida´k Z. (1968). “On multivariate normal probabilities of rectangles: Their dependence on correlations.” Ann Math Statist., 39, 1425–1434.
[22] Sida´k Z. (1971). “On probabilities of rectangles in multivariate normal Student distributions: their dependence on correlations.” Ann Math Statist. 41, 169–175 [23] Siegel S and Castellan N J. (1988). Nonparametric Statistics.
[24] Siegel S. (1956). Nonparametric Statistics. McGraw-Hill. London.
[25] Snedecor, George W and Cochran, William G. (1989). Statistical methods. English Edition, Iowa Stat University Press.
[26] SunYong K, Imoto S and Miyano S. (2003). “Dynamic Bayesian Network and
Nonparametric Regression for Nonlinear Modeling of Gene Networks from Time Series Gene Expression Data.” Proc. 1st International Workshop on Computational Methods in System Biology. Lecture Note in Computer Science, 2602, Springer-Verlag. 104-113.
[27] Tamada Y, Kim S, Bannai H, Imoto S, Tashiro K, Kuhara S, and Miyano S. (2003).
“Estimating gene networks from gene expression data by combining Bayesian network model with promoter element detection.” Bioinformatics, 19 Suppl 2, II227-II236.
[28] Xu H, Wu P, Wu CF, Tidwell C, and Wang Y. (2002). “A smooth response surface algorithm for constructing a gene regulatory network.” Physiol Genomics, 11(1), 11-20.
Tables
Table 2: The summary of larger sample size. i.e., Figures of right below the described compared with Figure 6.
Sample size
K Rank Rate/missing/extra/wrong
Range
Rate/missing/extra/wrong
Score
1000 8 77.8~82.2%/2/6/2
Figure 22(B)
80~84.4%/2/6/1
Figure 22(C)
91%/0/2/2
Figure 22(A)
1000 8 80~82.2%/1/7/2
Figure 23(B)
82.2%~84.4%/1/7/1
Figure 23(C)
91%/2/1/1
Figure 23(A)
500 8 80%/0/7/2
Figure 24(B)
80%/0/7/2
Figure 24(C)
91%/0/3/2
Figure 24(A)
8 80%/2/7/2 Figure 25(B) 80%/2/7/2 Figure 25(C)
200
4 84.4%/2/6/0 Figure 25(D) 84.4%/2/6/0 Figure 25(E)
88.9%/0/3/2
Figure 25(A)
8 82.2%/3/4/1 Figure 26(B) 80%/3/4/2 Figure 26(C)
100
4 80%/3/4/2 Figure 26(D) 75.6%/3/4/4 Figure 26(E)
86.7%/1/3/2 S=0.9
Figure 26(A)
65%/1/12/3 S=0.9
Figure 27(A)
100 8 75.6%/3/6/2
Figure 27(C)
77.8%/3/6/1
Figure 27(D)
84.4%/1/4/2 S2=0.73
Figure 27(B)
K: the number of class of ordinal data.
Rate: the similar rate compared with Figure 1. If both the direction and the connected
way (repressive or activate) are the same with Fugure 1, then we say that is correct.
Missing: the number of missing pathways compare with Figure 1.
Extra: the number of extra pathways compared with Figure 1.
Wrong: the number of wrong pathway directions compared with Figure 1.
S= 0.9, the threshold of SCORE.
S2= the threshold decided by consider Figure 1 ( adjusted by us).
Table 3: The summary of larger sample size. The resulting graph compared with Figure 7.
Sample size
K Rank Rate/missing/extra/wrong
Range
Rate/missing/extra/wrong
Score
1000 8 71.1%/7/4/2 73.3%/7/4/1 68.9%/13/0/1
1000 8 77.8%/7/1/2 80%/7/1/1 73.3%/11/0/1
500 8 75.6%/7/2/2 75.6%/7/2/2 75.6%/9/0/2
8 75.6%/7/2/2 75.6%/7/2/2
200
4 80%/7/2/0 80%/7/2/0
75.6%/9/0/2
8 68.9%/12/1/1 66.7%/12/1/2
100
4 66.7%/12/1/2 62.2%/12/1/4
68.9%/11/1/2 S=0.9 64.4%/8/5/3
S=0.9
100 8 64.4%/11/3/2 66.7%/11/3/1
71.1%/10/1/2 S2=0.73
K: the number of class of ordinal data.
Rate: the similar rate compared with Figure 7. If both the direction and the connected way (repressive or activate) are the same with Fugure 7, then we say that is correct.
Missing: the number of missing pathways compared with Figure 7.
Extra: the number of extra pathways compared with Figure 7.
Wrong: the number of wrong pathway directions compared with Figure 7.
S= 0.9, the threshold of SCORE.
S2= the threshold of Table 6 (adjusted by us).
Table 4: The summary of smaller sample size. Figures of right below the described compared with Figure 6.
Sample size
K lambda Rank Range Score
N 80%/2/6/1
Figure 28(C)
77.8%/0/5/5
Figure 28(A)
4
Y 82.2%/2/6/0
80%/2/6/1
Figure 28(D)
80%/0/4/5
Figure 28(B)
N 100
2
Y
80%/2/6/1
Figure 28(E)
75.56%/2/6/3
Figure 28(F)
N 80%/3/3/3
Figure 29(C)
77.7%/3/3/4
Figure 29(D)
82.2%/0/6/2
Figure 29(A)
4
Y 75.6%/3/3/5
Figure 29(E)
75.6%/3/3/5
Figure 29(F)
86.7%/1/4/1
Figure 29(B)
50
2 N 75.6%/3/3/5
Figure 30(A)
75.6%/3/3/5
Figure 30(B)
Y 73.3%/3/3/6
Figure 30(C)
75.6%/3/3/5
Figure 30(D)
N 84.4%/4/1/2
Figure 31(C)
84.4%/4/1/2
Figure 31(D)
64%/2/10/4
Figure 31(A)
4
Y 82.2%/4/1/3
Figure 31(E)
77.7%/4/1/4
Figure 31(F)
71%/0/10/3
Figure 31(B)
N 77.7%/4/1/5
Figure 32(A)
82.2%/4/1/3
Figure 32(B)
30
2
Y 80%/4/1/4
Figure 32(C)
80%/4/1/4
Figure 32(D)
N 77.8%/5/3/2
Figure 33(C)
73.3%/2/7/3
Figure 33(A)
4
Y 80%/4/4/1
Figure 33(E)
75.6%5/3/3
Figure 33(D)(F)
62.2%/1/12/4
Figure 33(B)
N 75.6%/5/3/3
Figure 34(A)
17
2
Y 75.6%/5/3/3
Figure 34(C)
73.3%/5/3/4
Figure 34(B)(D)
N: we ignore whether the value of lambda is too small or not.
Y: If λ1 <10−6 or λ2 <10−6, then we adjust the lambda by
2 ) (λ1 λ2 λnew = + .
Table 5: The summary of smaller sample size. The resulting graph compared with Figure 7.
Sample size
K lambda Rank Range Score
N 71.1%/10/2/1 68.8%/8/1/5
4
Y 74.4%/10/2/0
71.1%/10/2/1
66.7%/6/4/5
N 100
2
Y
71.1%/10/2/1 71.1%/10/2/1
N 62.2%/13/1/3 60%/13/1/4 68.9%/9/3/2
4
Y 57.7%/13/1/5 57.7%/13/1/5 71.1%/10/2/1
N 57.7%/13/1/5 57.7%/13/1/5
50
2
Y 55.6%/13/1/6 57.7%/13/1/5
N 62.2%/15/0/2 62.2%/15/0/2 68.8%/7/3/4 4
Y 60%/15/0/3 57.8%/15/0/4 75.6%/6/2/3
N 55.6%/15/0/5 60%/15/0/3
30
2
Y 57.8%/15/0/4 57.8%/15/0/4
N 66.7%/13/0/2 55.6%/12/5/3
4
Y 68.9%/13/0/1
64.4%13/0/3
62.2%/7/6/4
N 64.4%13/0/3
17
2
Y 64.4%13/0/3
62.2%/13/0/4
N: we ignore whether the value of lambda is too small or not.
Y: If λ1 <10−6 or λ2 <10−6, then we adjust the lambda by
2 ) (λ1 λ2 λnew = + .
Table 6: The threshold of SCORE of our simulation.
Sample size lambda threshold
Larger than 200 Adjust 0.9
100 Adjust 0.73/0.9
Adjust 0.76 50
Without adjusting 0.76
Adjust 0.76 30
Without adjusting 0.65
Adjust 0.72 17
Without adjusting 0.43
Table 7: The accurate times of 1000 trials.
( /num: the number of two-way directions)
Step two
RANK RANGE
Step one
K=8 K=4 K=2 K=8 K=4 K=2
Y =eX 1000 772 815 617/17 387 735 668
Y =X21/ 3 1000 763 774 671/42 761 854 620
Y =X2−1/ 3 1000 851 800 717/65 651 775 678/3
Y =X 1000 486 478 486/71 409 492 479/2
sin( 2)
Y = X 994 861 868 889/19 783 912 843
Y =X2 609 504 528 340/48 573 592 328
Table 8: The accurate rates of our methods (include step one and step two).
Our methods
RANK RANGE
K=8 K=4 K=2 K=8 K=4 K=2
Y =eX 0.772 0.815 0.617 0.387 0.735 0.668 Y =X21/ 3 0.763 0.774 0.671 0.761 0.854 0.620 Y =X2−1/ 3 0.851 0.800 0.717 0.651 0.775 0.678
Y =X 0.486 0.478 0.486 0.409 0.492 0.479
sin( 2)
Y = X 0.861 0.868 0.889 0.783 0.912 0.843
Y =X2 0.504 0.528 0.340 0.573 0.592 0.328
Table 9: The accurate rates of only use Step two.
Step two
RANK RANGE
K=8 K=4 K=2 K=8 K=4 K=2
Y =eX 0.772 0.815 0.617 0.387 0.735 0.668 Y =X21/ 3 0.763 0.774 0.671 0.761 0.854 0.620 Y =X2−1/ 3 0.851 0.800 0.717 0.651 0.775 0.678
Y =X 0.486 0.478 0.486 0.409 0.492 0.479
sin( 2)
Y = X 0.866 0.873 0.894 0.788 0.918 0.848
Y =X2 0.828 0.867 0.558 0.941 0.972 0.539
Table 10: The accurate times of 1000 trials.
Y =eX Y = X21/ 3 Y =X2−1/ 3 Y = X Y =sin(X2) Y =X2
score 997 988 996 467 989 831
Table 11: N=200, K=8 (“0” is “<0.00001”)
pair 12 13 14 15 16 17
p-value of the rank correlation test (Kendall) 0 0 0.06 0 0.11 0 p-value of the rank correlation test (Spearman) 0 0 0.06 0 0.11 0 Bonferroni correction (p-value=0.0975) 0 0 0.1164 0 0.2079 0 Kendall’s Tau 0.16 0.12 0.06 0.1 -0.05 -0.52 Spearman Rho 0.24 0.18 0.08 0.15 -0.07 -0.7
Step one
sign + + + -
The p-value of R1&X
0 0.54 0.89 0 0.62 0
RANK
The p-value of R2&Y
0.78 0.81 0.94 0.77 0.55 0.13 The p-value of R1&X
0.43 0.6 0.07 0 0.7 0.01
RANGE
The p-value of R2&Y
0.930.53 0.13 0.99 0.76 0.35
Lambda1 0.03 5.69 0.25 5.7 5.67 0.02
S(g1) 0.83 0.97 0.99 0.99 0.99 0.49
Lambda2 0.02 0.03 0 0 9.31 11.17
SCORE
S(g2) 0.720.96 0.98 0.82 0.99 0.56
18 19 110 23 24 25 26 27 28 29 210
0 0.51 0.69 0 0 0 0.87 0.14 0.08 0.39 0.26
0 0.49 0.69 0 0 0 0.87 0.15 0.08 0.39 0.27
0 0.7501 0.9039 0 0 0 0.9831 0.269 0.1536 0.6279 0.4598 -0.25 0.02 -0.01 0.26 -0.09 0.13 0 0.04 0.05 0.03 0.03 -0.37 0.03 -0.02 0.39 -0.13 0.19 -0.01 0.06 0.08 0.04 0.05
- + - +
0.53 0.58 0.56 0.23 0.74 0.54 0.87 0.96 0.14 0.7 0.38 0.08 0.58 0.47 0.78 0.9 0.53 0.92 0.6 0.37 0.36 0.08 0.68 0.78 0.46 0.51 0.13 0.83 0.43 0.95 0.38 0.22 0.29 0.79 0.7 0.75 0.92 0.91 0.05 0.46 0.86 0.36 0.53 0.87 0.02 5.67 0.51 9.07 9.07 9.09 1.29 9.09 0.44 9.07 0.01
0.88 1 1 0.86 0.99 0.97 1 0.99 0.99 1 0.98
0.01 9.65 8.66 0.01 0.07 0.05 9.32 11.18 9.86 9.62 8.64
0.86 1 1 0.83 0.98 0.96 1 0.99 0.99 1 1
34 35 36 37 38 39 310 45 46 47 48
0 0 0.68 0.27 0.13 0.89 0.91 0 0.05 0.03 0
0 0 0.65 0.28 0.12 0.87 0.95 0 0.06 0.03 0
0 0 0.888 0.4744 0.2344 0.9857 0.9955 0 0.107 0.0591 0 -0.15 0.4 0.01 0.03 0.05 0 0 -0.39 -0.06 -0.06 -0.16 -0.23 0.57 0.02 0.05 0.07 -0.01 0 -0.53 -0.08 -0.09 -0.24
- + - - -
0.04 0.78 0.33 0.36 0.72 0.24 0.77 0.994769 0 0.88 0.95 0.86 0.01 0.35 0.72 0.94 0.42 0.89 0.987262 0 0.29 0.18
0.2 0.79 0.49 0.57 0.27 0.21 0.17 0 0 0.38 0.76
0.17 0.43 0.36 0.63 0.16 0.19 0.13 0.6 0.01 0.35 0.11
0 0.01 9.4 0.53 0 0.49 9.39 0 0 0.04 0
0.77 0.68 1 0.99 0.94 1 1 0.49 0.9 0.99 0.88
0 0 0.17 7.41 0.01 1.84 0.77 0 0 1.59 9.86
0.91 0.62 0.99 0.99 0.98 1 1 0.1 0.27 1 0.97
49 410 56 57 58 59 510 67 68 69 610 0.48 0.86 0.24 0.09 0 0.36 0.53 0.15 0.08 0.07 0.87 0.51 0.86 0.22 0.09 0 0.38 0.54 0.16 0.09 0.06 0.87 0.7452 0.9804 0.4072 0.1719 0 0.6032 0.7838 0.286 0.1628 0.1258 0.9831
0.02 -0.01 0.03 0.05 0.09 -0.03 -0.02 0.04 0.05 -0.05 0 0.03 -0.01 0.05 0.08 0.13 -0.04 -0.03 0.06 0.08 -0.08 -0.01
+ -
0.73 0.59 0 0.62 0.95 0.15 0.92 0.91 0.57 0.77 0.26 0.14 0.74 0.85 0.64 0.62 0.01 0.87 0.47 0.5 0.91 0.62 0.48 1 0 0.14 0.73 0.65 0.87 0.82 0.9 0.13 0.27 0.28 0.98 0.23 0.46 0.16 0.6 0.98 0.34 0.93 0.6 0.29 0.1 0.1 0 0.09 0.01 0.15 0.58 9.29 0.14 0.03 9.28
0.99 1 0.97 0.99 0.96 0.99 1 0.99 0.99 0.99 1
0.06 0.61 0.01 11.21 9.83 0.01 8.64 1.23 0.16 9.63 0 0.99 1 0.5 0.99 0.98 0.98 1 0.99 0.99 0.99 0.99
78 79 710 89 810 910
0 0.73 0.78 0.87 0.69 0.96
0 0.72 0.75 0.87 0.69 0.96
0 0.9244 0.945 0.9831 0.9039 0.9984
0.42 -0.01 0.01 0 -0.01 0
0.61 -0.02 0.01 -0.01 -0.02 0 0.2 0.87 0.63 0.82 0.93 0.34
0.78 0.89 0.72 0.95 0.8 0
0.12 0.53 0.58 0.75 0.62 0.81
0.78 0.81 0.82 0.55 0.85 0
0.04 0.77 11.21 0.05 9.83 0.02
0.67 1 1 0.99 1 0.77
0.01 9.65 8.64 9.64 8.67 0.6
0.62 1 1 1 1 1
+ +
Figures
TEC1 PDS1
YGP1
PDS1
FAR1
INH1 NUF1
TEC1
+ +
+
+ + +
- -
+
(B) YGP1
FAR1
INH1
-
-- +
NUF1
+
Figure 16: A partial predicted gene regulatory network for the yeast data (CDC28) from Xu et al. (2002). This netwrok is constructed not only by the Smooth Response Surface algorithm also the exiting biological knowledge.
YGP1
PDS1
FAR1
INH1 NUF1
TEC1
+ +
+ -
-
- +
+ + (A) +
Figure 17: The resulting network constructed by SCORE (A) Without adjusting lambda. The threshold score is 0.45.
(B) When adjusting lambda. The threshold score is 0.72
Figure 18: The resulting network constructed by our methods. (A) RANK with K=4 and without adjusting lambda. (B) RANGE with K=4 and without adjusting lambda. (C) RANK with K=4 when adjusting lambda. (D) RANGE with K=4 when adjusting lambda. (E) RANK with K=2 and without adjusting lambda.
(F) RANGE with K=2 and without adjusting lambda. (G) RANK with K=2 when adjusting lambda. (H) RANGE with k=2 when adjusting lambda.
SMC1
Figure19: A partial predicted gene regulatory network for the yeast data (CDC28) from Xu et al. (2002). This network not only constructed by Smooth Response Surface algorithm but also adjusted by the exiting biological knowledge.
Figure 20: The resulting network constructed by SCORE (C) Without adjusting lambda. The threshold score is 0.45.
(D) When adjusting lambda. The threshold score is 0.72
+
RAD27 FAA1
SMC1
CDC45 ARE2
SMC1 SMC1
RAD27 FAA1
SMC1 CDC45 ARE2
TOP3
RAD27 RAD27
TOP3 HES1 CPS1
HAP1 CDC45 ARE2
TOP3 HES1 CPS1
HAP1 CDC45 ARE2
HES1 HES1
TOP3 CPS1 TOP3 CPS1
HAP1 HAP1
ARE2 ARE2
CDC45 CDC45
Figure 21: The resulting network constructed by our methods.
(A) RANK with K=4 and without adjusting lambda.
(B) RANGE with K=4 and without adjusting lambda.
(C) RANK with K=4 when adjusting lambda.
(D) RANGE with K=4 when adjusting lambda.
(E) RANK with K=2 without adjusting lambda.
(F) RANGE with K=2 without adjusting lambda.
(G) RANK with K=2 when adjusting lambda.
(H) RANGE with k=2 when adjusting lambda.
3 6
Accurate direction Wrong direction Extra
Missing
(B)
Figure 22: N=1000, K=8 (Table 2)
(A) The result of SCORE. (The threshold score = 0.9) (B) The result of RANK.
(C) The result of RANGE.
3 6
Accurate direction Wrong direction Extra
Missing
Figure 23: N=1000, K=8 (Table 2)
(A) The result of SCORE. (The threshold score = 0.9) (B) The result of RANK.
(C) The result of RANGE.
(B)
3 6
Accurate direction Wrong direction Extra
3 6
Accurate direction Wrong direction Extra
Figure 25: N=200 (Table 2)
(A) The result of SCORE. (The threshold score = 0.9) (B) The result of RANK with K=8.
(C) The result of RANGE with K=8.
(D) The result of RANK with K=4.
(E) The result of RANGE with K=4.
3 6
Figure 26: N=100 (Table 2)
(A) The result of SCORE. (The threshold score = 0.9) (B) The result of RANK with K=8.
(C) The result of RANGE with K=8.
(D) The result of RANK with K=4.
(E) The result of RANGE with K=4.
3 6
Figure 27: N=100 (Table 2)
(A) The result of SCORE. (The threshold score = 0.9) (B) The result of SCORE. (The threshold score = 0.73) (C) The result of RANK with K=8.
(D) The result of RANGE with K=8.
3 6 3 6
Figure 28: N=100 (Table 4)
(A) The result of SCORE without adjusting lambda. (The threshold score = 0.74) (B) The result of SCORE when adjusting lambda. (The threshold score = 0.74) (C) The result of RANK with K=4.
(D) The result of RANGE with K=4.
(E) The result of RANK with K=2.
(F) The result of RANGE with K=2.
(A) The result of SCORE without adjusting lambda. (The threshold score = 0.76) (B) The result of SCORE when adjusting lambda. (The threshold score = 0.76) (C) The result of RANK and K=4 without adjusting lambda.
(D) The result of RANGE and K=4 without adjusting lambda.
(E) The result of RANK and K=4 when adjusting lambda.
(F) The result of RANGE with K=4 when adjusting lambda.
(A) The result of RANK and K=2 without adjusting lambda.
(B) The result of RANGE and K=2 without adjusting lambda.
(C) The result of RANK and K=2 when adjusting lambda.
(D) The result of RANGE with K=2 when adjusting lambda.
3 6 3 6
(A) The result of SCORE without adjusting lambda. (The threshold score = 0.76) (B) The result of SCORE when adjusting lambda. (The threshold score = 0.65) (C) The result of RANK and K=4 without adjusting lambda.
(D) The result of RANGE and K=4 without adjusting lambda.
(E) The result of RANK and K=4 when adjusting lambda.
(F) The result of RANGE and K=4 when adjusting lambda.
The graph size of SCORE becomes very large compare with our methods.
3 6 3 6
(A) The result of RANK and K=2 without adjusting lambda.
(B) The result of RANGE and K=2 without adjusting lambda.
(C) The result of RANK and K=2 when adjusting lambda.
(D) The result of RANGE with K=2 when adjusting lambda.
3 6
(A) The result of SCORE without adjusting lambda. (The threshold score = 0.72) (B) The result of SCORE when adjusting lambda. (The threshold score = 0.43) (C) The result of RANK and K=4 without adjusting lambda.
(D) The result of RANGE and K=4 without adjusting lambda.
(E) The result of RANK and K=4 when adjusting lambda.
(F) The result of RANGE with K=4 when adjusting lambda.
The graph size of SCORE becomes very large compare with our methods.
3 6 3 6
(A) The result of RANK and K=2 without adjusting lambda.
(B) The result of RANGE and K=2 without adjusting lambda.
(C) The result of RANK and K=2 when adjusting lambda.
(D) The result of RANGE with K=2 when adjusting lambda.