Assessment of cyber security has been a long-standing challenge to the re-search community. Nevertheless, there is an imperative need for a practical security assessment method which is supportive of controlling and managing security. In this dissertation, we showed our first attempt at the quantitative assessments of cyber security. We proposed assessment methods to assist an administrator or a developer in assessing cyber security in a methodical manner, from establishing a formal representation to deriving a numerical assessment result.
Our assessment methods are separated along two dimensions, external and internal attacks, to meet specific requirements for distinct scenarios. We dived into the two dimensions and studied the deficiencies of the existing security assessment methods; then we presented a wireless risk assessment method and an evaluation method for estimating software robustness for each dimension.
Our wireless risk assessment method measures network risk in
considera-tion of dynamics of a wireless network. We designed a 4-layer risk analytical hierarchy to model wireless network risk from the perspectives of security re-quirements, external attacks and configurations. Due to the design of clearly separated layers and the design of a hierarchy per device, the computing load of assessing risk of a changing wireless network is reduced since only the re-lated layers and hierarchies have to be calcure-lated and developed. Our method diminishes the time complexity in the network assessment at a considerable sacrifice that wireless risk is assessed from a comparatively coarse-grained viewpoint. The assessment result can be used as the first perimeter of con-trolling and managing security of a network, especially a dynamic network.
Then, the administrator can further use other methods to probe potential attack paths and to mitigate security risk. A holistic security assessment and management can thus be achieved by combining the existing solutions with ours.
As for evaluating cyber security in terms of internal attacks, we started from evaluating software robustness in terms of control-flow obfuscation. We presented a framework for representing control-flow obfuscating transforma-tions and evaluating software robustness enhanced by the transformatransforma-tions.
We showed that, with a graph-based representation, many existing control-flow obfuscating transformations can be represented as a composition of atomic operators. The atomic operators can not only describe the present transformations but also help to design and construct new ones which may offer different levels of software robustness to a program. We have also pro-posed new metrics (distance and potency) for quantifying the effects of these transformations upon a program. The metrics has been designed from the
viewpoint of static analysis, and we recognize they serve merely heuristic, general indicators of security. However, we view our approach as a first step towards evaluating the trade-off between the robustness and overheads caused by control-flow obfuscating transformations. We believe our formal framework with the metrics is beneficial in avoiding suffering intolerable over-heads, which can be estimated at the design stage, prior to implementation of a more robust but too costly version of an obfuscated program.
We evaluated cyber security from multiple aspects and provided practical metrics for system administrators in a systematic manner. This dissertation is our first attempt, and we recognize there is still space for improvement to reach absolutely quantitative assessment. In order to fulfill the require-ments of realistic security assessment, there are, however, some issues that are interesting and need to be further explored. We summarize the issues as follows:
• Coherence of databases. To provide a fair or even close to fair eval-uation, risk assessment heavily depends on information collected from multiple databases and experts. A holistic risk assessment method should be able to consider the discrepancy between databases or expert opinions. More study is required to evaluate the consistency between the data, and to integrate the risk value with the consistency.
• Estimation of obfuscation overheads. The side effects of obfuscating a software program include not only the increased code size but also the slowed-down execution performance. Therefore, more study is needed on how best to compromise between security and performance
over-heads.
• Attacks, scenarios and protection mechanisms. We believe that no sin-gle security measurement or metric is able to satisfy the requirements of security control and management in the real world. A thorough security assessment has to be considered from aspects of different at-tacks, scenarios and protection mechanisms. In this dissertation, we simply discussed the evaluation of control-flow obfuscation, while a formal evaluation method for other types of obfuscation, such as data obfuscation and layout obfuscation, are also desired. Another interest-ing direction of future work would be the design of a framework for combining multiple security measurements or metrics.
• Absolutely quantitative assessment. In this dissertation, the range of a derived risk value varies with the number of attacks and number of de-vices because attacks are the threat sources impacting a network and more devices within a network sensibly imply more potential attack surfaces. We recognize the current design has not achieved absolutely quantitative assessment since we need extra information, such as a scale or a mapping table, to grasp the implication of a numerical risk value for an individual network. However, this is simply our first attempt at devising a quantitative and semantic reference for a network ad-ministrator. Further research on the design of a fixed-range variable representing risk of various networks could be conducted based on our present result.
• Human efforts. There is always a hope that system administrators
easily infer the realistic efforts that an attacker should invest, such as time and money, from the proposed security measurements and metrics.
The inference is a pressing need for system administrators to designate security strategies in a more effective way. A solution to determining the inference is required despite it is still an open issue.
Bibliography
[1] A. Jaquith, Security Metrics Replacing Fear, Uncertainty, and Doubt.
Addison Wesley, 2007.
[2] C. Phillips and L. P. Swiler, “A Graph-Based System for Network Vulnerability Analysis,” Proceedings of Workshop on New Security Paradigms (NSPW ’99), pp. 71–79, Jan. 1999.
[3] S. Jha, O. Sheyner, and J. Wing, “Two Formal Analyses of Attack Graphs,” Proceedings of the 15th Computer Security Foundation Work-shop, pp. 49–63, 2002.
[4] V. Mehta, C. Bartzis, H. Zhu, E. Clarke, and J. Wing, “Ranking Attack Graphs,” Proceedings of Recent Advances in Intrusion Detection, 2006.
[5] J. Pamula, S. Jajodia, P. Ammann, and V. Swarup, “A Weakest-adversary Security Metric for Network Configuration Security Analy-sis,” Proceedings of the 2nd ACM Workshop on Quality of Protection, pp. 31–38, 2006.
[6] L. Wang, A. Singhal, and S. Jajodia, “Toward Measuring Network Se-curity Using Attack Graphs,” Proceedings of the 2007 ACM Workshop on Quality of Protection, pp. 49–54, 2007.
[7] L. Wang, A. Singhal, and S. Jajodia, “Measuring the Overall Security of Network Configurations using Attack Graphs,” in Proceedings of the 21th IFIP WG 11.3 Working Conference on Data and Applications Se-curity, 2007.
[8] N. Poolsappasit, R. Dewri, and I. Ray, “Dynamic Security Risk Man-agement Using Bayesian Attack Graphs,” IEEE Transactions on De-pendable and Secure Computing (unpublished), 2011.
[9] D. M. Zhao, J. H. Wang, J. Wu, and J. F. Ma, “Using Fuzzy Logic and Entropy Theory to Risk Assessment of the Information Security,”
Proceedings of 4th International Conference on Machine Learning and Cybernetics, pp. 2248–2253, Aug. 2005.
[10] D. M. Zhao, J. H. Wang, J. Wu, and J. F. Ma, “Fuzzy Risk Assessment of the Network Security,” Proceedings of 2006 International Conference on Machine Learning and Cybernetics, pp. 4400–4405, Aug. 2006.
[11] D. Zhao, C. Wang, and J. Ma, “A Risk Assessment Method of the Wireless Network Security,” Journal of Electronics, vol. 24, no. 3, pp.
428–432, May 2007.
[12] Z. Wang and H. Zeng, “Study on the Risk Assessment Quantitative Method of Information Security,” Proceedings of 3rd International
Con-ference on Advanced Computer Theory and Engineering (ICACTE), pp.
529–553, 2010.
[13] X. Zhang, Z. Huang, G. Wei, and X. Zhang, “Information Security Risk Assessment Methodology Research: Group Decision Making and Ana-lytic Hierarchy Process ,” Proceedings of 2nd World Congress on Soft-ware Engineering (WCSE), pp. 157–160, 2010.
[14] T. L. Saaty, “How to Make a Decision: The Analytic Hierarchy Process,”
European Journal of Operational Research, vol. 48, no. 1, pp. 9–26, 1990.
[15] O. S. Vaidya and S. Kumar, “Analytic Hierarchy Process: An Overview of Applications,” European Journal of Operational Research, vol. 169, no. 1, pp. 1–29, 2006.
[16] P. Falcarin, C. Collberg, M. Atallah, and M. Jakubowski, “Guest Edi-tor’s Introduction: Software Protection,” IEEE Software, vol. 28, no. 2, pp. 24–27, March–April 2011.
[17] C. Collberg, C. Thomborson, and D. Low, “A Taxonomy of Obfuscating Transformations,” Univ. Auckland, New Nealand, Tech. Rep. 148, 1997.
[18] C. Wang, J. Davidson, J. Hill, and J. Knight, “Protection of Software-Based Survivability Mechanisms,” Foundations of Intrusion Tolerant Systems, pp. 273–282, 2003.
[19] F. Kazuhide, K. Shinsaku, and T. Toshiaki, “An Obfuscation Scheme Using Affine Transformation and Its Implementation,” Transactions of
Information Processing Society of Japan, vol. 47, no. 8, pp. 2556–2570, 2006.
[20] I. V. Popov, S. K. Debray, and G. R. Andrews, “Binary Obfuscation Using Signals,” Proceedings of the 16th USENIX Security Symposium, pp. 275–290, 2007.
[21] A. Majumdar, S. Drape, and C. Thomborson, “Slicing Obfuscations:
Design, Correctness and Evaluation,” Proceedings of the 2007 ACM workshop on Digital Rights Management (DRM’07), pp. 70–81, Oct.
2007.
[22] C. Collberg and C. Thomborson, “Watermarking, Tamper-proofing, and Obfuscation - Tools for Software Protection,” IEEE Transactions on Software Engineering, vol. 28, no. 8, pp. 735–746, 2002.
[23] C. Collberg and J. Nagra, Surreptitious Software: Obfuscation, Water-marking, and Tamperproofing for Software Protection. Addison Wesley, 2009.
[24] M. E. Pate-Cornell, “Fault Trees vs. Event Trees in Reliability Analyssi,”
Risk Analysis, vol. 4, no. 3, pp. 177–186, 1984.
[25] W. S. Lee, D. L. Grosh, F. A. Tillman, and C. H. Lie, “Fault Tree Analysis, Methods, and Applications: A Review,” IEEE Transactions on Reliability, vol. 34, no. 3, pp. 194–203, 1985.
[26] O. Sheyner, J. Haines, S. Jha, R. Lippmann, and J. M. Wing, “Auto-mated Generation and Analysis of Attack Graph,” Proceedings of IEEE Symposium on Security and Privacy, pp. 273–284, May 2002.
[27] L. P. Swiler, C. Phillips, D. Ellis, and S. Chakerian, “Computer-Attack Graph Generation Tool,” Proceedings of Information Survivability Con-ference & Exposition II, pp. 307–321, June 2001.
[28] S. Brin and L. Page, “The Anatomy of a Large-Scale Hypertextual Web Search Engine,” Computer Networks and ISDN Systems, vol. 30, pp.
107–117, 1998.
[29] Y. Liu and H. Man, “Network Vulnerability Assessment using Bayesian Networks,” Proceedings of the SPIE, vol. 5812, pp. 61–71, 2005.
[30] M. Frigault and L. Wang, “Measuring Network Security Using Bayesian Network-Based Attack Graphs,” Proceedings of 32nd Annual IEEE In-ternational Computer Software and Applications Conference, pp. 698–
703, 2008.
[31] R. Dantu, K. Loper, and P. Kolan, “Risk Management Using Behav-ior Based Attack Graphs,” Proceedings of International Conference on Information Technology: Coding and Computing, vol. 1, pp. 445–449, 2004.
[32] R. Dantu, P. Kolan, and J. ao Cangussu, “Network Risk Manage-ment Using Attacker Profiling,” Security and Communication Networks, vol. 2, pp. 83–96, 2009.
[33] X. Ou, W. F. Boyer, and M. A. McQueen, “A Scalable Approach to Attack Graph Generation,” Proceedings of 13th ACM Conference on Computer and Communications Security (CCS’06), pp. 336–345, 2006.
[34] P. Ammann, D. Wijesekera, and S. Kaushik, “Scalable, Graph-Based Network Vulnerability Analysis,” Proceedings of 9th ACM Conference on Computer and Communication Security, pp. 217–224, Nov. 2002.
[35] P. Ammann, J. Pamula, and R. Ritchey, “A Host-Based Approach to Network Attack Chaining Analysis,” 21st Annual Computer Security Applications Conference (ACSAC’05), pp. 72–84, 2005.
[36] B. Barak, O. Goldreich, R. Impagliazzo, S. Rudich, S. Vadhan, and K. Yang, “On the (Im)possibility of Obfuscating Programs ,” Lecture Notes in Computer Science, pp. 1–18, 2001.
[37] B. Lynn, M. Prabhakaran, and A. Sahai, “Positive Results and Tech-niques for Obfuscation,” in EUROCRYPT ’04, 2004.
[38] T. Hou, H. Chen, and M. Tsai, “Three Control Flow Obfuscation Meth-ods for Java Software,” IEE Proceedings Software, vol. 153, no. 2, pp.
80–86, Jan 2006.
[39] T. L´aszl´o and A. Kiss, “Obfuscating C++ Programs via Control Flow Flattening,” Annales Universitatis Scientiarum de Rolando Etvs Nom-inatae - Sectio Computatorica, May 2008.
[40] J. Cappaert and B. Preneel, “A General Model for Hiding Control Flow,”
Proceedings of 10th ACM Workshop on Digital Rights Management, pp.
35–42, 2010.
[41] S. Drape, A. Majumdar, and C. Thomborson, “Slicing Aided Design of Obfuscating Transformations,” Proceedings of the International Con-ference on Computng and Information Systems (ICIS 2007), pp. 1019–
1024, 2007.
[42] “Dotfuscator,” PreEmptive Solutions Inc. [Online]. Available:
http://www.preemptive.com/products/dotfuscator
[43] “Dasho,” PreEmptive Solutions Inc. [Online]. Available:
http://www.preemptive.com/products/dasho
[44] “Zelix klassmaster,” http://www.zelix.com/klassmaster/.
[45] “Proguard,” http://proguard.sourceforge.net/.
[46] N. Naeem, M. Batchelder, and L. Hendren, “Metrics for Measuring the Effectiveness of Decompilers and Obfuscators,” Proceedings of 15th IEEE International Conference on Program Comprehension (ICPC’07), pp. 253–258, 2007.
[47] B. Anckaert, M. Madou, B. D. Sutter, B. D. Bus, K. D. Bosschere, and B. Preneel, “Program Obfuscation: A Quantitative Approach,” Pro-ceedings of 3rd Workshop on Quality of Protection (QoP’07), pp. 15–20, 2007.
[48] A. Majumdar, S. Drape, and C. Thomborson, “Metrics-Based Evalua-tion of Slicing ObfuscaEvalua-tions,” Proceedings of 3rd InternaEvalua-tional Sympo-sium on Information Assurance and Security, pp. 472–477, 2007.
[49] M. Ceccato, M. D. Penta, J. Nagra, P. Falcarin, F. Ricca, M. Torchiano, and P. Tonella, “Towards Experimental Evaluation of Code Obfuscation Techniques,” Proceedings of 4th Workshop on Quality of Protection, pp.
39–46, 2008.
[50] M. Ceccato, M. D. Penta, J. Nagra, P. Falcarin, F. Ricca, M. Torchi-ano, and P. Tonella, “The Effectiveness of Source Code Obfuscation:
An Experimental Assessment,” Proceedings of IEEE 17th International Conference on Program Comprehension (ICPC’09), pp. 178–187, May 2009.
[51] M. Madou, B. Anckaert, B. D. Bus, K. D. Bosschere, J. Cappaert, and B. Preneel, “On the Effectiveness of Source Code Transformations for Binary Obfuscation,” Proceedings of the International Conference on Software Engineering Research and Practice (SERP06), 2006.
[52] S. Udupa, S. Debray, and M. Madou, “Deobfuscation: Reverse Engi-neering Obfuscated Code,” Proceedings of 12th Working Conference on Reverse Engineering (WCRE 2005), 2005.
[53] A. Majumdar, “Design and Evaluations of Software Obfuscations,”
Ph.D. dissertation, Department of Computer Science, University of Auckland, New Zealand, 2008.
[54] M. D. Preda and R. Giacobazzi, “Control Code Obfuscation by Abstract Interpretation,” Proceedings of the 3rd IEEE International Conference on Software Engineering and Formal Methods (SEFM), pp. 301–310, 2005.
[55] G. Stonebumer, A. Goguen, and A. Feringa, “Risk Management Guide for Information Technology Systems,” National Institute of Standards and Technology, Special Publication 800-30, 2002.
[56] ISO/IEC 27001:2005 – Information Technology – Security Techniques – Information Security Management Systems – Requirements, ISO/IEC Std.
[57] ISO/IEC 27005:2008 – Information Technology – Security Techniques – Information Security Risk Management, ISO/IEC Std.
[58] “IEEE Standard for Information Technology-Telecommunications and Information Exchange between Systems-Local and Metropolitan Area Networks-Specific Requirements - Part 11: Wireless LAN Medium Ac-cess Control (MAC) and Physical Layer (PHY) Specifications,” IEEE Std 802.11-2007 (Revision of IEEE Std 802.11-1999), 2007.
[59] A. H. Lashkari, F. Towhidi, and R. S. Hosseini, “Wired Equivalent Pri-vacy (WEP),” Proceedings of International Conference on Future Com-puter and Communication (ICFCC 2009), pp. 492–495, 2009.
[60] A. H. Lashkari, M. Mansoor, and A. S. Danesh, “Wired Equivalent Privacy (WEP) versus Wi-Fi Protected Access (WPA),” Proceedings of
2009 International Conference on Signal Processing Systems, pp. 445–
449, 2009.
[61] M. S. Ahmed, E. Al-Shaer, and L. Khan, “A Novel Quantitative Ap-proach for Measuring Network Security,” Proceedings of 27th IEEE International Conference on Computer Communications (INFOCOM 2008), pp. 13–18, Apr. 2008.
[62] B. A. Cota and R. G. Sargent, “Automatic Lookahead Computation for Conservative Distributed Simulation,” CASE Center, Tech. Rep. 8916, 1989.
[63] B. Cota, D. Fritz, and R. Sargent, “Control Flow Graphs as a Represen-tation Language,” Proceedings of Simulation Conference, pp. 555–559, 1994.
[64] P. D. Stotts and Z. N. Cai, “Hierarchical Graph Models of Concurrent CIM Systems,” Proceedings of IEEE Workshop on Languages for Au-tomation: Symbiotic and Intelligent Robots, pp. 100–105, 1988.
[65] J. Cheng, “Complexity Metrics for Distributed Programs,” Proceedings of 4th International Symposium on Software Reliability Engineering, pp.
132–141, Nov. 1993.
[66] C. Collberg, C. Thomborson, and D. Low, “Manufacturing Cheap, Re-silient, and Stealthy Opaque Constructs,” Proceedings of the 25th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Lan-guages (POPL’98), pp. 184–196, 1998.
[67] T. Toyofuku, T. Tabata, and K. Sakurai, “Program Obfuscation Scheme using Random Numbers to Complicate Control Flow,” IEIC Technical Report, Jan 2005.
[68] J. Ge, S. Chaudhuri, and A. Tyagi, “Control Flow Based Obfuscation,”
Proceedings of 5th ACM Workshop on Digital Rights Management, pp.
83–92, Nov 2005, USA.
[69] “Chilling Effects Clearinghourse,” Last updated: Dec. 12, 2008.
[Online]. Available: http://www.chillingeffects.org
[70] H. Bunke and K. Shearer, “A Graph Distance Metric Based on the Maximal Common Subgraph,” Pattern Recognition Letters, vol. 19, pp.
255–259, March 1998.
[71] W. Wallis, P. Shoubridge, M. Kraetz, and D. Ray, “Graph Distances Using Graph Union,” Pattern Recognition Letters, Jan 2001.
[72] H. Zuse, Software Complexity: Measures and Methods. Hawthorne, NJ, USA: Walter de Gruyter & Co., 1991.
[73] S. M. Shatz, “Towards Complexity Metrics for Ada Tasking,” IEEE Transactions on Software Engineering, vol. 14, no. 8, pp. 1122–1127, 1988.
[74] W. A. Arbaugh, N. Shankar, and Y. C. J. Wan, “Your 802.11 Wireless Network Has No Clothes,” IEEE Wireless Communications, vol. 9, no. 6, pp. 44–51, Dec. 2002.
[75] T. Karygiannis and L. Owens, “Wireless Network Security: 802.11, Bluetooth and Handheld Devices,” National Institute of Standards and Technology, Special Publication 800-48, 2002.
[76] J. Bellardo and S. Savage, “802.11 Denial-of-Service Attacks: Real Vul-nerabilities and Practical Solutions,” Proceedingsof 12th USENIX Secu-rity Symposium (SSYM’03), pp. 15–28, 2003.
[77] D. Welch and S. Lathrop, “Wireless Security Threat Taxonomy,” Pro-ceedings of IEEE Workshop on Information Assurance United States Military Academy, pp. 76–83, June 2003.
[78] “National Vulnerability Database,” Last updated: 09/25/2011. [Online].
Available: http://nvd.nist.gov/
[79] “Common Vulnerabilities and Exposures,” Last updated: 09/07/2011.
[Online]. Available: http://cve.mitre.org/