SecML
From RAD Lab
Contents |
Learning in Security Sensitive Environments
People
| Faculty | Collaborators |
Alumni
|
Summary: Security of Adaptive Systems
Machine learning is becoming prevalent in the systems domain as a detection and analysis tool for problems amenable to adaptive techniques. However, the adaptivity and flexibility that are machine learning's biggest assets are also qualities that an attacker might exploit. Thus, it is important to study the security of learning systems.
One research direction is to experimentally and theoretically analyze existing systems. In [2], the authors use PCA for detecting anomalous point-to-point flows based on link volume data. We are investigating the effect an adversary can have on the normal subspace of link volume vectors learned under various realistic models of control. In a similar vein, we are exploring the vulnerabilities of the spam filter, SpamBayes [3].
Another focus is security as a property of families of learners. Universal sequence prediction [1] considers the loss of a learner in the presence of an adversary. This approach is appropriate for security, as the adversary is modeled in a general way. Robust statistics is another appropriate framework, which quantifies the effect of outliers. For security, it is important to quantify the cost of an attack, possibly in the presence of non-adversarial data.
[1] N. Cesa-Bianchi and G. Lugosi. Prediction, Learning, and Games. Cambridge University Press, 2006.
[2] A. Lakhina, M. Crovella and C. Diot. Diagnosing network-wide traffic anomalies. ACM SIGCOMM Computer Communication Review, 34(4), 2004.
[3] T. A. Meyer and B. Whateley. SpamBayes: Effective open-source, Bayesian based, email classification system. Conference on Email and Anti-Spam, 2004.
Publications
- Query Strategies for Evading Convex-Inducing Classifiers. Blaine Nelson, Benjamin I. P. Rubinstein, Ling Huang, Anthony D. Joseph, Steven J. Lee, Satish Rao, and J. D. Tygar. arXiv report 1007.0484v1, accepted to the Journal of Machine Learning Research, 2011.
- Classifier Evasion: Models and Open Problems. Blaine Nelson, Benjamin I. P. Rubinstein, Ling Huang, Anthony D. Joseph, and J. D. Tygar. In Privacy and Security Issues in Data Mining and Machine Learning, volume 6549 of Lecture Notes in Computer Science, 2011, pages 92-98.
- Behavior of Machine Learning Algorithms in Adversarial Environments (PhD dissertation). Blaine Nelson. UC Berkeley, Department of EECS technical report UCB/EECS-2010-140, November 23 2010.
- Secure Learning and Learning for Security: Research in the Intersection (PhD dissertation). Benjamin Rubinstein. UC Berkeley, Department of EECS technical report UCB/EECS-2010-71, May 13 2010.
- The Security of Machine Learning. Marco Barreno, Blaine Nelson, Anthony D. Joseph, and J. D. Tygar. In Machine Learning Journal, 81(2), 2010, pg. 121-148. Technical Report.
- Near-Optimal Evasion of Convex-Inducing Classifiers. Blaine Nelson, Benjamin I. P. Rubinstein, Ling Huang, Anthony D. Joseph, Shing-hon Lau, Steven Lee, Satish Rao, Anthony Tran and J. D. Tygar. In the Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics (AISTATS), pg. 549-556, 2010.
- ANTIDOTE: Understanding and Defending against Poisoning of Anomaly Detectors. Benjamin I. P. Rubinstein, Blaine Nelson, Ling Huang, Anthony D. Joseph, Shing-hon Lau, Satish Rao, Nina Taft, and J. D. Tygar. In the Proceedings of the Ninth Internet Measurement Conference (IMC), pg. 1-14, 2009.
- Stealthy Poisoning Attacks on PCA-based Anomaly Detectors. Benjamin I. P. Rubinstein, Blaine Nelson, Ling Huang, Anthony D. Joseph, Shing-hon Lau, Satish Rao, Nina Taft, and J. D. Tygar. In ACM SIGMETRICS Performance Evaluation Review, 37(2), 2009, pg. 73-74.
- Misleading learners: Co-opting your spam filter. Blaine Nelson, Marco Barreno, Fuching Jack Chi, Anthony D. Joseph, Benjamin I. P. Rubinstein, Udam Saini, Charles Sutton, J. D. Tygar, and Kai Xia. Book chapter in Jeffrey J. P. Tsai and Philip S. Yu (eds.) Machine Learning in Cyber Trust: Security, Privacy, and Reliability, pg. 17-51, 2009.
- Open Problems in the Security of Learning. Marco Barreno, Peter L. Bartlett, Fuching Jack Chi, Anthony D. Joseph, Blaine Nelson, Benjamin I. P. Rubinstein, Udam Saini, and J. D. Tygar. In the Proceedings of the First ACM Workshop on Security and Artificial Intelligence (AISec), pg. 19-26, 2008.
- Evading Anomaly Detection through Variance Injection Attacks on PCA (Extended Abstract). Benjamin I. P. Rubinstein, Blaine Nelson, Ling Huang, Anthony D. Joseph, Shing-hon Lau, Nina Taft, and J. D. Tygar. In the 11th International Symposium on Recent Advances in Intrusion Detection (RAID), pg. 394-395, 2008. Winner of the RAID08 Best Poster Award.
- Compromising PCA-based anomaly detectors for network-wide traffic. Benjamin I. P. Rubinstein, Blaine Nelson, Ling Huang, Anthony D. Joseph, Shing-hon Lau, Nina Taft, and Doug Tygar. UC Berkeley, Department of EECS technical report UCB/EECS-2008-73, May 29 2008.
- Evaluating the security of machine learning algorithms (PhD dissertation). Marco Antonio Barreno. UC Berkeley, Department of EECS technical report UCB/EECS-2008-63, May 20 2008.
- Machine Learning in the Presence of an Adversary: Attacking and Defending the SpamBayes Spam Filter (Masters thesis). Udam Saini. UC Berkeley, Department of EECS technical report UCB/EECS-2008-62, May 20 2008.
- Exploiting machine learning to subvert your spam filter. Blaine Nelson, Marco Barreno, Fuching Jack Chi, Anthony D. Joseph, Benjamin I. P. Rubinstein, Udam Saini, Charles Sutton, J. D. Tygar, and Kai Xia. In Proceedings of the First Usenix Workshop on Large-Scale Exploits and Emergent Threats (LEET), April 2008.
- Bounding an attack's complexity for a simple learning model. Blaine Nelson and Anthony D. Joseph. In Proceedings of the First Workshop on Tackling Computer Systems Problems with Machine Learning Techniques (SysML), Saint-Malo, France, June 2006.
- Can machine learning be secure? (Invited paper). Marco Barreno, Blaine Nelson, Russell Sears, Anthony D. Joseph, and J. D. Tygar. In Proceedings of the ACM Symposium on InformAtion, Computer and Communications Security (ASIACCS), Taipei, Taiwan, March 2006.
