As a part of an ongoing effort to maintain you knowledgeable about our newest work, this weblog publish summarizes some current publications from the SEI within the areas of counter synthetic intelligence (AI), coordinated vulnerability disclosure for machine studying (ML) and AI, safe growth, cybersecurity, and synthetic intelligence engineering.
These publications spotlight the most recent work from SEI technologists in these areas. This publish features a itemizing of every publication, authors, and hyperlinks the place they are often accessed on the SEI web site.
Counter AI: What Is It and What Can You Do About It?
By Nathan M. VanHoudnos, Carol J. Smith, Matt Churilla, Shing-hon Lau, Lauren McIlvenny, and Greg Touhill
Because the strategic significance of AI will increase, so too does the significance of defending these AI programs. To grasp AI protection, it’s mandatory to know AI offense—that’s, counter AI. This paper describes counter AI. First, we describe the applied sciences that compose AI programs (the AI Stack) and the way these programs are in-built a machine studying operations (MLOps) lifecycle. Second, we describe three sorts of counter-AI assaults throughout the AI Stack and 5 risk fashions detailing when these assaults happen inside the MLOps lifecycle.
Lastly, based mostly on Software program Engineering Institute analysis and observe in counter AI, we give two suggestions. In the long run, the sector ought to put money into AI engineering analysis that fosters processes, procedures, and mechanisms that scale back the vulnerabilities and weaknesses being launched into AI programs. Within the close to time period, the sector ought to develop the processes essential to effectively reply to and mitigate counter-AI assaults, reminiscent of constructing an AI Safety Incident Response Crew and increasing current cybersecurity processes just like the Pc Safety Incident Response Crew Providers Framework.
Learn the SEI white paper.
Classes Realized in Coordinated Disclosure for Synthetic Intelligence and Machine Studying Methods
by Allen D. Householder, Vijay S. Sarvepalli, Jeff Havrilla, Matt Churilla, Lena Pons, Shing-hon Lau, Nathan M. VanHoudnos, Andrew Kompanek, and Lauren McIlvenny
On this paper, SEI researchers incorporate a number of classes realized from the coordination of synthetic intelligence (AI) and machine studying (ML) vulnerabilities on the SEI’s CERT Coordination Heart (CERT/CC). In addition they embody their observations of public discussions of AI vulnerability coordination instances.
Danger administration inside the context of AI programs is a quickly evolving and substantial house. Even when restricted to cybersecurity danger administration, AI programs require complete safety, reminiscent of what the Nationwide Institute of Requirements and Know-how (NIST) describes in The NIST Cybersecurity Framework (CSF).
On this paper, the authors give attention to one a part of cybersecurity danger administration for AI programs: the CERT/CC’s classes realized from making use of the Coordinated Vulnerability Disclosure (CVD) course of to reported “vulnerabilities” in AI and ML programs.
Learn the SEI white paper.
On the Design, Growth, and Testing of Trendy APIs
by Alejandro Gomez and Alex Vesey
Software programming interfaces (APIs) are a elementary element of recent software program purposes; thus, practically all software program engineers are designers or shoppers of APIs. From meeting instruction labels that present reusable code to the highly effective web-based software programming interfaces (APIs) of at the moment, APIs allow highly effective abstractions by making the system’s operations out there to customers, whereas limiting the small print of how the APIs are carried out and thus enhancing flexibility of implementation and facilitating replace.
APIs present entry to sophisticated performance inside massive codebases labored on by dozens if not a whole bunch of individuals, usually rotating out and in of initiatives whereas concurrently coping with altering necessities in an more and more adversarial surroundings. Below these situations, an API should proceed to behave as anticipated; in any other case, calling purposes inherit the unintended habits the API system gives. As programs develop in complexity and dimension, the necessity for clear, concise, and usable APIs will stay.
On this context, this white paper addresses the next questions regarding APIs:
- What’s an API?
- What elements drive API design?
- What qualities do good APIs exhibit?
- What particular socio-technical points of DevSecOps apply to the event, safety, and operational help of APIs?
- How are APIs examined, from the programs and software program safety patterns standpoint?
- What cybersecurity and different greatest practices apply to APIs?
Embracing AI: Unlocking Scalability and Transformation By Generative Textual content, Imagery, and Artificial Audio
by Tyler Brooks, Shannon Gallagher, and Dominic A. Ross
The potential of generative synthetic intelligence (AI) extends properly past automation of current processes, making “digital transformation” a risk for a quickly rising set of purposes. On this webcast, Tyler Brooks, Shannon Gallagher, and Dominic Ross purpose to demystify AI and illustrate its transformative energy in attaining scalability, adapting to altering landscapes, and driving digital innovation. The audio system discover sensible purposes of generative textual content, imagery, and artificial audio, with an emphasis on showcasing how these applied sciences can revolutionize many sorts of workflows.
What attendees will be taught:
- Sensible purposes of generative textual content, imagery, and artificial audio
- Impression on the scalability of instructional content material supply
- How artificial audio is remodeling AI schooling
Evaluating Giant Language Fashions for Cybersecurity Duties: Challenges and Greatest Practices
by Jeff Gennari and Samuel J. Perl
How can we successfully use massive language fashions (LLMs) for cybersecurity duties? On this podcast, Jeff Gennari and Sam Perl focus on purposes for LLMs in cybersecurity, potential challenges, and suggestions for evaluating LLMs.
Take heed to/view the podcast.
Utilizing High quality Attribute Eventualities for ML Mannequin Take a look at Case Era
by Rachel Brower-Sinning, Grace Lewis, Sebastián Echeverría, and Ipek Ozkaya
Testing of machine studying (ML) fashions is a rising problem for researchers and practitioners alike. Sadly, present observe for testing ML fashions prioritizes testing for mannequin perform and efficiency, whereas usually neglecting the necessities and constraints of the ML-enabled system that integrates the mannequin. This restricted view of testing can result in failures throughout integration, deployment, and operations, contributing to the difficulties of transferring fashions from growth to manufacturing. This paper presents an strategy based mostly on high quality attribute (QA) situations to elicit and outline system- and model-relevant take a look at instances for ML fashions. The QA-based strategy described on this paper has been built-in into MLTE, a course of and gear to help ML mannequin take a look at and analysis. Suggestions from customers of MLTE highlights its effectiveness in testing past mannequin efficiency and figuring out failures early within the growth course of.
Learn the convention paper.