Q&A: AWS’ Chad Lorenc Looks at the Effectiveness of AI-based Analytics

AI has become an important tool for cybersecurity and popular in such modern security controls as managed detection and response (MDR), extended detection and response (XDR) and endpoint detection and response (EDR).  AI-based predictive analytics and problem detection can be used to automate  a variety of IT operations. Chad Lorenc, security delivery practice manager at Amazon Web Services and a former CISO, addresses the effectiveness of AI-based analytics and identifies what still needs to be done to improve it.

Q. AI has been a part of the cybersecurity landscape since the late 1990s when it was introduced in intrusion detection systems – it’s not exactly the new kid on the block. Today, cyber criminals — state-sponsored and otherwise — also are implementing AI-based attacks on critical infrastructure and businesses alike. How effective are today’s AI-based tools at identifying an AI-based attack as opposed to a traditional attack?

A. AI-based tools are in a perpetual state of catch-up, a trend exacerbated by the recent disruption in the GenAI industry. Prior to this, adversaries had already honed techniques for evasion, utilizing polymorphic attacks and adaptive strategies. With dedicated labs, they refine these methods, outpacing industry-leading defenses. While AI-based attacks don't necessarily exhibit heightened sophistication, they do empower adversaries to swiftly deploy advanced evasion tactics.

Q. What are the key metrics that are used to determine the effectiveness of AI-based defensive tools? At what point are companies tipping the scale from benefiting by relying on AI-based tools and overreliance on AI to the point of creating vulnerabilities?

A. The paramount metric remains the mean time to detection. Integrating AI alongside skilled workers in the security operations center and deploying AI discovery tools in the field can expedite these outcomes. However, there's a looming concern of overreliance akin to the antivirus dilemma. AI necessitates training and isn't inherently adept at recognizing novel patterns, mirroring the perpetual cycle we experienced with antivirus solutions.

Q. How are AI-based analytics used to enable such tasks as threat hunting and predicting potential threats based on historical and real-time data? What are some of the other security controls enabled by AI-based analytics and how effective are they?

A. AI analytics possess the capability to connect a multitude of data points and perform rapid correlations. When guided by skilled prompts, this synergy can be exceptionally potent, facilitating real-time data processing. The extent of these capabilities remains largely uncharted, presenting the pivotal question: what is the tangible business value, and does it warrant the investment in these nascent stages of maturity?

Q. Describe some of the advantages and disadvantages of using AI-based analytics today, such as user and entity behavior analytics? What still needs to be done to improve AI-based analytics?

A. These models require rigorous training and validation, with costs often presenting a significant barrier to adoption. Nevertheless, I'm optimistic about the potential for companies to address these challenges and unlock the value of AI at scale. By leveraging innovative approaches and technologies, such as cost-effective training methodologies and scalable deployment strategies, they hold the promise of democratizing AI benefits across security domains.

Q. What are the most important points a CISO needs to know about AI-based analytics that they probably don’t know? Conversely, what do CISOs and other security pros absolutely “know” about AI-based analytics that’s incorrect?

A. Investing in AI requires considerable time, energy, and expertise in the relevant subject matter. Additionally, the substantial costs of acquiring and testing models present a significant challenge. Solutions like Amazon's Bedrock, which facilitate experimentation and testing to identify the optimal models, will be crucial. Companies must skillfully navigate these obstacles to unlock the transformative power of AI and ensure that their investments deliver tangible benefits.