04-733 AIOps: Continuous and Automated IT and AI Monitoring
12 units, Carnegie Mellon University
This course builds on and integrates S/W engineering, AI/ML, and IT skills in support of automated methods for assuring the highest level of system availability and resilience. Students will apply tools including Docker and AI-based models for anomaly detection to monitor and correct hybrid cloud applications as they experience simulated disruptions and outages. Lab exercises will deploy such monitoring tools in continuous integration/delivery pipelines for pro-active control as opposed to traditional reactive and manual incident response. The course will focus on automated monitoring of both IT components (e.g. code-based implementations of a distributed application) and AI/ML components (e.g. models and their associated pre- and post- deployment pipelines). The course requires prior background in DevOps including Docker, CI/CD pipelines, and the microservices architectures used in hybrid cloud deployments. This background provides preparation for deep dives on real-time operational data gathering and the automated tools available for anomaly detection, reporting, and even self-correction. For IT monitoring, multiple types of performance data will be used in automated monitoring including structured metrics such as CPU, memory, and network utilization as well as emerging methods using large language models for analyzing unstructured content such as textual information in application logs. For AI monitoring, extensions of DevIOps methods for AI will introduce ModelOps and its application to multiple types of quality metrics for automated model monitoring. Model accuracy, precision, recall, and bias will be evaluated for initial deployment and tracked for post-deployment drift over time leading to predicted violation of quality standards.
Prerequisites: none listed
Courseweave needs JavaScript to show sections, workloads and planning tools.