Machine learning (ML) systems promise disruptive capabilities in multiple industries. Building ML systems can be complicated and challenging, however, especially since best practices in the nascent field of AI engineering are still coalescing.
Consequently, a surprising fraction of ML projects fail or underwhelm. Behind the hype, there are three essential risks to analyze when building an ML system:
- Poor problem solution alignment,
- Excessive time or monetary cost, and
- Unexpected behavior once deployed. In this post I’ll discuss each risk and provide a way of thinking about risk analysis in ML systems