Managing Bias and Risk at Every Step of the AI-Building Process

Many machine-learning developers lack experience in building enterprise applications, and many business stakeholders have insufficient knowledge of machine learning to know what questions to ask as they scope and manage projects. To innovate effectively, project owners need to know what trade-offs and decisions they’ll face while building a machine learning system, and when they should assess these trade-offs to minimize frustration and wasted effort. Here are five steps to follow: Design: Define the problem and articulate the business case. Determine the business’ tolerance for error and ascertain which regulations, if any, could impact the solution. Exploration: Conduct a feasibility study on the available data. Determine whether the data are biased or imbalanced, and discuss the business’ need for explainability. May require re-iterating from design phase depending on the answers to these questions. Refinement: Train and test the model (or several potential model variants). Gauge the impact of fairness and privacy enhancements on accuracy. Build and ship: Implement a production-grade version of the model. Determine how frequently the model must be retrained and whether its output must be stored, and how these requirements affect infrastructure needs. Measure: Document and learn from the model’s ongoing performance. Scale it to new contexts and incorporate new features. Discuss how to manage model errors and unexpected outcomes. May require re-iterating from build-and-ship phase depending on the answers to these questions.