When AI is used in sectors such as medicine, military defense, or data privacy, AI must produce transparent and understandable results that reflect the ethical standards of our society. XAI helps us understand how right, how fair, and how just an AI’s output, outcome, and impact are. This capability is transforming machine learning (ML) to build trust, clarity, and transparency into an AI’s decisions and the factors that produce those decisions.
With traditional modeling techniques (such as regression models and surrogate decision tree-based models), the relationship between inputted data and outputted decision models is traceable and explainable. These are called white box models. More complex modeling techniques like deep learning and ensemble models promise increases in model accuracy, but often at the expense of model explainability. These are called black box models because it is much harder to observe and understand the relationship between the model input data and output decisions. Sources of bias, such as unbalanced training data, can impact how reliable these outputs are. Compared to white box models, black box models introduce risk because there is no way to observe what is occurring in a black box model on its own. This is where the value of XAI comes in.
With a suite of emerging techniques, XAI explains decisions for five main purposes:
- Analyst Insight: Helping analysts handle high-volume alerts and augmenting their performance with guidance around where to focus their attention and when
- Bias Detection & Mitigation: Investigating the presence of harmful bias and its potential effects on an algorithm's decision process in order to help mitigate throughout the entire AI development process
- Continuous Improvement: Fostering continuous optimization and building a symbiotic relationship between human and machine, with a recognition that human understanding of processes can lead to imperfect decisions
- Quality Assurance: Providing consistent, easy-to-read, explainable insights for the interpretation of data and decision outputs of machines
- Regulatory Transparency: Documenting decision factors and actions to ensure confidence that the machine is doing what it’s supposed to do