Alexa, Explain Yourself!

On August 17, 2020, NIST released a draft of its paper outlining a framework for developing "explainable" artificial intelligence (AI): Four Principles of Explainable Artificial Intelligence (NISTIR 8312). The agency is seeking feedback and comments on the draft proposal that articulates and defines the four principles capturing the "fundamental properties" of explainable AI systems. NIST is seeking comment on this proposal prior to October 15, 2020.

What Is "Explainable" AI?

The concept of explainable AI revolves around the idea that AI systems (including machine learning, neural networks, and other forms of AI) which make recommendations, draw conclusions, or take actions affecting individual rights or liberties, should be able to provide a meaningful explanation of how an AI system arrived at its output and how the system actually works. These systems should provide sufficient information for individuals to understand what decision the AI system made, why it was made, and on what basis.

To achieve these principles, NIST is proposing that a framework for explainable AI be based on the following:

• 1. Explanation – AI systems must deliver accompanying evidence, explanations, or reason(s) for all of their outputs.
• 2. Meaningful – AI explanations must be meaningful, i.e., within the context of the framework of the system, and which individuals can understand.
• 3. Accuracy – AI explanations must accurately reflect the system's process for generating the output.
• 4. Knowledge Limits – AI explanations must acknowledge and disclose that systems can only operate under conditions for which they are designed or when the system reaches a sufficient confidence in its output.

Why Pursue a Common Framework for Explainable AI?

According to NIST, building and deploying explainable AI systems will increase understanding, trust, and adoption of new AI technologies and societal acceptance of the decisions and guidance they produce. As the NIST Press Release (rel. August 18, 2020) explains:

An understanding of the reasons behind the output of an AI system can benefit everyone the output touches. If an AI contributes to a loan approval decision, for example, this understanding might help a software designer improve the system. But the applicant might want insight into the AI's reasoning as well, either to understand why she was turned down, or, if she was approved, to help her continue acting in ways that maintain her good credit rating.

The draft proposal follows other recent NIST actions aimed at developing a framework to support the creation of "trustworthy" AI systems, as we touched on in prior DWT AI Law Advisor blog posts on exploring bias and appropriate privacy frameworks in AI. Notably, this proposal joins a number of other initiatives focused on developing standards and protocols for explainable AI, including DARPA and the UK Information Commissioner's Office (ICO).

The comment period on NIST's draft proposal is open until October 15, 2020. Interested persons can submit comments by email to explainable-AI@nist.gov; and the agency encourages commenters to submit comments using the form provided on this page.

For further information about this proposal, contact K.C. Halm or any member of DWT's AI Team.