Published:
|
Updated:
Published on INFORMS Analytics Magazine (Joseph Byrum)
Validation is the critical component in keeping AI ethical.
You just filed for a loan. With a solid credit history and a steady job, you are confident everything will be approved. Yet, moments after submitting your application, an email from your lender appears in your inbox that says, “We are sorry to inform you that your application has been denied.” You want to know why, but you realize from the instantaneous reply that your plans have just been dashed by some automated routine that considers you too great a risk. There’s no explanation, and you have no opportunity to appeal to reason or explain just how important it is for you to get that loan.
That’s what the future might look like if machines are let loose to make decisions without first grounding them with strong ethical guidelines to prevent algorithmic bias – an unintended consequence of developing machine learning systems without stringent validation procedures.
Fortunately, IEEE is ahead of the game with its ethically aligned design guidelines [1], which were released last year to help companies keep their autonomous systems in check. The document sets out eight principles to guide developers in prioritizing human well-being when creating intelligent systems.
The Black Box Problem
Without a strong ethical foundation, it’s easy for artificial intelligence (AI) to make mistakes. For instance, deep learning systems are “trained” by reviewing a tagged dataset from which the system on its own comes up with rules to identify the key features of each tag. That algorithm is then set free to apply those rules to fresh, untagged information. The common example is that you train an AI image recognition algorithm by showing it pictures of cats with appropriate labels, and pictures of dogs. Once trained, the system should be able to distinguish between cats and dogs in brand new photographs within a certain level of accuracy.
Deep learning systems like this are a “black box,” meaning the end user really has no idea how the algorithm is making its choices. All that matters is that it works to the acceptable degree of accuracy. The danger comes in letting such systems loose without any human supervision.
No self-respecting engineer goes to work with the intention of designing a skewed algorithm. The problem is that a finite dataset used in training does not reflect complex reality. The humans who label the items in the training dataset items can impart their own biases to the information, and the algorithm could draw conclusions from the faulty assumptions, creating false positives.
Without proper constraints built in from the start, a financial deep-learning system could, on its own, develop lending criteria based on race, which would be unethical – and illegal. But algorithmic bias can extrapolate from limitations of training a dataset. Perhaps applicants in a very small training dataset with even account numbers were 80% more likely to default than those with odd-numbered accounts. Obviously, that’s a random occurrence and the machine is making a correlation without causation. Such a system would not be effective in the real world.
Creating More Ethical AI
The principles the IEEE set out address all the likely ethical dilemmas, not just the black box problem. They do so by encouraging intelligent system creators to think about the right things before starting the design for their intelligent system. Those who follow the principles will take the necessary care when creating the training dataset, and they will focus on continuously validating results with humans actively monitoring what’s going on.
As laid out by IEEE, the eight general principles fall under the following topics: human rights, well-being, data agency, effectiveness, transparency, accountability, awareness of misuse and competence.
IEEE begins by pointing out that the AI should be created with a respect for “human rights.” You don’t want an intelligent system to attempt to enslave humanity. IEEE intends all of the various United Nations conventions and the Geneva Conventions to apply to ensure AI does not discriminate or diminish the freedom of humans. Think of it like Google’s original slogan: “Don’t be evil.” As a corollary, this means the AI will always be subordinate to humans.
AI should have the goal of benefitting human “well-being” in a measurable way. Economic well-being, for instance, could be quantified in terms of increasing the GDP, or environmental well-being by reducing particulate matter in the atmosphere or psychological well-being by enhancing emotional stability [2].
“Data agency” is similar to data privacy, but it includes the power of individuals to access, control and share their own personal data.
An AI that hurts its user or doesn’t get the job done would never be trusted. Thus, the “effectiveness” principle is meant to encourage creators to verify that the AI is suited for its purpose by measuring outcomes in accordance with proper engineering practices.
Rather than blindly trust that an AI algorithm is doing the right thing, the “transparency” principle seeks to have systems built in a way open to inspection by humans so that they can understand why each particular AI action or decision was made.
“Accountability” builds upon transparency to ensure the public can figure out who is responsible for whatever the AI does. If something goes wrong, they’ll know who to blame, which helps keep AI creators on their toes.
Even an intelligent system designed by an unquestionably ethical team could go wrong if hackers hijack the AI routines. Thus, the “awareness of misuse” principle encourages AI creators to keep in mind their responsibility to minimize any opportunity for exploitation and manipulation of their system.
Finally, IEEE wants to ensure that standards exist to guarantee the “competence” of the human operators of AI systems. These operators need to be skilled enough to recognize when the AI’s decisions are flawed and need to be overruled.
What Makes the Principles Effective?
Nothing is going to prevent rogue actors from developing an AI system to terrorize humanity if that’s their actual intent. The real trick is keeping honest developers from creating tools that unexpectedly go wrong. Everything we design can fail due to unintended consequences. With complex systems, there is simply no way to account for all possible variables or foresee all the ways they can go wrong. IEEE’s ethical principles are right on target with the imperative to constantly measure results and monitor what the system is doing. Requiring transparency from the start is also the answer to the black box dilemma, ensuring openness in the way the system operates.
And for consumers, it means less worry about being arbitrarily rejected by impersonal AI algorithms. For businesses, the emphasis on validation means ethical AI should yield superior results.
References
Joseph Byrum is an accomplished executive leader, innovator, and cross-domain strategist with a proven track record of success across multiple industries. With a diverse background spanning biotech, finance, and data science, he has earned over 50 patents that have collectively generated more than $1 billion in revenue. Dr. Byrum’s groundbreaking contributions have been recognized with prestigious honors, including the INFORMS Franz Edelman Prize and the ANA Genius Award. His vision of the “intelligent enterprise” blends his scientific expertise with business acumen to help Fortune 500 companies transform their operations through his signature approach: “Unlearn, Transform, Reinvent.” Dr. Byrum earned a PhD in genetics from Iowa State University and an MBA from the Stephen M. Ross School of Business, University of Michigan.
