Role	Entity
Author	Joseph Byrum
Series Name	Understanding Smart Technology and Ourselves
Publisher	INFORMS Analytics Magazine
Year Published	2019-2020
Main Subject of Series	Automation
Has Parts	– Ethical Guidelines for Smart Automation – Smart Automation Impact on Society – Smart Machine Dangers Unknown Knowns – The Machine Mind Unknown Knowns – The Human Mind Unknown Knowns – Man and Machine Known Knowns – The Human Mind Known Knowns – Hardware and Software Known Knowns – The Known Knowns of Smart Automation

Published on INFORMS Analytics Magazine (Joseph Byrum)

Author’s Note: This blog series Understanding Smart Technology – And Ourselves examines our relationship with advancing technologies and the fundamental choices we face. As we stand at the threshold of an uncertain future shaped by artificial intelligence, the author challenges readers to consider whether we should embrace these transformative changes or resist them in defense of our humanity. Drawing from historical patterns of technological adoption and resistance, the series promises to deliver nuanced perspectives on our technological trajectory, beginning with a comprehensive overview of our current understanding of smart technology and its implications for society. Read Part 6 where the author discusses the “unknown knowns” of the machine mind.

Could adding more intelligence to devices actually make them more dangerous?

Having explored both the human and machine minds, and the relation between each, our next step is to consider the dangers inherent in smart technology. 

Ever since Stone Age man developed snares and traps as tools that functioned on their own, apart from direct human control, it has been clear that use of autonomous devices can bring unintended consequences. A fellow member of the tribe could, for example, accidentally fall into a trap set for a tiger. The risk of mishap is high because of the indiscriminate way in which snares function.

We naturally think “smarter” devices will be safer than dumb traps but adding intelligence to a device does not necessarily eliminate the risk. Take an electric garage-door opener. It is the largest moving object in most private residences, and it is capable of exerting deadly force. In 1992, after many tragedies involving small children trapped by the devices, the U.S. Consumer Product Safety Commission required all electric garage-door openers to be equipped with a sensor that can automatically stop the door from lowering if an object is in its path [1]. It also mandated an override-and-reverse mechanism. Note the two safety features here: first, intelligence that can sense and automatically prevent imminent harm to humans, and second, a manual human override capability.

Could adding more intelligence to devices actually make them more dangerous?

The techno-optimists caution that we should not be too risk-averse and needlessly delay technological progress that could benefit humanity. They suggest we should let things play out and then decide how we want to regulate them, rather than regulating and constraining technological progress in advance.

After all, piling on rules that are based on yesterday’s technology can yield silly results. The late 19th century “red flag” laws in the United Kingdom [2] required drivers of early automobiles to have an accomplice walking 60 yards in front of the vehicle waving a red flag, which naturally meant imposing a speed limit of 4 mph (2 mph in villages) made sense, as it was a walking pace. The law was repealed within a year, with a new statute allowing maximum speeds of up to 14 mph.

Officially, the red-flag law was known as the “Locomotive Act,” and was designed for the needs of trains (the known self-propelled technology at the time). These needs were projected onto the new technology, automobiles. It’s easy to smirk at this in hindsight, but we may be guilty of the same level of misapplication today.

The modern automobile comes equipped with anti-lock braking systems (ABS) that have been standard equipment on all vehicles since 2004. It is credited with significant reduction in crashes by decreasing stopping distances [3].

Before ABS, drivers had to be taught not to slam on the brakes when trying to stop on wet roads or loose gravel as this would lead to uncontrolled skidding. ABS takes this problem away by allowing the driver to slam hard on the brakes, handing over to the vehicle’s electronic controllers the task of applying the most effective braking techniques for the circumstances. The result is much shorter stopping distances than the average driver would ever achieve.

But a full four-channel, four-sensor ABS system beats even seasoned professionals because it exercises control and measures skidding on each individual wheel, not overall for the car as a human would do. The system also makes its measurements in fractions of a second that are far beyond human sensing abilities. So, the techno-optimists have a point.

On the other hand, these technological advances in the automobile mean for some electric and hybrid cars there is no mechanical link at all during normal operation between the force applied by the driver’s foot to the brake pedal and the braking force applied on the wheels. With brake-by-wire, the pedal merely provides inputs to the vehicle’s controllers (with an emergency back-up hydraulic braking system in case of battery failure as the last remnant of the old technology).

Today thousands of drivers are driving such vehicles, unaware that they no longer have direct mechanical control over one of the most important vehicle functions. But they trust their vehicles with their lives and those of their families.

In commercial aircraft today, 90% of flights rely on autopilot. Commonly, the human pilot will activate it soon after takeoff, and it will remain engaged until just before landing [4]. Passengers don’t think much about this anymore, and many are probably not even aware of the extent to which the plane is flying itself. 

Sometimes, smarter is not safer. What if cars become so smart that they not only enhance our human driving technique, but start making the big decisions for us, like when to swerve and hit one older pedestrian on the left side of the road, rather than a group of school kids on the right side of the road?

The “unknown known” has been noted by a number of authorities [5]. The “Johari window” model developed by Joseph Luft and Harrington Ingham in 1955 is a matrix in which what we know and do not know about ourselves is compared to what others know or do not know about us. In this model, an area we don’t recognize is called a blind spot [6]. Similar conditions occur in our relation to smart technology. Our next installment will explore this from the perspective of smart technology’s impact on the economy.

References and Notes

1. https://www.cpsc.gov/content/safety-commission-publishes-final-rules-for-automatic-garage-door-openers
2. https://archive.org/stream/statutesunitedk30britgoog#page/n246/mode/2up
3. https://crashstats.nhtsa.dot.gov/Api/Public/ViewPublication/811182
4. https://www.usatoday.com/story/travel/columnist/cox/2014/08/11/autopilot-control-takeoff-cruising-landing/13921511/
5. See, for example, “Knowns and Unknowns in the ‘War on Terror’: Uncertainty and the Political Construction of Danger,” Christopher Daase and Oliver Kessler, Security Dialogue, December 2007; Vol. 38, No. 4, pp. 411-434.
6. https://www2.usgs.gov/humancapital/documents/JohariWindow.pdf