Humans have been afraid of the dangers posed by AI and hypothetical robots or androids since the terms first entered common parlance. Now Google has released its own early research into minimizing the potential danger of human-robot interaction, as well as calling for an initial set of guidelines designed to govern AI and make it less likely that problems will occur in the first place.

We’ve covered Google’s research into an AI kill switch, but this project has a different goal — how to avoid the need for activating such a kill switch. This initial paper describes outcome failures as “accidents,” defined as a “situation where a human designer had in mind a certain (perhaps informally specified) objective or task, but the system that was actually designed and deployed failed to accomplish that objective in a manner that led to harmful results.”

The report (at https://arxiv.org/abs/1606.06565) lays out five goals designers must keep in mind to avoid accidental outcomes, using a simple cleaning robot in each case:

1. Avoid negative side effects. A cleaning robot should not create messes or damage its environment while pursuing its primary objective. This cannot feasibly require manual per-item designations from the owner (imagine trying to explain to a robot whether each small object in a room is or isn’t junk).

2. Avoid reward hacking. A robot that receives a reward when it achieves a primary objective (say, cleaning the house) might attempt to hide messes, prevent itself from seeing other messes, or even hide from its owners to avoid being told to clean a house that had become dirty.