>Even the mere existence of this kind of predictive control will help authoritarians. Self-censorship was perhaps the East German Stasi's most important disciplinary mechanism. AI will make the tactic dramatically more effective. People will know that the omnipresent monitoring of their physical and digital activities will be used to predict undesired behavior, even actions they are merely contemplating. From a technical perspective, such predictions are no different from using AI health-care systems to predict diseases in seemingly healthy people before their symptoms show. In order to prevent the system from making negative predictions, many people will begin to mimic the behaviors of a "responsible" member of society. These may be as subtle as how long one's eyes look at different elements on a phone screen. This will improve social control not only by forcing people to act in certain ways, but also by changing the way they think. A central finding in the cognitive science of influence is that making people perform behaviors can change their attitudes and lead to self-reinforcing habits. Making people expound a position makes them more likely to support it, a technique used by the Chinese on U.S. prisoners of war during the Korean War. Salespeople know that getting a potential customer to perform small behaviors can change attitudes to later, bigger requests. More than 60 years of laboratory and fieldwork have shown humans' remarkable capacity to rationalize their behaviors.

[Show 3 more lines]