projection

in the early 1900s, a person trained a horse to answer math questions, and took him to tour the world¹

as far as everyone could see—including the trainer—the horse answered on his own

but studies revealed that people unknowingly signal answers to the horse, fully believing that the horse answers on his own

in 1966, a creator of a simple chatbot found that "extremely short exposures to a relatively simple computer program could induce powerful delusional thinking in quite normal people"²

presented with an abstract interface, people tend to see motivation, understanding, and reasoning

they tend to not see, or be shown, the material reality behind the interface

as complexity grows, the effect sharpens, while more is obfuscated

there is more abstraction to project understanding, reasoning, knowing onto

and more material reality to be hidden

1. Crawford, Kate. Atlas of AI. Yale, 2021. "Introduction: The Smartest Horse in the World."
2. Weizenbaum, Joseph. Computer Power and Reason: From Judgment to Calculation. W. H. Freeman and Company, 1976.