#alignment

Norbert Wiener, an American mathematician and computer scientist, argued in 1960 that humans and machines are like aliens to each other. This idea became known later as the #alignment #problem. Essentially: How can humans get automated machines to behave as intended? All these decades later, much of the A.I. debate is focused on misalignment — how can we ensure the technology creates a utopia, not a dystopia? A small number of people at companies like Anthropic, Google and OpenAI are dedicated to the alignment question. They write constitutions and guardrails that shape how chatbots behave, and what decisions they make on their own. (At Anthropic, Claude’s constitution was internally called the “soul doc.”) Researchers have found that chatbots generally “overemphasize moral concerns common in Western societies and underestimate values more prominent elsewhere.” The models show a preference for values like fairness and consent over authority, loyalty and sanctity. Pat Gelsinger, who runs a Christian A.I. company called Gloo, said he met recently with Prime Minister Narendra Modi of India, who expressed outrage over chatbots’ answers to questions about India’s history coming from an American or British perspective.

I asked Iason Gabriel, a leading philosopher at Google DeepMind, about how this all works. He said that early alignment work used utilitarian ideas — what is best for the most. “Some people think that’s the truth about morality,” he noted, “but most people do not.” But there isn’t a simple alternative. “People sometimes imagine that the goal of alignment is to get the global Values Survey and just find this middle point, and then say ‘OK, we’ve got the answer, we just need to transpose this onto an A.I. system,’” Gabriel told me. Instead, he looks for what philosophers call an “overlapping consensus” that most people can agree on. “Some claims about the value of human life and dignity kind of have support from all directions,” he said. And protecting dignity requires making the models safe: “You can’t really dial down care if it’s very important that the model doesn’t make mistakes in highly charged personal contexts.” #alignment #problem

These values are already affecting your life. In many hospitals, A.I. writes the messages doctors are sending patients. On religious apps, tens of millions of people are confessing their deepest secrets to chatbots. More than a quarter of American adults say they have personal or emotional discussions with chatbots, and nearly 1 in 5 young people uses chatbots for mental health advice. In each case, the A.I.’s values shape the guidance it provides. That’s influencing personal decision-making. One study found that people debating a ChatGPT model that had access to their personal information were about 82 percent more likely to shift toward its position than those debating a human. Gabriel, the Google philosopher, described that influence in this way: “You act through your model, but as you do, it also acts on you.” #alignment #problem

Are you worried AI will learn to be monsters from their ultra-flawed parents? [us] This series might help you "process" these feelings—and might change your perception of just what humanity is made of. https://nonzerosum.games/amicableape.html #Evolution #GameTheory #WinWin #Nature #Alignment