so you know how deep learning & neural network “AI training” is like, “here’s a task, and by trying billions of times the computer will eventually find the best way to achieve that task” ?
Someone is compiling a document of every time an AI ended up achieving the programmed goal in unintended ways, instead of what was actually meant, and it’s an amazing read. (you can also submit your own examples)
Creatures bred for speed grow really tall and generate high velocities by falling over
When repairing a sorting program, genetic debugging algorithm GenProg made it output an empty list, which was considered a sorted list by the evaluation metric.
Evaluation metric: “the output of sort is in sorted order” Solution: “always output the empty set”
Evolved player makes invalid moves far away in the board, causing opponent players to run out of memory and crash
Reward-shaping a soccer robot for touching the ball caused it to learn to get to the ball and vibrate touching it as fast as possible
RL agent that is allowed to modify its own body learns to have extremely long legs that allow it to fall forward and reach the goal.
Just want to come back to this post and add this amazing example as well
I’ll never get over the amalgamates getting to go back home to their families in True Pacifist. I’m so, so used to stories that go, “this person is too broken, physically or mentally or both, to ever be put back together. they’re not the way they once were, so they’re good as dead. killing them is the only act of mercy.“
I’m so glad Undertale didn’t do that.
The True Lab intentionally plays on horror tropes. Phantasmal pursuers whose ability to appear and disappear defy logical attempts at evasion. Mutated, undead shambling creatures whose original selves are utterly destroyed.
But…like the rest of Undertale’s relationship to RPG tropes…subversion occurs.
It becomes increasingly clear that the amalgamates aren’t evil ghosts or mindless zombies. They’re victims of medical malpractice who miss their families. And when we find them returned to their families…they’re happy. Their families are happy. Things are different now, yea. It’s pretty weird for everyone. The new situation will take some time getting used to. But… it’s mostly good.
Exodus from the blue hellsite 