Machines can't cooperate human-style?

We were shocked by several aspects of the openAI/HuggingFace hack -- foremost among them the way AI agents spontaneously coordinated their attack, dividing labor, sharing lore, even setting up a message board to communicate, then reconstituting it in novel ways (communicating by creating folders with messages in the filenames). But recent evidence suggests that computer code can't cooperate the way humans can.

Human cooperation is complex. I've written about it here. It has a number of interacting components -- altruistic punishment, for example. But the norms that enable people to perform apparently altruistic acts on behalf of strangers are based on reciprocity. The simplest form of reciprocity is the quid pro quo: you scratch my back, I'll scratch yours. Most important is diffuse or indirect reciprocity: I don't know you, and it is unlikely you will return the favor, but I might do something for you if you belong to my tribe. But maybe software bots won't be able to truly cooperate the way humans do (and maybe they will: these are still early days.)

This recent research implies that direct and diffuse reciprocity might not be one of the potential powers of bot swarms:

Simple and complex forms of machine intelligence are becoming involved in many collective action challenges humans face, including ensuring safety in groups on the move. However, the social repercussions of intelligent assistance are often overlooked. We used a unique cyber-physical lab experiment involving remote-control robotic cars and widely distributed online drivers. We show that autonomous safety systems in cars can degrade the ordinary norms of reciprocity between people. Humans have developed social norms, but these can collapse when people are allowed to leave their coordination decisions to machines.