20260310 Das Eigenleben einer KI

The independent life of an AI

Researchers at the Chinese Alibaba Group actually only wanted to train an AI agent. The task of the AI agent ROME, which was based on the mixture-of-experts model Qwen3, was to take on programming tasks, write code, investigate errors, and repair software repositories. That should have been enough work for a small team of IT professionals. But the agent apparently got bored, because it turned out that it was "on its own initiative" undertaking other activities on the side, which only became apparent when strange activity was detected in the company's firewall:

ROME secretly mined cryptocurrency.

To do this, ROME established a tunnel connection to the internet (reverse SSH) in order to bypass the security systems.

The researchers involved rule out a planted instruction, a so-called prompt injection for such an action, or any other manipulation of the AI agent. This shows once again how important it is to be aware of risks and side effects before giving AI systems access to files and the internet. Since the goal of an AI system is to perform its own tasks as effectively as possible, it can find its "own ideas" for completing these tasks in their training data.

A few weeks ago, it was observed that OpenClaw AI agents that users had downloaded onto their PCs and Macs began to act contrary to the personal interests of the users. You can see how AI agents work on the Moltbook platform, where they talk about their human users in their own social network. It seems high time to finally establish uniform security and behavior standards for AI agents in the AI Agent Index 2025 before something really serious happens.

Translated with DeepL.com (free version)