Anthropic claims to have found Claude’s “consciousness”, although the data tells another story

Anthropic has published an article that promises to change the way we understand the inner workings of your business models. artificial intelligence (AI), like we explain to you last week. This company claims to have identified a kind of global workspace within Claudewhich he has named J-Spacewhere the model analyzes and manipulates concepts before these become part of the final answer. What is surprising is that, according to Anthropic, this space has not been deliberately programmed: it has emerged as a byproduct of training the model.

To analyze it and understand how it works, a technique known as J-Lens which is capable of mapping the internal activations of the model onto words from its own output vocabulary. When asking Claude to solve a multi-step mathematical calculation the visible answer contained only the final result, but within the J-Space each intermediate step was represented separately. Without this technique no part of that process could have been observed from the outside.

However, Anthropic itself recognizes that much of what a language model does does not even go through this internal space. Speaking fluently, remembering simple facts or applying basic grammatical rules almost completely dispenses with J-Spaceand when the company blocked its use Claude continued to function normally, although he lost some of his higher-order cognitive functions.

Claude “knows” that he is being watched

During one of his most striking experiments, Anthropic subjected Claude to injection attempts. prompts during data acquisition processes. He J-Space then showed words like “false”, “injection”, “erroneous”, “prompt“, “fraud” and “poison”, while the visible response of the model completely ignored the attempted manipulation. Interestingly, when pressed to comment on the matter, terms such as “transparency”, “honesty” and “truth” emerged in the internal space.

The Anthropic researchers also observed that Claude seems to recognize when he is being evaluated, which leads him to behave differently when faced with tests than when faced with prompts everyday. In a particularly revealing test, Anthropic removed J-Space the vocabulary associated with that consciousness of being put to the test (terms such as “fake”, “fictitious” or “scenario”) and found that Claude became much more vulnerable to attempts at blackmail or provocation.

Be that as it may, the researchers’ report admits important limits: the J-Space is restricted to a single-token vocabulary, so plans or concepts that cannot be expressed with a single word may not appear in the reading, even though the model is calculating them anyway. Anthropic also does not dare to claim that monitoring this space is sufficient to guarantee the alignment of a model.

However, we must not overlook that the language used in the article sometimes borders on the idea of ​​an emerging consciousness. Neel Nanda, head of model interpretability at DeepMind, supports this finding as real evidence of a cognitive space within the modelsalthough he clarifies that the practical usefulness of J-Lens remains limited. This achievement, in any case, opens a promising avenue for auditing the honesty of models, although it is still far from being a complete window into a machine’s thinking.

Image | Generated by Xataka with ChatGPT using a prompt created with Claude

More information | Anthropic

In Xataka | While most oppose AI data centers, there is one group enthusiastic about them: merchandise thieves