The Hands Know More Than the Words: Why Embodied Meaning Is Where AI Stalls at the Therapy Door
- Tversky argues that much human thinking is not in words at all: gesture, sketches, maps, graphs and diagrams represent thought more directly than language, and a string of related gestures can carry a whole system of ideas.
- She claims that making or seeing a gesture or a drawing does not merely express an already-finished thought but changes thought, in the person who produces it and in the person who watches.
- She holds that these meanings are frequently internal, invisible, unique to the moment and not easily broken into parts, with shared and private meaning emerging and shifting in real time during an exchange.
- From these premises she concludes that exactly these features of human cognition present a structural challenge to current multi-modal AI, which is trained to map across already-encoded text, image and sound.
This is a "Focus Article" in a bioethics symposium on artificial intelligence – an argument, not an experiment. It matters to our field because the thing Tversky describes is the raw material of a session: a hand that opens and stops halfway, a body that leans back as the words say "I'm fine," a shape drawn in the air to stand for a relationship the patient cannot yet name.
What the argument actually claims
Tversky's case is that language is a late, lossy translation of thinking that mostly happens in space and movement. We point, we shape, we arrange ideas with our hands before we have sentences for them. The drawing on the napkin is not a record of the idea; it is part of how the idea forms. And the meaning of all this is not fixed in a dictionary – it is built between two people in the moment and changes as they go.
The bridge to AI is the load-bearing claim, and it is worth stating carefully. Multi-modal systems are powerful precisely because they operate on encoded modalities: tokens, pixels, waveforms. Tversky's point is that the meaning a clinician reads off a half-gesture is none of these. It is unique, transient, co-constructed, and resists being decomposed into the features a model is trained to recognise. That is an argument about a limit, not a measurement of one.
For your practice
Hold this next to the products now being sold into mental health. Most "AI therapy" runs on transcript: it sees the words, sometimes the audio, increasingly the video frame – but it reads them as encoded signal, not as meaning made in a shared moment. Tversky names the gap between those two. The clinical skill that is hardest to script is also the one such systems are structurally worst at: catching the meaning that the body shows and the words deny.
The honest conclusion is sober, not triumphant. This is a single essay, and the absence of embodied reading in a tool is a reason to keep that tool in a narrow lane – intake, psychoeducation, between-session practice – not proof that no future system will close the gap. Use the argument to sharpen your own evaluation question: when a vendor claims a model "understands" a patient, ask what, exactly, it is reading – and what, in the room, it cannot see.
The meaning a clinician catches in a half-finished gesture is exactly the kind that does not survive being turned into tokens.
This is a conceptual focus article, not empirical evidence; it offers a theoretical case about a limit of multi-modal AI rather than a measured demonstration, and its clinical relevance is an extension we draw, not a claim Tversky tests.