When Models Manipulate Manifolds: The Geometry of a Counting Task

Language models perform visual reasoning on text despite receiving only token sequences. This paper reveals how Claude 3.5 Haiku accomplishes linebreaking in fixed-width text through geometric transformations of curved feature manifolds.

The Challenge

When you write on paper, you instinctively know when to start a new line. Language models face this same challenge but see only integer tokens, not visual layout. To predict linebreaks correctly, models must:

1. Count characters in the current line
2. Estimate the line width constraint
3. Compare remaining space with the next word’s length
4. Decide whether to break the line

How Models See Position

The model represents character counts on curved, one-dimensional manifolds embedded in low-dimensional subspaces. These manifolds exhibit “rippling”—a pattern that optimally balances capacity constraints with the need to distinguish different count values.

Character count features activate in overlapping ranges, similar to place cells in mammalian brains. Early features respond to positions 1-15, later features to 35-55, with progressively wider receptive fields. This creates a sparse, distributed representation where position is determined by which features activate and how strongly.

Geometric Boundary Detection

To detect approaching line boundaries, attention heads perform geometric “twisting” operations. They rotate the character count manifold to align with the line width manifold at specific offsets. When character count approaches line width, this alignment creates large inner products that signal boundary proximity.

Multiple boundary heads work together with different offsets, implementing a “stereoscopic” algorithm. Each head’s response peaks at different distances from the line end—one at 5-10 characters remaining, another at 10-20 characters. Their combined outputs provide high-resolution estimates across the full range.

The Final Decision

The model arranges character-remaining estimates and next-word lengths in near-orthogonal subspaces. This geometric structure makes the linebreak decision linearly separable: when remaining characters minus next-word length falls below zero, predict a newline.

Break predictor features activate only when the next word would exceed the line limit. Break suppressor features activate when the next word would barely fit, downweighting newline predictions.

Distributed Construction

Individual attention heads cannot generate sufficient curvature alone. Instead, multiple heads across layers cooperatively construct the counting manifold:

• Layer 0 heads each contribute ray-like outputs that sum to form initial curvature
• Layer 1 heads refine the representation, sharpening receptive fields
• Each head specializes in different positional offsets, tiling the counting space

Heads use previous newlines as “attention sinks,” attending primarily to newlines for a fixed number of tokens, then spreading attention over their receptive fields. The output combines token position estimates with corrections based on actual token lengths.

Visual Illusions

The model’s spatial perception can be hijacked. Inserting “@@ " into text disrupts linebreak predictions because counting heads mistakenly attend to these characters instead of newlines—a learned behavior from git diff contexts.

Code-related character pairs like >>, }}, and || also disrupt spatial perception, while random character pairs have minimal effect. This demonstrates how learned priors about contextual cues can modulate spatial estimates.

Key Insights

Manifold Representations: Models use curved manifolds rather than orthogonal dimensions to represent scalar quantities efficiently. The rippling pattern emerges from optimal packing of many values into low-dimensional spaces.

Geometric Computation: Linear transformations on manifolds enable complex comparisons impossible with one-dimensional representations. Rotation, alignment, and orthogonal arrangement create geometric structures where decisions become linearly separable.

Distributed Algorithms: Complex representations require coordination across multiple components. No single attention head can generate the necessary geometric structure alone.

Feature-Manifold Duality: The same computation can be understood through discrete features (attribution graphs) or continuous manifolds (geometric transformations). The geometric view often provides simpler explanations.

This work demonstrates that language models develop sophisticated spatial reasoning capabilities, using geometric principles similar to biological vision systems to perceive and manipulate the visual structure of text.