Against text as interface

The manos in manipulation is underappreciated. The manipulation that one has when drawing with a pen: thoughtless yet precise. It's a pity that the word has a negative connotation. It emphasizes shaping the world in an embodied manner. I think this is close to a friend's provocation—

He says that the true joy of programming comes when you can do it without a reference, offline. Perhaps programming on a flight is prototypical. For him, it was the Trans-Siberian Railway. I think it may be the reason he's built so many little tools for himself, which compose into wholes. Its programming as direct expression of the mind, channeled through the body.

The new text-to-image models have text as their interface. One makes a request, sees the response, and then tweaks to continue. It's a game of hermeneutics, trying to align ones desires with the language of the machine. The latent space is composed out of "that-which-has-been" i.e. images that already exist.

If one emphasizes content, then the users manipulation returns to a negative connotation. One must coerce the black-box in the direction one desires. Often the mirrored effect dominates: the black-box coerces ones desires to fit into its latent image. One cannot operate "offline," as the system is never under ones control. The modernist artist dies, once again, replaced by the curator.

Instead, "prompt engineers" are structuralist anthropologists, teasing apart the linguistic system embedded in the latent space. The critical dimension provided is outside the generated (kitsch) content. Like a photographer who frames reality, the prompteur can frame artifacts from the latent-space of the dataset. Though here it is second-order: a framing of a framing of reality.

The most compelling demos involve collaging (stable-diffusion Photoshop). One generates isolated items, and then composes them with an interactive tool. It's returns to manos, but with new raw material and verbs: inpaint, re-light, &c. For this, the interface remains a GUI, not text.

Cristóbal Sciutto, September 2022.