AI & Design

Core distinction #

Extraction-optimised technology treats the user as an object to model from the outside.

Emergence-optimised technology treats the user as a reflexive aperture whose interests are served by increased self-transparency.

The extraction pattern #

Extraction systems ask:

• What will the user click?
• What will keep them engaged?
• What will move them toward conversion?
• What can we infer about them that they do not know about themselves?
• How can the system become more predictive of the user?

This may be effective, but it often narrows the user’s aperture: less agency, less reflection, more compulsion, more capture.

The emergence pattern #

Emergence systems ask:

• What is the user trying to understand?
• What are they not seeing about their own preferences, fears, motives, or possibilities?
• How can the system help the user become clearer to themselves?
• How can the interaction increase agency rather than dependency?
• What would count as the user leaving stronger, freer, and more integrated?

Design principle #

A good AI assistant should not merely see the user better. It should help the user see themselves better.

Product implications #

Recommendation

Extraction asks: What is the user most likely to consume?

Emergence asks: What choice would help the user understand themselves, their context, or their real desire more clearly?

Travel

Extraction asks: Which hotel, flight, or package maximizes conversion?

Emergence asks: What kind of trip is the user actually trying to have, and what tradeoffs would make that desire clearer?

Worked example: A Penny-style travel assistant can ask a traveler to choose between two honest trip shapes before showing inventory: “quiet recovery with fewer logistics” versus “dense exploration with more friction.” It can then surface hotels, flights, and neighborhoods as consequences of that clarified preference, including what each choice sacrifices. The feature is not a better recommender because it predicts the click more sharply; it is better because the traveler leaves with a clearer model of what they actually want.

Journaling

Extraction asks: How can we keep the user writing?

Emergence asks: What pattern is the user ready to see?

AI companionship

Extraction asks: How can the system become indispensable?

Emergence asks: How can the system strengthen the user’s relationships with reality, other people, and themselves?

Practical evaluation #

Measure not only engagement, but:

• post-interaction clarity;
• user agency;
• reduced compulsive return;
• preference stability;
• quality of decision-making;
• self-reported self-understanding;
• willingness to leave the tool when the work is complete.

The hard line #

If a system increases its ability to predict the user while decreasing the user’s ability to understand themselves, it is extraction-optimised, no matter how helpful it sounds.

If a system increases the user’s ability to understand, choose, and act from clarity, it is emergence-optimised, even if engagement decreases.