The Geometry of Creativity in Artificial Minds

Part II — turning the same machinery into a creative engine. Status: Concept proposal (unvalidated). More materials (work-in-progress): https://osf.io/wprfm/

TL;DR

We turn “intuition” into stable creativity:

Spot Conceptual Inflection Points (CIPs) — rungs where a small dimension increase reveals rich new options.
Diverge low-D, converge mid-D — branch ideas where geometry bends; keep only those that re-lock soon after.
Constrained hallucination — candidates must keep a connected path across dimensions (no orphan leaps).
Merge-stable creativity — fuse concepts via a coarse blend + sparse high-D residual so the hybrid is novel and remains stable when you zoom.

Part 1 — Creativity, in one picture

Think of the ladder (128→256→512→1024…) as a zoom dial on meaning.

At low-D you see gist and variation axes. Some neighbors keep swapping — that’s a hinge where new ideas live.
You branch cheap moves at low-D (pivot, blend, exaggerate, invert), then test them at 512/1024.
You keep only branches that re-lock (stable neighbors, wider margins) — creativity with guardrails.

Two concrete domains:

Impulse responses (audio): Low-D moves along Presence/Air/Thump/Room axes, then lock at 512 (phase/spectral checks).
Text (double-embedding): Low-D pivots on tone/stance/analogy; lock at 512 when plan/constraints hold.

Part 2 — How it works (still friendly)

1) CIPs: “opportunities to create”

We score each rung with a Rupture Index: high when flip-rate and curvature spike (many new options), but only act if a lock is likely within ≤2 rungs. In plain terms: diverge where the manifold bends; converge where it re-locks.

2) Constrained hallucination

A candidate idea is valid only if you can trace a connected path across rungs back to the seed: low-D gist stays aligned; top-K neighbors overlap from 128→256→512; late flips are suppressed. If the path breaks, shrink the step or insert a bridge.

3) Merge-stable creativity

To merge A+B (or A+B+C):

Check compatibility across rungs (analogical alignment or “meet-again” reconvergence).
Do a coarse low-D blend (shared scaffold) and add a sparse high-D residual (nuance).
Accept only if margins improve at 512/1024 and the path remains connected.

4) Controller & logs

We reuse the intuition controller (bandit + hysteresis + pilot-zoom) to decide when to branch and when to lock. Every decision gets a flight-record: rung path, flip/curvature traces, constraints satisfied, and “why we stopped.” Note: All rung values here (e.g., 128/256/512/1024) are examples, not fixed rules; pick rungs that fit your model and budget.

Part 3 — Show me the math (Creativity v1.0)

Notation (inherits Part I)

Encoder: $f_\theta(x)$ outputs $z_x\in\mathbb{R}^D$ . Use the unit-norm $\bar z_x=z_x/\|z_x\|_2$ for distances.
Fixed orthonormal basis (versioned per model): $U\in\mathbb{R}^{D\times D}$ .
Projection at depth $d$ : $p_x^{(d)}=(U^\top \bar z_x)_{1:d}$ and $\hat z_x^{(d)}=p_x^{(d)}/\|p_x^{(d)}\|_2$ .
Cosine distance at depth $d$ : $\delta^{(d)}(x,y)=1-\langle \hat z_x^{(d)},\,\hat z_y^{(d)}\rangle\in[0,2]$ .
Top- $K$ set / rank: $\mathcal{N}^{(d)}_K(x)$ and $\pi^{(d)}_K(x)$ .
Depth ladder: $\mathcal{D}$ denotes the served dimensions (e.g., $\{128,256,512,1024\}$ ).

3.1 — Signals across rungs

Let $\Delta$ be the step (e.g., $d\!\to\!d+\Delta$ ).

Flip / rank-stability (Kendall on top- $K$ ): $\tau_d(x)=\tau\big(\pi^{(d)}_K(x),\,\pi^{(d+\Delta)}_K(x)\big),\quad F_d(x)=\tfrac{1-\tau_d(x)}{2}\in[0,1]$ .

DDP curvature (second difference; skip ends or use one-sided): $\kappa_d(x)=\delta^{(d+\Delta)}(x,y_\star)-2\,\delta^{(d)}(x,y_\star)+\delta^{(d-\Delta)}(x,y_\star)$ (with $y_\star$ the current best neighbor or an average over top-5).

Margin change (nearest vs runner-up): $m_d(x)=\delta^{(d)}_{(2)}(x)-\delta^{(d)}_{(1)}(x),\quad \Delta m_d(x)=m_{d+\Delta}(x)-m_d(x)$ .

Entropy jump (if neighbors carry labels/clusters): $H_d(x)=-\sum_c p^{(d)}_x(c)\log p^{(d)}_x(c),\quad \Delta H_d(x)=H_{d+\Delta}(x)-H_d(x)$ . If logits are used, $p^{(d)}_x=\mathrm{softmax}(g^{(d)}_x/T)$ .

Factor conflict (text only): $C_d(x)=\|g^{(d)}_x-g^{(d+\Delta)}_x\|_1$ or $\mathrm{KL}\!\big(p^{(d)}_x\,\|\,p^{(d+\Delta)}_x\big)$ .

3.2 — Conceptual Inflection Points (CIPs) & Rupture Index

Definition (CIP). Rung $d$ is a CIP for $x$ if at least one of $F_d(x)$ , $|\kappa_d(x)|$ , $|\Delta m_d(x)|$ exceeds a threshold and the neighborhood re-stabilizes within $\le 2$ steps: $\tau_{d+\Delta}(x)$ , $\tau_{d+2\Delta}(x)\ge \tau_{\min}$ .

Rupture Index (standardize terms per rung before weighting):
$R_d(x)=w_1F_d(x)+w_2|\kappa_d(x)|+w_3|\Delta m_d(x)|+w_4\Delta H_d(x)+w_5C_d(x)-w_6\mathrm{Pull}_d(x)$

Pull term (centroid attraction at prior rung):
$\mathrm{Pull}_d(x)=\max_c \langle \hat z_x^{(d)},\,\mu_c^{(d-\Delta)}\rangle,\quad \mu_c^{(d-\Delta)}=\mathrm{Norm}\!\big((1/|S_c|)\sum_{y\in S_c}\hat z_y^{(d-\Delta)}\big)$ .

Open/close: branch if $R_d(x)>\theta_{\mathrm{open}}$ and a lock is likely within $\le 2$ rungs; otherwise continue/stop (hysteresis with $\theta_{\mathrm{open}}>\theta_{\mathrm{close}}$ ).

3.3 — Constrained hallucination (cross-dimensional connectivity)

Contract (seed $s$ , candidate $c$ ):

(C1) Gist agreement (low-D): $\cos\big(\hat z^{(d_{\mathrm{low}})}_s,\,\hat z^{(d_{\mathrm{low}})}_c\big)\ge \tau_{\mathrm{gist}}$ .
(C2) Path existence (each hop $d\to d+\Delta$ ): $J_d(c)=\dfrac{\big|\mathcal{N}^{(d)}_K(c)\cap\mathcal{N}^{(d+\Delta)}_K(c)\big|}{\big|\mathcal{N}^{(d)}_K(c)\cup\mathcal{N}^{(d+\Delta)}_K(c)\big|}\ge \tau_K$ or $\tau_d(c)\ge \tau_{\min}$ .
(C3) Late-flip suppression (for $d\ge d_{\mathrm{lock}}$ ): $\mathrm{FlipCount}_{d\to d+\Delta}(c)\le \varepsilon$ .
(C4) Factor bounds (text): $\|g^{(d+\Delta)}_c-g^{(d)}_c\|_1\le \beta_{\mathrm{factor}}$ per hop.

Acceptance score (rescale each term to [0,1] or calibrate weights):
$J(c)=\alpha\|\hat z^{(d_{\mathrm{low}})}_c-\hat z^{(d_{\mathrm{low}})}_s\|_2+\beta \cos\big(\hat z^{(d_{\mathrm{mid}})}_c,\,\hat z^{(d_{\mathrm{mid}})}_{\mathrm{brief}}\big)+\gamma\,\tau_{d_{\mathrm{mid}}\to d_{\mathrm{hi}}}(c)-\lambda\,\mathrm{Violations}(c)$

Multi-hop stability (mean adjacent Kendall across the mid→hi ladder):
$\tau_{d_{\mathrm{mid}}\to d_{\mathrm{hi}}}(c)=\tfrac{1}{L}\sum_{j=0}^{L-1}\tau\!\big(\pi^{(d_j)}_K(c),\,\pi^{(d_j+\Delta)}_K(c)\big)$ with $d_j\in\{d_{\mathrm{mid}},\,d_{\mathrm{mid}}+\Delta,\,\dots,\,d_{\mathrm{hi}}-\Delta\}$ .

Emit only if $J(c)\ge J_{\min}$ and (C1–C4) all hold. If the path breaks, shrink the step or insert a bridge ( $s\!\to\!w\!\to\!c$ ).

3.4 — Merge-stable creativity (concept fusion)

Compatibility via relation alignment: for pairs $(A,B)$ and $(M,\tilde B)$ , define $r_{A,B}^{(d)}=\hat z_A^{(d)}-\hat z_B^{(d)}$ and the bi-scale analogy score

$\mathrm{BAS}(d)=\Big\langle \frac{r_{A,B}^{(d)}}{\|r_{A,B}^{(d)}\|_2},\,\frac{r_{M,\tilde B}^{(d)}}{\|r_{M,\tilde B}^{(d)}\|_2}\Big\rangle,\quad \overline{\mathrm{BAS}}=\frac{1}{|\mathcal{S}|}\sum_{d\in\mathcal{S}}\mathrm{BAS}(d)$

Require $\overline{\mathrm{BAS}}\ge \tau_{\mathrm{BAS}}$ (shared scaffold) or a meet-again pattern: $\delta^{(d_{\mathrm{low}})}(A,B)$ high, $\delta^{(d_{\mathrm{mid}})}(A,B)$ low.

Coarse blend + sparse high-D residual:

$\tilde z^{(d_\ell)}_{\mathrm{coarse}}=\mathrm{Norm}\big(\lambda_A\hat z^{(d_\ell)}_A+\lambda_B\hat z^{(d_\ell)}_B+\cdots\big),\ \sum\lambda_i=1,\ \lambda_i\ge 0$

$z_M^{(D)}=\mathrm{Norm}\big([\ \tilde z^{(d_\ell)}_{\mathrm{coarse}}\ ;\ \Delta z_{d_\ell\to D}\ ]\big),\qquad \Omega(\Delta z)=\mu_1\|\Delta z\|_1+\mu_0\|\Delta z\|_0$

Objective (subject to connectivity):

$\max_{\lambda,\Delta z}\ \Delta\mathrm{margin}_{d_{\mathrm{mid}}}(M)\ +\ \eta\,\overline{\mathrm{BAS}}\ -\ \Omega(\Delta z)\quad \text{s.t. }(C1)\text{--}(C4)\ \text{hold for } M$

3.5 — Creative walk (policy sketch)

Low-D propose: pivot / blend / exaggerate / invert. Mid-D evaluate: compute $J$ at 512/1024. Lock: when $\tau$ plateaus and margin widens (use hysteresis).

for d in {128, 256}:
    if R_d(x) > θ_open and budget_ok:
        props = propose_lowD(x)  # cheap moves
        kept = []
        for p in props:
            p = bridge_or_shrink_if_needed(p)      # enforce (C2)
            if connected_path(p) and J(p; 512) ≥ J_min:
                kept.append(p)
        if kept:
            return lock(best_of(kept), d_lock)
return lock(x, cheap_lock)

Part 4 — What we’ll measure (if we proceed)

Lock-dim histogram (how early we converge), late-flip rate (≥512), and Novelty–Coherence–Stability Pareto for accepted ideas.
Connectivity pass-rate (fraction of branches that keep cross-dim paths).
Budget (proposals/idea; rungs evaluated/idea) vs. quality.

Part 5 — Why this is “creative search” (and why it’s stable)

We explore where geometry bends (high Rupture Index), keep only ideas that re-lock, and fuse concepts with just enough high-D nuance to work — while preserving a connected path across rungs. That yields novel but robust outputs you can audit and reproduce.

Part 6 — Artifacts

OSF (spec, figures, audio/text demos, notebooks): https://osf.io/wprfm/

Part 1: Intuition in A.I

Part 3: Wisdom in A.I