The encoder is written \(E\) here (and \(\varphi_\psi\) in the GRL literature — the same object). The downstream Operator World Models series uses the same conventions.
a raw input data point — an image, a behavioral-signal window, a sequence
\(s\)
the underlying system state in state space \(\mathcal{S}\) — the physically real thing an action transforms
\(E\)
the encoder: a neural network mapping an input to an embedding (written \(\varphi_\psi\) in GRL)
\(z = E(x)\)
the latent embedding of \(x\) — a vector capturing meaning, not surface form. Lives in latent space \(\mathcal{Z}\)
\(E_{\text{target}}\)
the target encoder: a slow exponential-moving-average (EMA) copy of \(E\), updated by averaging, not by gradients (stop-gradient). Produces JEPA's prediction targets
\(\mathcal{S},\ \mathcal{Z}\)
the state space and the latent space. When actions operate on latents, \(\mathcal{S}=\mathcal{Z}\)
\(\mathrm{Pred}(\cdot)\)
JEPA's predictor — maps a context embedding plus a query to the predicted embedding of the target
\(q\)
a query / position token — tells the predictor which target to predict (where, or how far ahead)
an action operator: a function \(\hat O:\mathcal{S}\to\mathcal{S}\) that turns a state into its successor. The hat marks "operator (a function), not a number"
\(\hat{O}_\theta\)
the specific state operator configured by parameters \(\theta\) — the physically meaningful (often inaccessible) transformation acting on \(s\)
\(\theta\)
operator / action parameters — the knobs that configure the operator (a complete description of how it transforms the state)
\(\Theta\)
the space of all operator parameters \(\theta\)
\(f_\theta\)
the latent operator: how \(\hat O_\theta\) acts on the latent \(z = E(s)\) instead of on the raw state. The object you compute with
\(\Phi(\theta, s)\)
the operator generator: builds an operator from parameters, \(\hat O_\theta(s)=\Phi(\theta,s)\)
\(E(\hat{O})\ge 0\)
the energy functional: how large a transformation the operator is — a least-action / parsimony penalty. (Distinct from the encoder \(E\); the argument disambiguates)
The symbol \(E\) does double duty: the encoder\(E\) (and its EMA copy \(E_{\text{target}}\)) and the energy functional\(E(\hat O)\). They never appear in the same role, and the argument disambiguates — \(E(x)\) encodes an observation, \(E(\hat O)\) scores an operator.