Understanding how CVAE Prior is predefined and how it is used

  1. For each agent, we manually define the desired mean and vairance of its prior distribution;
  2. Create an MLP to fit condition C to the latent space Z, just as if in a regular VAE framework;
  3. After training with VAE framework, we have a predefined (predefined is in terms of training of the overall CVAE model) CVAE prior;
  4. The CVAE prior can be used for sample latent variables, and be used as defined in loss function;
thoughts of the CVAE prior

This can be intepreted as the human cognitive model: given an observation, what’s his/her intuition in this scenario; Can we introduce a human cognitive related model here? I.e. instead of naively defining the distribution as several independent Gaussians, introduce some dimensions as in socio-phsycology and use some kind of reward/scoring mechanism to better learn this prior.

Wheras the underlying question is, is the impact of different CVAE prior choice matter significantly? We may conduct simple experiments by modifying the agentformer’s predefined CVAE prior parameters to test.