Flow matching between gaussian distributions of different space

[ttsesm]

Hi @DoronHav,

Congratulations for your work, it looks indeed quite interesting and possibly could be helpful for my study of research. Thus, I would be interested to your opinion considering that in the flow matching field I am quite new.

I will try to give you a brief description of my problem and then possibly we could discuss about it. Imagine that you have the following test case where I have a set of 3D gaussian distributions (also defined as ellipsoids) by their mean value μ (being a 1x3 vector) and their corresponding covariance matrices Σ1 (being a 3x3 matrix). Then I have a set of 2D gaussian distributions (also defined as ellipses) again by their mean value ν (being a 1x2 vector) and their corresponding covariance matrices Σ2 (being a 2x2 matrix). So in practice the ellipses are the projected ellipsoids in a plane through a linear map P, $ℝ^3$ → $ℝ^2$.

Having this setup, where my input is the 2D and 3D gaussian distributions I would like to recover the projection matrix P and/or the correspondences/coupling between the two set of distributions.

Initially I started playing with optimal transport and the Gromov-Wasserstein distance in the Bures-Wasserstein space in order to try to find out the possible couplings/transport plan without success though, since apparently the distortion created from the perspective projections is not easy to be handled. I believe the fundamental challenge lies in the nature of GW distance: rather than directly comparing individual 3D and 2D points, GW instead seeks to align the internal distance structures of the two distributions. ​
However, this assumption does not hold well in our case, as projection inherently distorts internal distances. Specifically, in a 3D-to-2D projection, two points that are far apart in 3D space may become overlapping or closer together in the 2D projection. This distortion disrupts the internal distance relationships, making GW-based matching unreliable. For example, a ring-like structure formed by 3D Gaussian ellipsoids may appear as a stretched elliptical shape when projected from a specific angle.

Thus, I would be interested to hear your opinion whether I could use the Wasserstein (or Burres-Wasserstein) Flow Matching from your work here to create a model that will learn how to apply this mapping/projection through a flow matching interpolation.

Thanks.