Neural Netw. 2026 Aug 3;205(Pt B):109461. doi: 10.1016/j.neunet.2026.109461. Online ahead of print.
ABSTRACT
This work revisits diffusion Transformers for relative-depth-conditioned and joint image-depth synthesis, focusing on two bottlenecks: (1) cross-modal attention degrades around edges and structural regions, causing geometric distortions; (2) the depth branch can be overly influenced by the semantic branch, leading to over-coupling and instability. To address these issues, we propose JointConn-v2 with GCM-WFM (Gated Cross-Modal Weighted Flow Matching), a unified framework for joint and depth-conditioned image synthesis. JointConn-v2 strengthens bidirectional guidance via Swap-Q cross-attention, explicitly injects 2D relative positions and edge energy into the attention logits through a Geometric Mask Bias, performs token-level spatial selection of where and in which direction to fuse cross-modal features via Regional Routing, and controls whether and how strongly cross-branch injection occurs through a sample-level Content Gate with residual fusion. On the training side, we introduce GCM-WFM, which regresses a joint vector field in the packed sequence space and incorporates temporal, geometric, gating, and routing terms into the objective with diagonal weights. Our approach achieves a better balance between edge controllability and cross-modal consistency. The current formulation treats depth as a normalized relative geometric signal and is not intended to preserve absolute metric scale. Code is available at https://github.com/haizhu12/JointConn-v2.
PMID:42574824 | DOI:10.1016/j.neunet.2026.109461
