STAR-VAE: A Scalable Latent-Variable Transformer for Controllable Molecular Generation

Many molecular Transformers lack probabilistic latent variables for posterior inference and latent interpolation. We introduce STAR-VAE, a SELFIES-encoded, Transformer-based, AutoRegressive Variational AutoEncoder combining a bidirectional encoder with an autoregressive decoder pretrained on 79 million PubChem molecules. A property signal jointly conditions the prior, posterior, and decoder, while LoRA adapters support fine-tuning on small datasets without modifying the backbone. STAR-VAE achieves 100% validity and near-perfect novelty under unconditional MOSES sampling, the lowest KL divergence on five of ten GuacaMol descriptors, Spearman ρ = 0.62 at 98% validity for synthetic-accessibility conditioning, and directional docking-score control for three Tartarus protein targets. Across four ChEMBL targets, seed-based posterior sampling recovers target-associated held-out scaffolds while label-conditioned sampling produces structurally diverse outputs. Code is available at https://github.com/BiomedSciAI/STAR-VAE.