Editing Antibody Design by AI (section)

== Variational autoencoder model for antibody sequence optimization ==

Another alternative approach to Ens-Grad we also applied for optimizing antibodies was a framework based around a recurrent variational autoencoder (VAE) generative model that has been proposed to revise natural language and other discrete sequence structures 30. This approach is based around the idea that combinatorial optimization of discrete sequences may be easier in an alternate space of learned continuous representations, in which both the fitness function and the region of high-performing sequences adhere to a smoother and simpler geometry. Before optimization of antibody sequences, the VAE approach first learns a probabilistic generative model of the antibodies, in which continuous latent vectors are responsible for producing both the antibody CDR-H3 sequence as well as the corresponding binding affinity. Both the likelihood function of sequences given latent factors (decoder) as well as an inference model of latent values responsible for generating a particular sequence (encoder) are parameterized using GRU recurrent neural networks 31, which are jointly trained with a feedforward neural network that predicts binding affinity from inferred-values of the latent factors (Fig. S4). After these components are learned, we leverage this model to optimize one of the seed CDR-H3 sequences as follows: the sequence is encoded into the latent space by our inference network, then constrained gradient-ascent is applied to these continuous latent variable values with respect to the predicted binding affinity until a locally optimal latent-factor configuration is found. Finally, the most likely sequence (according to our decoder network) associated with this optimal latent vector is returned as the optimized sequence. This method simultaneously leverages the generative capacities of the recurrent decoder (to ensure naturallooking CDR-H3 sequences are produced), the compressed sequence representation learned by the recurrent encoder network (to enable the application of lower-dimensional continuous optimization algorithms), and the posterior Kullback-Leibler penalty of the VAE (to ensure the encoded sentence representations follow a simple geometry).

Our VAE method was applied with the same architecture, training, and sequenceoptimization settings detailed in 30. We employed 128-dimensional vectors as latent factors together with a learned 8-dimensional continuous representation of each amino acid upon which our inference network was trained to operate. Training proceeded with the mean-squared error loss applied to binding affinity labels equal to the R2-to-R3 enrichment, as in the previously described regression task. From each seed sequence, this method was applied to produce at most 20 new sequences, where generated sequences that violated our set of structural conditions were subsequently omitted as in the case of the other methods.