Synthetic polymers are crucial in diverse industries, but current AI-driven design methodologies primarily target linear homopolymers, with limited emphasis on developing customized approaches for copolymers. To address this gap, we introduce a generative model for goal-directed synthetic copolymer design using reinforcement learning. Our model operates in a graph generation environment, facilitating efficient monomer unit design while incorporating domain-specific constraints to ensure high validity rates. In a case study optimizing for Hydrogen Evolution Rate (HER) and synthetic accessibility, our approach showcases the efficacy of reinforcement learning in advancing copolymer design. Furthermore, experimental results underscore the challenges in designing effective scoring functions due to the sparse nature of polymer datasets, emphasizing the need for robust property predictors in polymer design methodologies before integrating more complex generative models into the design process.

Recent advancements in machine learning (ML) have shown promise in accelerating polymer discovery by aiding in tasks such as virtual screening via property prediction, and the design of new polymer materials with desired chemical properties. However, progress in polymer ML is hampered by the scarcity of high-quality, labelled datasets, which are necessary for training supervised ML models. In this work, we study the use of the very recent ’Joint Embedding Predictive Architecture’ (JEPA) type for self-supervised learning (SSL) on polymer molecular graphs, to understand whether pretraining with the proposed SSL strategy improves downstream performance when labelled data is scarce. By doing so, this study aims to shed light on this new family of architectures in the molecular graph domain and provide insights and directions for future research on JEPAs. Our experimental results indicate that JEPA self-supervised pretraining enhances downstream performance, particularly when labelled data is very scarce, achieving improvements across all tested datasets.

Predicting properties, such as toxicity or water solubility of unknown molecules with Graph Neural Networks has applications in drug research. Because of the ethical concerns associated with using artificial intelligence techniques in the medical field, explainable artificial intelligence techniques are used to explain how GNNs make their decisions. To evaluate the performance of those techniques, different metrics are used. The BAGEL benchmark proposes four such metrics, designed to be useable with any GNN explainer. Of these, the applicability of faithfulness was investigated in molecular property prediction, measured by the submetrics of comprehensiveness and sufficiency. While comprehensiveness and sufficiency were designed to be task agnostic, several shortcomings were identified that make it unsuitable for molecular property prediction. Future recommendations are to investigate other pre-established faithfulness metrics or to develop ones that do not require splitting molecules.

As graph neural networks (GNNs) become more frequently used in the biomedical field, there is a growing need to provide insight into how their predictions are made. An algorithm that does this is GNN-SubNet, developed with the aim of detecting disease subnetworks in protein-protein interaction (PPI) networks. GNN-SubNet makes use of a sampling scheme to generate a global explanation in the form of a node mask which indicates each node's importance for all of the GNN's predictions on a dataset. The aim of this study is to validate GNN-SubNet by comparing it with an alternative approach of obtaining global explanations. Instead of obtaining the node mask via a sampling scheme, multiple (local) explanations are optimized per dataset sample, then the node masks are aggregated by either the mean (Mean Aggregation) or the median value (Median Aggregation) per node.
GNN-SubNet is compared with its two modifications firstly by analyzing which disease subnetworks each algorithm detects, and secondly by leveraging metrics devised to assess explainers for GNNs. The results show that all algorithms detect subnetworks associated with cancer. In terms of the metric scores, Mean Aggregation obtains explanations with the highest fidelity, however no algorithm obtains sparse explanations. The study also indicates that GNN-SubNet obtains variate outcomes over multiple runs, and as such the results may not be reproducible.

AI explainers are tools capable of approximating how a neural network arrived at a given predic- tion by providing parts of the input data most rel- evant for the model’s choice. These tools have become a major point of research due to a need for human-verifiable predictions in multiple differ- ent fields, such as biomedical engineering. Graph Neural Networks (GNNs) are often used for such tasks, which led to the development of GNNSub- net, a tool capable of finding disease subnetworks on models trained with protein-protein interaction (PPI) data. This tool has been tested with only a single GNN architecture, which left a knowledge gap about the performance of the tool under dif- ferent models, which can differ significantly in the way they operate. Here the question ”How does the explainer perfor- mance vary with change in architectures of training models?” is answered. This paper explores this knowledge gap by train- ing and evaluating two other models (GCN and GraphSAGE) to see if the explanation performance of GNNSubnet changes. The performance is eval- uated with BAGEL metrics, a tool developed for XAI analysis. These metrics allow for comparison of explanations on multiple benchmarks. Three of these - Fidelity, Validity- and Validity+ - measure how accurate an explanation was in terms of iden- tifiying important nodes. The last one - Sparsity - assesses the nontriviality of an explanation by mea- suring how few nodes have been identified as im- portant. The experimental process shows low performance changes with different GNN architectures for accuracy-related metrics - RDT-Fidelity, Validity- and Validity+, which means that GNNSubnet is highly generalizable and not tied to a specific GNN model. However, the Sparsity score differs across models, with GIN being able to provide the most concise - and therefore useful - explanations.

Graph neural networks (GNNs), while effective at various tasks on complex graph-structured data, lack interpretability. Post-hoc explainability techniques developed for these GNNs in order to overcome their inherent uninterpretability have been applied to the additional task of detecting important subnetworks in graphs. For example, the GNN-SubNet program uses explanations of protein-protein interaction networks to detect the most important disease subnetworks for specific types of cancer. However, when using a post-hoc explanation for such additional tasks, evaluating the quality of the explanation becomes critical.
This study implements four explainability evaluation metrics to provide a fast and accurate way of evaluating explainability, using the GNN-SubNet program as a case study of explainable GNNs for subnetwork detection. Fidelity and sparsity metrics are implemented as defined in existing literature, while validity+ and validity- are newly defined. The results show that GNN-SubNet finds robust and faithful but highly dense explanations.

Proteins are fundamental biological macromolecules essential for cellular structure, enzymatic catalysis, and immune defense, making the generation of novel proteins crucial for advancements in medicine, biotechnology, and material sciences. This study explores protein design using deep generative models, specifically Denoising Diffusion Probabilistic Models (DDPMs). While traditional methods often focus on either protein structure or sequence design independently, recent trends emphasize a co-design approach addressing both aspects simultaneously. We propose a novel methodology utilizing Equivariant Graph Neural Networks (EGNNs) within the diffusion framework to co-design protein structures and sequences. We modify the EGNN architecture to improve its effectiveness in learning intricate data patterns. Experimental results show that our approach effectively generates high-quality protein sequences, although challenges remain in producing plausible protein backbones and ensuring strong sequence-structure correlation.

This study evaluates how the explainer for a Graph Neural Network creates explanations for chemical property prediction tasks. Explanations are masks over input molecules that indicate the importance of atoms and bonds toward the model output. Although these explainers have been evaluated for accuracy, no information exists on how faithful they are to the model (faithfulness), or how closely they correspond to human rationale (plausibility). Using explainability metrics to measure this, the per formance of the explainer is evaluated on different subsets based on the presence of benzene rings and halogens respectively, and on molecular weight. This study reveals that benzene rings influence the plausibility performance of the explainer, showing that performance is better at higher thresholds but worse at lower thresholds. Molecular weight and the presence of halogens do have no impact on plausibility. The ratio of positive samples in a set is shown to influence the metrics used for faithfulness. To ac
curately evaluate the faithfulness of different subsets, they should be changed to have equal positive rates or different metrics should be used. This research can be used as a starting point to research the influence of dataset properties on explainer performance. This is useful to create better explainers, leading to better acceptance of these models.

Advancing protein design is crucial for breakthroughs in medicine and biotechnology, yet traditional approaches often fall short by focusing solely on representing protein sequences using the 20 canonical amino acids. This thesis explores discrete diffusion models for generating novel protein sequences with an all-atom representation, specifically SELFIES a widely used molecular string representation. This all-atom approach considers the atomic composition of each amino acid in the protein. Enabling the inclusion of non-canonical amino acids and post-translational modifications. Using a modified ByteNet architecture and the D3PM framework, we compare the effects of this all-atom representation to the standard amino acid representation on the generated proteins' quality, diversity and novelty. Additionally, we see how a uniform or absorbing noise process affects the results. While models trained on the all-atom representation struggle to generate fully valid proteins consistently, those successfully designed showed improved novelty and diversity. Moreover, the all-atom representation can achieve comparable structural reliability results from OmegaFold to the amino acid models. Lastly, our results show that the use of an absorbing noise schedule is the most effective for both the all-atom and amino acid representation.

Accurately predicting enzyme-substrate interactions is critical for applications in drug discovery, biocatalysis and protein engineering. Building upon the ProSmith algorithm, a machine learning framework with a multimodal transformer for protein-small molecule interaction prediction, this study introduces protein 3D structural data as an additional modality. To integrate this data, we explore additive and multiplicative modality fusion strategies without requiring retraining the original transformer from scratch. Our experiments demonstrate that while the incorporation of structural data does not offer improved performance in random splits, it has the potential to surpass ProSmith in challenging data splits involving unseen small molecules. Notably, the model shows better generalization for underrepresented substrates.

Large chemical reaction databases often suffer from incompleteness, such as missing molecules or stoichiometric information. Concurrently, numerous computational models are being developed in predictive chemistry that rely on reaction databases and would hugely benefit from complete reaction equations. Also, research in sustainable chemistry often focuses on automated mass balance tasks, which require a full reaction to properly evaluate. In this work, we present a hybrid approach for computational completion of reaction equations. Specifically, we combine a rule-based method and a machine learning (ML) model to complete reactions. The rule-based approach constructs a balance of atoms and charge on either side of the reaction in an attempt to find missing molecules. We tailor the pre-trained transformer model on the chemical language domain to take partial reactions as inputs and predict missing molecules. Furthermore, we present a novel approach to measure the correctness of our model, which is useful when we apply it to the uncurated dataset and the ground-truth is unknown.

Dataset discovery techniques originally required datasets to have the same domain which made them unsuitable to be used on a larger scale. To avoid this requirement, newer techniques use additional information, aside from the datasets being processed, to better understand the data. They might rely on a knowledge base that describes the meaning of data, or a lexical database that defines meaningful relations between the words contained within the data. The main problem with this approach is that these types of additional information have poor coverage of the data being analyzed.

I propose to use a type of information that I call dataset domain terms. These are terms, or data values, that represent the domain of a dataset. I provide a technique that can derive these dataset domain terms automatically from existing datasets which means they are easily available. The problem of poor coverage can also be mitigated by only discovering datasets for the domain represented by these dataset domain terms. I provide a dataset discovery technique that takes this approach with these dataset domain terms.

Through an evaluation, I show that these dataset domain terms are sufficiently representative of the domain to be used for dataset discovery. The accuracy of the dataset discovery technique is also shown to be comparable to state-of-the-art dataset discovery techniques, though its precision is lacking.

This makes it highly suitable to filter datasets before other dataset discovery techniques can be performed on them. The data from these filtered datasets should also have a limited range of domains. So subsequent dataset discovery techniques should be less affected by the poor coverage of the additional information they use to understand the data. This allows dataset discovery to be performed on a larger scale.