Peptone AI team presents our research during NeurIPS 2023
Peptone's researchers are investigating novel methods of structural disorder modelling at our UK headquarters in London, UK.
Our AI team, led by Dr Istvan Redl, has extensively used protein language models (pLMs) in in-silico protein engineering tasks and observed impressive results. We introduced a set of fine-tuning techniques based on Sentence Transformers (STs) that were integrated with a novel data augmentation procedure. Our team demonstrated how this can lead to new state-of-the-art performance.
Although STs were initially developed for classic NLP space, we found them to hold natural appeal in pLM-related applications, mainly due to their use of sequence pairs and triplets in the process.
We showcased this conceptual approach in two different settings that frequently occur in this domain: a residue and sequence-level prediction task.
Apart from demonstrating how these tools can extract more and higher quality information from pLMs, our team will also discuss the main differences between their applications in NLP and protein spaces.
