Review DNA BERT

[Shuyib]

BERT models are encoders which are good for natural language understanding. This is based on the documentation available on HuggingFace NLP course. We need to assess this. You'll need to find out how to move from raw text -> tokenize text -> model -> logits -> Prediction to find motifs which is a capability that has been indicated in their README.

MEME is still a black box and very computationally intensive. But since the source code of BERT is available we can assess it.

Link to DNA BERT

[Shuyib]

🤔 reviewing the tokenizer. Should have start,stop codons and promoter regions should be marked. My assumption.

[Shuyib]

Looking into this now? 🏃🏿