ConvBert-PyTorch 🤗An unofficial PyTorch implementation of ConvBert based on huggingface/transformers. 🤗基于huggingface/transformers的ConvBert的非官方PyTorch实现。 requirements transformers==3.5.1 torch>=1.5.0 wandb pretrain python run_language_modeling.py Reference ConvBERT: Improving BERT with Span-based Dynamic Convolution yitu-opensource/ConvBert huggingface/transformers JunnYu/ConvBert_huggingface sjpark9503/ELECTRA-pretraining ThilinaRajapakse/simpletransformers