Domain Specific Language Model Pretraining For Biomedical

Additionally the study of model size on domain. We show that for high volume high value domains such as biomedicine such a strategy outperforms all prior language models and obtains state of the art results across the board in biomedical nlp applications. A prevailing assumption is that even domain specific pretraining can benefit by starting from general domain language models.