​SBERT-WK: A Sentence Embedding Method by Dissecting BERT-based Word Models

Sentence embedding is an important research topic in natural language processing (NLP) since it can transfer knowledge to downstream tasks. Meanwhile, a contextualized word representation, called BERT, achieves the state-of-the-art performance in quite a few NLP tasks.

Yet, it is an open problem to generate a high quality sentence representation from BERT-based word models. It was shown in previous study that different layers of BERT capture different linguistic properties. This allows us to fusion information across layers to find better sentence representation.

[GitHub]

https://github.com/BinWang28/SBERT-WK-Sentence-Embedding

[arXiv]
https://arxiv.org/abs/2002.06652

#ai #artificialintelligence #deeplearning #nlp #nlproc #machinelearning

🔗 BinWang28/SBERT-WK-Sentence-Embedding
Code for Paper: SBERT-WK: A Sentence Embedding Method By Dissecting BERT-based Word Models - BinWang28/SBERT-WK-Sentence-Embedding