[SEP] token

A special marker that separates two pieces of text fed in together.

[CLS] sentence A [SEP] sentence B [SEP]

The [SEP] token marks a boundary between segments when you give the encoder two chunks of text at once — for example a question and a passage, or two sentences being compared. It typically also closes the sequence. Paired with segment embeddings, it lets a single encoder reason about the RELATIONSHIP between two texts (does B answer A? does A imply B?) rather than just one text in isolation.

Appears in

Two sentences at once Masked Language Models · pt 4

Nearby in the brain

BERT [CLS] token dot product natural language inference next sentence prediction positional encoding segment embedding