Token

In the context of prompting, a "token" refers to a unit of text that is used in natural language processing. In English, a token can be a word, a punctuation mark, or even a subword (like "unhappiness" being tokenized into "un" and "happiness"). When writing a prompt, each word or segment of text is typically considered a token. It helps structure the input for the language model, allowing it to process and generate responses based on these units of text.