What are tokens in the context of generative AI models?

What are tokens in the context of generative AI models?
A . Tokens are the basic units of input and output that a generative AI model operates on, representing words, subwords, or other linguistic units.
B . Tokens are the mathematical representations of words or concepts used in generative AI models.
C . Tokens are the pre-trained weights of a generative AI model that are fine-tuned for specific tasks.
D . Tokens are the specific prompts or instructions given to a generative AI model to generate output.

View Answer

Answer: A