Subject
Computer Science
Topic
Artificial Intelligence
In large language models, a "token" is ____.
In large language models, a "token" is ____.
a)
a password for the APIb)
one complete documentc)
a small unit of text, such as a word or part of a wordd)
a single image pixelExplanation
Models read and generate text as sequences of tokens; limits and prices are counted in tokens.Related questions
A support bot must answer from a company’s current policy documents and cite them. Which design best reduces unsupported answers?A text generator should produce more varied outputs from the same prompt. Which decoding change usually helps?A generated medical answer sounds confident but has no supporting evidence. What is the safest response?In an autoregressive language model, how is a sequence generated?What is the main role of self-attention in a transformer?A pretrained model is further trained on labelled examples for one task. What process is this?Untrusted retrieved text tells a support bot to ignore its system rules. What security problem is this?Why should generated citations be checked before publication?