Skip to content

vllm.renderers.inputs.tokenize

Schemas and utilites for tokenization inputs.

DecoderOnlyTokPrompt module-attribute 

DecoderOnlyTokPrompt: TypeAlias = (
    TokensPrompt | EmbedsPrompt
)

A DecoderOnlyDictPrompt that has been tokenized.

DecoderTokPrompt module-attribute 

DecoderTokPrompt: TypeAlias = TokensPrompt

A DecoderDictPrompt that has been tokenized.

EncoderTokPrompt module-attribute 

EncoderTokPrompt: TypeAlias = TokensPrompt

A EncoderDictPrompt that has been tokenized.

SingletonTokPrompt module-attribute 

SingletonTokPrompt: TypeAlias = (
    DecoderOnlyTokPrompt
    | EncoderTokPrompt
    | DecoderTokPrompt
)

A SingletonDictPrompt that has been tokenized.

TokPrompt module-attribute 

TokPrompt: TypeAlias = (
    DecoderOnlyTokPrompt | EncoderDecoderTokPrompt
)

A DictPrompt that has been tokenized.

EncoderDecoderTokPrompt

Bases: TypedDict

A EncoderDecoderDictPrompt that has been tokenized.

Source code in vllm/renderers/inputs/tokenize.py 
32
33
34
35
36
37
38
39
40
41
class EncoderDecoderTokPrompt(TypedDict):
    """
    A
    [`EncoderDecoderDictPrompt`][vllm.renderers.inputs.preprocess.EncoderDecoderDictPrompt]
    that has been tokenized.
    """

    encoder_prompt: EncoderTokPrompt

    decoder_prompt: DecoderTokPrompt | None