"Toward a Theory of Tokenization in LLMs."

Nived Rajaraman, Jiantao Jiao, Kannan Ramchandran (2024)

Details and statistics

DOI: 10.48550/ARXIV.2404.08335

access: open

type: Informal or Other Publication

metadata version: 2024-05-16

a service of  Schloss Dagstuhl - Leibniz Center for Informatics