• GPU Instances
  • Cluster Engine
  • Application Platform
  • NVIDIA H200
  • NVIDIA GB200 NVL72
  • Products
    
    GPU InstancesCluster EngineInference EngineApplication Platform
  • GPUs
    
    H200NVIDIA GB200 NVL72NVIDIA HGX™ B200
  • Pricing
  • Company
    
    About usBlogDiscoursePartnersCareers
  • About Us
  • Blog
  • Discourse
  • Partners
  • Contact Us
  • Get started
English
English

English
日本語
한국어
繁體中文
Get startedContact Sales

Tokenization

Get startedfeatures

Related terms

Large Language Model (LLM)
BACK TO GLOSSARY

Tokenization is the process of breaking text into smaller pieces called tokens—such as words or subwords—that a language model can understand. For example, “ChatGPT” might become “Chat” and “GPT.” These tokens are then converted into numbers the model uses to process language. Tokenization affects how much text a model can handle at once, how fast it runs, and how accurate its output is. In short, it’s the first step in helping AI read and work with language.

Sign up for our newsletter

Empowering humanity's AI ambitions with instant GPU cloud access.

[email protected]

278 Castro St, Mountain View, CA 94041

  • GPU Cloud
  • Cluster Engine
  • Inference Engine
  • Pricing
  • Glossary
  • About Us
  • Blog
  • Partners
  • Careers
  • Contact Us

© 2025 All Rights Reserved.

Privacy Policy

Terms of Use