75 words, word rule
- Your prompt or text
- word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word
- Estimate by
- Words ÷ 0.75
- Context window
- 128000
Result Estimated tokens: 100
Characters: 374
Words: 75
Tokens by character rule: 94
Tokens by word rule: 100
Share of context window: 0.08%
Tokens left in window: 127,900
- Character rule: 0 CJK characters + 374 other characters ÷ 4 = 93.5 tokens.
- Word rule: 75 words ÷ 0.75 = 100 tokens.
- Estimate used (words): 100 tokens, 0.08% of a 128000-token window.
OpenAI's guide puts 75 English words at about 100 tokens, and the word rule reproduces that exactly.