# What is Tokenizer?

**Category:** ai

## Overview
It is a tool that divides texts into small numerical pieces that artificial intelligence can understand.

---
Source: TreScout Tech Dictionary · https://trescout.com/en/dictionary/tokenizer/