Option 2: Custom Token Limit

If you require more control over how your document is divided, select Custom Token Limit.

This option allows you to specify the maximum number of tokens in each chunk before the document is indexed.

A token is a small unit of text used by AI models. Larger chunks provide more context, while smaller chunks enable more precise retrieval of specific information.

After selecting Custom Token Limit, enter the desired value in the Max Tokens field.