tokenizers with a focus on performance
An implementation of today's most used tokenizers, with a focus on
performance.

Maintainer: The OpenBSD ports mailing-list <ports@openbsd.org>

WWW: https://pypi.org/project/tokenizers
