OpenBSD ports

The textproc/py-tokenizers port

py3-tokenizers-0.23.1 – tokenizers with a focus on performance

Description

An implementation of today's most used tokenizers, with a focus on
performance.

Only for arches

aarch64 amd64 riscv64 sparc64

Categories:

lang/python lang/rust textproc

Library dependencies

Build dependencies

• devel/cargo-generate-vendor
• devel/maturin
• devel/py-build
• devel/py-installer
• lang/python/3
• lang/rust
• textproc/oniguruma

Run dependencies

• lang/python/3