OpenBSD ports

The textproc/py-tokenizers port

py3-tokenizers-0.23.1 – tokenizers with a focus on performance

Description

An implementation of today's most used tokenizers, with a focus on
performance.

Only for arches

aarch64 aarch64 alpha amd64 amd64 arm arm hppa i386 i386 mips64 mips64 mips64el mips64el powerpc powerpc powerpc64 powerpc64 riscv64 riscv64 sparc64

Categories:

lang/python lang/rust textproc

Library dependencies

Build dependencies

• STEM->=1.1:devel/cargo-generate-vendor
• devel/maturin
• devel/py-build
• devel/py-installer
• lang/python/3
• lang/rust
• textproc/oniguruma

Run dependencies

• lang/python/3