Metadata-Version: 2.4
Name: pyTokenizer
Version: 1.1.3
Summary: A streaming tokenizer.
Home-page: https://github.com/Paebbels/pyTokenizer
Author: Patrick Lehmann
Author-email: Paebbels@gmail.com
Project-URL: Documentation, https://pyTokenizer.readthedocs.io/en/latest/
Project-URL: Source Code, https://github.com/Paebbels/pyTokenizer
Project-URL: Issue Tracker, https://github.com/Paebbels/pyTokenizer/issues
Keywords: Python3 Token Tokenizer Streaming
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.5
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: Topic :: Utilities
Requires-Python: >=3.5
Description-Content-Type: text/markdown
Dynamic: author
Dynamic: author-email
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: home-page
Dynamic: keywords
Dynamic: project-url
Dynamic: requires-python
Dynamic: summary

[![PyPI - License](https://img.shields.io/pypi/l/pyTokenizer?logo=PyPI)](LICENSE.md)
[![GitHub tag (latest SemVer incl. pre-release)](https://img.shields.io/github/v/tag/Paebbels/pyTokenizer?logo=GitHub&include_prereleases)](https://github.com/Paebbels/pyTokenizer/tags)
[![GitHub release (latest SemVer incl. including pre-releases)](https://img.shields.io/github/v/release/Paebbels/pyTokenizer?logo=GitHub&include_prereleases)](https://github.com/Paebbels/pyTokenizer/releases/latest)
[![GitHub release date](https://img.shields.io/github/release-date/Paebbels/pyTokenizer?logo=GitHub&)](https://github.com/Paebbels/pyTokenizer/releases)
[![Libraries.io status for latest release](https://img.shields.io/librariesio/release/pypi/pyTokenizer)](https://libraries.io/github/Paebbels/pyTokenizer)
[![Requires.io](https://img.shields.io/requires/github/Paebbels/pyTokenizer)](https://requires.io/github/Paebbels/pyTokenizer/requirements/?branch=master)  
[![Travis](https://img.shields.io/travis/com/Paebbels/pyTokenizer?logo=Travis)](https://travis-ci.com/Paebbels/pyTokenizer)
[![PyPI](https://img.shields.io/pypi/v/pyTokenizer?logo=PyPI)](https://pypi.org/project/pyTokenizer/)
![PyPI - Status](https://img.shields.io/pypi/status/pyTokenizer?logo=PyPI)
![PyPI - Python Version](https://img.shields.io/pypi/pyversions/pyTokenizer?logo=PyPI)
[![Dependent repos (via libraries.io)](https://img.shields.io/librariesio/dependent-repos/pypi/pyTokenizer)](https://github.com/Paebbels/pyTokenizer/network/dependents)  
[![Codacy Badge](https://api.codacy.com/project/badge/Grade/7ea6c9eea5b94493837ea55c07de3562)](https://www.codacy.com/manual/Paebbels/pyTokenizer)
[![Libraries.io SourceRank](https://img.shields.io/librariesio/sourcerank/pypi/pyTokenizer)](https://libraries.io/github/Paebbels/pyTokenizer/sourcerank)
[![Read the Docs](https://img.shields.io/readthedocs/pytokenizer)](https://pyTokenizer.readthedocs.io/en/latest/)

# pyTokenizer

A streaming tokenizer.


## Contributors

* [Patrick Lehmann](https://github.com/Paebbels) (Maintainer)


## License

This library is licensed under [Apache License 2.0](LICENSE.md)

-------------------------

SPDX-License-Identifier: Apache-2.0
