SplitByAnyOf Class Reference

The SplitByAnyOf class tokenizes a string using a set of delimiters. More...

#include <split_by_any_of.hpp>

Public Types
using	MaskType = std::array< bool, 1<< CHAR_BIT >
	A convenient alias for the mask type. More...
using	TokenType = boost::string_view
	The type of the token which the tokenizer extracts. More...

Public Member Functions
	SplitByAnyOf (const boost::string_view delimiters)
	Construct the object from the given delimiters. More...
MaskType &	Mask ()
	Modify the mask. More...
const MaskType &	Mask () const
	Return the mask. More...
boost::string_view	operator() (boost::string_view &str) const
	The function extracts the first token from the given string view and then removes the prefix containing the token from the view. More...

Static Public Member Functions
static bool	IsTokenEmpty (const boost::string_view token)
	The function returns true if the given token is empty. More...

Detailed Description

The SplitByAnyOf class tokenizes a string using a set of delimiters.

Definition at line 26 of file split_by_any_of.hpp.

Member Typedef Documentation

MaskType

using MaskType = std::array<bool, 1 << CHAR_BIT>

A convenient alias for the mask type.

Definition at line 33 of file split_by_any_of.hpp.

TokenType

using TokenType = boost::string_view

The type of the token which the tokenizer extracts.

Definition at line 30 of file split_by_any_of.hpp.

Constructor & Destructor Documentation

SplitByAnyOf()

SplitByAnyOf ( const boost::string_view  delimiters )

inline

Construct the object from the given delimiters.

Parameters

delimiters

The given delimiters.

Definition at line 40 of file split_by_any_of.hpp.

Member Function Documentation

IsTokenEmpty()

static bool IsTokenEmpty ( const boost::string_view  token )

inlinestatic

The function returns true if the given token is empty.

Parameters

token

The given token.

Definition at line 78 of file split_by_any_of.hpp.

Mask() [1/2]

MaskType & Mask ( )

inline

Modify the mask.

Definition at line 86 of file split_by_any_of.hpp.

Mask() [2/2]

const MaskType & Mask ( ) const

inline

Return the mask.

Definition at line 84 of file split_by_any_of.hpp.

operator()()

boost::string_view operator() ( boost::string_view &  str ) const

inline

The function extracts the first token from the given string view and then removes the prefix containing the token from the view.

Parameters

str	String view to retrieve the token from.

Definition at line 54 of file split_by_any_of.hpp.

---

The documentation for this class was generated from the following file:

• src/mlpack/core/data/tokenizers/split_by_any_of.hpp