Skip to contents

Tokenize text

Source: R/text.R
tokenize_text.Rd

This function generates a character vector for a given text string

Usage

tokenize_text(text, sep = "\\s+")

Arguments

text

A character vector or object that can be coerced to a character string. Represents the input text to be cleaned.

sep

= "\s+"

Value

A character vector