{"name":"text","intro":"Text operations a tokeniser cannot do. Counts are by Unicode codepoint, so the “how many r in strawberry” class of question is answered by counting rather than by guessing, and an emoji that renders as one glyph is reported as the several codepoints and many bytes it really is. Regex, diffs, natural sorting, dedupe and entity extraction round it out.","when_to_use":["Counting characters, words, lines, sentences or occurrences of a substring.","Running or testing a regex — including checking that a pattern compiles at all.","Producing an exact diff between two versions of a string.","Sorting strings in natural order (`file2` before `file10`) or removing near-duplicates.","Pulling emails, URLs, money, dates, PAN/GSTIN and other entities out of free text."],"network":false,"related":"[`collections`](/docs/tools/collections) for the same operations over records rather than strings · [`validate`](/docs/tools/validate) to check the identifiers `extract` finds · [`encode`](/docs/tools/encode) for hashing and encoding the text itself.","docs_url":"/docs/tools/text","modes":[{"name":"count","purpose":"Count characters, words, lines — or one substring.","description":"With `what: all` (the default) returns every count at once: characters, characters without spaces, letters, digits, words, unique words, lines, non-empty lines, sentences, paragraphs, UTF-8 bytes, a rough token estimate, and `graphemes` — what a reader actually sees, so a ZWJ family emoji is one character rather than five. Text containing bidi controls or zero-width characters is flagged in `warnings`: they are invisible, they count towards length, and they are how a filename is made to read backwards. Ask for one statistic by name, or use `what: occurrences` with a `substring` to count and locate a specific string, optionally overlapping.","parameters":[{"name":"text","type":"string","required":true,"doc":"The text to measure.","default":null},{"name":"what","type":"string \\| string[]","required":false,"doc":"`all`, `occurrences`, or the name of one statistic.","default":"`all`"},{"name":"substring","type":"string","required":false,"doc":"The needle, for `what: occurrences`.","default":null},{"name":"case_sensitive","type":"boolean","required":false,"doc":"Match case when counting occurrences.","default":"`true`"},{"name":"overlapping","type":"boolean","required":false,"doc":"Count overlapping matches.","default":"`false`"}],"examples":[{"mode":"count","text":"👩‍👩‍👧 family"},{"mode":"count","text":"strawberry","what":"occurrences","substring":"r"},{"mode":"count","text":"Café 👨‍👩‍👧‍👦"},{"mode":"count","text":"aaaa","what":"occurrences","substring":"aa","overlapping":true},{"mode":"count","text":"2025-08-26 09:00 INFO started\n2025-08-26 09:05 WARN retrying\n2025-08-26 09:06 INFO ready","what":"lines"},{"mode":"count","text":"abc","what":"vowels"},{"mode":"count","text":"abc","what":"occurrences"}]},{"name":"regex_match","purpose":"Find every match, with positions and groups.","description":"Runs a regular expression over the text and returns each match with its span, positional groups and named groups, plus whether the pattern matched the whole string. Flags are given as letters: `i`, `m`, `s`, `x`, `u`, `a`. `count` is the total number of matches in the text, not the number listed: when there are more than `limit`, `returned` says how many came back, `truncated` is `true` and `warnings` says so. A pattern that can backtrack exponentially — `(a+)+$`, `(a|aa)+` — is refused with `unsupported` before it runs, because once stdlib `re` starts on one nothing can stop it.","parameters":[{"name":"text","type":"string","required":true,"doc":"The text to search.","default":null},{"name":"pattern","type":"string","required":true,"doc":"A Python regular expression.","default":null},{"name":"flags","type":"string","required":false,"doc":"Any of `imsxua`.","default":null},{"name":"limit","type":"integer","required":false,"doc":"Maximum matches returned.","default":"1000"}],"examples":[{"mode":"regex_match","text":"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaab","pattern":"(a+)+$"},{"mode":"regex_match","text":"Order 1234 shipped 2025-08-26 to PIN 560001","pattern":"\\d{4}"},{"mode":"regex_match","text":"2025-08-26","pattern":"(?P<year>\\d{4})-(?P<month>\\d{2})-(?P<day>\\d{2})"},{"mode":"regex_match","text":"Error: ERROR while erroring","pattern":"error","flags":"i"},{"mode":"regex_match","text":"abc","pattern":"([a-z"},{"mode":"regex_match","text":"abc","pattern":"a","flags":"z"}]},{"name":"regex_replace","purpose":"Substitute matches, counting what changed.","description":"Replaces matches and reports how many substitutions were made and whether anything changed at all — the part a blind `sub()` never tells you. Backreferences (`\\1`) and named references work in the replacement; `count` limits how many are replaced. A replacement that would produce more than 200 000 characters is refused with `too_large` rather than returned, and a pattern that can backtrack exponentially is refused with `unsupported`.","parameters":[{"name":"text","type":"string","required":true,"doc":"The text to transform.","default":null},{"name":"pattern","type":"string","required":true,"doc":"A Python regular expression.","default":null},{"name":"replacement","type":"string","required":true,"doc":"Replacement, with `\\1`-style backreferences.","default":null},{"name":"flags","type":"string","required":false,"doc":"Any of `imsxua`.","default":null},{"name":"count","type":"integer","required":false,"doc":"Replace at most this many; 0 means all.","default":"0"}],"examples":[{"mode":"regex_replace","text":"call 98765 43210 now","pattern":"\\d","replacement":"#"},{"mode":"regex_replace","text":"2025-08-26","pattern":"(\\d{4})-(\\d{2})-(\\d{2})","replacement":"\\3/\\2/\\1"},{"mode":"regex_replace","text":"a a a a","pattern":"a","replacement":"b","count":2},{"mode":"regex_replace","text":"abc","pattern":"a"},{"mode":"regex_replace","text":"abc","pattern":"a","replacement":"\\9"},{"mode":"regex_replace","text":"abc","pattern":"a(","replacement":"x"}]},{"name":"diff","purpose":"An exact diff between two texts.","description":"Compares two strings by line, word or character and returns a similarity ratio, the number of units added and removed, an operation list with both sides and their ranges, and — for line granularity — a unified diff. Use it instead of asking a model whether two documents differ. Comparison is quadratic, so each side is capped at 10 000 lines, words or characters.","parameters":[{"name":"a","type":"string","required":true,"doc":"The original text.","default":null},{"name":"b","type":"string","required":true,"doc":"The changed text.","default":null},{"name":"granularity","type":"string","required":false,"doc":"Unit of comparison.","default":"`line`"}],"examples":[{"mode":"diff","a":"2025-08-26 09:00 INFO started\n2025-08-26 09:05 WARN retrying\n2025-08-26 09:06 INFO ready","b":"2025-08-26 09:00 INFO started\n2025-08-26 09:05 ERROR timed out\n2025-08-26 09:06 INFO ready\n2025-08-26 09:07 INFO done"},{"mode":"diff","a":"the quick brown fox","b":"the quiet brown dog","granularity":"word"},{"mode":"diff","a":"x","b":"y","granularity":"sentence"}]},{"name":"sort","purpose":"Sort strings naturally and case-insensitively.","description":"Sorts a list with natural ordering by default, so `file2` comes before `file10`, and case-insensitively, so `Apple` and `apple` sit together. Mixed types are ordered deterministically — numbers, then strings, then nulls — and `changed` tells you whether the input was already sorted. Ordering is by code point with no locale collation, so `éclair` sorts after `Zebra`; that is stated in `assumptions`.","parameters":[{"name":"items","type":"array","required":true,"doc":"The list to sort.","default":null},{"name":"key","type":"string","required":false,"doc":"Field to sort on, when the items are objects.","default":null},{"name":"order","type":"string","required":false,"doc":"Sort direction.","default":"`asc`"},{"name":"natural","type":"boolean","required":false,"doc":"Digit runs compare numerically.","default":"`true`"},{"name":"case_insensitive","type":"boolean","required":false,"doc":"Fold case before comparing.","default":"`true`"}],"examples":[{"mode":"sort","items":["file10.txt","file2.txt","File1.txt","file20.txt"]},{"mode":"sort","items":[{"n":"b","v":2},{"n":"a","v":10},{"n":"c","v":7}],"key":"v","order":"desc"},{"mode":"sort","items":["file10.txt","file2.txt"],"natural":false},{"mode":"sort","items":"a,b,c"}]},{"name":"dedupe","purpose":"Remove duplicates and report which ones went.","description":"Removes duplicates while preserving order, and returns what it removed and where the first occurrence was — so a dedupe can be reviewed rather than trusted. Whitespace is normalised by default (`\" a  b \"` equals `\"a b\"`); `case_insensitive` folds case; `key` dedupes objects on one field.","parameters":[{"name":"items","type":"array","required":true,"doc":"The list to dedupe.","default":null},{"name":"key","type":"string","required":false,"doc":"Field to compare, when the items are objects.","default":null},{"name":"case_insensitive","type":"boolean","required":false,"doc":"Fold case before comparing.","default":"`false`"},{"name":"normalize_whitespace","type":"boolean","required":false,"doc":"Collapse runs of whitespace before comparing.","default":"`true`"}],"examples":[{"mode":"dedupe","items":["Apple","  apple ","APPLE","banana","banana"],"case_insensitive":true},{"mode":"dedupe","items":[{"id":1,"n":"a"},{"id":2,"n":"b"},{"id":1,"n":"c"}],"key":"id"},{"mode":"dedupe","items":{"a":1}}]},{"name":"extract","purpose":"Pull entities out of free text.","description":"Runs a library of regexes over the text and returns what it found, deduplicated and in order. Kinds: `emails`, `urls`, `phones`, `numbers`, `dates`, `times`, `hashtags`, `mentions`, `ips`, `money`, `pan`, `gstin`, `uuids`. Extraction is regex-based and says so — check anything it finds with [`validate`](/docs/tools/validate) before trusting it.","parameters":[{"name":"text","type":"string","required":true,"doc":"The text to scan.","default":null},{"name":"what","type":"string \\| string[]","required":false,"doc":"One kind, a list of kinds, or `all`.","default":"`all`"},{"name":"unique","type":"boolean","required":false,"doc":"Collapse repeated hits.","default":"`true`"}],"examples":[{"mode":"extract","text":"Ping ops@example.com or billing@mailinator.com, docs at https://leftbrain.dev/docs, invoice ₹1,25,000 due 2025-09-15, GST 19ABCDE1234F1ZX, call +91 98765 43210. #urgent @sayantan","what":["emails","urls","money"]},{"mode":"extract","text":"Ping ops@example.com or billing@mailinator.com, docs at https://leftbrain.dev/docs, invoice ₹1,25,000 due 2025-09-15, GST 19ABCDE1234F1ZX, call +91 98765 43210. #urgent @sayantan"},{"mode":"extract","text":"abc","what":"vehicles"}]},{"name":"find","purpose":"Locate a substring with line numbers and context.","description":"Finds every occurrence of a substring and returns its offset, line number and surrounding context — the grep-shaped answer, rather than a yes/no. Matching is case-insensitive by default; results stop at 200 hits.","parameters":[{"name":"text","type":"string","required":true,"doc":"The text to search.","default":null},{"name":"substring","type":"string","required":true,"doc":"The string to find.","default":null},{"name":"case_sensitive","type":"boolean","required":false,"doc":"Match case.","default":"`false`"},{"name":"context","type":"integer","required":false,"doc":"Characters of context on each side.","default":"40"}],"examples":[{"mode":"find","text":"2025-08-26 09:00 INFO started\n2025-08-26 09:05 ERROR timed out\n2025-08-26 09:06 INFO ready\n2025-08-26 09:07 INFO done","substring":"info","context":12},{"mode":"find","text":"2025-08-26 09:00 INFO started\n2025-08-26 09:05 ERROR timed out\n2025-08-26 09:06 INFO ready\n2025-08-26 09:07 INFO done","substring":"info","case_sensitive":true},{"mode":"find","substring":"abc"}]},{"name":"similarity","purpose":"Edit distance, and the best match from a list.","description":"Levenshtein distance by codepoint and a 0–1 `ratio` (1 minus distance over the longer length). Give `a` and `b` for one pair, or `text` and `items` to rank a list of candidates and return the `best` one with its index — for mapping what a user typed onto a menu, or spotting near-duplicate names. Case is folded and whitespace normalised by default; both are stated in `assumptions`. Strings are capped at 5,000 characters.","parameters":[{"name":"a","type":"string","required":false,"doc":"One string of a pair.","default":null},{"name":"b","type":"string","required":false,"doc":"The other.","default":null},{"name":"text","type":"string","required":false,"doc":"The input to match against `items`.","default":null},{"name":"items","type":"array","required":false,"doc":"Candidate strings to rank.","default":null},{"name":"case_insensitive","type":"boolean","required":false,"doc":"Fold case before comparing.","default":"`true`"},{"name":"normalize_whitespace","type":"boolean","required":false,"doc":"Collapse runs of whitespace first.","default":"`true`"},{"name":"limit","type":"integer","required":false,"doc":"How many ranked candidates to return.","default":"5"}],"examples":[{"mode":"similarity","a":"kitten","b":"sitting"},{"mode":"similarity","text":"bengaluru","items":["Mumbai","Bangalore","Bengaluru","Bengal","Chennai"],"limit":3},{"mode":"similarity","a":"Delhi","b":"delhi","case_insensitive":false},{"mode":"similarity","a":"kitten"}]}]}