ALPHA: breaking changes expected. What that means
NanoC

tokens builtin

tokens(text)

Splits NanoC or C text into tokens and returns the token list.

Phaseall

Example

nanoc
@tt
  log(str(str(tokens("a + 1"))))

Logs: a + 1

All builtins