`go/token`

Guided tour · Go Tooling · pkg.go.dev →

Tokens (IDENT, INT, +, func, ...) and position info (FileSet, Pos). Thread a FileSet through every parser/format call.

FileSet and positions

Position of a node

fset := token.NewFileSet()
file, _ := parser.ParseFile(fset, "x.go", nil, 0)
ast.Inspect(file, func(n ast.Node) bool {
    if id, ok := n.(*ast.Ident); ok {
        fmt.Println(id.Name, fset.Position(id.Pos()))
    }
    return true
})

Token constants

Categories

token.IDENT token.INT token.STRING   // literals
token.ADD   token.SUB                  // operators
token.FUNC  token.VAR  token.RETURN    // keywords
tok.IsLiteral() / tok.IsOperator() / tok.IsKeyword()