Software Tools

Do one thing well, and work together.

Update Go scanner to accept non-ASCII operators

with one comment

Read the discussion of this change on the golang-nuts mailing list. See this notice in UTF-8 format.

For each line, the scanner accepts, in place of the first operator, any of the remaining operators, and outputs a token whose string (set in $GOROOT/src/go/token/token.go) is the first operator.

! ¬

!=

&

&&

&= ∧=

&^ ∧¬

&^= ∧¬=

:=

<-

<<

<<= ≪=

<=

== =?

>=

>>

>>= ≫=

^

^= ⊻=

|

|= ∨=

|| ⋁

To install, copy $GOROOT/src/pkg/go/scanner/scanner.go to another file.

Replace scanner.go with scanner.go

Run $GOROOT/src/all.bash and check for 0 unexpected errors.

Changes to scanner.go update gofmt, which accepts UTF-8 operators and outputs their ASCII equivalents.  This mkfile production rule uses gofmt as a preprocessor to create a sharable and compilable file.

%.go: %.ℊℴ

cat $stem.ℊℴ | gofmt > $stem.go

See these files for an example of each new operator form in the context of a simple Go program.

myscanner.ℊℴ

chan.ℊℴ

Please mail me privately if something doesn’t work with this code, to avoid noise on the golang-nuts list, since we’re no longer discussing officially-released code.

Written by catena

10 February 2010 at 0116

Posted in Uncategorized

One Response

Subscribe to comments with RSS.

  1. From only my point of view: I intended my scanner change to enable gofmt to preprocess out the non-ASCII symbols before the compiler, or another human, sees the code (and to encourage gofmt use). So someone who is comfortable with the non-ASCII characters, and wants to use them, can, and can also look at a version with only ASCII characters, and send this version to other people and the compiler without changing how they expect to see Go source text. The scanner as I changed it accepts both the ASCII and non-ASCII operators, and translates them both to the same ASCII output representation (so I don’t have to also change gc). For many reasons, I (do not speak for the Go authors) do not feel it’s the right time to force anyone to view Go source text with non-ASCII operators, but it is a good time to let people use them if they have an environment in which they’re comfortable with them (which I am).

    catena

    25 February 2010 at 1423


Leave a comment