A high-performance Go package for measuring the monospace display width of strings, UTF-8 bytes, and runes.
go get github.com/clipperhouse/displaywidthpackage main
import (
"fmt"
"github.com/clipperhouse/displaywidth"
)
func main() {
width := displaywidth.String("Hello, 世界!")
fmt.Println(width)
width = displaywidth.Bytes([]byte("🌍"))
fmt.Println(width)
width = displaywidth.Rune('🌍')
fmt.Println(width)
}For most purposes, you should use the String or Bytes methods. They sum
the widths of grapheme clusters in the string or byte slice.
Note: in your application, iterating over runes to measure width is likely incorrect; the smallest unit of display is a grapheme, not a rune.
If you need the individual graphemes:
import (
"fmt"
"github.com/clipperhouse/displaywidth"
)
func main() {
g := displaywidth.StringGraphemes("Hello, 世界!")
for g.Next() {
width := g.Width()
value := g.Value()
// do something with the width or value
}
}There is one option, displaywidth.Options.EastAsianWidth, which defines
how East Asian Ambiguous characters
are treated.
When false (default), East Asian Ambiguous characters are treated as width 1.
When true, they are treated as width 2.
You may wish to configure this based on environment variables or locale.
go-runewidth, for example, does so
during package initialization.
displaywidth does not do this automatically, we prefer to leave it to you.
You might do something like:
var width displaywidth.Options // zero value is default
func init() {
if os.Getenv("EAST_ASIAN_WIDTH") == "true" {
width = displaywidth.Options{EastAsianWidth: true}
}
// or check locale, or any other logic you want
}
// use it in your logic
func myApp() {
fmt.Println(width.String("Hello, 世界!"))
}This package implements the Unicode East Asian Width standard (UAX #11), and handles version selectors, and regional indicator pairs (flags). We implement Unicode TR51. We are keeping an eye on emerging standards.
clipperhouse/displaywidth, mattn/go-runewidth, and rivo/uniseg will
give the same outputs for most real-world text. Extensive details are in the
compatibility analysis.
If you wish to investigate the core logic, see the lookupProperties and width
functions in width.go. The essential trie generation logic is in
buildPropertyBitmap in unicode.go.
cd comparison
go test -bench=. -benchmemgoos: darwin
goarch: arm64
pkg: github.com/clipperhouse/displaywidth/comparison
cpu: Apple M2
BenchmarkString_Mixed/clipperhouse/displaywidth-8 10400 ns/op 162.21 MB/s 0 B/op 0 allocs/op
BenchmarkString_Mixed/mattn/go-runewidth-8 14296 ns/op 118.00 MB/s 0 B/op 0 allocs/op
BenchmarkString_Mixed/rivo/uniseg-8 19770 ns/op 85.33 MB/s 0 B/op 0 allocs/op
BenchmarkString_EastAsian/clipperhouse/displaywidth-8 10593 ns/op 159.26 MB/s 0 B/op 0 allocs/op
BenchmarkString_EastAsian/mattn/go-runewidth-8 23980 ns/op 70.35 MB/s 0 B/op 0 allocs/op
BenchmarkString_EastAsian/rivo/uniseg-8 19777 ns/op 85.30 MB/s 0 B/op 0 allocs/op
BenchmarkString_ASCII/clipperhouse/displaywidth-8 1032 ns/op 124.09 MB/s 0 B/op 0 allocs/op
BenchmarkString_ASCII/mattn/go-runewidth-8 1162 ns/op 110.16 MB/s 0 B/op 0 allocs/op
BenchmarkString_ASCII/rivo/uniseg-8 1586 ns/op 80.69 MB/s 0 B/op 0 allocs/op
BenchmarkString_Emoji/clipperhouse/displaywidth-8 3017 ns/op 240.01 MB/s 0 B/op 0 allocs/op
BenchmarkString_Emoji/mattn/go-runewidth-8 4745 ns/op 152.58 MB/s 0 B/op 0 allocs/op
BenchmarkString_Emoji/rivo/uniseg-8 6745 ns/op 107.34 MB/s 0 B/op 0 allocs/op
BenchmarkRune_Mixed/clipperhouse/displaywidth-8 3381 ns/op 498.90 MB/s 0 B/op 0 allocs/op
BenchmarkRune_Mixed/mattn/go-runewidth-8 5383 ns/op 313.41 MB/s 0 B/op 0 allocs/op
BenchmarkRune_EastAsian/clipperhouse/displaywidth-8 3395 ns/op 496.96 MB/s 0 B/op 0 allocs/op
BenchmarkRune_EastAsian/mattn/go-runewidth-8 15645 ns/op 107.83 MB/s 0 B/op 0 allocs/op
BenchmarkRune_ASCII/clipperhouse/displaywidth-8 257.8 ns/op 496.57 MB/s 0 B/op 0 allocs/op
BenchmarkRune_ASCII/mattn/go-runewidth-8 267.3 ns/op 478.89 MB/s 0 B/op 0 allocs/op
BenchmarkRune_Emoji/clipperhouse/displaywidth-8 1338 ns/op 541.24 MB/s 0 B/op 0 allocs/op
BenchmarkRune_Emoji/mattn/go-runewidth-8 2287 ns/op 316.58 MB/s 0 B/op 0 allocs/op
BenchmarkTruncateWithTail/clipperhouse/displaywidth-8 3689 ns/op 47.98 MB/s 192 B/op 14 allocs/op
BenchmarkTruncateWithTail/mattn/go-runewidth-8 8069 ns/op 21.93 MB/s 192 B/op 14 allocs/op
BenchmarkTruncateWithoutTail/clipperhouse/displaywidth-8 3457 ns/op 66.24 MB/s 0 B/op 0 allocs/op
BenchmarkTruncateWithoutTail/mattn/go-runewidth-8 10441 ns/op 21.93 MB/s 0 B/op 0 allocs/op
Here are some notes on how to make Unicode things fast.
