Skip to content

Add Grapheme cluster support#5

Open
adammw wants to merge 3 commits intoliggitt:masterfrom
adammw:grapheme
Open

Add Grapheme cluster support#5
adammw wants to merge 3 commits intoliggitt:masterfrom
adammw:grapheme

Conversation

@adammw
Copy link

@adammw adammw commented Dec 27, 2023

No description provided.

go.sum Outdated
@@ -0,0 +1,2 @@
github.com/rivo/uniseg v0.4.4 h1:8TfxU8dW6PdqD27gjM8MVNuicgxIjxpm4K7x4jp8sis=
github.com/rivo/uniseg v0.4.4/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88=
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm in favor of fixing the width counting issue but not in favor of picking up a dependency outside the stdlib (or maaaaaybe golang.org/x/text). It looks like this was proposed to be upstreamed into golang.org/x/text in golang/go#14820 (comment).

I'd accept a change that used an upstreamed version of this in golang.org/x/text or unicode/utf8, or a direct implementation of the Grapheme Cluster Boundary Rules in a private function in this library.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated PR to attempt to use a direct implementation from the uniseg code but there's an awful lot of lookup tables that are required to implement them, and the golang.org/x/text issue does not seem to have progressed since 2019.

@adammw
Copy link
Author

adammw commented Jan 3, 2024

This has become a rabbit hole.. most terminals seem to support Unicode 8 or 9 rather than the latest and treat most Emoji that should be double-width as single width, with additional advanced settings to only make them double-width in the presence of Emoji-presentation variation selectors. Therefore I'm not sure a standard approach can be reached without numerous flags to support all options.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants