Support full-text search DDLs

[apstndb]

This PR includes CREATE SEARCH INDEX, ALTER SEARCH INDEX, and DROP SEARCH INDEX statements.

Depends on helper functions in #101, so this branch includes implementation of CREATE PROPERTY GRAPH, and DROP PROPERTY GRAPH.
It needs rebase after merge that.
rebased.

• CREATE SEARCH INDEX

• ALTER SEARCH INDEX

• DROP SEARCH INDEX

• Ensure CREATE TABLE in full-text search examples can be parsed.

  CREATE TABLE Albums (
    AlbumId STRING(MAX) NOT NULL,
    SingerId INT64 NOT NULL,
    ReleaseTimestamp INT64 NOT NULL,
    AlbumTitle STRING(MAX),
    Rating FLOAT64,
    AlbumTitle_Tokens TOKENLIST AS (TOKENIZE_FULLTEXT(AlbumTitle)) HIDDEN,
    Rating_Tokens TOKENLIST AS (TOKENIZE_NUMBER(Rating)) HIDDEN
  ) PRIMARY KEY(AlbumId);

  • Make STORED optional in generated columns
    • https://cloud.google.com/spanner/docs/release-notes#June_17_2024

    Generated columns no longer require the STORED attribute. Without this, the generated column is evaluated at query or index time and doesn't require additional storage or write overhead. For more information, see Create and manage generated columns.

  • Support HIDDEN column

• Implement ast.SearchIndexOptions on ast.GenericOptions Generalize OPTIONS behavior #137

  • Need to merge after Generalize OPTIONS #142 is merged.

• Add Pos.Add() and firstValid to refactor End() 2127e0e

fix #108

depends on #120

[makenowjust]

I have some minor suggestions and give a major issue. Please check it.

[apstndb]

This PR still depends on #142, so I turn this PR as draft.

[apstndb]

Merge OPTIONS and conflict is resolved. Re-opened.

[apstndb]

Merged to fix conflicts.

[makenowjust]

LGTM!