Skip to content

feat(gapic): support mTLS certificates when available #658

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
parthea merged 2 commits into from
Dec 16, 2025
Merged

feat(gapic): support mTLS certificates when available #658

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
0e41885
chore(librarian): rename owlbot.py to librarian.py
parthea Dec 16, 2025
cee35e7
feat: update image to us-central1-docker.pkg.dev/cloud-sdk-librarian-…
parthea Dec 16, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
0 .librarian/generator-input/owlbot.py → .librarian/generator-input/librarian.py
View file
Open in desktop
File renamed without changes.
4 changes: 0 additions & 4 deletions .librarian/generator-input/setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -97,10 +97,6 @@
install_requires=dependencies,
extras_require=extras,
python_requires=">=3.7",
scripts=[
"scripts/fixup_datastore_v1_keywords.py",
"scripts/fixup_datastore_admin_v1_keywords.py",
],
include_package_data=True,
zip_safe=False,
)
2 changes: 1 addition & 1 deletion .librarian/state.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
image: us-central1-docker.pkg.dev/cloud-sdk-librarian-prod/images-prod/python-librarian-generator@sha256:ce48ed695c727f7e13efd1fd68f466a55a0d772c87b69158720cec39965bc8b2
image: us-central1-docker.pkg.dev/cloud-sdk-librarian-prod/images-prod/python-librarian-generator@sha256:b8058df4c45e9a6e07f6b4d65b458d0d059241dd34c814f151c8bf6b89211209
libraries:
- id: google-cloud-datastore
version: 2.22.0
Expand Down
104 changes: 104 additions & 0 deletions google/cloud/datastore_admin_v1/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,18 @@
#
from google.cloud.datastore_admin_v1 import gapic_version as package_version

import google.api_core as api_core
import sys

__version__ = package_version.__version__

if sys.version_info >= (3, 8): # pragma: NO COVER
from importlib import metadata
else: # pragma: NO COVER
# TODO(https://github.com/googleapis/python-api-core/issues/835): Remove
# this code path once we drop support for Python 3.7
import importlib_metadata as metadata


from .services.datastore_admin import DatastoreAdminClient
from .services.datastore_admin import DatastoreAdminAsyncClient
Expand All @@ -43,6 +53,100 @@
from .types.migration import MigrationState
from .types.migration import MigrationStep

if hasattr(api_core, "check_python_version") and hasattr(
api_core, "check_dependency_versions"
): # pragma: NO COVER
api_core.check_python_version("google.cloud.datastore_admin_v1") # type: ignore
api_core.check_dependency_versions("google.cloud.datastore_admin_v1") # type: ignore
else: # pragma: NO COVER
# An older version of api_core is installed which does not define the
# functions above. We do equivalent checks manually.
try:
import warnings
import sys

_py_version_str = sys.version.split()[0]
_package_label = "google.cloud.datastore_admin_v1"
if sys.version_info < (3, 9):
warnings.warn(
"You are using a non-supported Python version "
+ f"({_py_version_str}). Google will not post any further "
+ f"updates to {_package_label} supporting this Python version. "
+ "Please upgrade to the latest Python version, or at "
+ f"least to Python 3.9, and then update {_package_label}.",
Comment on lines +72 to +76
Copy link

@gemini-code-assist gemini-code-assist bot Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The warning message is constructed using a mix of string concatenation with + and an f-string. This can be simplified into a single multi-line string using implicit concatenation for better readability.

                ("You are using a non-supported Python version "
                f"({_py_version_str}).  Google will not post any further "
                f"updates to {_package_label} supporting this Python version. "
                "Please upgrade to the latest Python version, or at "
                f"least to Python 3.9, and then update {_package_label}.")

FutureWarning,
)
if sys.version_info[:2] == (3, 9):
warnings.warn(
f"You are using a Python version ({_py_version_str}) "
+ f"which Google will stop supporting in {_package_label} in "
+ "January 2026. Please "
+ "upgrade to the latest Python version, or at "
+ "least to Python 3.10, before then, and "
+ f"then update {_package_label}.",
Comment on lines +81 to +86
Copy link

@gemini-code-assist gemini-code-assist bot Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The warning message is constructed using a mix of string concatenation with + and an f-string. This can be simplified into a single multi-line string using implicit concatenation for better readability.

                (f"You are using a Python version ({_py_version_str}) "
                f"which Google will stop supporting in {_package_label} in "
                "January 2026. Please "
                "upgrade to the latest Python version, or at "
                "least to Python 3.10, before then, and "
                f"then update {_package_label}.")

FutureWarning,
)

def parse_version_to_tuple(version_string: str):
"""Safely converts a semantic version string to a comparable tuple of integers.
Example: "4.25.8" -> (4, 25, 8)
Ignores non-numeric parts and handles common version formats.
Args:
version_string: Version string in the format "x.y.z" or "x.y.z<suffix>"
Returns:
Tuple of integers for the parsed version string.
"""
parts = []
for part in version_string.split("."):
try:
parts.append(int(part))
except ValueError:
# If it's a non-numeric part (e.g., '1.0.0b1' -> 'b1'), stop here.
# This is a simplification compared to 'packaging.parse_version', but sufficient
# for comparing strictly numeric semantic versions.
break
return tuple(parts)

def _get_version(dependency_name):
try:
version_string: str = metadata.version(dependency_name)
parsed_version = parse_version_to_tuple(version_string)
return (parsed_version, version_string)
except Exception:
# Catch exceptions from metadata.version() (e.g., PackageNotFoundError)
# or errors during parse_version_to_tuple
return (None, "--")

_dependency_package = "google.protobuf"
_next_supported_version = "4.25.8"
_next_supported_version_tuple = (4, 25, 8)
_recommendation = " (we recommend 6.x)"
(_version_used, _version_used_string) = _get_version(_dependency_package)
if _version_used and _version_used < _next_supported_version_tuple:
warnings.warn(
f"Package {_package_label} depends on "
+ f"{_dependency_package}, currently installed at version "
+ f"{_version_used_string}. Future updates to "
+ f"{_package_label} will require {_dependency_package} at "
+ f"version {_next_supported_version} or higher{_recommendation}."
+ " Please ensure "
+ "that either (a) your Python environment doesn't pin the "
+ f"version of {_dependency_package}, so that updates to "
+ f"{_package_label} can require the higher version, or "
+ "(b) you manually update your Python environment to use at "
+ f"least version {_next_supported_version} of "
+ f"{_dependency_package}.",
Comment on lines +127 to +138
Copy link

@gemini-code-assist gemini-code-assist bot Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The warning message is constructed using a mix of string concatenation with + and an f-string. This can be simplified into a single multi-line string using implicit concatenation for better readability.

                (f"Package {_package_label} depends on "
                f"{_dependency_package}, currently installed at version "
                f"{_version_used_string}. Future updates to "
                f"{_package_label} will require {_dependency_package} at "
                f"version {_next_supported_version} or higher{_recommendation}."
                " Please ensure "
                "that either (a) your Python environment doesn't pin the "
                f"version of {_dependency_package}, so that updates to "
                f"{_package_label} can require the higher version, or "
                "(b) you manually update your Python environment to use at "
                f"least version {_next_supported_version} of "
                f"{_dependency_package}.")

FutureWarning,
)
except Exception:
warnings.warn(
"Could not determine the version of Python "
+ "currently being used. To continue receiving "
+ "updates for {_package_label}, ensure you are "
+ "using a supported version of Python; see "
+ "https://devguide.python.org/versions/"
)

__all__ = (
"DatastoreAdminAsyncClient",
"CommonMetadata",
Expand Down
46 changes: 32 additions & 14 deletions google/cloud/datastore_admin_v1/services/datastore_admin/client.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -207,6 +207,34 @@ def _get_default_mtls_endpoint(api_endpoint):
_DEFAULT_ENDPOINT_TEMPLATE = "datastore.{UNIVERSE_DOMAIN}"
_DEFAULT_UNIVERSE = "googleapis.com"

@staticmethod
def _use_client_cert_effective():
"""Returns whether client certificate should be used for mTLS if the
google-auth version supports should_use_client_cert automatic mTLS enablement.

Alternatively, read from the GOOGLE_API_USE_CLIENT_CERTIFICATE env var.

Returns:
bool: whether client certificate should be used for mTLS
Raises:
ValueError: (If using a version of google-auth without should_use_client_cert and
GOOGLE_API_USE_CLIENT_CERTIFICATE is set to an unexpected value.)
"""
# check if google-auth version supports should_use_client_cert for automatic mTLS enablement
if hasattr(mtls, "should_use_client_cert"): # pragma: NO COVER
return mtls.should_use_client_cert()
else: # pragma: NO COVER
# if unsupported, fallback to reading from env var
use_client_cert_str = os.getenv(
"GOOGLE_API_USE_CLIENT_CERTIFICATE", "false"
).lower()
if use_client_cert_str not in ("true", "false"):
raise ValueError(
"Environment variable `GOOGLE_API_USE_CLIENT_CERTIFICATE` must be"
" either `true` or `false`"
)
return use_client_cert_str == "true"

@classmethod
def from_service_account_info(cls, info: dict, *args, **kwargs):
"""Creates an instance of this client using the provided credentials
Expand Down Expand Up @@ -372,20 +400,16 @@ def get_mtls_endpoint_and_cert_source(
)
if client_options is None:
client_options = client_options_lib.ClientOptions()
use_client_cert = os.getenv("GOOGLE_API_USE_CLIENT_CERTIFICATE", "false")
use_client_cert = DatastoreAdminClient._use_client_cert_effective()
use_mtls_endpoint = os.getenv("GOOGLE_API_USE_MTLS_ENDPOINT", "auto")
if use_client_cert not in ("true", "false"):
raise ValueError(
"Environment variable `GOOGLE_API_USE_CLIENT_CERTIFICATE` must be either `true` or `false`"
)
if use_mtls_endpoint not in ("auto", "never", "always"):
raise MutualTLSChannelError(
"Environment variable `GOOGLE_API_USE_MTLS_ENDPOINT` must be `never`, `auto` or `always`"
)

# Figure out the client cert source to use.
client_cert_source = None
if use_client_cert == "true":
if use_client_cert:
if client_options.client_cert_source:
client_cert_source = client_options.client_cert_source
elif mtls.has_default_client_cert_source():
Expand Down Expand Up @@ -417,20 +441,14 @@ def _read_environment_variables():
google.auth.exceptions.MutualTLSChannelError: If GOOGLE_API_USE_MTLS_ENDPOINT
is not any of ["auto", "never", "always"].
"""
use_client_cert = os.getenv(
"GOOGLE_API_USE_CLIENT_CERTIFICATE", "false"
).lower()
use_client_cert = DatastoreAdminClient._use_client_cert_effective()
use_mtls_endpoint = os.getenv("GOOGLE_API_USE_MTLS_ENDPOINT", "auto").lower()
universe_domain_env = os.getenv("GOOGLE_CLOUD_UNIVERSE_DOMAIN")
if use_client_cert not in ("true", "false"):
raise ValueError(
"Environment variable `GOOGLE_API_USE_CLIENT_CERTIFICATE` must be either `true` or `false`"
)
if use_mtls_endpoint not in ("auto", "never", "always"):
raise MutualTLSChannelError(
"Environment variable `GOOGLE_API_USE_MTLS_ENDPOINT` must be `never`, `auto` or `always`"
)
return use_client_cert == "true", use_mtls_endpoint, universe_domain_env
return use_client_cert, use_mtls_endpoint, universe_domain_env

@staticmethod
def _get_client_cert_source(provided_cert_source, use_cert_flag):
Expand Down
104 changes: 104 additions & 0 deletions google/cloud/datastore_v1/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,18 @@
#
from google.cloud.datastore_v1 import gapic_version as package_version

import google.api_core as api_core
import sys

__version__ = package_version.__version__

if sys.version_info >= (3, 8): # pragma: NO COVER
from importlib import metadata
else: # pragma: NO COVER
# TODO(https://github.com/googleapis/python-api-core/issues/835): Remove
# this code path once we drop support for Python 3.7
import importlib_metadata as metadata


from .services.datastore import DatastoreClient
from .services.datastore import DatastoreAsyncClient
Expand Down Expand Up @@ -69,6 +79,100 @@
from .types.query_profile import ExplainOptions
from .types.query_profile import PlanSummary

if hasattr(api_core, "check_python_version") and hasattr(
api_core, "check_dependency_versions"
): # pragma: NO COVER
api_core.check_python_version("google.cloud.datastore_v1") # type: ignore
api_core.check_dependency_versions("google.cloud.datastore_v1") # type: ignore
else: # pragma: NO COVER
# An older version of api_core is installed which does not define the
# functions above. We do equivalent checks manually.
try:
import warnings
import sys

_py_version_str = sys.version.split()[0]
_package_label = "google.cloud.datastore_v1"
if sys.version_info < (3, 9):
warnings.warn(
"You are using a non-supported Python version "
+ f"({_py_version_str}). Google will not post any further "
+ f"updates to {_package_label} supporting this Python version. "
+ "Please upgrade to the latest Python version, or at "
+ f"least to Python 3.9, and then update {_package_label}.",
Comment on lines +98 to +102
Copy link

@gemini-code-assist gemini-code-assist bot Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The warning message is constructed using a mix of string concatenation with + and an f-string. This can be simplified into a single multi-line string using implicit concatenation for better readability.

                ("You are using a non-supported Python version "
                f"({_py_version_str}).  Google will not post any further "
                f"updates to {_package_label} supporting this Python version. "
                "Please upgrade to the latest Python version, or at "
                f"least to Python 3.9, and then update {_package_label}.")

FutureWarning,
)
if sys.version_info[:2] == (3, 9):
warnings.warn(
f"You are using a Python version ({_py_version_str}) "
+ f"which Google will stop supporting in {_package_label} in "
+ "January 2026. Please "
+ "upgrade to the latest Python version, or at "
+ "least to Python 3.10, before then, and "
+ f"then update {_package_label}.",
Comment on lines +107 to +112
Copy link

@gemini-code-assist gemini-code-assist bot Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The warning message is constructed using a mix of string concatenation with + and an f-string. This can be simplified into a single multi-line string using implicit concatenation for better readability.

                (f"You are using a Python version ({_py_version_str}) "
                f"which Google will stop supporting in {_package_label} in "
                "January 2026. Please "
                "upgrade to the latest Python version, or at "
                "least to Python 3.10, before then, and "
                f"then update {_package_label}.")

FutureWarning,
)

def parse_version_to_tuple(version_string: str):
"""Safely converts a semantic version string to a comparable tuple of integers.
Example: "4.25.8" -> (4, 25, 8)
Ignores non-numeric parts and handles common version formats.
Args:
version_string: Version string in the format "x.y.z" or "x.y.z<suffix>"
Returns:
Tuple of integers for the parsed version string.
"""
parts = []
for part in version_string.split("."):
try:
parts.append(int(part))
except ValueError:
# If it's a non-numeric part (e.g., '1.0.0b1' -> 'b1'), stop here.
# This is a simplification compared to 'packaging.parse_version', but sufficient
# for comparing strictly numeric semantic versions.
break
return tuple(parts)

def _get_version(dependency_name):
try:
version_string: str = metadata.version(dependency_name)
parsed_version = parse_version_to_tuple(version_string)
return (parsed_version, version_string)
except Exception:
# Catch exceptions from metadata.version() (e.g., PackageNotFoundError)
# or errors during parse_version_to_tuple
return (None, "--")

_dependency_package = "google.protobuf"
_next_supported_version = "4.25.8"
_next_supported_version_tuple = (4, 25, 8)
_recommendation = " (we recommend 6.x)"
(_version_used, _version_used_string) = _get_version(_dependency_package)
if _version_used and _version_used < _next_supported_version_tuple:
warnings.warn(
f"Package {_package_label} depends on "
+ f"{_dependency_package}, currently installed at version "
+ f"{_version_used_string}. Future updates to "
+ f"{_package_label} will require {_dependency_package} at "
+ f"version {_next_supported_version} or higher{_recommendation}."
+ " Please ensure "
+ "that either (a) your Python environment doesn't pin the "
+ f"version of {_dependency_package}, so that updates to "
+ f"{_package_label} can require the higher version, or "
+ "(b) you manually update your Python environment to use at "
+ f"least version {_next_supported_version} of "
+ f"{_dependency_package}.",
Comment on lines +153 to +164
Copy link

@gemini-code-assist gemini-code-assist bot Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The warning message is constructed using a mix of string concatenation with + and an f-string. This can be simplified into a single multi-line string using implicit concatenation for better readability.

                (f"Package {_package_label} depends on "
                f"{_dependency_package}, currently installed at version "
                f"{_version_used_string}. Future updates to "
                f"{_package_label} will require {_dependency_package} at "
                f"version {_next_supported_version} or higher{_recommendation}."
                " Please ensure "
                "that either (a) your Python environment doesn't pin the "
                f"version of {_dependency_package}, so that updates to "
                f"{_package_label} can require the higher version, or "
                "(b) you manually update your Python environment to use at "
                f"least version {_next_supported_version} of "
                f"{_dependency_package}.")

FutureWarning,
)
except Exception:
warnings.warn(
"Could not determine the version of Python "
+ "currently being used. To continue receiving "
+ "updates for {_package_label}, ensure you are "
+ "using a supported version of Python; see "
+ "https://devguide.python.org/versions/"
)

__all__ = (
"DatastoreAsyncClient",
"AggregationQuery",
Expand Down
Loading