v3.34.0

Added

* New reporter ``github_issue`` creates a GitHub issue for changes detected.

  - Kindly contributed by `Dmitry Vasiliev <https://github.com/swimmwatch>`__ in `#105
    <https://github.com/mborsetti/webchanges/issues/105>`__.
  - Implemented as a GitHub Action ``webchanges-action`` available `here
    <https://github.com/swimmwatch/webchanges-action>`__.

* The ``wdiff`` filter now handles ``html`` text.
* New ``suppress_error_ended`` and ``suppress_errors``` Job sub-directives to control error notifications job-by-job.

  - Suggested by `Marcos Alano <https://github.com/mhalano>`__ in `#101
    <https://github.com/mborsetti/webchanges/issues/101>`__.

* New ``ntfy`` reporter to support `ntfy <https://ntfy.sh>`__ (pronounced _notify_), an open-source fee simple
  HTTP-based `pub-sub <https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern>`__ notification service (also
  for upstream compatibility).
* Filters ``execute`` and ``shellpipe`` now have an ``escape_characters`` sub-directive to automatically escape Windows
  command caracters (e.g. ``%`` becomes ``%%``, ``!`` becomes ``^!``, etc.).

Fixed
`````
* All databases, keep-alive connections, and underlying SSL sockets are now closed correctly before exit. Fixes the
  ``ResourceWarning: unclosed database`` and ``ResourceWarning: <ssl.SSLSocket>`` messages when run with environment
  variable ``PYTHONWARNINGS=all``.

v3.33.0

Added

* ``ai_google`` differ has new ``thinking_level`` and ``media_resolution`` sub-directives.
* ``command`` differ has a new ``context_lines`` sub-directive for commands starting with wdiff for backwards 
  compatibility (but use the built-in ``wdiff`` differ instead).

Changed

• ai_google differ is no longer considered BETA.
• Improved logging for the evaluate directive in URL Jobs with browser: true.
• --dump-history JOB command line argument will now match any job, even one that is not in the --jobs file.

Fixed

* Regression: ``http_ignore_error_codes`` not being applied to ``TransientHTTPError`` Exceptions such as '429 Too Many
  Requests' (issue #`119 <https://github.com/mborsetti/webchanges/issues/119>`__).
* ``http_credentials`` directive not being applied to URL jobs with ``browser: true`` and ``user_data_dir``.
* When running with command line argument ``-vv``, browser pages will open with DevTools open.
* Problem parsing Playwright exceptions in BrowserJob class retrieve method  (issue #`141 
  <https://github.com/mborsetti/webchanges/issues/141>`__)..

Internals for ``hooks.py``

• The BrowserJob class' retrieve method has been modularized, and exposes response_handler (a callable which
  replaces the built-in page.goto() directive), content_handler (a callable which replaces the built-in content
  extractor from the Page), and return_data (a callable which replaces all of the built-in functionality after
  the browser is launched).`

Internals

* Code type checking is now performed using ``ty`` instead of ``mypy``.
* Improved logging and the saving of snapshots when a browsing error is encountered for URL jobs with ``browser: true``.
* Use ``uv`` to build PyPi distributable.

v3.31.4

Reminder

Older Python versions are supported for 3 years after being obsoleted by a new major release. As Python 3.10 was
released on 24 October 2022, the codebase will be streamlined by removing support for Python 3.10 on or after 24 
October 2025.

Added
`````
* Support for Python 3.14.

Fixed
`````
* Fixed ``deepdiff`` differ to handle text strings correctly (e.g. when an API typically returning JSON starts
  returning an error in HTML).

Internals (impacting hooks.py)

• In the Differ Class' process method, the report_kind's value text has been renamed plain for
  clarity and to align with IANA's media type nomenclature for different types of text.

Internals (other)

* Implemented testing for Python 3.14.
* Implemented OpenSSF Scorecard and improved code security.
* Enabled additional ``ruff check`` linters and improved code quality.

v3.31.3

Reminder

Older Python versions are supported for 3 years after being obsoleted by a new major release. As Python 3.10 was
released on 24 October 2022, the codebase will be streamlined by removing support for Python 3.10 on or after 24 
October 2025.

Fixed
`````
* Certain job Exceptions would fail with a yaml Exception.

Internals

• Removed non-unique elements in pyproject.toml's classifiers,
• Updated run-gemini-cli to fix GitHub error.
• Fixed pre-commit.ci failing checks on new PRs

v3.31.2

Fixed

* Fixed UnboundLocalError when using new ``utf-8`` sub-directive within the ``smtp`` emailer (``email`` report).
  Reported in #`104 <https://github.com/mborsetti/webchanges/issues/110>`__.

Internals

• Removed workaround for Python 3.9, which is no longer supported.

v3.31.1.post2

Reminder

Older Python versions are supported for 3 years after being obsoleted by a new major release. As Python 3.10 was
released on 24 October 2022, the codebase will be streamlined by removing support for Python 3.10 on or after 24 
October 2025.

Added
`````
* Documented the ``deepdiff`` differ for side-effects when using ``ignore_order: true``.
* Added the ``utf-8`` configuration sub-directive within the ``smtp`` emailer (``email`` report) to enable turning off 
  RFC 6531 Internationalized Email, aka SMTPUTF8 service extension, for backward compatibility with old SMTP servers.
  Requested in #`108 <https://github.com/mborsetti/webchanges/issues/108>`__.

Fixed
`````
* Fixed regresion: getting messages with deletion when the content still seems to be there. Reported in #`104
  <https://github.com/mborsetti/webchanges/issues/104>`__.
* Fixed handling of Playwright exceptions for ``browser: true`` jobs. Reported in #`106
  <https://github.com/mborsetti/webchanges/issues/106>`__.
* Fixed ``--detailed-versions`` CLI, which raised an Exception if playwright was installed but no Chromium browser was
  available.
* Fixed ``monospace: true`` in HTML reports, which was overly applied to include Comparison Type banners.
* Fixed ``deepdiff`` differ, which would not convert Markdown to HTML when needed.
* Fixed url jobs with ``no_redirects: true``, who would not report the redirect response correctly.
* Fixed ``--delete-snapshot`` CLI, which would not display the snapshot time in the timezone set for reports (if one is
  set).
* Fixed ``xpath`` filter, which would throw an Exception with an XPath ``concat()`` string function.
* Fixed default port for SMTP email to ``port: 587``, as is the correct one given the default ``starttls: true``.

Internals

• Code is now formated with ruff format instead of black.
• Code is now linted with ruff check instead of isort and flake8 (and its extensions).
• Packages in test environments are now installed with uv.
• Experimenting with Gemini CLI GitHub Action <https://github.com/google-github-actions/run-gemini-cli/>__ to triage
  issues and perform pull request reviews (thanks to Google's generous free-of-charge quotas).
• Starting to implement lazy loading of packages and modules to improve startup time to execute simple command line
  arguments.

v3.31.0

⚠ Breaking Changes

* Differ ``ai_google`` (BETA) now expects the API key to be in the environment variable named ``GEMINI_API_KEY`` to
  maintain consistency with the new API documentation from Google and is interoperable with Gemini CLI. The deprecated
  ``GOOGLE_AI_API_KEY`` will be read until the end of 2025.

Added
`````
* Directive ``wait_for_selector`` of URL jobs with ``browser: true`` (Playwright) can now be a list, enabling waiting
  for multiple selectors in the order given.
* Differ ``deepdiff`` has a new sub-directive ``compact`` which produces a much less wordy report in YAML and ignores
  changes of data type (e.g. "type changed from NoneType to str").
* The command-line arguments ``--jobs``, ``--config``, and ``--hooks`` feature a "smart file specification" capability.
  This allows you to provide a shorthand name for your files, and  **webchanges** will automatically search for several
  variations of that name.

Changed
```````
* Differ ``deepdiff``'s report has been improved by indenting multi-line value changes.
* Command line ``--test-job``: Improved display by adding to the report the media type (fka MIME type), ETag (when
  present) and GUID (internal identifier).
* HTML reports now treat the job directive ``note`` as Markdown.

Deprecated
``````````
* Environment variable ``GOOGLE_AI_API_KEY`` for the API key used by differ ``ai_google`` (BETA) is deprecated; use
  ``GEMINI_API_KEY`` instead.

Fixed
`````
* Job directive ``ignore_connection_errors``: Did not work as expected with the default ``httpx`` HTML client library
  (#`100 <https://github.com/mborsetti/webchanges/issues/100>`__.).
* Command line argument ``--error``: Was reporting jobs with sites returning HTTP response code 304 Not Modified as if
  they returned no data (#`102 <https://github.com/mborsetti/webchanges/issues/102>`__).
* Command line argument ``--database``: If filename was not not present in the current directory, it was not searched in
  the default database directory.
* Command line argument ``--test-job``:

  - Output is no longer colorized as if it were a diff.
  - When used in conjunction with ``--test-reporter browser`` or any other HTML reporter and the job has
    ``monospace: true``, the output now uses a monospaced font.
* Configuration ``differ_defaults``: Was not being applied correctly in certain circumstances.
* Differ ``ai_google`` (BETA): Improved reporting of upstream API errors.
* Differ ``images`` (BETA):

  - Report now includes the old image;
  - Minor fixes to the ``ai_google`` summary (ALPHA), including proper application of defaults from the config file
    and the inclusion in the footnote of the actual generative AI model used (vs. the one specified).

Internals / hooks.py

• GUID is now assigned to the job when it's loaded, enabling for a hook to programmatically change job.url,
  job.user_visible_url and/or job.command without causing processing errors downstream.
• Jobs with a directive kind belonging to a Class defined in hooks.py, and which inherits from UrlJob, BrowserJob or
  ShellJob, were not initialized with the default configurations for the parent class (fixed).
• Updated vendored code providing httpx.Headers (when httpx is not installed) to mirror version 0.28.1.
• Updated vendored code providing packaging.versions (when packaging is not installed) to mirror version 24.2.
• Minor code fixes resulting from newly using Pyright / Pylance in the IDE.

v3.30.0

Added

• README links to a new Docker implementation which includes the Chrome browser. Generously offered by and maintained
  by Jeff Hedlund <https://github.com/jhedlund>__ as per #96 <https://github.com/mborsetti/webchanges/issues/96>__.
• New filter jsontoyaml to convert JSON to YAML, which is generally more readable in a report for humans.
• New yaml data type for the deepdiff differ (in addition to json and xml).

Changed

• The deepdiff differ will now try to derive the data-type (when it is not specified) from the data's media
  type (fka MIME type) before defaulting to json.

Fixed

• Fixed confusing warning when no default hooks.py file exists. Thanks to Marcos Alano <https://github.com/mhalano>__
  for reporting in #97 <https://github.com/mborsetti/webchanges/issues/97>__.
• The format-json filter now uses JSON text instead of plain text to report errors caused by it not receiving
  valid JSON data, to be compatible with downstream filters or differs.
• Fixed the differ ai_google (BETA), which under certain circumstances was omitting the footnote with the version
  of the GenAI model used.

v3.29.0

⚠ Breaking Changes

• The differ command now requires that the name: command subdirective of differ be specified.

Changed

• The differ command now has a sub-directive is_html to indicate when output is in HTML format. Thanks to Jeff Hedlund <https://github.com/jhedlund>__ for requesting this enhancement in
  #95 <https://github.com/mborsetti/webchanges/issues/95>__.
• Added a tip in the documentation on how to add bullet points <https://webchanges.readthedocs.io/en/stable/advanced.html#bullet-points>__ to improve the legibility of HTML
  reports.

Fixed

• Fixed reporting of errors arising from filters or reporters.
• Fixed reporting of repeated errors (i.e. when the same error occurs multiple times).
• Fixed header and colorization of the differ command.

v3.28.2

Changed

• The filter format-json will no longer raise an error when it is not fed JSON data to facilitate
  troubleshooting, and will now report the JSONDecodeError details and the full string causing this.

• Documentation for the css and xml filters has been split into two separate entries for ease of reference.

• Minor word editing of error messages to improve clarity.

• Various updates to the differ ai_google (BETA):

  • top_p is set to 1.0 if temperature is 0.0 (its default value) to eliminate randomness in results.
  • Wordsmitting of the default system prompt leading to small improvements.
  • The footnote now shows the model actually used vs. the one specified in the model sub-directive, useful when
    omitting the version when using an experimental version (e.g. specifying gemini-2.0-pro-exp instead of
    gemini-2.0-pro-exp-02-05).

Internals

• Tested the image differ's ai_google directive (ALPHA, undocumented), which uses GenAI to summarize
  differences between two images, with the new gemini-2.0-pro-exp-02-05 experimental <https://ai.google.dev/gemini-api/docs/models/experimental-models#available-models>__ and improved default system
  prompt. While the new model shows improvements by producing a plausible-sounding summary instead of gibberish, the
  summary is highly inaccurate and therefore unusable. Development paused again until model accuracy improves.