Oracle perf #4187

akostadinov · 2025-12-10T21:08:53Z

Improve oracle performance by

disabling array fetch for queries with lobs
fetch lobs using piecewise fetch LONG interface
clean-up prepared statements when connection returns to the pool to free memory

Also added a couple of small fixes.

Supersedes #4176

akostadinov · 2025-12-10T21:10:27Z

config/initializers/oracle.rb

 ActiveSupport.on_load(:active_record) do
  if System::Database.oracle?
-    require 'arel/visitors/oracle12_hack'
+    require 'arel/visitors/oracle12_hack' || next # once done, we can skip setup


I believe on application reload we don't need any of this code to be executed again. Using this require statement as a state tracking tool know whether we ever run or not. If your local development environment breaks on code reload, this would be the culprit. Be my guests any time!

akostadinov · 2025-12-10T21:10:43Z

features/support/hooks.rb


 After do
-  ActiveRecord::Base.clear_active_connections!
+  ActiveRecord::Base.connection_handler.clear_active_connections!


avoid DEPRECATION warning

akostadinov · 2025-12-10T21:13:06Z

Gemfile

 # NOTE: Use ENV['DB'] only to install oracle dependencies
 group :oracle do
  oracle = -> { (ENV['ORACLE'] == '1') || ENV.fetch('DATABASE_URL', ENV['DB'])&.start_with?('oracle') }
+  ENV['NLS_LANG'] ||= 'AMERICAN_AMERICA.UTF8' if oracle


Another attempt at setting reliably NLS_LANG before oracle connection is established and only if using oracle. Might be a hack but I'm not sure any other place would be less of a hack and I'm tired of trying to figure out where that other place might be. So here I'm sure it's gonna be set prior loading the oracle gems.

Did we have encoding problems before this?

You can see always a complaint that we fallback to some 7bit ASCII encoding if you don't set this variable separately in your environment. But we have always tried to set a default in the initializer. But that has been too late. So this one should be early enough always.

you can see that line removed from the initializer

qltysh · 2025-12-15T18:55:09Z

❌ 9 blocking issues (9 total)

Tool	Category	Rule	Count
reek	Lint	OracleLobLargeUpdateTest has the variable name 'large_policy_data_v2'	3	❌
reek	Lint	OracleLobLargeUpdateTest#generate_large_policies_config calls '[test_policy].to_json' 2 times	2	❌
rubocop	Style	Incorrect formatting, autoformat by running `qlty fmt`.	1	❌
rubocop	Lint	Models should subclass `ApplicationRecord`.	1	❌
rubocop	Style	Avoid using `OpenStruct`; use `Struct`, `Hash`, a class or test doubles instead.	1	❌
rubocop	Style	Surrounding space missing in default value assignment.	1	❌

@qltysh one-click actions:

Auto-fix formatting (qlty fmt && git push)

jlledom

I would ask you to add better description to complicated PRs like this. The amount of investigation required to understand any single line in that oracle initializer is big enough for the PR to not be reviewable in practice. Now you have claude, you can tell it to explain what you did, and verify it's correct before pasting it in the PR description.

Also, creating one single commit for each conceptually different work would make reviewer's life easier as well.

jlledom · 2025-12-17T09:15:11Z

Gemfile

 # NOTE: Use ENV['DB'] only to install oracle dependencies
 group :oracle do
  oracle = -> { (ENV['ORACLE'] == '1') || ENV.fetch('DATABASE_URL', ENV['DB'])&.start_with?('oracle') }
+  ENV['NLS_LANG'] ||= 'AMERICAN_AMERICA.UTF8' if oracle


Did we have encoding problems before this?

config/initializers/oracle.rb

jlledom · 2025-12-19T13:31:08Z

See the errors in the Oracle cucumbers:

Message:could not obtain a connection from the pool within 5.000 seconds (waited 5.000 seconds); all pooled connections were in use (ActiveRecord::ConnectionTimeoutError)

They might be related to the changes 🤕.

akostadinov · 2025-12-19T14:25:56Z

They might be related to the changes 🤕.

Could be, although I have no idea what might be the case as we don't change the connection management. I wonder if this closing of cursors might take too long sometimes. But then, that can also explain issues we have previously seen with random slowdowns. Will need to try with and without it and then whateever else needed. This is a nightmare.

jlledom · 2026-01-07T09:32:22Z

They might be related to the changes 🤕.

Could be, although I have no idea what might be the case as we don't change the connection management. I wonder if this closing of cursors might take too long sometimes. But then, that can also explain issues we have previously seen with random slowdowns. Will need to try with and without it and then whateever else needed. This is a nightmare.

My thoughts are with you

akostadinov · 2026-01-08T11:58:36Z

config/initializers/oracle.rb

+      def close_and_clear_statements
+        start_time = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+        statement_count = @statements&.length || 0
+        @statements&.clear


Some AI explanation why clearing cursors from app end may not be much of a performance penalty because oracle server still caches these for reuse.

Why Oracle Keeps "Closed" Cursors

Oracle's V$OPEN_CURSOR view is somewhat misleadingly named - it shows cursors Oracle is tracking for the session, not just actively open ones. Oracle keeps them for performance:

Session Cursor Cache: When you "close" a cursor from the application side, Oracle often keeps it cached in case you run the same SQL again. This is controlled by the SESSION_CACHED_CURSORS parameter.

Soft vs Hard Close:
- Soft close: Application closes the cursor, but Oracle keeps it in its cache
- Hard close: Oracle actually deallocates it

akostadinov · 2026-01-08T17:19:01Z

@jlledom , now I'm not sure the changes are related to the failures. I can't reproduce the exact errors but other failure types randomly appear.

akostadinov added 2 commits December 10, 2025 23:06

remove DEPRECATION

f0f2b7e

oracle performance

49d6769

akostadinov commented Dec 10, 2025

View reviewed changes

akostadinov mentioned this pull request Dec 10, 2025

THREESCALE-12061: Monkey patch ruby-oci8 to fix memory bloat (master) #4176

Closed

akostadinov added 2 commits December 12, 2025 21:52

proper fix for writing lobs

8762d66

add tests

4c3fd23

jlledom reviewed Dec 17, 2025

View reviewed changes

jlledom previously approved these changes Dec 18, 2025

View reviewed changes

akostadinov dismissed jlledom’s stale review via c4f6939 January 8, 2026 10:23

akostadinov force-pushed the oracle-perf branch from c4f6939 to 6768407 Compare January 8, 2026 10:37

akostadinov commented Jan 8, 2026

View reviewed changes

debug oracle cursor cleanup

0044b2f

akostadinov force-pushed the oracle-perf branch from 6768407 to 0044b2f Compare January 8, 2026 14:57

Oracle perf #4187

Are you sure you want to change the base?

Oracle perf #4187

Conversation

akostadinov commented Dec 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

qltysh bot commented Dec 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jlledom left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jlledom commented Dec 19, 2025

Uh oh!

akostadinov commented Dec 19, 2025

Uh oh!

jlledom commented Jan 7, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

akostadinov commented Jan 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

akostadinov commented Dec 10, 2025 •

edited

Loading

qltysh bot commented Dec 15, 2025 •

edited

Loading