Bm::Instrumentations

Provides Prometheus metrics collectors and integrations for Sequel, Rack, S3, Roda and etc.

• Installation
• Rack Metrics
• Sequel Metrics
• AWS Client Metrics
• Ruby Method Metrics
• Endpoint Name Roda Plugin
• Management Server Puma plugin
• Ruby VM & GC Metrics
• System Process Metrics

---

Installation

Add this line to your application’s Gemfile:

gem 'bm-instrumentations'

And then execute:

$ bundle

Or install it yourself as:

$ gem install bm-instrumentations

---

Rack Metrics

BM::Instrumentations::Rack is a Rack middleware that collect metrics for HTTP request and responses.

# config.ru
require 'bm/instrumentations'
use BM::Instrumentations::Rack, exclude_path: %w[/metrics /ping]

The middleware has some optional parameters:

Parameter	Type	Description
exclude_path	String Array<String>	a list of ignored path names, for that paths the metrics won’t be record
registry	Prometheus::Client::Registry	override the default Prometheus registry

Collected metrics

Metrics	Type	Labels	Description
http_server_requests_total	counter	method path status status_code	the total number of HTTP requests handled by the Rack application
http_server_exceptions_total	counter	method path exception	the total number of uncaught exceptions raised by the Rack application
http_server_request_duration_seconds	histogram	method path status status_code	the HTTP response times in seconds of the Rack application

Labels

• method is a HTTP method name such as GET or POST
• path is an endpoint’s path that handled a request, none if undefined
• status is a cumulative value of a HTTP status code like 2xx or 5xx
• status_code is a HTTP status code from response such as 200 or 500
• exception is an uncaught exception class name such as RuntimeError or Errno::ENOENT

---

Sequel Metrics

Sequel::Extensions::PrometheusInstrumentation is a Sequel extension that instrument a database queries and write metrics into Prometheus.

# Apply an extension
db = Sequel.connect(database_url)
db.extension(:prometheus_instrumentation)

Collected metrics

Metrics	Type	Labels	Description
sequel_queries_total	counter	database query status	how many Sequel queries processed, partitioned by status
sequel_query_duration_seconds	histogram	database status status	the duration in seconds that a Sequel queries spent

Labels

• database is a database name that a connection connected
• query is a query statement name such as select, update
• status one of success or failure

---

AWS Client Metrics

BM::Instrumentations::Aws.plugin is an AWS client plugin that instrument API calls and write metrics into Prometheus.

require 'bm/instrumentations'

# Apply a plugin
Aws::S3::Client.add_plugin(BM::Instrumentations::Aws.plugin)
s3_client = Aws::S3::Client.new(options)

Collected metrics

Metrics	Type	Labels	Description
aws_sdk_client_requests_total	counter	service api status	the total number of successful or failed API calls to AWS services
aws_sdk_client_request_duration_seconds	histogram	service api status	the total time in seconds for the AWS Client to make a call to AWS services
aws_sdk_client_retries_total	counter	service api	the total number retries of failed API calls from AWS client to AWS services
aws_sdk_client_exceptions_total	counter	service api exception	the total number of AWS API calls that fail

Labels

• service is an AWS service name such as S3
• api is an AWS api method method such as ListBuckets or GetObject
• status is an HTTP status code returned by API call such as 200 or 500
• exception is an exception class name such as Seahorse::Client::NetworkingError

---

Ruby Methods Metrics

BM::Instrumentations::Timings is an observer that watch on specified ruby method and write metrics about the method invocations.

require 'bm/instrumentations'

class QueryUsers
  include BM::Instrumentations::Timings[:user_queries] # (1)
  
  def query_one(params)
    # ... any ruby code to instrument ...
  end
  timings :query_one # (2)
  
  def query_all(params)
    # ... any ruby code to instrument ...
  end
  timings :query_all # (2)
end

1. Includes a module with the user_queries metrics prefix, so each metric will have the user_queries_ prefix
2. Attach to methods, each time when any observed method invokes a corresponding counter and a histogram will be updated

Collected metrics

Metrics	Type	Labels	Description
<metrics_prefix>_calls_total	counter	class method status	the total number of of successful or failed calls by ruby's method
<metrics_prefix>_call_duration_seconds	histogram	class method status	the time in seconds which spent at ruby's method calls

Labels

• class is a ruby class where the module included
• method is an observed ruby's method name
• status is one of success or failure

---

Endpoint Name Roda plugin

The endpoint plugin adds an endpoint name to the Rack's request env.

Roda lacks of "controller and action" abstractions, so it cannot be obtain a some useful information about who was handled a request. This plugin fixes the issue by exporting an endpoint name (a function which handled a request) to the Rack's request env as a x.rack.endpoint key.

This plugin is useful with BM::Instrumentations::Rack::Collector. When applied the rack collector could be able to determine which function handled a request and correctly writes a path label to metrics.

# Apply a plugin
class API < Roda
  plugin(:endpoint) # (1)

  endpoint def pong # (2)
    'Pong'
  end

  route do |r|
    r.get('ping') { pong }
  end
end

1. Include a plugin, after included Roda has a class level method endpoint
2. Use the endpoint to mark a specified method as a function that may handle a request. When a function will be invoked a key x.rack.endpoint with a value pong will be exported into Rack env.

---

Management Server Puma plugin

Requires: puma > 5.4.0

The management_server plugin provides monitoring and metrics on different HTTP port, it starts a separated Puma::Server that serves requests.

The plugin exposes few endpoints

• /ping - a liveness probe, always return HTTP 200 OK when the server is running
• /metrics - metrics list from the current Prometheus registry
• /gc-status - print ruby GC statistics as JSON
• /threads - print running threads, names and backtraces as JSON

By default the server is running on 0.0.0.0:9990, the default configuration values could be override in puma configuration file.

# config/puma.rb
plugin(:management_server)

# or override default configuration
plugin(:management_server)
management_server(host: '127.0.0.1', port: 9000, logger: Logger.new(IO::NULL))

Collected metrics

Metrics	Type	Labels	Description
puma_thread_pool_max_size	gauge	-	The preconfigured maximum number of worker threads in the Puma server
puma_thread_pool_size	gauge	-	The number of spawned worker threads in the Puma server
puma_thread_pool_active_size	gauge	-	The number of worker threads that actively executing requests in the Puma server
puma_thread_pool_queue_size	gauge	-	The number of queued requests that waiting execution in the Puma server
puma_server_socket_backlog_size	gauge	listener	Linux only The current size of the pending connection queue of the Puma listener
puma_server_socket_backlog_max_size	gauge	listener	Linux only The preconfigured maximum size of the pending connections queue of the Puma listener

---

Ruby VM & GC metrics

BM::Instrumentation::RubyVM is a custom metrics collector that captures ruby's VM and GC stats. Due to the official prometheus client for ruby isn't yet support that types of collectors, so the collectors only works with BM::Instrumentations::Management::Server together.

require 'bm/instrumentations'

# It installs a custom collector to the default prometheus registry
BM::Instrumentations::RubyVM.install

# Or if you don't want to activate GC::Profiler
BM::Instrumentations::RubyVM.install(enable_gc_profiler: false)

Collected metrics

Metrics	Type	Labels	Description
ruby_version	gauge	ruby version	The current ruby engine name and version
ruby_gc_time_seconds	summary	-	The total time that Ruby GC spends for garbage collection in seconds
ruby_gc_heap_slots_size	gauge	slots	The size of available heap slots of Ruby GC partitioned by slots type (free or live)
ruby_gc_allocated_objects_total	gauge	-	The total number of allocated objects by Ruby GC
ruby_gc_freed_objects_total	gauge	-	The total number of freed objects by Ruby GC
ruby_gc_counts_total	gauge	counts	The total number of Ruby GC counts partitioned by counts type (minor or major)
ruby_vm_global_cache_state	gauge	cache	The Ruby VM global cache state (version) for methods and constants, partitioned by cache type
ruby_threads_count	gauge	-	The number of running threads

System Process Metrics

BM::Instrumentations::Process is a custom metrics collector that captures process' RSS memory and the number of open files. Due to the official prometheus client for ruby isn't yet support that types of collectors, so the collectors only works with BM::Instrumentations::Management::Server together.

require 'bm/instrumentations'

# It installs a custom collector to the default prometheus registry
BM::Instrumentations::Process.install

Collected metrics

Metrics	Type	Labels	Description
process_rss_memory_bytes_count	gauge	-	The number of bytes is allocated to this process
process_open_fds_count	gauge	-	The number of open files by this process

License

The gem is available as open source under the terms of the MIT License.

Code of Conduct

Everyone interacting in the Bm::Instrumentations project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the code of conduct.