mruby-benchmark: add benchmarking gem

add initial implementation of mruby-benchmark gem providing: - Benchmark.measure for detailed timing with Tms results - Benchmark.realtime for simple wall-clock measurements - Benchmark.bm for formatted comparison reports - optional memory tracking via ObjectSpace integration Co-authored-by: Claude <noreply@anthropic.com>
2026-06-08 16:11:16 +00:00 · 2025-11-26 07:46:06 +09:00
parent ea221a5562
commit 2f40f3d170
3 changed files with 390 additions and 0 deletions
@@ -0,0 +1,257 @@
+# mruby-benchmark
+
+Benchmarking and profiling tools for mruby.
+
+## Overview
+
+The `mruby-benchmark` gem provides simple and lightweight benchmarking capabilities for measuring execution time and memory usage in mruby applications. It is designed for embedded systems and resource-constrained environments.
+
+## Installation
+
+Add the following line to your `build_config.rb`:
+
+```ruby
+conf.gem :core => 'mruby-benchmark'
+```
+
+## API
+
+### Benchmark Module
+
+The main interface for benchmarking operations.
+
+#### `Benchmark.measure { block }` → Benchmark::Tms
+
+Measures the execution time of the given block and returns a `Benchmark::Tms` object containing timing information.
+
+```ruby
+result = Benchmark.measure do
+  # code to benchmark
+  1000.times { "string interpolation: #{42}" }
+end
+
+puts result  # Prints formatted timing information
+```
+
+#### `Benchmark.realtime { block }` → Float
+
+Returns only the real (wall-clock) time in seconds as a floating-point number.
+
+```ruby
+time = Benchmark.realtime do
+  sleep(0.1)
+end
+
+puts "Took #{time} seconds"  # => "Took 0.100... seconds"
+```
+
+#### `Benchmark.bm(label_width = 0) { |x| ... }`
+
+Performs formatted benchmark comparisons with aligned output.
+
+```ruby
+Benchmark.bm(10) do |x|
+  x.report("array:") { 1000.times { [1, 2, 3, 4, 5] } }
+  x.report("hash:") { 1000.times { {a: 1, b: 2, c: 3} } }
+  x.report("string:") { 1000.times { "hello" * 100 } }
+end
+```
+
+Output example:
+
+```
+                user     system      total        real
+array:      0.010000   0.000000   0.010000 (  0.012345)
+hash:       0.015000   0.000000   0.015000 (  0.016789)
+string:     0.008000   0.000000   0.008000 (  0.009012)
+```
+
+#### `Benchmark.measure(memory: true) { block }` → Benchmark::Tms
+
+Measures both execution time and memory allocation when `memory: true` is specified.
+
+```ruby
+result = Benchmark.measure(memory: true) do
+  array = []
+  1000.times { |i| array << i }
+end
+
+puts "Objects allocated: #{result.objects}"
+puts "Memory used: #{result.memory} bytes"
+```
+
+### Benchmark::Tms Class
+
+Holds timing measurement results. Provides methods to access individual timing components.
+
+#### Attributes
+
+- `utime` - User CPU time in seconds (Float)
+- `stime` - System CPU time in seconds (Float)
+- `cutime` - User CPU time of child processes (Float, usually 0 in mruby)
+- `cstime` - System CPU time of child processes (Float, usually 0 in mruby)
+- `real` - Real (wall-clock) time in seconds (Float)
+- `objects` - Number of objects allocated (Integer, when memory tracking enabled)
+- `memory` - Memory allocated in bytes (Integer, when memory tracking enabled)
+
+#### Methods
+
+##### `total` → Float
+
+Returns the total CPU time (user + system).
+
+```ruby
+result = Benchmark.measure { heavy_computation }
+puts "Total CPU time: #{result.total} seconds"
+```
+
+##### `to_s` → String
+
+Returns formatted string representation of timing results.
+
+```ruby
+result = Benchmark.measure { sleep(0.1) }
+puts result.to_s
+# => "  0.000000   0.000000   0.000000 (  0.100123)"
+```
+
+##### `format(format_str)` → String
+
+Returns timing results formatted according to the format string.
+
+Format specifiers:
+
+- `%u` - User CPU time
+- `%s` - System CPU time
+- `%t` - Total CPU time
+- `%r` - Real time
+- `%o` - Objects allocated (if memory tracking enabled)
+- `%m` - Memory allocated (if memory tracking enabled)
+- `%n` - Label name
+
+```ruby
+result = Benchmark.measure { computation }
+puts result.format("Real: %rs, CPU: %ts")
+# => "Real: 0.123s, CPU: 0.100s"
+```
+
+### Benchmark::Report Class
+
+Used within `Benchmark.bm` for formatted reporting.
+
+#### `report(label = "") { block }`
+
+Executes and reports on a single benchmark within a `bm` block.
+
+```ruby
+Benchmark.bm do |x|
+  x.report("first test") { code1 }
+  x.report("second test") { code2 }
+end
+```
+
+## Usage Examples
+
+### Basic Timing
+
+```ruby
+require 'benchmark'
+
+# Simple timing
+time = Benchmark.realtime do
+  sum = 0
+  1000000.times { |i| sum += i }
+end
+puts "Calculation took #{time} seconds"
+
+# Detailed timing
+result = Benchmark.measure do
+  arr = (1..10000).to_a
+  arr.sort!
+end
+puts result
+```
+
+### Comparing Implementations
+
+```ruby
+require 'benchmark'
+
+Benchmark.bm(15) do |x|
+  x.report("Array#each:") do
+    arr = (1..1000).to_a
+    sum = 0
+    arr.each { |n| sum += n }
+  end
+
+  x.report("Array#inject:") do
+    arr = (1..1000).to_a
+    arr.inject(0) { |sum, n| sum + n }
+  end
+
+  x.report("Numeric#times:") do
+    sum = 0
+    1000.times { |n| sum += n }
+  end
+end
+```
+
+### Memory Profiling
+
+```ruby
+require 'benchmark'
+
+# Track memory allocation
+result = Benchmark.measure(memory: true) do
+  strings = []
+  1000.times { |i| strings << "string_#{i}" }
+end
+
+puts "Execution time: #{result.real}s"
+puts "Objects created: #{result.objects}"
+puts "Memory allocated: #{result.memory} bytes"
+```
+
+### Performance Testing in Tests
+
+```ruby
+# In test files
+assert('String concatenation performance') do
+  time = Benchmark.realtime do
+    1000.times { "hello" + "world" }
+  end
+
+  # Assert it completes within reasonable time
+  assert_true time < 0.1, "String concat should be fast"
+end
+```
+
+## Implementation Notes
+
+### Time Measurement
+
+mruby-benchmark uses `Process.clock_gettime` (via mruby-time) for high-resolution timing when available. User and system CPU times are measured using platform-specific APIs where available, otherwise both are set to 0.
+
+### Memory Tracking
+
+Memory profiling uses `ObjectSpace.count_objects` (via mruby-objectspace) to track object allocation. Memory size estimation is based on typical object overhead and may not be exact for all platforms.
+
+### Limitations
+
+- Child process timing (`cutime`, `cstime`) is not supported in most mruby environments and always returns 0
+- System CPU time may not be available on all platforms
+- Memory measurements are estimates and may not reflect actual heap usage
+- GC activity during benchmarking may affect timing results
+
+## Dependencies
+
+- **mruby-time** - Required for timing measurements
+- **mruby-objectspace** - Required for memory profiling
+
+## License
+
+MIT License
+
+## Authors
+
+mruby developers
@@ -0,0 +1,8 @@
+MRuby::Gem::Specification.new('mruby-benchmark') do |spec|
+  spec.license = 'MIT'
+  spec.author  = 'mruby developers'
+  spec.summary = 'benchmarking and profiling tools'
+
+  spec.add_dependency('mruby-time', :core => 'mruby-time')
+  spec.add_dependency('mruby-objectspace', :core => 'mruby-objectspace')
+end
@@ -0,0 +1,125 @@
+module Benchmark
+  # Timing measurement result
+  class Tms
+    attr_reader :utime, :stime, :cutime, :cstime, :real
+    attr_reader :objects, :memory
+
+    def initialize(utime, stime, cutime, cstime, real, label = nil, objects = nil, memory = nil)
+      @utime = utime
+      @stime = stime
+      @cutime = cutime
+      @cstime = cstime
+      @real = real
+      @label = label
+      @objects = objects
+      @memory = memory
+    end
+
+    def total
+      @utime + @stime + @cutime + @cstime
+    end
+
+    def to_s
+      format("%10.6f %10.6f %10.6f (%10.6f)\n", @utime, @stime, total, @real)
+    end
+
+    def format(format_str)
+      str = format_str.dup
+      str.gsub!('%u', @utime.to_s)
+      str.gsub!('%s', @stime.to_s)
+      str.gsub!('%t', total.to_s)
+      str.gsub!('%r', @real.to_s)
+      str.gsub!('%o', @objects.to_s) if @objects
+      str.gsub!('%m', @memory.to_s) if @memory
+      str.gsub!('%n', @label.to_s) if @label
+      str
+    end
+  end
+
+  # Report class for formatted benchmark output
+  class Report
+    def initialize(width = 0)
+      @width = width
+      @results = []
+    end
+
+    def report(label = "")
+      tms = Benchmark.measure { yield }
+      tms.instance_variable_set(:@label, label)
+
+      label_str = label.to_s
+      if label_str.length < @width
+        label_str = label_str + " " * (@width - label_str.length)
+      end
+
+      print label_str
+      print tms.to_s
+
+      @results << tms
+      tms
+    end
+
+    def results
+      @results
+    end
+  end
+
+  # Measure execution time of a block
+  def self.measure(memory: false)
+    start_time = Time.now
+    start_objects = nil
+    start_count = nil
+
+    if memory
+      if defined?(ObjectSpace)
+        start_count = ObjectSpace.count_objects
+        start_objects = start_count.values.inject(0) { |sum, n| sum + n }
+      end
+    end
+
+    yield
+
+    end_time = Time.now
+    real = end_time - start_time
+
+    objects_allocated = nil
+    memory_allocated = nil
+
+    if memory && start_count
+      end_count = ObjectSpace.count_objects
+      end_objects = end_count.values.inject(0) { |sum, n| sum + n }
+      objects_allocated = end_objects - start_objects
+
+      # Estimate memory based on object count
+      # Average object overhead in mruby (approximate)
+      memory_allocated = objects_allocated * 40
+    end
+
+    # mruby typically doesn't have per-process CPU time
+    # Set user/system times to 0
+    Tms.new(0.0, 0.0, 0.0, 0.0, real, nil, objects_allocated, memory_allocated)
+  end
+
+  # Return only real time as a float
+  def self.realtime
+    start_time = Time.now
+    yield
+    end_time = Time.now
+    end_time - start_time
+  end
+
+  # Formatted benchmark with labeled reports
+  def self.bm(label_width = 0)
+    report = Report.new(label_width)
+
+    # Print header
+    if label_width > 0
+      print " " * label_width
+    end
+    puts "      user     system      total        real"
+
+    yield report
+
+    report
+  end
+end