Skip to content

rspockframework/rspock

Repository files navigation

CI codecov

RSpock

RSpock is a testing and specification framework built on top of Minitest. It intends to give back productivity to its users with its incredibly simple yet highly expressive specification language.

Note: RSpock is heavily inspired by Spock for the Groovy programming language.

Goals

  • High readability, expressiveness, maintainability and productivity: Take back your very precious developer time!
  • Encourage code reuse through expressive data-driven tests

Features

  • BDD-style code blocks: Given, When, Then, Expect, Cleanup, Where
  • Data-driven testing with incredibly expressive table-based Where blocks
  • Expressive assertions: Use familiar comparison operators == and != for assertions!
  • Interaction-based testing, i.e. 1 * object.receive("message") in Then blocks, with optional return value stubbing via >>, exception stubbing via >> raises(...), and block forwarding verification via &block
  • (Planned) BDD-style custom reporter that outputs information from Code Blocks
  • (Planned) Capture all Then block violations

Installation

Add this line to your application's Gemfile:

gem 'rspock'

And then execute:

$ bundle

Or install it yourself as:

$ gem install rspock

Add this to the very beginning of your script or application to install the ASTTransform hook:

require 'ast_transform'
ASTTransform.install

If you're in a non-Rails project, add this to your Rakefile to enable the Truth Table Generator Rake task:

require 'rspock'

spec = Gem::Specification.find_by_name 'rspock'
rakefile = "#{spec.gem_dir}/lib/Rakefile"
load rakefile

Rails

If you are using Rails, it is necessary to add a filter to Rails.backtrace_cleaner for source mapping to work, so that you get proper line numbers in Minitest backtraces. For your convenience, we've built a Rails Generator just for that:

$ rails g rspock:install

Note: If you are not using Rails, you don't have anything to do, as RSpock includes a Minitest plugin that will set its own backtrace filter.

Usage

Getting started using RSpock is extremely easy!

require 'test_helper'

transform!(RSpock::AST::Transformation)
class MyTest < Minitest::Test
  # Feature Methods go here
end

Note: transform! is an annotation added by the ast_transform module that allows executing AST transformations on the annotated code. See here for more info.

Example With Feature Method and Code Blocks

require 'test_helper'

transform!(RSpock::AST::Transformation)
class MyTest < Minitest::Test
  test "adding 1 and 2 results in 3" do
    When "Adding 1 and 2"
    actual = 1 + 2

    Then "We get the expected result 3"
    actual == 3
  end
end

Feature Methods

test "adding 1 and 2 results in 3" do
  # Code Blocks go here
end

A feature method consists of four main conceptual phases:

  • Setup
  • Provide a stimulus to the system under test
  • Describe the response expected from the system
  • Cleanup

The first and last steps are optional, however the stimulus and response phases are always present.

Code Blocks

Blocks Phases
Given Setup
When Stimulus
Then Response
Expect Stimulus + Response
Cleanup Cleanup
Where Repeat

RSpock has support for each conceptual phase of a feature method. As such, feature methods are structured into Code Blocks, each representing a phase.

See below diagram for how you may arrange code blocks. Any directed path from Start to End is valid.

Given Block

Given "An empty Cart and a Product"
cart = Cart.new
product = Product.new

The Given block is where you do any special setup for the Feature Method. It otherwise doesn't have any special semantics.

When Block

When "Adding a product"
cart.add_product(Product.new)

The When block describes the stimulus to be applied to the system under test. It is always followed by a Then block.

Then Block

Then "The product is added to the cart"
cart.products.size == 1
cart.products.first == product

The Then block describes the response from the stimulus. Any comparison operators used in the Then block (== or !=) is transformed to assert_equal / refute_equal under the hood. By convention, the LHS operand is considered the actual value, while the RHS operand is considered the expected value.

Expect Block

The Expect block is useful when expressing the stimulus and the response in one statement is more natural. For example, let's compare two equivalent ways of describing some behaviour:

When + Then
When "Calling #abs on a negative number"
actual = -2.abs

Then "Value is positive"
actual == 2
Using Expect
Expect "absolute of -2 is 2"
-2.abs == 2

A good rule of thumb is using When + Then blocks to describe methods with side-effects and Expect blocks to describe purely functional methods.

Cleanup Block

Given "Open the file"
file = File.new("/invalid/file/path") # raises

# other blocks...

Cleanup
file&.close # Use safe navigation operator, since +file+ is nil if an error occurred.

The Cleanup block is where you free any resources used by a Feature Method. It runs even if a previous part of the Feature Method produced an exception. This means that Cleanup blocks must be coded defensively so as to not raise NoMethodError. A good way to do this in Ruby is demonstrated above by using the &. safe navigation operator.

This is also useful to ensure a shared resource is cleaned in between Feature Methods / test runs so as to not leak test state.

Where Block

Where blocks have very special semantics in RSpock. They take the form of a data table, for readability.

Take a look at the following Feature Method for an example of how to use it:

test "Adding #{a} and #{b} results in #{c}" do
  When "Adding two numbers"
  actual = a + b

  Then "We get the expected result"
  actual == c

  Where
  a  | b  | c
  -1 | -1 | -2
  -1 | 0  | -1
  -1 | 1  | 0
  0  | -1 | -1
  0  | 0  | 0
  0  | 1  | 1
  1  | -1 | 0
  1  | 0  | 1
  1  | 1  | 2
end

The first row in the Where block is considered the Header. The names of columns will expose a local variable of the same name in the scope of the Feature Method. The header column names have the same constraints as method names in Ruby. Each other row defines one test case that will be generated, binding each column's data to the appropriate variable.

This effectively creates one version of the Feature Method for each data row. Note how we've listed test cases as if this was a truth table, ordering them by boolean increment. This makes it very easy to ensure all cases have been covered.

Note: Although the Where block is declared last, it is evaluated first. This means that it cannot access local variables previously defined in the test method. It is evaluated in Class scope, so it is possible to use generators or methods for column values, provided they are class methods, not instance methods.

Tip: By convention, the expected_result or output column should be the rightmost column.

Test Name Interpolation

You might have noticed above that the test name contains string interpolations, that's one of the features of RSpock! You can interpolate test names and use Where block header variables to parameterize the test name using the test data.

Truth Table

Formatting Where Blocks as a truth table is the recommended way to organize your Where Blocks in RSpock. It makes test cases more maintainable since it creates an order between the different test cases and makes it easier to spot missing or duplicated test cases. It also makes it easier to add a column:

  • Add your new column to the left with the first possible value
  • Copy all current Where Block's data lines
  • For each other possible value for the column you're adding, paste the copied lines
  • Fill the new column by iterating over its possible values and you're done!
Truth Table Generator

Are you a lazy programmer? Good, so are we!

If you end up in a situation where the code under test has many different inputs / states that make it hard to manually generate a Truth Table, or if you simply want to make sure you don't omit test cases, we got you covered! You can use our truth table generator to generate all the different test cases automatically!

Run the following command, listing columns in the order you want them to appear, and their possible values in the order you want them to be iterated on:

$ rake rspock:truth_table -- a=-1,0,1 b=-1,0,1 expected_result="'?'"

The above command outputs the following formatted table:

a  | b  | expected_result
-1 | -1 | '?'
-1 | 0  | '?'
-1 | 1  | '?'
0  | -1 | '?'
0  | 0  | '?'
0  | 1  | '?'
1  | -1 | '?'
1  | 0  | '?'
1  | 1  | '?'

Tip: When generating a truth table, we recommend to include the expected_result column with only one possible value, which you can then fill manually. Unless of course all your test cases have the same result, in which case you don't need said column.

Escaping Delimiter

You may escape the , delimiter, which can be useful when i.e. you're calling a method with multiple arguments to generate or build the data for a certain column.

$ rake rspock:truth_table -- a=0,1 b="generator(1\, 2)","generator(3\, 4)" expected_result="'?'"

The above command outputs the following formatted table:

a | b               | expected_result
0 | generator(1, 2) | '?'
0 | generator(3, 4) | '?'
1 | generator(1, 2) | '?'
1 | generator(3, 4) | '?'

Mocking with Interactions

Interaction-based testing is a testing practice which focuses on the messages sent to objects, rather than their state. It explores how the object(s) under specification interact, through method calls, with their collaborators.

RSpock allows a natural way of expressing these interactions, using the expected method calls as code directly. An example is worth a thousand words here:

test "#publish sends a message to all subscribers" do
  Given
  subscriber1 = Subscriber.new
  subscriber2 = Subscriber.new
  publisher = Publisher.new(subscriber1, subscriber2)

  When
  publisher.publish("message")

  Then
  1 * subscriber.receive("message")
  1 * subscriber2.receive("message")
end

The above Then block contains 2 interactions, each of which has 4 parts: the cardinality, the receiver, the message and its arguments. Optionally, an outcome can be specified using the >> operator (either a return value or raises(...) for exceptions), and block forwarding can be verified using the & operator.

1 * receiver.message('hello', &blk) >> "result"
|   |          |       |       |       |
|   |          |       |       |       outcome (optional): value or raises(...)
|   |          |       |       block forwarding (optional)
|   |          |       argument(s) (optional)
|   |          message
|   receiver
cardinality

Note: Interactions are supported in the Then block only.

Execution Order

Although interactions are declared in the Then block, they are effectively active before the When block executes. RSpock ensures the following order:

  1. Before the stimulus — mock expectations and block captures are installed on the receiver, so they are ready to intercept calls.
  2. Stimulus — the When block runs.
  3. After the stimulus — assertions such as block identity checks run alongside other Then assertions. Cardinality is verified at teardown.

Simply declare what should happen in a natural order — RSpock handles the when.

Cardinality

The cardinality of an interaction describes how often a method is expected to be called. It can be a fixed number of times, or a range.

1 * receiver.message('hello')       # exactly once
0 * receiver.message('hello')       # must never be called
(1..3) * receiver.message('hello')  # between one and three times (inclusive)
(1...4) * receiver.message('hello') # between one and four times (exclusive) => same as (1..3) above
(1.._) * receiver.message('hello')  # at least once
(_..3) * receiver.message('hello')  # at most three times
_ * receiver.message('hello')       # any number of calls, including zero

Receiver

The receiver describes which object is expected to receive the method call.

1 * subscriber.receive('hello') # a call to 'subscriber'

Message

The message of an interaction describes which method is expected to be called on the receiver.

1 * subscriber.receive('hello') # a method named 'receive'

Arguments

The arguments of an interaction describe which arguments of the method call are expected.

1 * subscriber.receive('hello') # an argument that is equal to the String 'hello'

Stubbing Return Values

Interactions can be combined with return value stubbing using the >> operator. This is useful when the code under test relies on the return value of a collaborator method.

1 * subscriber.receive("hello") >> "ok"

The >> operator is placed at the end of the interaction and specifies the value that the method call will return. This means you can set up expectations and stub return values in a single, expressive statement.

class Service
  # ...

  def initialize(repository)
    @repository = repository
  end

  def foo(param)
    # ...
    result = @repository.bar(param)
    # Do something with +result+ ...

    result
  end
end

test "Service#bar" do
  Given
  repository = Repository.new
  service = Service.new(repository)

  When
  result = service.foo(42)

  Then
  1 * repository.foo(42) >> { name: "item", status: "active" }
  result == "active"
end

The return value can be any expression — a literal, a variable, or a complex object:

1 * repository.find(42) >> "result"          # a String
1 * repository.all >> [item1, item2]         # an Array
1 * service.call >> { status: :ok }          # a Hash
_ * cache.fetch("key") >> expensive_result   # a variable

Note: Without >>, an interaction sets up an expectation only (the method will return nil by default). Use >> when the code under test depends on the return value.

Stubbing Exceptions

When the code under test needs a collaborator to raise an exception, use >> raises(...) instead of a return value. This sets up the mock to raise the given exception when called.

test "#fetch raises when the record is not found" do
  Given
  repository = mock
  service = Service.new(repository)

  When
  service.fetch(42)

  Then
  1 * repository.find(42) >> raises(RecordNotFound)
end

You can also pass a message or an exception instance:

1 * repository.find(42) >> raises(RecordNotFound, "not found")  # class + message
1 * repository.find(42) >> raises(RecordNotFound.new("not found"))  # instance

This works with all interaction features — cardinality, arguments, and ranges:

(1..3) * service.call(anything) >> raises(TimeoutError)

Note: Without raises(...), the >> operator stubs a return value. With raises(...), it stubs an exception instead.

Block Forwarding Verification

When the code under test forwards a block (or proc) to a collaborator, you may want to verify that the exact block was passed through. RSpock supports this with the & operator in interactions, performing an identity check on the block reference.

test "#frame forwards the block to CLI::UI.frame" do
  Given "a block to forward"
  my_block = proc { puts "hello" }

  When "we call frame with that block"
  @ui.frame("Build", &my_block)

  Then "the block was forwarded to cli_ui.frame"
  1 * @cli_ui.frame("Build", to: @out, &my_block)
end

Note: Inline blocks (do...end or { }) are not supported in interactions and will raise an InteractionError. Use a named proc or lambda variable with & instead, since block forwarding verification requires a reference to compare against.

Debugging

Pry

Let's be honest, at some point you will need to debug your tests. Because RSpock requires transforming the AST, the executed code is slightly different from the source code that you wrote. Although we have plans to add Source Mapping support in Pry so that you can see the exact source code you wrote, this is not currently available. This means that code shown in the Pry console will be slightly different. We think the value of using RSpock greatly outweighs this current limitation. We encourage you to try debugging a test to see what the transformed code looks like, or look through tmp/rspock for the transformed files.

Backtraces

RSpock supports Source Mapping so that backtraces for the executed code point to the correct line numbers in your source code, and so that the correct files are referenced. This is achieved through wrapping the executed code in rescue blocks, processing the backtraces (source mapping) and re-raising the error.

Tips and Tricks

test_index and line_number

The generated test name for each test case will contain the test index and the line number, corresponding to the Where Block data row for that case, which is available in the test scope as _test_index_ and _line_number_ respectively. This can be leveraged to conditionally break on certain test cases, so that you can have a more granular debugging session.

test "Adding #{a} and #{b} results in #{c}" do
  When "Adding two numbers"
  actual = a + b

  Then "We get the expected result"
  # Breaks on the first test case
  binding.pry if _test_index_ == 0
  # Breaks on the second test case
  binding.pry if _line_number_ == 15
  actual == c

  Where
  a | b | c
  1 | 2 | 3
  4 | 5 | 9 # Line 15
end

A few notes:

  • Comparison with _test_index_ and _line_number_ is not transformed to assertions in Then and Expect Code Blocks
  • _test_index_ is zero-based, meaning the index of the first test case is 0

line_number

The Line number is extremely useful for figuring out exactly which test case failed in your Where Block, especially if you have many rows in your Where Block data table.

More info

RSpock Syntax Pre-Processor

transform!(RSpock::AST::Transformation)
class MyTest < Minitest::Test
  # ...
end

RSpock, although having valid Ruby syntax, has different semantics in certain contexts, so that we can offer a more expressive and readable syntax. As such, we perform AST transformations on the code to produce semantically valid-in-context Ruby code.

This is achieved by the transform! "method call" above. This doesn't actually call anything, it instead is an annotation exposed by the ast_transform module that picks up whatever AST transformations are passed as arguments and runs them before compilation into Ruby instructions. The transformations required by RSpock are encapsulated in RSpock::AST::Transformation.

Using RSpock Alongside Regular Minitest Tests

Although we strongly encourage being consistent in your code bases and using only RSpock Tests in the same test file, we understand that there might be a transition period until all tests in a file can be migrated to using RSpock.

To that effect, we support disabling the strict mode on the transform! annotation:

transform!(RSpock::AST::Transformation.new(strict: false))
class MyTest < Minitest::Test
  test "non-RSpock tests work" do
    assert_equal true, true
  end

  test "RSpock tests also work" do
    Expect
    1 + 2 == 3
  end
end

Development

After checking out the repo, run bin/setup to install dependencies. Then, run bundle exec rake test to run the tests. You can also run bin/console for an interactive prompt that will allow you to experiment.

To install this gem onto your local machine, run bundle exec rake install.

Releasing a New Version

There are two ways to create a release. Both require that version.rb has already been updated and merged to main.

Via GitHub UI

  1. Update VERSION in lib/rspock/version.rb and run bundle install to regenerate Gemfile.lock, commit, open a PR, and merge to main
  2. Go to the repo on GitHub → ReleasesDraft a new release
  3. Enter a new tag (e.g. v2.0.0), select main as the target branch
  4. Add a title and release notes (GitHub can auto-generate these from merged PRs)
  5. Click Publish release

Via CLI

  1. Update VERSION in lib/rspock/version.rb and run bundle install to regenerate Gemfile.lock, commit, open a PR, and merge to main
  2. Tag and push:
    git checkout main && git pull
    git tag v2.0.0
    git push origin v2.0.0
    

In both cases, the release workflow validates that the tag matches version.rb, builds the gem, and publishes it to rubygems.org via Trusted Publishing (no API key needed). If there's a mismatch, the workflow fails before publishing.

One-time setup

Configure the gem as a trusted publisher on rubygems.org so that the release workflow can publish automatically. See the Trusted Publishing guide for details.

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/rspockframework/rspock. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the Contributor Covenant code of conduct.

License

The gem is available as open source under the terms of the MIT License.

Code of Conduct

Everyone interacting in the RSpock project’s codebases, issue trackers, chat rooms and mailing lists is expected to follow the code of conduct.