Skip to content

Commit c42ca0e

Browse files
committed
Update README with notes on compare_errors
1 parent 1bcbab1 commit c42ca0e

1 file changed

Lines changed: 33 additions & 1 deletion

File tree

README.md

Lines changed: 33 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -108,6 +108,38 @@ class MyWidget
108108
end
109109
```
110110

111+
If either the control block or candidate block raises an error, Scientist compares the two observations' classes and messages using `==`. To override this behavior, use `compare_error` to define how to compare observed errors instead:
112+
113+
```ruby
114+
class MyWidget
115+
include Scientist
116+
117+
def slug_from_login(login)
118+
science "slug_from_login" do |e|
119+
e.use { User.slug_from_login login } # returns String instance or ArgumentError
120+
e.try { UserService.slug_from_login login } # returns String instance or ArgumentError
121+
122+
compare_error_message_and_class = -> (control, candidate) do
123+
control.class == candidate.class &&
124+
control.message == candidate.message
125+
end
126+
127+
compare_argument_errors = -> (control, candidate) do
128+
control.class == ArgumentError &&
129+
candidate.class == ArgumentError &&
130+
control.message.start_with?("Input has invalid characters") &&
131+
candidate.message.star_with?("Invalid characters in input")
132+
end
133+
134+
e.compare_error do |control, candidate|
135+
compare_error_message_and_class.call(control, candidate) ||
136+
compare_argument_errors.call(control, candidate)
137+
end
138+
end
139+
end
140+
end
141+
```
142+
111143
### Adding context
112144

113145
Results aren't very useful without some way to identify them. Use the `context` method to add to or retrieve the context for an experiment:
@@ -225,7 +257,7 @@ def admin?(user)
225257
end
226258
```
227259

228-
The ignore blocks are only called if the *values* don't match. If one observation raises an exception and the other doesn't, it's always considered a mismatch. If both observations raise different exceptions, that is also considered a mismatch.
260+
The ignore blocks are only called if the *values* don't match. Unless a `compare_error` comparator is defined, two cases are considered mismatches: a) one observation raising an exception and the other not, b) observations raising exceptions with different classes or messages.
229261

230262
### Enabling/disabling experiments
231263

0 commit comments

Comments
 (0)