An In-Depth Guide to Testing Ethereum Smart Contracts
Part Five: Tools for Effective Testing
This article is part of a series. If you haven’t yet, check out the previous articles:
Part One: Why we Test
Part Two: Core Concepts of Testing
Part Three: Writing Basic Tests
Part Four: Running Your Tests
Part Five: Tools and Techniques for Effective Testing
Part Six: Parametrization and Property-Based Testing
Part Seven: Stateful Testing
Dealing with Tests that cannot Pass
At some point you may find yourself dealing with a test that cannot pass. Maybe you are unsure of the reason why but need to focus on something else for now, maybe you’re in the middle of a large-scale refactor and so the failure is expected. Either way it can be annoying to see the test fail repeatedly.
In these cases, pytest provides markers to help you deal with the situation:
@pytest.mark.skip(reason="no way of currently testing this")
skip marker tells pytest not to run the test. You can optionally provide a reason why you are skipping it, to help remember why when you come back to it later.
xfail marker indicates that the test should still run, but the outcome is inverted. No traceback is given for the failing test, and the console output is
x for “expected to fail”. If the test passes it is interpreted as a failure, or “unexpected pass”.
chain object provides methods for fast forwarding the clock and mining new blocks. These are invaluable when testing time dependent behaviors.
chain.sleep fast forwards the clock in the local environment. It accepts one argument — the number of seconds to advance.
chain.mine mines 1 or more new, empty blocks.
To explore developer revert comments in greater detail, check out this article.
When testing we want to ensure that every branch is correctly hit — and the best way to do this with reverting transactions is to expect a specific error string.
Unfortunately, each revert string adds a minimum 20,000 gas to your contract deployment cost, and increases the cost for a function to execute. Including an error message for every
revert statement is often impractical and sometimes simply not possible due to the block gas limit.
Brownie solves this issue by letting you include revert strings as source code comments. The comments are not included in the bytecode, but are still accessible via
TransactionReceipt.revert_msg or the
brownie.reverts context manager.
Using developer revert comments is simple. At the end of a line of code that includes a potential revert, add a comment starting with
// dev: in Solidity or
# dev: in Vyper. Brownie detects these comments and adds them automatically during testing.
Here is an example admin function where revert comments are used in place of actual error strings:
And here is a test that targets a specific revert comment:
In “Part Six: Parametrization and Property-Based Testing”, we explore how parametrization can strengthen our test cases and help us find unexpected edge cases.
You can also follow the Brownie Twitter account, read my other Medium articles, and join us on Gitter.