Forge – How to Use Forge Coverage Effectively?

forgefoundrytestingunittesting

I am trying to get the test coverage for our Smart Contracts. Couple of questions I have.

  1. While running the coverage command, it doesn't include all the test cases in the folder, is it because we have written test cases in a modular form, i.e. there are folders inside folders and then test cases, so that the structure remains clean when the codebase grows.

The files included in the report:
enter image description here

The actual files I have:

enter image description here

  1. While using the --report summary flag it creates a JSON file that is not practically readable to a human eye, what is the best way to extract the details given in that JSON file?

JSON File :

enter image description here

  1. Lastly, are there some docs, etc. for more information on the usage of forge coverage, as the foundry book is pretty limited.

Best Answer

The answers below refer to forge 0.2.0, the latest release at the time of the writing.

foundryup: installed - forge 0.2.0 (2b2a499 2024-01-31T17:41:20.967549332Z)
foundryup: installed - cast 0.2.0 (2b2a499 2024-01-31T17:41:20.972189858Z)
foundryup: installed - anvil 0.2.0 (2b2a499 2024-01-31T17:41:20.965275165Z)
foundryup: installed - chisel 0.2.0 (2b2a499 2024-01-31T17:41:20.969731196Z)

1. Running tests to get the coverage

The coverage operation executes all the tests in the test files, including those in subfolders of the test folder. But you need to put the tests in the test folder (or subfolders) and the source to be tested in the src folder (or subfolders).

When working correctly, what you see in the report generated by the forge coverage command are the results of testing the source code files present in the src folder running the tests in the test folder.

You should not see the file containing the tests listed in that report.

In your case, looking at the screenshot, I see the test cases files (.t.sol) listed in the report, so it means Forge thinks those files must also be included in the coverage report. And as a side note, nothing is executing tests over the test files, so this is the reason why you get 0% for those files.

You need to fix your project structure: remove those test files from being the target of the coverage, put them under a dedicated test folder, move the source code under a src folder, change the importing path accordingly to find the source .sol contracts again.

You project should look something like this:

/src/contracts/PushComm/PushCommV2_5.sol
...
/test/BaseTest.t.sol
/test/PushCore/...
...
/foundry.toml
/README.md
...

Using the src folder as the root folder for your code and the test folder on the same level for the test case is a best practice, but you can change that editing the foundry.toml configuration file; still, I suggest sticking with the best practices to have fewer problems with the repo maintenance.

[profile.default]
src = "src"
out = "out"
libs = ["lib"]

See more config options at the official repo.

2. Coverage reporting options

I'm not sure how you produced that JSON, but the coverage --report option works like this:

  • summary: the default output in the form of a table with colors and data
  • lcov: coverage in LCOV format; you can use multiple tools to decode that info (i.e., generating HTML reports from that) and also dedicated VS Code plugins like Coverage Gutters
  • debug: text format, with some details about the uncovered code parts and what's happening under the hood
  • bytecode: it does not work correctly for me, and I see I'm not alone having issues, and this value is not mentioned in the doc even if it's already mentioned in the command options, so I suggest not to use it until it is stable.

3. Documentation and tutorials

Official documentation is not super detailed, but you can find on the Internet articles like this one from Rohanzarathustra, that let you play around with all the possibilities of the coverage command.

Please consider the program is labelled 0.2 for a reason, so expect improvements, changes and breaking changes in the future. ;)

Related Topic