Reporting is an essential element of BDD as it allows to monitor the outcome of the scenarios that have been run. At the heart of JBehave's reporting is the ScenarioReporter, which is used to report events as they occur.
Currently, the scenario reporters supported are:
Let's see an example of how we can configure the scenario reporters. We use the DelegatingScenarioReporter as a proxy for the reporters we want to configure, in the example below a plain text reporter (both to System.out and to a file) and an HTML reporter to a file:
public TraderScenario(final Class scenarioClass) { final ScenarioNameResolver resolver = new UnderscoredCamelCaseResolver(".scenario"); useConfiguration(new MostUsefulConfiguration() { @Override public ScenarioDefiner forDefiningScenarios() { return new ClasspathScenarioDefiner(resolver, new PatternScenarioParser(keywords())); } @Override public ScenarioReporter forReportingScenarios() { return new ScenarioReporterBuilder(new FilePrintStreamFactory(scenarioClass, resolver)) .outputTo("target/jbehave-reports").outputAsAbsolute(true) // only required with Ant (cf note below) .withDefaultFormats() // add default formats .with(CONSOLE) .with(TXT) .with(HTML) .with(XML) .build(); } }); StepsConfiguration configuration = new StepsConfiguration(); addSteps(createSteps(configuration)); } protected CandidateSteps[] createSteps(StepsConfiguration configuration) { return new StepsFactory(configuration).createCandidateSteps(new TraderSteps(new TradingService()), new BeforeAfterSteps()); }For a fully-working example of reporting configuration you may consult the trader example. Note that the configuration only needs to be done in one place, i.e. the parent Scenario class, and not repeated in every Scenario class that maps to a textual scenario. In general, the specification of the output directory is optional. It is necessary when running with Ant as a workaround for Ant's classloading strategy which uses the Ant Project.class ProtectionDomain, which in turn leads to the $ANT_HOME/lib as the code source location. A better solution would be to override the hacky AntClassLoader behaviour, as also noted by Ant folks in the AntClassLoader#defineClassFromData method. A user-friendly way to do is being investigated.
Note that we use the ScenarioReporterBuilder to implement a builder pattern for file-based reporters via the FilePrintStreamFactory, in which we inject the ScenarioNameResolver to derive the report file names from the scenario class, using the same name resolution mechanism used for mapping Java classes and textual scenarios. So, e.g., if the scenario class is com.example.MyScenario, we'll end up with file report outputs of the form: com.example.my_scenario.[format] (where format is any of txt,html,xml in the example above).
The file-based print stream factory for the scenario reporting requires the scenario class to be provided in order to derive from it the output file paths (with the appropriate format extension). If you want to migrate multiple scenarios to using file-based reporting, an intermediate stepping stone might be to provide as a scenario class the parent class in which you build the scenario reporter (TraderScenario.class in the example), which will result in all reporting (for a given format) to be appended to the same file. Then gradually you can refactor your scenario classes to provide their class to the parent (as the example shows). Otherwise, you may want to keep output in a single file, the choice is yours.The builder provides defaults for all the formats supported, but if the user needs to create a bespoke instance of a reporter for a given format, it can be easily done by overriding the default. E.g. to override the reporter for TXT format to use a ".text" extension (a possibly keywords for a different Locale):
ScenarioReporter reporter = new ScenarioReporterBuilder(factory){ public ScenarioReporter reporterFor(Format format){ switch (format) { case TXT: factory.useConfiguration(new FileConfiguration("text")); return new PrintStreamScenarioReporter(factory.getPrintStream(), new Properties(), new I18nKeyWords(Locale.ITALIAN), true); default: return super.reporterFor(format); } }
The generation of the reports is only the first part of a complete HTML-based reporting solution. Next we need to render the reports, aggregating all the ones that have been configured and generated in a given output directory, and presenting a collective index view for all formats configured. Moreover, we need to style the view, both for HTML and non-HTML report formats.
The rendering is the responsibility of the ReportRenderer. JBehave provides an implementation (FreemarkerReportRenderer) based on Freemarker to allow a templateable and easily styleable way to render the report views.
The default resources required for the report rendering are bundled in the jbehave-core.jar (extract ftl/*.ftl, js/*.js and style/*.css) but can be overridden. The FTL files need to be the classpath for the FreemarkerReportRenderer to find them, while the look and feel resources (js/*.js and style/*.css files) need to be copied to the target/jbehave-reports/rendered directory (or wherever the index page has been rendered to) Also note that the default style makes use of images found in the jbehave-site-resources.jar.
Also, note that the report formats configured should match ones found in the Ant or Maven execution for the report rendering task or goal (c.f. running scenarios for examples).
The scenario statistics report above is treated in the rendering slightly differently from other reports, in that the statistics are displayed on the index page, if available. To ensure they are always available the scenario reporter builder is configured to have stats as a default format (although the default formats need to be added to the builder).
Default formats allow users to define a builder that can be used in multiple configurations without having to repeat tediously all the formats required, if these are used consistently. By default, "stats" is the only default format (used for collecting reporting statistics). To modify simply override the method withDefaultFormats(). E.g. to add "txt" as a default format:
new ScenarioReporterBuilder(factory){ protected ScenarioReporterBuilder withDefaultFormats() { return with(Format.STATS).with(Format.TXT); } }
By default, JBehave outputs file reports to the directory jbehave-reports (relative to the scenario class code source location, e.g. the target directory in Maven), but this can be changed via the builder:
new ScenarioReporterBuilder(factory).outputTo("my-reports")
In some cases, i.e. when using with Ant, the class code source location may not be properly set, or different from what is expected. In this case, one can tell the builder to treat the output directory as an absolute path:
new ScenarioReporterBuilder(factory).outputTo("target/my-reports").outputAsAbsolute(true)
If the default formats or the output file directory are modified via the builder, then correspondingly we need to inform the report render of these changes. See running scenarios for details on how to configure these changes.