net.thucydides.core.model
Class TestOutcome

java.lang.Object
  net.thucydides.core.model.TestOutcome

public class TestOutcome
extends Object

Represents the results of a test (or "scenario") execution. This includes the narrative steps taken during the test, screenshots at each step, the results of each step, and the overall result. A test scenario can be associated with a user story using the UserStory annotation. A TestOutcome is stored as an XML file after a test is executed. When the aggregate reports are generated, the test outcome XML files are loaded into memory and processed.

Author:

johnsmart

Nested Class Summary

static class

TestOutcome.StepCountBuilder

Constructor Summary

protected	TestOutcome(long startTime, long duration, String title, String methodName, Class<?> testCase, List<TestStep> testSteps, Set<String> issues, Set<String> additionalIssues, Set<TestTag> tags, Story userStory, Throwable testFailureCause, TestResult annotatedResult, com.google.common.base.Optional<String> qualifier)
	TestOutcome(String methodName) The title is immutable once set.
	TestOutcome(String methodName, Class<?> testCase)
protected	TestOutcome(String methodName, Class<?> testCase, Story userStory) A test outcome should relate to a particular test class or user story class.

Method Summary

void	addIssues(List<String> issues)
TestOutcome	andStep(TestStep step)
Integer	countTestSteps()
void	endGroup() Finish the current group.
static TestOutcome	forTest(String methodName, Class<?> testCase) Create a new test outcome instance for a given test class or user story.
static TestOutcome	forTestInStory(String testName, Class<?> testCase, Story story)
static TestOutcome	forTestInStory(String testName, Story story)
TestStep	getCurrentGroup()
TestStep	getCurrentStep()
Long	getDuration()
double	getDurationInSeconds()
EnvironmentVariables	getEnvironmentVariables()
Integer	getFailureCount()
ApplicationFeature	getFeature() Get the feature that includes the user story tested by this test.
List<TestStep>	getFlattenedTestSteps()
String	getFormattedIssues()
Integer	getIgnoredCount()
List<String>	getIssueKeys()
Set<String>	getIssues()
TestStep	getLastStep()
List<TestStep>	getLeafTestSteps()
String	getMethodName()
Integer	getNestedStepCount()
double	getOverallStability()
Integer	getPendingCount()
String	getQualifiedMethodName()
com.google.common.base.Optional<String>	getQualifier()
int	getRecentFailCount()
int	getRecentPassCount()
int	getRecentPendingCount()
double	getRecentStability()
Long	getRecentTestRunCount()
String	getReportName()
String	getReportName(ReportType type)
TestResult	getResult() The outcome of the acceptance test, based on the outcome of the test steps.
String	getScreenshotReportName()
List<Screenshot>	getScreenshots()
String	getSessionId()
String	getSimpleReportName(ReportType type)
Integer	getSkippedCount()
Integer	getSkippedOrIgnoredCount()
TestStatistics	getStatistics()
Integer	getStepCount()
String	getStoryTitle()
Integer	getSuccessCount()
Set<TestTag>	getTags()
Class<?>	getTestCase()
Throwable	getTestFailureCause()
List<TestStep>	getTestSteps() An acceptance test is made up of a series of steps.
String	getTitle() Return the human-readable name for this test.
String	getTitleWithLinks()
Story	getUserStory()
String	getVideoLink() Returns the link to the associated video (e.g.
Boolean	isFailure()
Boolean	isPending()
void	isRelatedToIssue(String issue)
Boolean	isSkipped()
Boolean	isSuccess()
void	lastStepFailedWith(StepFailure failure)
void	recordDuration()
TestOutcome	recordStep(TestStep step) Add a test step to this acceptance test.
void	setAnnotatedResult(TestResult annotatedResult)
void	setDuration(long duration)
void	setEnvironmentVariables(EnvironmentVariables environmentVariables)
void	setSessionId(String sessionId)
void	setStatistics(TestStatistics statistics)
void	setTags(Set<TestTag> tags)
void	setTestFailureCause(Throwable cause)
void	setTitle(String title)
void	setUserStory(Story story)
void	startGroup() Turns the current step into a group.
void	startGroup(String groupName) Deprecated.
String	toString()
TestOutcome	usingIssueTracking(IssueTracking issueTracking)
TestOutcome	withQualifier(String qualifier)
TestOutcome	withStep(TestStep step)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

TestOutcome

public TestOutcome(String methodName)

The title is immutable once set. For convenience, you can create a test run directly with a title using this constructor.

Parameters:

methodName - The name of the Java method that implements this test.

TestOutcome

public TestOutcome(String methodName,
                   Class<?> testCase)

TestOutcome

protected TestOutcome(String methodName,
                      Class<?> testCase,
                      Story userStory)

A test outcome should relate to a particular test class or user story class.

Parameters:

methodName - The name of the Java method implementing this test, if the test is a JUnit or TestNG test (for example)

testCase - The test class that contains this test method, if the test is a JUnit or TestNG test

userStory - If the test is not implemented by a Java class (e.g. an easyb story), we may just use the Story class to represent the story in which the test is implemented.

TestOutcome

protected TestOutcome(long startTime,
                      long duration,
                      String title,
                      String methodName,
                      Class<?> testCase,
                      List<TestStep> testSteps,
                      Set<String> issues,
                      Set<String> additionalIssues,
                      Set<TestTag> tags,
                      Story userStory,
                      Throwable testFailureCause,
                      TestResult annotatedResult,
                      com.google.common.base.Optional<String> qualifier)

Method Detail

usingIssueTracking

public TestOutcome usingIssueTracking(IssueTracking issueTracking)

setEnvironmentVariables

public void setEnvironmentVariables(EnvironmentVariables environmentVariables)

getEnvironmentVariables

public EnvironmentVariables getEnvironmentVariables()

forTest

public static TestOutcome forTest(String methodName,
                                  Class<?> testCase)

Create a new test outcome instance for a given test class or user story.

Parameters:

methodName - The name of the Java method implementing this test,

testCase - The JUnit or TestNG test class that contains this test method

Returns:

A new TestOutcome object for this test.

withQualifier

public TestOutcome withQualifier(String qualifier)

getMethodName

public String getMethodName()

Returns:

The name of the Java method implementing this test, if the test is implemented in Java.

forTestInStory

public static TestOutcome forTestInStory(String testName,
                                         Story story)

forTestInStory

public static TestOutcome forTestInStory(String testName,
                                         Class<?> testCase,
                                         Story story)

toString

public String toString()

Overrides:

toString in class Object

getTitle

public String getTitle()

Return the human-readable name for this test. This is derived from the test name for tests using a Java implementation, or can also be defined using the Title annotation.

Returns:

the human-readable name for this test.

getTitleWithLinks

public String getTitleWithLinks()

getStoryTitle

public String getStoryTitle()

getReportName

public String getReportName(ReportType type)

getSimpleReportName

public String getSimpleReportName(ReportType type)

getReportName

public String getReportName()

getScreenshotReportName

public String getScreenshotReportName()

getTestSteps

public List<TestStep> getTestSteps()

An acceptance test is made up of a series of steps. Each step is in fact a small test, which follows on from the previous one. The outcome of the acceptance test as a whole depends on the outcome of all of the steps.

Returns:

A list of top-level test steps for this test.

getScreenshots

public List<Screenshot> getScreenshots()

getFlattenedTestSteps

public List<TestStep> getFlattenedTestSteps()

getLeafTestSteps

public List<TestStep> getLeafTestSteps()

getResult

public TestResult getResult()

The outcome of the acceptance test, based on the outcome of the test steps. If any steps fail, the test as a whole is considered a failure. If any steps are pending, the test as a whole is considered pending. If all of the steps are ignored, the test will be considered 'ignored'. If all of the tests succeed except the ignored tests, the test is a success. The test result can also be overridden using the 'setResult()' method.

Returns:

The outcome of this test.

recordStep

public TestOutcome recordStep(TestStep step)

Add a test step to this acceptance test.

Parameters:

step - a completed step to be added to this test outcome.

Returns:

this TestOucome insstance - this is a convenience to allow method chaining.

withStep

public TestOutcome withStep(TestStep step)

andStep

public TestOutcome andStep(TestStep step)

getFeature

public ApplicationFeature getFeature()

Get the feature that includes the user story tested by this test. If no user story is defined, no feature can be returned, so the method returns null. If a user story has been defined without a class (for example, one that has been reloaded), the feature will be built using the feature name and id in the user story.

Returns:

The Feature defined for this TestOutcome, if any

setTitle

public void setTitle(String title)

startGroup

@Deprecated
public void startGroup(String groupName)

Deprecated.

Creates a new step with this name and immediately turns it into a step group. TODO: Review where this is used, as it is mainly for backward compatibility.

getQualifier

public com.google.common.base.Optional<String> getQualifier()

startGroup

public void startGroup()

Turns the current step into a group. Subsequent steps will be added as children of the current step.

endGroup

public void endGroup()

Finish the current group. Subsequent steps will be added after the current step.

getCurrentStep

public TestStep getCurrentStep()

Returns:

The current step is the last step in the step list, or the last step in the children of the current step group.

getLastStep

public TestStep getLastStep()

getCurrentGroup

public TestStep getCurrentGroup()

setUserStory

public void setUserStory(Story story)

setTestFailureCause

public void setTestFailureCause(Throwable cause)

getTestFailureCause

public Throwable getTestFailureCause()

setAnnotatedResult

public void setAnnotatedResult(TestResult annotatedResult)

getIssues

public Set<String> getIssues()

getTestCase

public Class<?> getTestCase()

addIssues

public void addIssues(List<String> issues)

getFormattedIssues

public String getFormattedIssues()

isRelatedToIssue

public void isRelatedToIssue(String issue)

lastStepFailedWith

public void lastStepFailedWith(StepFailure failure)

getTags

public Set<TestTag> getTags()

setTags

public void setTags(Set<TestTag> tags)

getIssueKeys

public List<String> getIssueKeys()

getQualifiedMethodName

public String getQualifiedMethodName()

getStepCount

public Integer getStepCount()

getNestedStepCount

public Integer getNestedStepCount()

getSuccessCount

public Integer getSuccessCount()

getFailureCount

public Integer getFailureCount()

getIgnoredCount

public Integer getIgnoredCount()

getSkippedOrIgnoredCount

public Integer getSkippedOrIgnoredCount()

getSkippedCount

public Integer getSkippedCount()

getPendingCount

public Integer getPendingCount()

isSuccess

public Boolean isSuccess()

isFailure

public Boolean isFailure()

isPending

public Boolean isPending()

isSkipped

public Boolean isSkipped()

getUserStory

public Story getUserStory()

recordDuration

public void recordDuration()

setDuration

public void setDuration(long duration)

getDuration

public Long getDuration()

getDurationInSeconds

public double getDurationInSeconds()

Returns:

The total duration of all of the tests in this set in milliseconds.

getVideoLink

public String getVideoLink()

Returns the link to the associated video (e.g. from Saucelabs) for this test.

Returns:

a URL.

getSessionId

public String getSessionId()

setSessionId

public void setSessionId(String sessionId)

countTestSteps

public Integer countTestSteps()

setStatistics

public void setStatistics(TestStatistics statistics)

getStatistics

public TestStatistics getStatistics()

getOverallStability

public double getOverallStability()

getRecentStability

public double getRecentStability()

getRecentTestRunCount

public Long getRecentTestRunCount()

getRecentPassCount

public int getRecentPassCount()

getRecentFailCount

public int getRecentFailCount()

getRecentPendingCount

public int getRecentPendingCount()