Package net.thucydides.core.model

Class TestOutcome

java.lang.Object
net.thucydides.core.model.TestOutcome
public class TestOutcome
extends java.lang.Object
Represents the results of a test (or "scenario") execution. This includes the narrative steps taken during the test, screenshots at each step, the results of each step, and the overall result. A test getscenario can be associated with a user story using the UserStory annotation. A TestOutcome is stored after a test is executed. When the aggregate reports are generated, the test outcome files are loaded into memory and processed.
Author:
johnsmart

  • Constructor Details

    • TestOutcome

      public TestOutcome​(java.lang.String name)
      The title is immutable once set. For convenience, you can create a test run directly with a title using this constructor.
      Parameters:
      name - The name of the Java method that implements this test.

    • TestOutcome

      public TestOutcome​(java.lang.String name, java.lang.Class<?> testCase)

    • TestOutcome

      public TestOutcome​(java.lang.String name, java.lang.Class<?> testCase, EnvironmentVariables environmentVariables)
      Create a test outcome based on a test method in a test class. The requirement type will be derived if possible using the class package.
      Parameters:
      name -
      testCase -

    • TestOutcome

      protected TestOutcome​(java.lang.String name, java.lang.Class<?> testCase, Story userStory)

    • TestOutcome

      protected TestOutcome​(java.lang.String name, java.lang.Class<?> testCase, Story userStory, EnvironmentVariables environmentVariables)
      A test outcome should relate to a particular test class or user story class.
      Parameters:
      name - The name of the Java method implementing this test, if the test is a JUnit or TestNG test (for example)
      testCase - The test class that contains this test method, if the test is a JUnit or TestNG test
      userStory - If the test is not implemented by a Java class (e.g. an easyb story), we may just use the Story class to represent the story in which the test is implemented.

    • TestOutcome

      protected TestOutcome​(java.time.ZonedDateTime startTime, long duration, java.lang.String title, java.lang.String description, java.lang.String name, java.lang.String id, java.lang.Class<?> testCase, java.util.List<TestStep> testSteps, java.util.List<java.lang.String> issues, java.util.List<java.lang.String> additionalIssues, java.util.List<CastMember> actors, java.util.Set<TestTag> tags, Story userStory, FailureCause testFailureCause, java.lang.String testFailureClassname, java.lang.String testFailureMessage, java.lang.String testFailureSummary, TestResult annotatedResult, DataTable dataTable, java.util.Optional<java.lang.String> qualifier, java.lang.String driver, boolean manualTest, boolean isManualTestingUpToDate, java.lang.String lastTested, java.util.List<java.lang.String> testEvidence, java.lang.String projectKey, EnvironmentVariables environmentVariables, ExternalLink externalLink, java.lang.String context)

  • Method Details

    • inEnvironment

      public static net.thucydides.core.model.TestOutcome.TestOutcomeWithEnvironmentBuilder inEnvironment​(EnvironmentVariables environmentVariables)

    • calculateDynamicFieldValues

      public void calculateDynamicFieldValues()
      Fix the values of synthetic fields for serialization purposes

    • usingIssueTracking

      public TestOutcome usingIssueTracking​(IssueTracking issueTracking)

    • setToManual

      public TestOutcome setToManual()

    • withTestSource

      public TestOutcome withTestSource​(java.lang.String testSource)

    • withFlagProvider

      protected TestOutcome withFlagProvider​(FlagProvider flagProvider)
      Set the current flag provider; only used for testing purposes.

    • setEnvironmentVariables

      public void setEnvironmentVariables​(EnvironmentVariables environmentVariables)

    • getEnvironmentVariables

      public EnvironmentVariables getEnvironmentVariables()

    • copy

      public TestOutcome copy()

    • forTest

      public static TestOutcome forTest​(java.lang.String methodName, java.lang.Class<?> testCase)
      Create a new test outcome instance for a given test class or user story.
      Parameters:
      methodName - The name of the Java method implementing this test,
      testCase - The JUnit or TestNG test class that contains this test method
      Returns:
      A new TestOutcome object for this test.

    • withQualifier

      public TestOutcome withQualifier​(java.lang.String qualifier)

    • withIssues

      public TestOutcome withIssues​(java.util.List<java.lang.String> issues)

    • withTags

      public TestOutcome withTags​(java.util.Set<TestTag> tags)

    • withMethodName

      public TestOutcome withMethodName​(java.lang.String methodName)

    • getName

      public java.lang.String getName()
      Returns:
      The name of the Java method implementing this test, if the test is implemented in Java.

    • forTestInStory

      public static TestOutcome forTestInStory​(java.lang.String testName, Story story)

    • forTestInStory

      public static TestOutcome forTestInStory​(java.lang.String testName, java.lang.Class<?> testCase, Story story)

    • toString

      public java.lang.String toString()
      Overrides:
      toString in class java.lang.Object

    • getTitle

      public java.lang.String getTitle()
      Return the human-readable name for this test. This is derived from the test name for tests using a Java implementation, or can also be defined using the Title annotation.
      Returns:
      the human-readable name for this test.

    • getTitle

      public java.lang.String getTitle​(boolean qualified)

    • getUnqualified

      public TitleBuilder getUnqualified()

    • getQualified

      public TitleBuilder getQualified()

    • setAllStepsTo

      public void setAllStepsTo​(TestResult result)

    • addDataFrom

      public void addDataFrom​(DataTable newDataTable)

    • clearForcedResult

      public void clearForcedResult()

    • setDriver

      public void setDriver​(java.lang.String driver)

    • getDriver

      public java.lang.String getDriver()

    • resetFailingStepsCausedBy

      public void resetFailingStepsCausedBy​(java.lang.Class<? extends java.lang.Throwable> expected)

    • replace

      public TestOutcome.StepReplacer replace​(java.util.List<TestStep> stepsToMerge)

    • mergeMostRecentSteps

      public void mergeMostRecentSteps​(int maxStepsToMerge)

    • updateOverallResults

      public void updateOverallResults()

    • getFailingStep

      public java.util.Optional<TestStep> getFailingStep()

    • getId

      public java.lang.String getId()

    • getParentId

      public java.lang.String getParentId()

    • withId

      public TestOutcome withId​(java.lang.String id)

    • updateTopLevelStepResultsTo

      public void updateTopLevelStepResultsTo​(TestResult result)

    • getTestFailureSummary

      public java.lang.String getTestFailureSummary()

    • getFlakyTestFailureCause

      public TestFailureCause getFlakyTestFailureCause()

    • setFlakyTestFailureCause

      public void setFlakyTestFailureCause​(TestFailureCause flakyTestFailureCause)

    • hasTagWithName

      public boolean hasTagWithName​(java.lang.String tagName)

    • hasTagWithType

      public boolean hasTagWithType​(java.lang.String tagType)

    • hasTagWithTypes

      public boolean hasTagWithTypes​(java.util.List<java.lang.String> tagTypes)

    • getDataTableRowCount

      public int getDataTableRowCount()

    • getTestStepCount

      public int getTestStepCount()

    • castActor

      public void castActor​(java.lang.String name)

    • assignFact

      public void assignFact​(java.lang.String name, java.lang.String fact)

    • assignAbility

      public void assignAbility​(java.lang.String name, java.lang.String ability)

    • assignDescriptionToActor

      public void assignDescriptionToActor​(java.lang.String name, java.lang.String description)

    • setManualTestEvidence

      public void setManualTestEvidence​(java.util.List<java.lang.String> manualTestEvidence)

    • getManualTestEvidence

      public java.util.List<java.lang.String> getManualTestEvidence()

    • getRenderedManualTestEvidence

      public java.util.List<ManualTestEvidence> getRenderedManualTestEvidence()

    • setLink

      public void setLink​(ExternalLink externalLink)

    • hasNoSteps

      public boolean hasNoSteps()

    • withSteps

      public TestOutcome withSteps​(java.util.List<TestStep> childSteps)

    • isUnsuccessful

      public boolean isUnsuccessful()

    • isTitleWithIssues

      public boolean isTitleWithIssues()

    • setDescription

      public void setDescription​(java.lang.String description)

    • setBackgroundDescription

      public void setBackgroundDescription​(java.lang.String description)

    • setBackgroundTitle

      public void setBackgroundTitle​(java.lang.String title)

    • getDescription

      public java.lang.String getDescription()

    • getBackgroundDescription

      public java.lang.String getBackgroundDescription()

    • getBackgroundTitle

      public java.lang.String getBackgroundTitle()

    • getDescriptionText

      public java.util.Optional<java.lang.String> getDescriptionText()
      Tests may have a description. This can be defined with the scenarios (e.g. in the .feature files for Cucumber) or defined elsewhere, such as in JIRA for manual tests.

    • toJson

      public java.lang.String toJson()

    • getTitleWithLinks

      public java.lang.String getTitleWithLinks()

    • getStoryTitle

      public java.lang.String getStoryTitle()

    • getPath

      public java.lang.String getPath()

    • getPathId

      public java.lang.String getPathId()

    • getReportName

      public java.lang.String getReportName​(ReportType type)

    • getSimpleReportName

      public java.lang.String getSimpleReportName​(ReportType type)

    • getHtmlReport

      public java.lang.String getHtmlReport()

    • getReportName

      public java.lang.String getReportName()

    • getScreenshotReportName

      public java.lang.String getScreenshotReportName()

    • getTestSteps

      public java.util.List<TestStep> getTestSteps()
      An acceptance test is made up of a series of steps. Each step is in fact a small test, which follows on from the previous one. The outcome of the acceptance test as a whole depends on the outcome of all of the steps.
      Returns:
      A list of top-level test steps for this test.

    • getLatestTopLevelTestStep

      public java.util.Optional<TestStep> getLatestTopLevelTestStep()

    • hasScreenshots

      public boolean hasScreenshots()

    • hasRestQueries

      public boolean hasRestQueries()

    • getScreenshotAndHtmlSources

      public java.util.List<ScreenshotAndHtmlSource> getScreenshotAndHtmlSources()

    • getScreenshots

      public java.util.List<Screenshot> getScreenshots()

    • getStepScreenshots

      public java.util.List<Screenshot> getStepScreenshots()
      Find the first and last screenshots for each aggregate step, and every screenshots for leaf steps.

    • hasNonStepFailure

      public boolean hasNonStepFailure()

    • getFlattenedTestSteps

      public java.util.List<TestStep> getFlattenedTestSteps()

    • getLeafTestSteps

      public java.util.List<TestStep> getLeafTestSteps()

    • getResult

      public TestResult getResult()
      The outcome of the acceptance test, based on the outcome of the test steps. If any steps fail, the test as a whole is considered a failure. If any steps are pending, the test as a whole is considered pending. If all of the steps are ignored, the test will be considered 'ignored'. If all of the tests succeed except the ignored tests, the test is a success. The test result can also be overridden using the 'setResult()' method.
      Returns:
      The outcome of this test.

    • recordSteps

      public TestOutcome recordSteps​(java.util.List<TestStep> steps)

    • recordChildSteps

      public TestOutcome recordChildSteps​(java.util.List<TestStep> steps)

    • recordStep

      public TestOutcome recordStep​(TestStep step)
      Add a test step to this acceptance test.
      Parameters:
      step - a completed step to be added to this test outcome.
      Returns:
      this TestOucome insstance - this is a convenience to allow method chaining.

    • getFeature

      public ApplicationFeature getFeature()
      Get the feature that includes the user story tested by this test. If no user story is defined, no feature can be returned, so the method returns null. If a user story has been defined without a class (for example, one that has been reloaded), the feature will be built using the feature name and id in the user story.
      Returns:
      The Feature defined for this TestOutcome, if any

    • setTitle

      public void setTitle​(java.lang.String title)

    • startGroup

      @Deprecated public void startGroup​(java.lang.String groupName)
      Deprecated.
      Creates a new step with this name and immediately turns it into a step group.

    • getQualifier

      public java.util.Optional<java.lang.String> getQualifier()

    • startGroup

      public void startGroup()
      Turns the current step into a group. Subsequent steps will be added as children of the current step.

    • endGroup

      public void endGroup()
      Finish the current group. Subsequent steps will be added after the current step.

    • currentStep

      public java.util.Optional<TestStep> currentStep()
      Returns:
      The current step is the last step in the step list, or the last step in the children of the current step group.

    • lastStep

      public TestStep lastStep()

    • currentGroup

      public TestStep currentGroup()

    • setUserStory

      public void setUserStory​(Story story)

    • determineTestFailureCause

      public void determineTestFailureCause​(java.lang.Throwable cause)

    • appendTestFailure

      public void appendTestFailure​(TestFailureCause failureCause)

    • testStepWithDescription

      public java.util.Optional<TestStep> testStepWithDescription​(java.lang.String expectedDescription)

    • getTestFailureCause

      public FailureCause getTestFailureCause()

    • getTestFailureErrorType

      public java.lang.String getTestFailureErrorType()

    • getNestedTestFailureCause

      public FailureCause getNestedTestFailureCause()

    • firstStepWithErrorMessage

      public java.util.Optional<TestStep> firstStepWithErrorMessage()

    • testFailureMessage

      public java.util.Optional<java.lang.String> testFailureMessage()

    • getErrorMessage

      public java.lang.String getErrorMessage()

    • getConciseErrorMessage

      public java.lang.String getConciseErrorMessage()

    • setTestFailureMessage

      public void setTestFailureMessage​(java.lang.String testFailureMessage)

    • getTestFailureMessage

      public java.lang.String getTestFailureMessage()

    • getTestFailureClassname

      public java.lang.String getTestFailureClassname()

    • setAnnotatedResult

      public void setAnnotatedResult​(TestResult annotatedResult)

    • overrideAnnotatedResult

      public void overrideAnnotatedResult​(TestResult annotatedResult)

    • setResult

      public void setResult​(TestResult annotatedResult)

    • overrideResult

      public void overrideResult​(TestResult result)

    • withResult

      public TestOutcome withResult​(TestResult annotatedResult)

    • getAnnotatedResult

      public TestResult getAnnotatedResult()

    • getAdditionalVersions

      public java.util.List<java.lang.String> getAdditionalVersions()

    • getAdditionalIssues

      public java.util.List<java.lang.String> getAdditionalIssues()

    • getIssues

      public java.util.List<java.lang.String> getIssues()

    • getVersions

      public java.util.List<java.lang.String> getVersions()

    • getTestCase

      public java.lang.Class<?> getTestCase()

    • getTestCaseName

      public java.lang.String getTestCaseName()

    • addVersion

      public TestOutcome addVersion​(java.lang.String version)

    • addVersions

      public TestOutcome addVersions​(java.util.List<java.lang.String> versions)

    • forProject

      public TestOutcome forProject​(java.lang.String project)

    • getRule

      public Rule getRule()

    • setRule

      public void setRule​(Rule rule)

    • getProject

      public java.lang.String getProject()

    • inTestRunTimestamped

      public TestOutcome inTestRunTimestamped​(java.time.ZonedDateTime testRunTimestamp)

    • setTestRunTimestamp

      public void setTestRunTimestamp​(java.time.ZonedDateTime testRunTimestamp)

    • addIssues

      public void addIssues​(java.util.List<java.lang.String> issues)

    • getFormattedIssues

      public java.lang.String getFormattedIssues()

    • isRelatedToIssue

      public void isRelatedToIssue​(java.lang.String issue)

    • addFailingStepAsSibling

      public void addFailingStepAsSibling​(java.util.List<TestStep> testStepList, java.lang.Throwable testFailureCause)

    • lastStepFailedWith

      public void lastStepFailedWith​(StepFailure failure)

    • lastStepFailedWith

      public void lastStepFailedWith​(java.lang.Throwable testFailureCause)

    • testFailedWith

      public void testFailedWith​(java.lang.Throwable cause)

    • getTags

      public java.util.Set<TestTag> getTags()

    • getAllTags

      public java.util.Set<TestTag> getAllTags()

    • addUserStoryFeatureTo

      public void addUserStoryFeatureTo​(java.util.Set<TestTag> augmentedTags)

    • setTags

      public void setTags​(java.util.Set<TestTag> tags)

    • addTags

      public void addTags​(java.util.List<TestTag> tags)

    • addTag

      public void addTag​(TestTag tag)

    • getIssueKeys

      public java.util.List<java.lang.String> getIssueKeys()

    • getQualifiedMethodName

      public java.lang.String getQualifiedMethodName()

    • getQualifiedId

      public java.lang.String getQualifiedId()

    • getNonNullContext

      public java.lang.String getNonNullContext()

    • getContext

      public java.lang.String getContext()

    • setContext

      public void setContext​(java.lang.String context)
      Setting the context
      Parameters:
      context -

    • getCompleteName

      public java.lang.String getCompleteName()
      Returns the name of the test prefixed by the name of the story.

    • useExamplesFrom

      public void useExamplesFrom​(DataTable table)

    • addNewExamplesFrom

      public void addNewExamplesFrom​(DataTable table)

    • moveToNextRow

      public void moveToNextRow()

    • updateCurrentRowResult

      public void updateCurrentRowResult​(TestResult result)

    • dataIsPredefined

      public boolean dataIsPredefined()

    • addRow

      public void addRow​(java.util.Map<java.lang.String,​?> data)

    • addRow

      public void addRow​(DataTableRow dataTableRow)

    • getTestCount

      public int getTestCount()

    • getImplementedTestCount

      public int getImplementedTestCount()

    • countResults

      public int countResults​(TestResult expectedResult)

    • countResults

      public int countResults​(TestResult expectedResult, TestType expectedType)

    • typeCompatibleWith

      public boolean typeCompatibleWith​(TestType testType)

    • countNestedStepsWithResult

      public int countNestedStepsWithResult​(TestResult expectedResult, TestType testType)

    • getTagValue

      public java.util.Optional<java.lang.String> getTagValue​(java.lang.String tagType)

    • hasIssue

      public boolean hasIssue​(java.lang.String issue)

    • hasTag

      public boolean hasTag​(TestTag tag)

    • hasAMoreGeneralFormOfTag

      public boolean hasAMoreGeneralFormOfTag​(TestTag specificTag)

    • hasAMoreSpecificFormOfTag

      public boolean hasAMoreSpecificFormOfTag​(TestTag generalTag)

    • setStartTime

      public void setStartTime​(java.time.ZonedDateTime startTime)

    • clearStartTime

      public void clearStartTime()

    • isManual

      public boolean isManual()

    • getLastTested

      public java.lang.String getLastTested()

    • setLastTested

      public void setLastTested​(java.lang.String lastTested)

    • isManualTestingUpToDate

      public boolean isManualTestingUpToDate()

    • setManualTestingUpToDate

      public void setManualTestingUpToDate​(java.lang.Boolean upToDate)

    • getFlags

      public java.util.Set<? extends Flag> getFlags()

    • isStartTimeNotDefined

      public boolean isStartTimeNotDefined()

    • has

      public TestOutcome.OptionalElements has()

    • getStepCount

      public java.lang.Integer getStepCount()

    • getRunningStepCount

      public java.lang.Integer getRunningStepCount()

    • getNestedStepCount

      public java.lang.Integer getNestedStepCount()

    • getSuccessCount

      public java.lang.Long getSuccessCount()

    • getFailureCount

      public java.lang.Long getFailureCount()

    • getErrorCount

      public java.lang.Long getErrorCount()

    • getCompromisedCount

      public java.lang.Long getCompromisedCount()

    • getIgnoredCount

      public java.lang.Long getIgnoredCount()

    • getSkippedOrIgnoredCount

      public java.lang.Long getSkippedOrIgnoredCount()

    • getSkippedCount

      public java.lang.Long getSkippedCount()

    • getPendingCount

      public java.lang.Long getPendingCount()

    • isSuccess

      public java.lang.Boolean isSuccess()

    • isFailure

      public java.lang.Boolean isFailure()

    • isCompromised

      public java.lang.Boolean isCompromised()

    • isError

      public java.lang.Boolean isError()

    • isPending

      public java.lang.Boolean isPending()

    • isSkipped

      public java.lang.Boolean isSkipped()

    • getUserStory

      public Story getUserStory()

    • recordDuration

      public void recordDuration()

    • setDuration

      public void setDuration​(long duration)

    • getDuration

      public java.lang.Long getDuration()

    • getEndTime

      public java.time.ZonedDateTime getEndTime()

    • getDurationInSeconds

      public double getDurationInSeconds()
      Returns:
      The total duration of all of the tests in this set in milliseconds.

    • getVideoLink

      public java.lang.String getVideoLink()
      Returns the link to the associated video (e.g. from Saucelabs) for this test.
      Returns:
      a URL.

    • getSessionId

      public java.lang.String getSessionId()

    • setSessionId

      public void setSessionId​(java.lang.String sessionId)

    • countTestSteps

      public java.lang.Integer countTestSteps()

    • getStartTime

      public java.time.ZonedDateTime getStartTime()

    • getStartedAt

      public java.lang.String getStartedAt()

    • getTimestamp

      public java.lang.String getTimestamp()

    • getTimestamp

      public java.lang.String getTimestamp​(java.time.format.DateTimeFormatter formater)

    • isDataDriven

      public boolean isDataDriven()

    • getExampleFields

      public java.util.List<java.lang.String> getExampleFields()

    • useScenarioOutline

      public void useScenarioOutline​(java.lang.String scenarioOutline)

    • getDataDrivenSampleScenario

      public java.lang.String getDataDrivenSampleScenario()

    • getDataTable

      public DataTable getDataTable()

    • equals

      public boolean equals​(java.lang.Object o)
      Overrides:
      equals in class java.lang.Object

    • hashCode

      public int hashCode()
      Overrides:
      hashCode in class java.lang.Object

    • getFeatureTag

      public java.util.Optional<TestTag> getFeatureTag()

    • getFailureDetails

      public FailureDetails getFailureDetails()

    • getTestSource

      public java.lang.String getTestSource()

    • setTestSource

      public void setTestSource​(java.lang.String testSource)

    • getActors

      public java.util.List<CastMember> getActors()

    • hasEvidence

      public boolean hasEvidence()

    • getEvidence

      public java.util.List<ReportData> getEvidence()

    • withDataRowsfilteredbyTag

      public TestOutcome withDataRowsfilteredbyTag​(TestTag tag)

    • withDataRowsfilteredbyTagsFrom

      public TestOutcome withDataRowsfilteredbyTagsFrom​(java.util.Collection<TestTag> filterTags)

    • getExternalLink

      public ExternalLink getExternalLink()

    • setOrder

      public void setOrder​(int order)

    • getOrder

      public java.lang.Integer getOrder()

    • fromStep

      public TestOutcome fromStep​(int index)

    • getNestedTestPath

      public java.util.List<java.lang.String> getNestedTestPath()