e3.testsuite.driver

Submodules

Attributes

ResultQueue

Classes

`GAIAResultFiles`	Filenames for a given result in a GAIA report.
`TestResult`	Represent a result for a given test.
`TestResultSummary`	Basic data about a test result.
`CleanupMode`	Mode for working space cleanups.
`ResultQueueItem`	Information to integrate a test result in a testsuite report.
`TestDriver`	Testsuite Driver.
`BasicTestDriver`	Testsuite Driver.

Functions

dump_result_logs_if_needed(→ Optional[GAIAResultFiles])

Shortcut to call dump_result_logs if a GAIA report is requested.

Package Contents

class e3.testsuite.driver.GAIAResultFiles

Filenames for a given result in a GAIA report.

In a GAIA testsuite report, each result is described as one entry (line) in the “results” text file, and several optional files. This object contains the names for the files, when present, corresponding to a given result.

result: str | None

log: str | None

expected: str | None

out: str | None

diff: str | None

time: str | None

info: str | None

e3.testsuite.driver.dump_result_logs_if_needed(env: e3.env.Env, result: e3.testsuite.result.TestResult, output_dir: str) → GAIAResultFiles | None: Shortcut to call dump_result_logs if a GAIA report is requested.

class e3.testsuite.driver.TestResult(name: str, env: dict | None = None, status: TestStatus | None = None, msg: str = '')

Bases: yaml.YAMLObject

Represent a result for a given test.

yaml_loader

yaml_tag = '!e3.testsuite.result.TestResult'

test_name

env = None

status

msg: str | None = ''

log

processes: list = []

failure_reasons: Set[FailureReason]

expected: Log | None = None

out: Log | None = None

diff: Log | None = None

time: float | None = None

info: Dict[str, str]

set_status(status: TestStatus, msg: str | None = '') → None

Update the test status.

Parameters:

status – New status. Note that only test results with status set to ERROR can be changed.
msg – Optional short message to describe the result. Note that multiline strings are turned into single-line strings.

__str__() → str

save(results_dir: str) → str

Write this test results as a YAML file.

Parameters:: results_dir – Name of the directory in which to write the test result. When writing a testsuite report, this corresponds to the report’s results_dir (see e3.testsuite.report.index.ReportIndex).
Returns:: The base filename of the file written. It is generated from the testname.

property summary: TestResultSummary

class e3.testsuite.driver.TestResultSummary

Basic data about a test result.

This class holds a subset of the information available in TestResult instances. This subset is meant to be small enough to be included in result indexes while enabling efficient common processings such as “show logs for tests that failed”: no need to load all the test results for tests that succeeded.

test_name: str

status: TestStatus

msg: str | None

failure_reasons: Set[FailureReason]

time: float | None

info: Dict[str, str]

class e3.testsuite.driver.CleanupMode(*args, **kwds)

Bases: enum.Enum

Mode for working space cleanups.

NONE

PASSING

ALL

classmethod default() → CleanupMode

classmethod descriptions() → Dict[CleanupMode, str]

class e3.testsuite.driver.ResultQueueItem

Information to integrate a test result in a testsuite report.

Test drivers create test results. They travel from there to the testsuite final report through various queues. This class gathers all the information needed for the various stages of this pipeline.

result: e3.testsuite.result.TestResultSummary: Summary for this test result.

filename: str: Name of the file that contains test result data.

traceback: List[str]

Stack trace for this result’s push time.

This stack trace corresponds to the code that led the TestResult instance to be included in the testsuite report.

gaia_results: e3.testsuite.report.gaia.GAIAResultFiles | None

GAIA files for this result.

This is None if no GAIA report is requested.

e3.testsuite.driver.ResultQueue

class e3.testsuite.driver.TestDriver(env: e3.env.Env, test_env: Dict[str, Any])

Bases: object

Testsuite Driver.

All drivers declared in a testsuite should inherit from this class

Fore: Any

Style: Any

test_env_filename: str

env: e3.env.Env

testsuite_options: argparse.Namespace

test_env: Dict[str, Any]

test_name: str

result: e3.testsuite.result.TestResult

result_queue: ResultQueue = []: Queue of test results that this driver plans to integrate to the testsuite report.

push_result(result: e3.testsuite.result.TestResult | None = None) → None

Push a result to the testsuite.

This method should be called to push results to the testsuite report.

Parameters:: result – A TestResult object to push. If None push the current test result.

add_fragment(dag: e3.collection.dag.DAG, name: str, fun: e3.testsuite.fragment.FragmentCallback | None = None, after: List[str] | None = None) → None

Add a test fragment.

This function is a helper to define test workflows that do not introduce dependencies to other tests. For more complex operation use directly add_vertex method from the dag. See add_test method.

Parameters:

dag – DAG containing test fragments.
name – Name of the fragment.
fun – Callable that takes two positional arguments: a mapping from fragment names to return values for already executed fragments, and a slot ID. If None looks for a method inside this class called name.
after – List of fragment names that should be executed before this one.

working_dir(*args: str) → str: Build a filename in the test working directory.

test_dir(*args: str) → str: Build a filename in the testcase directory.

abstractmethod add_test(dag: e3.collection.dag.DAG) → None

Create the test workflow.

Amend a DAG with the test fragments that should be executed along with their dependencies. See BasicTestDriver for an example of workflow.

property working_dir_cleanup_enabled: bool

Return whether test drivers should cleanup their working directory.

Unless this returns False, test drivers should delete their working directory when the test has completed, so that temporaries for the whole testsuite are removed incrementally. This is necessary to avoid creating huge temporary directories when executing big testsuites.

class e3.testsuite.driver.BasicTestDriver(env: e3.env.Env, test_env: Dict[str, Any])

Bases: TestDriver

Testsuite Driver.

All drivers declared in a testsuite should inherit from this class

add_test(dag: e3.collection.dag.DAG) → None

Create a standard test workflow.

set up -> run -> analyze -> tear_down in which set up and tear_down are optional.

set_up(prev: Dict[str, Any], slot: int) → None: Execute operations before executing a test.

tear_up(prev: Dict[str, Any], slot: int) → None: Backwards-compatible name for the “set_up” method.

tear_down(prev: Dict[str, Any], slot: int) → None: Execute operations once a test is finished.

abstractmethod run(prev: Dict[str, Any], slot: int) → None: Execute a test.

abstractmethod analyze(prev: Dict[str, Any], slot: int) → None: Compute the test result.