e3.testsuite.driver
===================

.. py:module:: e3.testsuite.driver


Submodules
----------

.. toctree::
   :maxdepth: 1

   /autoapi/e3/testsuite/driver/adacore/index
   /autoapi/e3/testsuite/driver/classic/index
   /autoapi/e3/testsuite/driver/diff/index


Attributes
----------

.. autoapisummary::

   e3.testsuite.driver.ResultQueue


Classes
-------

.. autoapisummary::

   e3.testsuite.driver.GAIAResultFiles
   e3.testsuite.driver.TestResult
   e3.testsuite.driver.TestResultSummary
   e3.testsuite.driver.CleanupMode
   e3.testsuite.driver.ResultQueueItem
   e3.testsuite.driver.TestDriver
   e3.testsuite.driver.BasicTestDriver


Functions
---------

.. autoapisummary::

   e3.testsuite.driver.dump_result_logs_if_needed


Package Contents
----------------

.. py:class:: GAIAResultFiles

   Filenames for a given result in a GAIA report.

   In a GAIA testsuite report, each result is described as one entry (line) in
   the "results" text file, and several optional files. This object contains
   the names for the files, when present, corresponding to a given result.


   .. py:attribute:: result
      :type:  Optional[str]


   .. py:attribute:: log
      :type:  Optional[str]


   .. py:attribute:: expected
      :type:  Optional[str]


   .. py:attribute:: out
      :type:  Optional[str]


   .. py:attribute:: diff
      :type:  Optional[str]


   .. py:attribute:: time
      :type:  Optional[str]


   .. py:attribute:: info
      :type:  Optional[str]


.. py:function:: dump_result_logs_if_needed(env: e3.env.Env, result: e3.testsuite.result.TestResult, output_dir: str) -> Optional[GAIAResultFiles]

   Shortcut to call dump_result_logs if a GAIA report is requested.


.. py:class:: TestResult(name: str, env: Optional[dict] = None, status: Optional[TestStatus] = None, msg: str = '')

   Bases: :py:obj:`yaml.YAMLObject`


   Represent a result for a given test.


   .. py:attribute:: yaml_loader


   .. py:attribute:: yaml_tag
      :value: '!e3.testsuite.result.TestResult'



   .. py:attribute:: test_name


   .. py:attribute:: env
      :value: None



   .. py:attribute:: status


   .. py:attribute:: msg
      :type:  Optional[str]
      :value: ''



   .. py:attribute:: log


   .. py:attribute:: processes
      :type:  list
      :value: []



   .. py:attribute:: failure_reasons
      :type:  Set[FailureReason]


   .. py:attribute:: expected
      :type:  Optional[Log]
      :value: None



   .. py:attribute:: out
      :type:  Optional[Log]
      :value: None



   .. py:attribute:: diff
      :type:  Optional[Log]
      :value: None



   .. py:attribute:: time
      :type:  Optional[float]
      :value: None



   .. py:attribute:: info
      :type:  Dict[str, str]


   .. py:method:: set_status(status: TestStatus, msg: Optional[str] = '') -> None

      Update the test status.

      :param status: New status. Note that only test results with status set
          to ERROR can be changed.
      :param msg: Optional short message to describe the result.  Note that
          multiline strings are turned into single-line strings.



   .. py:method:: __str__() -> str


   .. py:method:: save(results_dir: str) -> str

      Write this test results as a YAML file.

      :param results_dir: Name of the directory in which to write the test
          result. When writing a testsuite report, this corresponds to the
          report's ``results_dir`` (see
          ``e3.testsuite.report.index.ReportIndex``).
      :return: The base filename of the file written. It is generated from
          the testname.



   .. py:property:: summary
      :type: TestResultSummary



.. py:class:: TestResultSummary

   Basic data about a test result.

   This class holds a subset of the information available in ``TestResult``
   instances. This subset is meant to be small enough to be included in result
   indexes while enabling efficient common processings such as "show logs for
   tests that failed": no need to load all the test results for tests that
   succeeded.


   .. py:attribute:: test_name
      :type:  str


   .. py:attribute:: status
      :type:  TestStatus


   .. py:attribute:: msg
      :type:  Optional[str]


   .. py:attribute:: failure_reasons
      :type:  Set[FailureReason]


   .. py:attribute:: time
      :type:  Optional[float]


   .. py:attribute:: info
      :type:  Dict[str, str]


.. py:class:: CleanupMode(*args, **kwds)

   Bases: :py:obj:`enum.Enum`


   Mode for working space cleanups.


   .. py:attribute:: NONE


   .. py:attribute:: PASSING


   .. py:attribute:: ALL


   .. py:method:: default() -> CleanupMode
      :classmethod:



   .. py:method:: descriptions() -> Dict[CleanupMode, str]
      :classmethod:



.. py:class:: ResultQueueItem

   Information to integrate a test result in a testsuite report.

   Test drivers create test results. They travel from there to the testsuite
   final report through various queues. This class gathers all the information
   needed for the various stages of this pipeline.


   .. py:attribute:: result
      :type:  e3.testsuite.result.TestResultSummary

      Summary for this test result.



   .. py:attribute:: filename
      :type:  str

      Name of the file that contains test result data.



   .. py:attribute:: traceback
      :type:  List[str]

      Stack trace for this result's push time.

      This stack trace corresponds to the code that led the TestResult instance
      to be included in the testsuite report.



   .. py:attribute:: gaia_results
      :type:  Optional[e3.testsuite.report.gaia.GAIAResultFiles]

      GAIA files for this result.

      This is None if no GAIA report is requested.



.. py:data:: ResultQueue

.. py:class:: TestDriver(env: e3.env.Env, test_env: Dict[str, Any])

   Bases: :py:obj:`object`


   Testsuite Driver.

   All drivers declared in a testsuite should inherit from this class


   .. py:attribute:: Fore
      :type:  Any


   .. py:attribute:: Style
      :type:  Any


   .. py:attribute:: test_env_filename
      :type:  str


   .. py:attribute:: env
      :type:  e3.env.Env


   .. py:attribute:: testsuite_options
      :type:  argparse.Namespace


   .. py:attribute:: test_env
      :type:  Dict[str, Any]


   .. py:attribute:: test_name
      :type:  str


   .. py:attribute:: result
      :type:  e3.testsuite.result.TestResult


   .. py:attribute:: result_queue
      :type:  ResultQueue
      :value: []


      Queue of test results that this driver plans to integrate to the
      testsuite report.



   .. py:method:: push_result(result: Optional[e3.testsuite.result.TestResult] = None) -> None

      Push a result to the testsuite.

      This method should be called to push results to the testsuite report.

      :param result: A TestResult object to push. If None push the current
          test result.



   .. py:method:: add_fragment(dag: e3.collection.dag.DAG, name: str, fun: Optional[e3.testsuite.fragment.FragmentCallback] = None, after: Optional[List[str]] = None) -> None

      Add a test fragment.

      This function is a helper to define test workflows that do not
      introduce dependencies to other tests. For more complex operation use
      directly add_vertex method from the dag. See add_test method.

      :param dag: DAG containing test fragments.
      :param name: Name of the fragment.
      :param fun: Callable that takes two positional arguments: a mapping
          from fragment names to return values for already executed
          fragments, and a slot ID. If None looks for a method inside this
          class called ``name``.
      :param after: List of fragment names that should be executed before
          this one.



   .. py:method:: working_dir(*args: str) -> str

      Build a filename in the test working directory.



   .. py:method:: test_dir(*args: str) -> str

      Build a filename in the testcase directory.



   .. py:method:: add_test(dag: e3.collection.dag.DAG) -> None
      :abstractmethod:


      Create the test workflow.

      Amend a DAG with the test fragments that should be executed along with
      their dependencies. See BasicTestDriver for an example of workflow.



   .. py:property:: working_dir_cleanup_enabled
      :type: bool


      Return whether test drivers should cleanup their working directory.

      Unless this returns False, test drivers should delete their working
      directory when the test has completed, so that temporaries for the
      whole testsuite are removed incrementally. This is necessary to avoid
      creating huge temporary directories when executing big testsuites.



.. py:class:: BasicTestDriver(env: e3.env.Env, test_env: Dict[str, Any])

   Bases: :py:obj:`TestDriver`


   Testsuite Driver.

   All drivers declared in a testsuite should inherit from this class


   .. py:method:: add_test(dag: e3.collection.dag.DAG) -> None

      Create a standard test workflow.

      set up -> run -> analyze -> tear_down in which set up and tear_down
      are optional.



   .. py:method:: set_up(prev: Dict[str, Any], slot: int) -> None

      Execute operations before executing a test.



   .. py:method:: tear_up(prev: Dict[str, Any], slot: int) -> None

      Backwards-compatible name for the "set_up" method.



   .. py:method:: tear_down(prev: Dict[str, Any], slot: int) -> None

      Execute operations once a test is finished.



   .. py:method:: run(prev: Dict[str, Any], slot: int) -> None
      :abstractmethod:


      Execute a test.



   .. py:method:: analyze(prev: Dict[str, Any], slot: int) -> None
      :abstractmethod:


      Compute the test result.



