e3.testsuite.driver.diff
========================

.. py:module:: e3.testsuite.driver.diff


Classes
-------

.. autoapisummary::

   e3.testsuite.driver.diff.OutputRefiner
   e3.testsuite.driver.diff.RefiningChain
   e3.testsuite.driver.diff.Substitute
   e3.testsuite.driver.diff.CanonicalizeLineEndings
   e3.testsuite.driver.diff.ReplacePath
   e3.testsuite.driver.diff.PatternSubstitute
   e3.testsuite.driver.diff.LineByLine
   e3.testsuite.driver.diff.DiffTestDriver


Module Contents
---------------

.. py:class:: OutputRefiner

   Bases: :py:obj:`Generic`\ [\ :py:obj:`AnyStr`\ ]


   Interface to refine a test output before baseline and actual comparison.

   Sometimes, the way a library/tool works forces it to have outputs that
   depends on the environment (for instance: the location of the testsuite on
   the filesystem, the current date, etc.). Refiners makes it possible for
   a testsuite to "hide" these discrepancies during the diff computation.

   Note that output refiners might get called bytes strings when the test
   drivers operate in binary mode.


   .. py:method:: refine(output: AnyStr) -> AnyStr
      :abstractmethod:


      Refine a test/baseline output.

      :param output: Output to refine.



.. py:class:: RefiningChain(refiners: List[OutputRefiner])

   Bases: :py:obj:`OutputRefiner`\ [\ :py:obj:`AnyStr`\ ]


   Simple wrapper for a sequence of output refiners applied in chain.


   .. py:attribute:: refiners


   .. py:method:: refine(output: AnyStr) -> AnyStr

      Refine a test/baseline output.

      :param output: Output to refine.



.. py:class:: Substitute(substring: AnyStr, replacement: Optional[AnyStr] = None)

   Bases: :py:obj:`OutputRefiner`\ [\ :py:obj:`AnyStr`\ ]


   Replace substrings in outputs.


   .. py:attribute:: substring
      :type:  AnyStr


   .. py:attribute:: replacement
      :type:  AnyStr


   .. py:method:: refine(output: AnyStr) -> AnyStr

      Refine a test/baseline output.

      :param output: Output to refine.



.. py:class:: CanonicalizeLineEndings

   Bases: :py:obj:`OutputRefiner`\ [\ :py:obj:`AnyStr`\ ]


   Replace \r\n with \n in outputs.


   .. py:method:: refine(output: AnyStr) -> AnyStr

      Refine a test/baseline output.

      :param output: Output to refine.



.. py:class:: ReplacePath(path: str, replacement: str = '')

   Bases: :py:obj:`RefiningChain`\ [\ :py:obj:`str`\ ]


   Return an output refiner to replace the given path.


.. py:class:: PatternSubstitute(pattern: AnyStr, replacement: Optional[AnyStr] = None)

   Bases: :py:obj:`OutputRefiner`, :py:obj:`Generic`\ [\ :py:obj:`AnyStr`\ ]


   Replace patterns in outputs.


   .. py:attribute:: regexp
      :type:  Pattern[AnyStr]


   .. py:attribute:: replacement
      :type:  AnyStr


   .. py:method:: refine(output: AnyStr) -> AnyStr

      Refine a test/baseline output.

      :param output: Output to refine.



.. py:class:: LineByLine(refiner: OutputRefiner)

   Bases: :py:obj:`OutputRefiner`\ [\ :py:obj:`AnyStr`\ ]


   Wrapper to apply an output refine line by line.


   .. py:attribute:: refiner


   .. py:method:: refine(output: AnyStr) -> AnyStr

      Refine a test/baseline output.

      :param output: Output to refine.



.. py:class:: DiffTestDriver(env: e3.env.Env, test_env: Dict[str, Any])

   Bases: :py:obj:`e3.testsuite.driver.classic.ClassicTestDriver`


   Test driver to compute test output against a baseline.


   .. py:property:: baseline_file
      :type: Tuple[str, bool]


      Return the test output baseline file.

      :return: The name of the text file (relative to test directories) that
          contains the expected test output and whether the baseline is a
          regexp.



   .. py:property:: baseline
      :type: Tuple[Optional[str], Union[str, bytes], bool]


      Return the test output baseline.

      Subclasses can override this method if they want to provide a baseline
      that does not come from a file, short-circuiting the baseline_file
      property.

      :return: The baseline absolute filename (if any), the baseline content,
          as a string or as a bytes string (depending on the default
          encoding), and whether the baseline is a regexp. The baseline
          filename is used to rewrite test output: leave it to None if
          rewriting does not make sense.



   .. py:property:: output_refiners
      :type: List[OutputRefiner]


      List of refiners for test baselines/outputs.

      This just returns a refiner to canonicalize line endings unless the
      test environment contains a "strict_line_endings" key associated to
      true.



   .. py:property:: refine_baseline
      :type: bool


      Whether to apply output refiners to the output baseline.



   .. py:property:: diff_ignore_white_chars
      :type: bool


      Whether to ignore white characters in diff computations.

      This returns whether the comparison between test output and baseline
      must ignore whitespaces (leading and trailing spaces, tabs and carriage
      returns on lines, and empty lines). Note that if we don't ignore them,
      we still canonicalize line separators (CRLF are replaced by LF before
      the comparison).

      Note that at some point, this mechanism should be unified with the
      ``output_refiners`` machinery. However, this relies on e3.diff's
      ignore_white_chars feature, which is not trivial to reimplement.



   .. py:property:: diff_context_size
      :type: int


      Positive number of context lines to include in diff computations.



   .. py:method:: set_up() -> None

      Run initialization operations before a test runs.

      Subclasses can override this to prepare testcase execution.

      Having a callback separate from "run" is useful when dealing with
      inheritance: overriding the "set_up" method in subclasses allows to
      append setup actions before the testcase execution actually takes place
      (in the "run" method).

      If everything happened in "run" method, that would not be possible
      unless re-implementing the "run" method in each subclass, with obvious
      code duplication issues.



   .. py:method:: compute_diff(baseline_file: Optional[str], baseline: AnyStr, actual: AnyStr, failure_message: str = 'unexpected output', ignore_white_chars: Optional[bool] = None, context_size: Optional[int] = None, truncate_logs_threshold: Optional[int] = None) -> List[str]

      Compute the diff between expected and actual outputs.

      Return an empty list if there is no diff, and return a list that
      contains an error message based on ``failure_message`` otherwise.

      :param baseline_file: Absolute filename for the text file that contains
          the expected content (for baseline rewriting, if enabled), or None.
      :param actual: Actual content to compare.
      :param failure_message: Failure message to return if there is a
          difference.
      :param ignore_white_chars: Whether to ignore whitespaces during the
          diff computation. If left to None, use
          ``self.diff_ignore_white_chars``.
      :param context_size: Positive number of context lines to include in
          diff computations. If left to None, use ``self.diff_context_size``.
      :param truncate_logs_threshold: Threshold to truncate the diff message
          in ``self.result.log``. See ``e3.testsuite.result.truncated``'s
          ``line_count`` argument. If left to None, use the testsuite's
          ``--truncate-logs`` option.



   .. py:method:: compute_regexp_match(regexp: Union[Pattern[AnyStr], AnyStr], actual: AnyStr, failure_message: str = 'output does not match expected pattern', truncate_logs_threshold: Optional[int] = None) -> List[str]

      Compute whether the actual output matches a regexp.

      Return an empty list if the acutal content matches, and return a list
      that contains an error message based on ``failure_message`` otherwise.

      :param regexp: Regular expression to use.
      :param actual: Actual content to match.
      :param failure_message: Failure message to return if there is a
          difference.
      :param truncate_logs_threshold: Threshold to truncate the diff message
          in ``self.result.log``. See ``e3.testsuite.result.truncated``'s
          ``line_count`` argument. If left to None, use the testsuite's
          ``--truncate-logs`` option.



   .. py:method:: compute_failures() -> List[str]

      Return a failure if ``self.output.log`` does not match the baseline.

      This computes a diff with the content of the baseline file, unless
      there is a "baseline_regexp" entry in the test environment that
      evaluates to true.

      Subclasses can override this if they need more involved analysis:
      for instance computing multiple diffs.



