XProc Test Suite Vocabulary

Norman Walsh

25 Sep 2007

Table of Contents

1. The test-suite element
2. The test element
2.1. Testing correct pipelines
2.2. Testing incorrect pipelines
3. The input element
4. The output element
5. The document element
6. The option element
7. The parameter element
8. The pipeline element
9. The compare-pipeline element
10. The description element

This document describes the format for XProc test suite documents. They are defined formally by a RELAX NG grammar (compact syntax).

All of the elements in the test suite vocabulary are in the test suite namespace, http://xproc.org/ns/testsuite.

1. The test-suite element

The test-suite element defines an entire test suite. A test suite must have a title and consists of one or more tests.

<t:test-suite
  xinclude? = boolean>
    t:title,
    t:test+
</t:test-suite>

If the xinclude attribute on the test-suite element has the value true, then XInclude processing must be applied to the test suite before attempting to run it.

Note

XInclude processing is recursive. If it is enabled for the test suite, then all tests in the test suite that wish to test the p:xinclude step must identify the source document for the p:xinclude step externally.

2. The test element

The test element identifies a single test. There are two kinds of tests, tests for pipelines that are expected to succeed and tests for pipelines that are expected to raise (static or dynamic) errors.

Both kinds of test element must have a title and may have a description.

2.1. Testing correct pipelines

The test element for a correct pipeline contains zero or more input elements, a pipeline, an optional compare-pipeline, and zero or more output elements.

<t:test
  xml:id? = ID
  ignore-whitespace-differences? = true|false>
    t:title,
    t:description?,
    (t:input |
     t:parameter |
     t:option)*,
    t:pipeline,
    t:compare-pipeline?,
    t:output*
</t:test>

Each input element specifies what should appear on the input port that it defines.

The pipeline element specifies the p:pipeline that will be evaluated with the specified inputs.

Each output element specifies what should appear on the output port that it defines.

If there is no compare-pipeline, then the test passes if and only if what appears on each output port of the pipeline is deep-equal to what is specified for that port in the test’s output element for that port.

If a compare-pipeline is present, then it is run before the comparison, as described in Section 9, “The compare-pipeline element”.

2.2. Testing incorrect pipelines

The test element for an incorrect pipeline has an error attribute which identifies the expected error. It contains zero or more input elements and a pipeline element.

<t:test
  xml:id? = ID
  ignore-whitespace-differences? = true|false
  error = QName>
    t:title,
    t:description?,
    t:input*,
    t:pipeline
</t:test>

It may not contain a compare-pipeline element or any output elements. The pipeline is not expected to succeed.

An incorrect pipeline test passes if and only if the implementation signals an error during processing. Ideally, the error will match the error specified in the test, but this is not always the case.

Conformance to an incorrect test requires only that the test fail, there is no requirement that it generate the expected error.

3. The input element

The input element identifies an input document or sequence of input documents. The document or documents will appear on the port specified by the port attribute.

<t:input
  port = NCName>
    (t:document* |
     anyElement)
</t:input>

If the input element contains one or more document elements, then those elements identify the document or sequence of documents that appear on the specified port.

As a syntactic convenience, if the input consists of exactly one document, that document can be provided inline without the document wrapper.

As a further syntactic convenience, if the input consists of a single, external document, the input element can point directly to it:

<t:input
  port = NCName
  href = anyURI />

4. The output element

The output element identifies an output document or sequence of output documents against which the pipeline output will be compared. The document or documents will be compared to what appears on the port specified by the port attribute.

<t:output
  port = NCName>
    (t:document* |
     anyElement)
</t:output>

If the output element contains one or more document elements, then those elements identify the document or sequence of documents that will be used for comparison.

As a syntactic convenience, if the output consists of exactly one document, that document can be provided inline without the document wrapper.

5. The document element

The document element identifies a single document for either input or output. Documents can be specified inline or by reference.

<t:document>
    anyElement
</t:document>

An inline document appears…inline inside the document element.

<t:document
  href = anyURI />

An external document is identified by an href attribute.

6. The option element

The option element specifies the value for a pipeline option:

<t:option
  name = QName
  value = string />

Note: Only the pipeline receives the options; the pipeline in the compare-pipeline (if there is one) does not receive any options.

7. The parameter element

The parameter element specifies the value for a pipeline parameter:

<t:parameter
  port? = NCName
  name = QName
  value = string />

Note: Only the pipeline receives the parameters; the pipeline in the compare-pipeline (if there is one) does not receive any parameters.

8. The pipeline element

The pipeline element identifies the p:pipeline that is the subject of the p:test. Pipelines can be specified inline or by reference.

<t:pipeline>
    

<div class="unknowntag">
<font color="red"><grammar></font>
<div class="unknowntag">
<font color="red"><include href="xproc.rng"></font>
<font color="red"></include></font>
</div>
<font color="red"></grammar></font>
</div>

</t:pipeline>

An inline pipeline appears…inline inside the pipeline element.

<t:pipeline
  href = anyURI />

An external pipeline is identified by an href attribute.

9. The compare-pipeline element

A compare-pipeline is used to preprocess the results of the primary pipeline.

If a compare-pipeline is present, then it is run before the comparison. The p:pipeline that is specified in a compare-pipeline must have an input port with the same name as the name of each of the output ports in the tested p:pipeline.

The test passes if and only if what appears on each output port of the compare-pipeline is deep-equal to what is specified for that port in the test’s output element for that port.

The compare-pipeline is only needed when the output of the tested pipeline cannot be directly compared to the putative outputs. Consider this example:

  1 <?xml-stylesheet type="text/xsl" href="/style/testcase.xsl"?>
  2 <t:test xmlns:t="http://xproc.org/ns/testsuite"
            xmlns:p="http://www.w3.org/ns/xproc"
  4         xmlns:c="http://www.w3.org/ns/xproc-step"
            xmlns:err="http://www.w3.org/ns/xproc-error">
  6 
    <t:title>Test the <t:code>p:store</t:code> step</t:title>
  8 
    <t:description xmlns="http://www.w3.org/1999/xhtml">
 10   <p>WARNING: This test writes to <code>file:///tmp/testout.xml</code>.
      </p>
 12 </t:description>
    
 14 <t:input port='source'>
      <document>
 16     <title>Some Title</title>
        <para>Some paragraph.</para>
 18   </document>
    </t:input>
 20 
    <t:pipeline>
 22   <p:declare-step version='1.0' name="main">
        <p:input port="source"/>
 24     <p:store name="store" href="file:///tmp/testout.xml"/>
      </p:declare-step>
 26 </t:pipeline>
    
 28 <t:compare-pipeline>
      <p:declare-step name="cmain" version="1.0">
 30     <p:output port="output"/>
        <p:load name="load" href="file:///tmp/testout.xml"/>
 32   </p:declare-step>
    </t:compare-pipeline>
 34 
    <t:output port="output">
 36   <document>
        <title>Some Title</title>
 38     <para>Some paragraph.</para>
      </document>
 40 </t:output>
    </t:test>

The test cannot directly test if the pipeline stored the correct document because the stored document is not part of the output. So the compare-pipeline is used to load the stored document for comparison.

10. The description element

The description element is provided for human-readable prose about the test. It must contain XHTML markup.

<t:description>
    anyHtmlElement*
</t:description>