This is the Version 1.5.0 Release of the Touchstone Testing Implementation Guide, based on FHIR Version 4.0.1. See the Directory of published versions

General Guidance

This section outlines important definitions and interpretations and requirements common to all AEGIS Touchstone Testing actors used in this guide. The conformance verbs used are defined in FHIR Conformance Rules.

Contents

• Touchstone Placeholders
  • Predefined User Unique Data Values
  • Functions for Dynamic Data Generation
    • CURRENTDATE[TIME]
    • DATE[TIME]
    • UUID[-ST|-NODASH|-ST-NODASH]
  • Usage
• Minimum ID (minimumID) Assert Processing
• Rule and RuleSet Extensions
• OAuth2 Capabilities
  • TestScript Extensions
  • Pre-defined Fixture Ids
  • Base64Encoding
• Bulk Data Capabilities
  • NDJSON File Evalutation and Validation
  • NDJSON Assertion Prefix Syntax and Usage
    • TestScript Example
    • TestScript.profile.reference
    • TestScript.test.action.assert.expression
    • TestScript.test.action.assert.path
    • TestScript.test.action.assert.resouce

Touchstone Placeholders

The Touchstone Placeholders are special predefined TestScript variables and may be used where standard TestScript variables are allowed: static fixtures and operation.params, operation.requestHeader.value, operation.url, and assert.value element values.

These Touchstone Placeholders fall into two (2) categories

• Predefined User Unique Data Values
• Functions for Dynamic Data Generation

Predefined User Unique Data Values

The predefined user unique data value placeholders are strings of varying lengths from 1 to 20 characters. They are defined of three (3) types:

• Character only; for example, ${C6}
• Digits only; for example, ${D9}
• Characters + Digits; for example, ${CD14}

Users may view their assigned placeholder variable values in Touchstone via the user name menu item ${} My Placeholders.

Guaranteed Value Uniqueness - All predefined user unique data value placeholder values with a character length of 6 or more are guaranteed to be unique.

Functions for Dynamic Data Generation

There are three (3) available functional constructs for dynamic data generation:

• CURRENTDATE and CURRENTDATETIME
• DATE and DATETIME (relative to dynamic variable)
• UUID, UUID-ST, UUID-NODASH and UUID-ST-NODASH

CURRENTDATE[TIME]

These function placeholder variables provide support for date and datetime values based on the current date (today) and current datetime (now). Optional comma separated offset arguments apply date and time arithmetic providing relative date and time generated values.

Syntax/Format ${<placeholder name>[, <datetime portion code>, <offset value>]} Where,

• <placeholder name> is CURRENTDATE or CURRENTDATETIME
• <datetime portion code> is the character indicating what portion of the date or datetime value will be offset. Valid characters are derived from the following Java SimpleDateFormat pattern string “yyMMddHHmmss”
• <offset value> is the signed integer value used to adjust the date or time
• <datetime portion code>, <offset value> may be repeated for more complex arithmetic

Examples

DATE[TIME]

These function placeholder variables provide support for date and datetime values based on a dynamic variable user input date or datetime value. Optional comma separated offset arguments apply date and time arithmetic providing relative date and time generated values.

Syntax/Format ${<placeholder name>, <relative value>[, <datetime portion code>, <offset value>]} Where,

• <placeholder name> is DATE or DATETIME
• <relative value> is a dynamic variable defined in the current TestScript that holds the relative date or datetime value
  • The dynamic variable must be defined in TestScript without path/expression; its value will be entered by the user during Test Setup
• <datetime portion code> is the character indicating what portion of the date or datetime value will be offset. Valid characters are derived from the following Java SimpleDateFormat pattern string “yyMMddHHmmss”
• <offset value> is the signed integer value used to adjust the date or time
• <datetime portion code>, <offset value> may be repeated for more complex arithmetic

Examples

UUID[-ST|-NODASH|-ST-NODASH]

These function placeholder variables provide support for generation of UUID values with predefined formatting.

Syntax/Format ${<placeholder name>} Where,

• <placeholder name> is UUID, UUID-ST, UUID-NODASH or UUID-ST-NODASH

Examples

Usage

The Touchstone Placeholder, both predefined values and functions, are typically defined in static fixtures used as an operation request payload; for example, create or update. Touchstone’s TestScript Execution interface provides both a Raw and Resolved view of static fixtures where the pre and post test execution contents can be examined.

Raw Example - Patient.name, Patient.birthDate

<name>
  <use value="official"/>
  <family value="Smith${C7}"/>
  <given value="John${C6}"/>
  <birthDate value="${CURRENTDATE,d,-7}"/>
</name>

Resolved Example - Patient.name, Patient.birthDate

<name>
  <use value="official"/>
  <family value="SmithMqCERSQ"/>
  <given value="JohnXRCnsc"/>
  <birthDate value="2021-01-27"/>
</name>

Minimum ID (minimumID) Assert Processing

The FHIR specification’s definition of FHIR Testing contains a brief description within the Test search operation section on how a Test Engine is to evaluate and process a TestScript assert using the minimumId element. This description states:

<action>
	<assert>
		<minimumId value="F1"/>
		<sourceId value="R1"/>
	</assert>
</action>

Test engines will parse the 'body' of the F1 fixture and verify that each element and its value matches the corresponding element in the R1 response body. In other words, R1 is verified to be a 'superset' of F1. The resource id element in the body will be ignored during comparison. The headers will also be ignored.

F1 can be statically defined or it can be the [responseId](http://hl7.org/fhir/testscript-definitions.html#TestScript.setup.action.operation.responseId) for another operation. If [sourceId](http://hl7.org/fhir/testscript-definitions.html#TestScript.setup.action.assert.sourceId) is not specified, then test engines will use the response of the last operation.

This is a high-level functional description that does not provide sufficient information regarding the required definition of how the detailed comparison is to be performed. In order to provide more clarity the Touchstone implementation of the minimumId assert comparison logic now provides the following structural comparison of the minimumId fixture and the fixture being evaluated behaviors:

• Attribute or element ordering - If the comparison encounters an order mismatch of an attribute or element between the minimumId fixture and the fixture being evaluated, this does not result in an error as long as the path to the attribute or element matches.
  • This is particularly important when comparing JSON formatted fixtures where attribute order is not significant.
  • For example, the following two fixtures are considered to be matching:

    {
      name: "hello",
      title: "goodbye"
    }
    
    {
      title: "goodbye",
      name: "hello"
    }
    

• Array value ordering - The order of an array of the same attribute or element is allowed to differ in the comparison; also, additional attribute or element instances are allowed in between the minimumId fixture attribute or element instances such that the index can differ.
  • For example, the following two fixtures are considered equal even though the order of array elements differs:

    min fixture:
    {
      names: ["hello", "goodbye"]
    }
    
    compare fixture:
    {
      names: ["goodbye", "hello"]
    }
    

  • For example, these fixtures would match even though there are extra values:

    min fixture:
    {
      names: ["hello", "goodbye"]
    }
    
    compare1 fixture:
    {
      names: ["hello", "greeting", "goodbye"]
    }
    
    compare2 fixture:
    {
      names: ["greeting", "hello", "goodbye"]
    }
    
    compare3 fixture:
    {
      names: ["hello", "goodbye", "greeting"]
    }
    

• Duplicate attributes or elements - When the minimumId fixture has duplicate attributes or elements (in a JSON attribute array or XML repeated element) the fixture being evaluated must have the same number of duplicated attributes or elements.
  • For example, the following two fixtures would not match as the compare fixture does not have the same number of duplicate repeating attributes as the min fixture:

    min fixture:
    {
      names: ["hello", "hello"]
    }
    
    compare1 fixture:
    {
      names: ["hello"]
    }
    

• Comparison results - The comparison results now include ALL inconsistencies between the minimumId fixture and the fixture being evaluated (previously only the first mismatch was reported back to the user).

• Attribute or element present regardless of value - The ability to allow for the existence of an attribute or element regardless of its value is possible with the XML syntax but not JSON due to the JSON requirement that an attribute value must be present.
  • For example, this minimumId fixture XML element without a value will match any same named XML element in the fixture being evaluated regardless of its value:

    <novalueelement/>
    

  • This concept cannot be expressed in JSON syntax.

Rule and RuleSet Extensions

The TestScript rule and ruleset extensions can be used to reference complex validation logic that goes beyond what the basic TestScript assert element supports. As such, rules are recommended only when a TestScript assert cannot be used in its basic form.

The Touchstone Rules-Engine supports rules written in the following languages:

• Groovy
• XSLT
• Schematron

Support for additional languages may be added in the future. Unless you plan on executing test scripts against a test system that only supports XML, it is highly recommended to write rules in Groovy as XSLT and Schematron rules can only be evaluated against requests and responses whose content is in XML while Groovy supports JSON as well.

OAuth2 Capabilities

Touchstone has the ability for a user to create and use a Test System that is OAuth2 enabled, and with that comes new features that are important to take note of when it comes to Test Executions performed against a Test System that has an OAuth2 Authorization service connected to it.

TestScript Extensions

With the addition of a more robust OAuth2, SMART-on-FHIR and Bulk Data support into Touchstone, there are new extensions under the TestScript.test and TestScript.test.operation elements to support the manual control of test execution behavior and execution of the various OAuth2 authorization flows that are available to test script authors.

• AEGIS Touchstone Testing TestScript Operation AuthorizeInNewTab Extension
• AEGIS Touchstone Testing TestScript Operation Oauth2AuthzRedirectId Extension
• AEGIS Touchstone Testing TestScript Operation Oauth2AuthzRequestId Extension
• AEGIS Touchstone Testing TestScript Operation PagesNext Extension
• AEGIS Touchstone Testing TestScript Operation SmartLaunchRequestId Extension
• AEGIS Touchstone Testing TestScript Test ManualCompletion Extension

Pre-defined Fixture Ids

With the addition of a more robust OAuth2 environment into Touchstone, there are new fixtures, and values under those fixtures, that are available to test script authors.

• One new fixture is the dest1SmartConfig (dest2SmartConfig, dest3SmartConfig, etc.). The fixture allows the test script author to access different variables coming from the JSON document that is at the .well-known/smart-configuration.json endpoint. It acts the same as retrieving variables from the capabilities statement. For example, to use dest1SmartConfig to retrieve the Client ID, you would use the sourceId and path in the testscript as sourceId = dest1SmartConfig, path = $.clientId.

• Test Systems have special Oauth2 values that can be be retrieved by test script authors. The fixtures have a naming convention of dest1SystemConfig, dest2SystemConfig, origin1SystemConfig, origin2SystemConfig, etc. These fixtures have the following attributes:

• Another set of fixtures that can be used by the user in the test scripts are oauth2AuthzRequest and oauth2AuthzRedirect. These two fixtures are used to access parts the the last OAuth2 Authroization Request sent to the OAuth2 server and the last OAuth2 Authorization response returned from the server.

Base64Encoding

Touchstone is configured to use a Base64Encoding on the operation.requestHeader.value when it includes Basic + A Single Space “ “ ahead of the value and when the operation.requestHeader.name is equal to Authorization in your Testscript executions. This is done for security and is the explanation as to why an Authorization value in the header will be different from the one that was originally coded.

Bulk Data Capabilities

Touchstone has the ability for a user to create and use a Test System that supports the Bulk Data specification. Touchstone provides additional features that provide support for the evaluation and validation of the NDJSON returned files.

NDJSON File Evalutation and Validation

The Bulk Data specification defines a new FHIR extended operation $export which generates bulk data output files containing multiple FHIR resources using the NDJSON format for FHIR - application/fhir+ndjson.

Evaluation and validation of this NDJSON format for FHIR requires extended capabilities be implemented on the FHIR Test Engine and extended definitions within the FHIR TestScript resource. Touchstone has implemented these extended capabilities through the use of NDJSON Assertion Prefix syntax within the values of the following TestScript elements:

• TestScript.profile.reference
• TestScript.test.action.assert.expression
• TestScript.test.action.assert.path
• TestScript.test.action.assert.resource

NDJSON Assertion Prefix Syntax and Usage

NDJSON Assertion Prefix syntax is specified within curly braces proceeding the normal expected content of these element values and is composed of 3 optional parts:

{ Evaluation-Operator | Filter-Index-Range | Filter-Path } Regular-Assert

The NDJSON Assertion Prefix syntax is applied in 4 stages:

1. Filter-Path evaluation. Example: .name[?(@.family=='Gracia')] - all Patients whose name.family is 'Gracia'. A resource is included if the JSON-Path evaluation results in a Truthy value (exists and is not false). If no Filter-Path is specified then all resources filter through.
2. Filter-Index-Range. Example: 1-10 - include resources indexed 1 through 10 in the assertion evaluation. If Filter-Index-Range is not specified then all resources filter through. If Filter-Path is specified then Filter-Index-Range operates on the resources that made it through the Filter-Path and not the original resources.
3. Regular-Assert. The Profile/Expression/Path/Resource evaluation on resources that made it past 1 and 2. Example: Patient (if assertion is resource).
4. Evaluation-Operator. 'any' or 'all' - If 'any' operator is used then overall assertion evaluation passes if a single resource passes. The default is 'all'.

TestScript Example

The following TestScript provides a comprehensive list of asserts that show various usage examples of the NDJSON Assertion Prefix Syntax.

• NDJSON Assertion Prefix Syntax Example

Here are some portions from this TestScript illustrating key examples of the NDJSON Assertion Prefix syntax.

TestScript.profile.reference

Using the NDJSON Assertion Prefix syntax in the TestScript.profile.reference element allows for specific profile definitions with this syntax to be used in any assert.validateProfileId evaluation; i.e. where the FHIR Validation Engine is invoked. The syntax stages are applied in the same sequence where the FHIR Validation Engine invocation is executed a step 3 Regular-Assert.

Here are three (3) profile definitions and corresponding asserts showing the use of each optional NDJSON Assertion Prefix syntax item.

1. { Evaluation-Operator 'any' } Regular-Assert

   <profile id="resource-profile-for-any">
     <!-- Assert passes as long as any resource in the NDJSON contents validates successfully against the FHIR Patient profile. -->
     <reference value="{any}http://hl7.org/fhir/StructureDefinition/Patient"/>
   </profile>
   ...
   <action>
     <assert>
       <extension url="http://touchstone.aegis.net/touchstone/fhir/testing/StructureDefinition/testscript-assert-stopTestOnFail">
         <valueBoolean value="false"/>
       </extension>
       <description value="Validate that the returned resource conforms to the corresponding FHIR resource profile."/>
       <direction value="response"/>
       <validateProfileId value="resource-profile-for-any"/>
       <warningOnly value="false"/>
     </assert>
   </action>

2. { Filter-Index-Range } Regular-Assert

   <profile id="resource-profile-for-first">
     <!-- Assert passes as long as the first five (5) resources in the NDJSON contents validates successfully against the FHIR Patient profile. -->
     <reference value="{1-5}http://hl7.org/fhir/StructureDefinition/Patient"/>
   </profile>
   ...
   <action>
     <assert>
       <extension url="http://touchstone.aegis.net/touchstone/fhir/testing/StructureDefinition/testscript-assert-stopTestOnFail">
         <valueBoolean value="false"/>
       </extension>
       <description value="Validate that the returned resource conforms to the corresponding FHIR resource profile."/>
       <direction value="response"/>
       <validateProfileId value="resource-profile-for-first"/>
       <warningOnly value="false"/>
     </assert>
   </action>

3. { Filter-Path } Regular-Assert

   <profile id="resource-profile-for-Gracia">
     <!-- Assert passes as long as the those resources where name.family equals 'Garcia' in the NDJSON contents validates successfully against the FHIR Patient profile. -->
     <reference value="{.name[?(@.family=='Gracia')]}http://hl7.org/fhir/StructureDefinition/Patient"/>
   </profile>
   ...
   <action>
     <assert>
       <extension url="http://touchstone.aegis.net/touchstone/fhir/testing/StructureDefinition/testscript-assert-stopTestOnFail">
         <valueBoolean value="false"/>
       </extension>
       <description value="Validate that the returned resource conforms to the corresponding FHIR resource profile."/>
       <direction value="response"/>
       <validateProfileId value="resource-profile-for-Gracia"/>
       <warningOnly value="false"/>
     </assert>
   </action>

TestScript.test.action.assert.expression

The following asserts show the use of various combinations of the NDJSON Assertion Prefix syntax items within the assert.expression element. Please refer to each assert.description element for details of expected behavior.

1. { Evaluation-Operator 'any' } Regular-Assert

   <action>
     <assert>
       <extension url="http://touchstone.aegis.net/touchstone/fhir/testing/StructureDefinition/testscript-assert-stopTestOnFail">
         <valueBoolean value="false"/>
       </extension>
       <description value="Assert passes as long as any resource in the NDJSON contents contains a Patient.name.family value in 'Allen,Gracia'"/>
       <direction value="response"/>
       <expression value="{any}Patient.name.family"/>
       <operator value="in"/>
       <value value="Allen,Gracia"/>
       <warningOnly value="false"/>
     </assert>
   </action>

2. { Evaluation-Operator 'all' } Regular-Assert

   <action>
     <assert>
       <extension url="http://touchstone.aegis.net/touchstone/fhir/testing/StructureDefinition/testscript-assert-stopTestOnFail">
         <valueBoolean value="false"/>
       </extension>
       <description value="Assert passes as long as all resource in the NDJSON contents contain a Patient.name.family value equal to 'Allen'"/>
       <direction value="response"/>
       <expression value="{all}Patient.name.family"/>
       <operator value="equals"/>
       <value value="Allen"/>
       <warningOnly value="false"/>
     </assert>
   </action>

3. { Filter-Index-Range } Regular-Assert

   <action>
     <assert>
       <extension url="http://touchstone.aegis.net/touchstone/fhir/testing/StructureDefinition/testscript-assert-stopTestOnFail">
         <valueBoolean value="false"/>
       </extension>
       <description value="Assert passes as long as the first three (3) resources in the NDJSON contents contain a Patient.name.family value in 'McKay,Gracia,Allen'"/>
       <direction value="response"/>
       <expression value="{1-3}Patient.name.family"/>
       <operator value="in"/>
       <value value="McKay,Gracia,Allen"/>
       <warningOnly value="false"/>
     </assert>
   </action>

4. { Filter-Path } Regular-Assert

   <action>
     <assert>
       <extension url="http://touchstone.aegis.net/touchstone/fhir/testing/StructureDefinition/testscript-assert-stopTestOnFail">
         <valueBoolean value="false"/>
       </extension>
       <description value="Assert passes as long as the resources in the NDJSON contents that contain a name.family value equal to 'Allen' contain a Patient.name.given value in 'Carol,G'"/>
       <direction value="response"/>
       <expression value="{.name[?(@.family=='Allen')]}Patient.name.given"/>
       <operator value="in"/>
       <value value="Carol,G"/>
       <warningOnly value="false"/>
     </assert>
   </action>

5. { Filter-Index-Range | Filter-Path } Regular-Assert

   <action>
     <assert>
       <extension url="http://touchstone.aegis.net/touchstone/fhir/testing/StructureDefinition/testscript-assert-stopTestOnFail">
         <valueBoolean value="false"/>
       </extension>
       <description value="Assert passes as long as the first three (3) resources in the NDJSON contents that contain a name.family value equal to 'McKay' contain a Patient.name.given value equal to 'George'"/>
       <direction value="response"/>
       <expression value="{1-3 | .name[?(@.family=='McKay')]}Patient.name.given"/>
       <operator value="equals"/>
       <value value="George"/>
       <warningOnly value="false"/>
     </assert>
   </action>

6. { Evaluation-Operator 'any' | Filter-Index-Range | Filter-Path } Regular-Assert

   <action>
     <assert>
       <extension url="http://touchstone.aegis.net/touchstone/fhir/testing/StructureDefinition/testscript-assert-stopTestOnFail">
         <valueBoolean value="false"/>
       </extension>
       <description value="Assert passes as long as the any of the first three (3) resources in the NDJSON contents that contain a name.family value equal to 'McKay' contain a Patient.name.given value equal to 'George'"/>
       <direction value="response"/>
       <expression value="{any | 1-3 | .name[?(@.family=='McKay')]}Patient.name.given"/>
       <operator value="equals"/>
       <value value="George"/>
       <warningOnly value="false"/>
     </assert>
   </action>

TestScript.test.action.assert.path

The following asserts show the use of various combinations of the NDJSON Assertion Prefix syntax items within the assert.path element. Please refer to each assert.description element for details of expected behavior.

1. { Evaluation-Operator 'any' } Regular-Assert

   <action>
     <assert>
       <extension url="http://touchstone.aegis.net/touchstone/fhir/testing/StructureDefinition/testscript-assert-stopTestOnFail">
         <valueBoolean value="false"/>
       </extension>
       <description value="Assert passes as long as any resource in the NDJSON contents contains a generalPractitioner.reference value"/>
       <direction value="response"/>
       <operator value="equals"/>
       <path value="{any}generalPractitioner.reference"/>
       <value value="Practitioner/2"/>
       <warningOnly value="false"/>
     </assert>
   </action>

2. { Filter-Index-Range } Regular-Assert

   <action>
     <assert>
       <extension url="http://touchstone.aegis.net/touchstone/fhir/testing/StructureDefinition/testscript-assert-stopTestOnFail">
         <valueBoolean value="false"/>
       </extension>
       <description value="Assert passes as long as the 2-5 resources in the NDJSON contents contain a generalPractitioner.reference value that contains 'Practitioner'"/>
       <direction value="response"/>
       <operator value="contains"/>
       <path value="{2-5}generalPractitioner.reference"/>
       <value value="Practitioner/"/>
       <warningOnly value="false"/>
     </assert>
   </action>

3. { Filter-Path } Regular-Assert

   <action>
     <assert>
       <extension url="http://touchstone.aegis.net/touchstone/fhir/testing/StructureDefinition/testscript-assert-stopTestOnFail">
         <valueBoolean value="false"/>
       </extension>
       <description value="Assert passes as long as the resources in the NDJSON contents that contain a generalPractitioner.reference value equal to 'Practitioner/3' contain a name.family value equal to 'Allen'"/>
       <direction value="response"/>
       <operator value="equals"/>
       <path value="{generalPractitioner[?(@.reference=='Practitioner/3')]}name.family"/>
       <value value="Allen"/>
       <warningOnly value="false"/>
     </assert>
   </action>

TestScript.test.action.assert.resouce

The following asserts show the use of various combinations of the NDJSON Assertion Prefix syntax items within the assert.resource element. Please refer to each assert.description element for details of expected behavior.

WARNING - THE USE OF THE NDJSON Assertion Prefix SYNTAX WITHIN THE assert.resource ELEMENT PREVENTS THE CONTAINING TESTSCRIPT RESOURCE FROM VALIDATING ON UPLOAD TO TOUCHSTONE.

1. { Evaluation-Operator 'any' } Regular-Assert

   <action>
     <assert>
       <extension url="http://touchstone.aegis.net/touchstone/fhir/testing/StructureDefinition/testscript-assert-stopTestOnFail">
         <valueBoolean value="false"/>
       </extension>
       <description value="Assert passes as long as any resource in the NDJSON contents is a FHIR Patient resource type"/>
       <direction value="response"/>
       <resource value="{any}Patient"/>
       <warningOnly value="false"/>
     </assert>
   </action>

2. { Filter-Index-Range } Regular-Assert

   <action>
     <assert>
       <extension url="http://touchstone.aegis.net/touchstone/fhir/testing/StructureDefinition/testscript-assert-stopTestOnFail">
         <valueBoolean value="false"/>
       </extension>
       <description value="Assert passes as long as the first five (5) resources in the NDJSON contents is a FHIR Patient resource type"/>
       <direction value="response"/>
       <resource value="{1-5}Patient"/>
       <warningOnly value="false"/>
     </assert>
   </action>

3. { Filter-Path } Regular-Assert

   <action>
     <assert>
       <extension url="http://touchstone.aegis.net/touchstone/fhir/testing/StructureDefinition/testscript-assert-stopTestOnFail">
         <valueBoolean value="false"/>
       </extension>
       <description value="Assert passes as long as all the resources in the NDJSON contents that contain a name.family value equal to 'Gracia' is a FHIR Patient resource type"/>
       <direction value="response"/>
       <resource value="{.name[?(@.family=='Gracia')]}Patient"/>
       <warningOnly value="false"/>
     </assert>
   </action>