AEGIS Touchstone Testing Implementation Guide

This is the Version 1.4.0 Release of the Touchstone Testing Implementation Guide, based on FHIR Version 4.0.1. See the Directory of published versions

General Guidance

This section outlines important definitions and interpretations and requirements common to all AEGIS Touchstone Testing actors used in this guide. The conformance verbs used are defined in FHIR Conformance Rules.

Contents

Touchstone Placeholders

The Touchstone Placeholders are special predefined TestScript variables and may be used where standard TestScript variables are allowed: static fixtures and operation.params, operation.requestHeader.value, operation.url, and assert.value element values.

These Touchstone Placeholders fall into two (2) categories

  • Predefined User Unique Data Values
  • Functions for Dynamic Data Generation

Predefined User Unique Data Values

The predefined user unique data value placeholders are strings of varying lengths from 1 to 20 characters. They are defined of three (3) types:

  • Character only; for example, ${C6}
  • Digits only; for example, ${D9}
  • Characters + Digits; for example, ${CD14}

Users may view their assigned placeholder variable values in Touchstone via the user name menu item ${} My Placeholders.

Guaranteed Value Uniqueness - All predefined user unique data value placeholder values with a character length of 6 or more are guaranteed to be unique.

Functions for Dynamic Data Generation

There are three (3) available functional constructs for dynamic data generation:

  • CURRENTDATE and CURRENTDATETIME
  • DATE and DATETIME (relative to dynamic variable)
  • UUID, UUID-ST, UUID-NODASH and UUID-ST-NODASH

CURRENTDATE[TIME]

These function placeholder variables provide support for date and datetime values based on the current date (today) and current datetime (now). Optional comma separated offset arguments apply date and time arithmetic providing relative date and time generated values.

Syntax/Format ${<placeholder name>[, <datetime portion code>, <offset value>]} Where,

  • <placeholder name> is CURRENTDATE or CURRENTDATETIME
  • <datetime portion code> is the character indicating what portion of the date or datetime value will be offset. Valid characters are derived from the following Java SimpleDateFormat pattern string “yyMMddHHmmss”
  • <offset value> is the signed integer value used to adjust the date or time
  • <datetime portion code>, <offset value> may be repeated for more complex arithmetic

Examples

Placeholder Example Description
${CURRENTDATE} The current date (today)
${CURRENTDATETIME} The current date and time (today)
${CURRENTDATE,d,-10} The date 10 days before the current date
${CURRENTDATETIME,d,10} The date and time 10 days after the current date and time
${CURRENTDATE,y,-1} The date 1 year before the current date
${CURRENTDATETIME,H,10} The date and time 10 hours after the current date and time

DATE[TIME]

These function placeholder variables provide support for date and datetime values based on a dynamic variable user input date or datetime value. Optional comma separated offset arguments apply date and time arithmetic providing relative date and time generated values.

Syntax/Format ${<placeholder name>, <relative value>[, <datetime portion code>, <offset value>]} Where,

  • <placeholder name> is DATE or DATETIME
  • <relative value> is a dynamic variable defined in the current TestScript that holds the relative date or datetime value
    • The dynamic variable must be defined in TestScript without path/expression; its value will be entered by the user during Test Setup
  • <datetime portion code> is the character indicating what portion of the date or datetime value will be offset. Valid characters are derived from the following Java SimpleDateFormat pattern string “yyMMddHHmmss”
  • <offset value> is the signed integer value used to adjust the date or time
  • <datetime portion code>, <offset value> may be repeated for more complex arithmetic

Examples

Placeholder Example Description
${DATE, medicationDate} The value of the user entered medicationDate will be used as-is
${DATETIME, medicationDateTime} The value of the user entered medicationDateTime will be used as-is
${DATE, medicationDate, d, -10} The value of the user entered medicationDate minus 10 days will be used
${DATETIME, medicationDateTime, M, -1} The value of the user entered medicationDateTime minus 1 month will be used

UUID[-ST|-NODASH|-ST-NODASH]

These function placeholder variables provide support for generation of UUID values with predefined formatting.

Syntax/Format ${<placeholder name>} Where,

  • <placeholder name> is UUID, UUID-ST, UUID-NODASH or UUID-ST-NODASH

Examples

Placeholder Example Description
${UUID} A single generated UUID string value without the standard prefix; e.g., '3ed6eb79-fc68-443a-996f-08167f5bdef0'
${UUID-ST} A single generated UUID string value including the standard prefix; e.g., 'urn:uuid:3ed6eb79-fc68-443a-996f-08167f5bdef0'
${UUID-NODASH} A single generated UUID string value without the standard prefix and dash characters removed; e.g., '3ed6eb79fc68443a996f08167f5bdef0'
${UUID-ST-NODASH} A single generated UUID string value including the standard prefix and dash characters removed; e.g., 'urn:uuid:3ed6eb79fc68443a996f08167f5bdef0'

Usage

The Touchstone Placeholder, both predefined values and functions, are typically defined in static fixtures used as an operation request payload; for example, create or update. Touchstone’s TestScript Execution interface provides both a Raw and Resolved view of static fixtures where the pre and post test execution contents can be examined.

Raw Example - Patient.name, Patient.birthDate

<name>
  <use value="official"/>
  <family value="Smith${C7}"/>
  <given value="John${C6}"/>
  <birthDate value="${CURRENTDATE,d,-7}"/>
</name>

Resolved Example - Patient.name, Patient.birthDate

<name>
  <use value="official"/>
  <family value="SmithMqCERSQ"/>
  <given value="JohnXRCnsc"/>
  <birthDate value="2021-01-27"/>
</name>

Rule and RuleSet Extensions

The TestScript rule and ruleset extensions can be used to reference complex validation logic that goes beyond what the basic TestScript assert element supports. As such, rules are recommended only when a TestScript assert cannot be used in its basic form.

The Touchstone Rules-Engine supports rules written in the following languages:

  • Groovy
  • XSLT
  • Schematron

Support for additional languages may be added in the future. Unless you plan on executing test scripts against a test system that only supports XML, it is highly recommended to write rules in Groovy as XSLT and Schematron rules can only be evaluated against requests and responses whose content is in XML while Groovy supports JSON as well.

OAuth2 Capabilities

Touchstone has the ability for a user to create and use a Test System that is OAuth2 enabled, and with that comes new features that are important to take note of when it comes to Test Executions performed against a Test System that has an OAuth2 Authorization service connected to it.

TestScript Extensions

With the addition of a more robust OAuth2, SMART-on-FHIR and Bulk Data support into Touchstone, there are new extensions under the TestScript.test and TestScript.test.operation elements to support the manual control of test execution behavior and execution of the various OAuth2 authorization flows that are available to test script authors.

Pre-defined Fixture Ids

With the addition of a more robust OAuth2 environment into Touchstone, there are new fixtures, and values under those fixtures, that are available to test script authors.

  • One new fixture is the dest1SmartConfig (dest2SmartConfig, dest3SmartConfig, etc.). The fixture allows the test script author to access different variables coming from the JSON document that is at the .well-known/smart-configuration.json endpoint. It acts the same as retrieving variables from the capabilities statement. For example, to use dest1SmartConfig to retrieve the Client ID, you would use the sourceId and path in the testscript as sourceId = dest1SmartConfig, path = $.clientId.

  • Test Systems have special Oauth2 values that can be be retrieved by test script authors. The fixtures have a naming convention of dest1SystemConfig, dest2SystemConfig, origin1SystemConfig, origin2SystemConfig, etc. These fixtures have the following attributes:

Attribute Name Description
name The Test System name
fullName The Organization name + the Test System name
baseUrl The Base URL of the Test System
supportsSmartOnFhir Set to true if the Test System supports SMART on FHIR, false if it does not
oauth2GrantType The Grant Type of the OAuth2 enables Test System, either Authorization Code or Client Credentials
clientId The Client ID of the Test System that is registered with the OAuth2 server.
clientSecret The Client Secret of the Test System that is registered with the OAuth2 server.
authEndpoint The Authorization Endpoint for the Test System.
tokenEndpoint The Token Endpoint for the Test System.
registerEndpoint The Registration Endpoint for the Test System.
introspectEndpoint The Introspection Endpoint for the Test System.
revocationEndpoint The Revocation Endpoint for the Test System.
scopesSupported The scopes that allowed for the server access to certain user scopes
  • Another set of fixtures that can be used by the user in the test scripts are oauth2AuthzRequest and oauth2AuthzRedirect. These two fixtures are used to access parts the the last OAuth2 Authroization Request sent to the OAuth2 server and the last OAuth2 Authorization response returned from the server.

Base64Encoding

Touchstone is configured to use a Base64Encoding on the operation.requestHeader.value when it includes Basic + A Single Space “ “ ahead of the value and when the operation.requestHeader.name is equal to Authorization in your Testscript executions. This is done for security and is the explanation as to why an Authorization value in the header will be different from the one that was originally coded.

Bulk Data Capabilities

Touchstone has the ability for a user to create and use a Test System that supports the Bulk Data specification. Touchstone provides additional features that provide support for the evaluation and validation of the NDJSON returned files.

NDJSON File Evalutation and Validation

The Bulk Data specification defines a new FHIR extended operation $export which generates bulk data output files containing multiple FHIR resources using the NDJSON format for FHIR - application/fhir+ndjson.

Evaluation and validation of this NDJSON format for FHIR requires extended capabilities be implemented on the FHIR Test Engine and extended definitions within the FHIR TestScript resource. Touchstone has implemented these extended capabilities through the use of NDJSON Assertion Prefix syntax within the values of the following TestScript elements:

  • TestScript.profile.reference
  • TestScript.test.action.assert.expression
  • TestScript.test.action.assert.path
  • TestScript.test.action.assert.resource

NDJSON Assertion Prefix Syntax and Usage

NDJSON Assertion Prefix syntax is specified within curly braces proceeding the normal expected content of these element values and is composed of 3 optional parts:

{ Evaluation-Operator | Filter-Index-Range | Filter-Path } Regular-Assert

The NDJSON Assertion Prefix syntax is applied in 4 stages:

  1. Filter-Path evaluation. Example: .name[?(@.family=='Gracia')] - all Patients whose name.family is 'Gracia'. A resource is included if the JSON-Path evaluation results in a Truthy value (exists and is not false). If no Filter-Path is specified then all resources filter through.
  2. Filter-Index-Range. Example: 1-10 - include resources indexed 1 through 10 in the assertion evaluation. If Filter-Index-Range is not specified then all resources filter through. If Filter-Path is specified then Filter-Index-Range operates on the resources that made it through the Filter-Path and not the original resources.
  3. Regular-Assert. The Profile/Expression/Path/Resource evaluation on resources that made it past 1 and 2. Example: Patient (if assertion is resource).
  4. Evaluation-Operator. 'any' or 'all' - If 'any' operator is used then overall assertion evaluation passes if a single resource passes. The default is 'all'.

TestScript Example

The following TestScript provides a comprehensive list of asserts that show various usage examples of the NDJSON Assertion Prefix Syntax.

Here are some portions from this TestScript illustrating key examples of the NDJSON Assertion Prefix syntax.

TestScript.profile.reference

Using the NDJSON Assertion Prefix syntax in the TestScript.profile.reference element allows for specific profile definitions with this syntax to be used in any assert.validateProfileId evaluation; i.e. where the FHIR Validation Engine is invoked. The syntax stages are applied in the same sequence where the FHIR Validation Engine invocation is executed a step 3 Regular-Assert.

Here are three (3) profile definitions and corresponding asserts showing the use of each optional NDJSON Assertion Prefix syntax item.

  1. { Evaluation-Operator 'any' } Regular-Assert 
    <profile id="resource-profile-for-any">
  <!-- Assert passes as long as any resource in the NDJSON contents validates successfully against the FHIR Patient profile. -->
  <reference value="{any}http://hl7.org/fhir/StructureDefinition/Patient"/>
</profile>
...
<action>
  <assert>
    <extension url="http://touchstone.aegis.net/touchstone/fhir/testing/StructureDefinition/testscript-assert-stopTestOnFail">
      <valueBoolean value="false"/>
    </extension>
    <description value="Validate that the returned resource conforms to the corresponding FHIR resource profile."/>
    <direction value="response"/>
    <validateProfileId value="resource-profile-for-any"/>
    <warningOnly value="false"/>
  </assert>
</action>
  2. { Filter-Index-Range } Regular-Assert 
    <profile id="resource-profile-for-first">
  <!-- Assert passes as long as the first five (5) resources in the NDJSON contents validates successfully against the FHIR Patient profile. -->
  <reference value="{1-5}http://hl7.org/fhir/StructureDefinition/Patient"/>
</profile>
...
<action>
  <assert>
    <extension url="http://touchstone.aegis.net/touchstone/fhir/testing/StructureDefinition/testscript-assert-stopTestOnFail">
      <valueBoolean value="false"/>
    </extension>
    <description value="Validate that the returned resource conforms to the corresponding FHIR resource profile."/>
    <direction value="response"/>
    <validateProfileId value="resource-profile-for-first"/>
    <warningOnly value="false"/>
  </assert>
</action>
  3. { Filter-Path } Regular-Assert 
    <profile id="resource-profile-for-Gracia">
  <!-- Assert passes as long as the those resources where name.family equals 'Garcia' in the NDJSON contents validates successfully against the FHIR Patient profile. -->
  <reference value="{.name[?(@.family=='Gracia')]}http://hl7.org/fhir/StructureDefinition/Patient"/>
</profile>
...
<action>
  <assert>
    <extension url="http://touchstone.aegis.net/touchstone/fhir/testing/StructureDefinition/testscript-assert-stopTestOnFail">
      <valueBoolean value="false"/>
    </extension>
    <description value="Validate that the returned resource conforms to the corresponding FHIR resource profile."/>
    <direction value="response"/>
    <validateProfileId value="resource-profile-for-Gracia"/>
    <warningOnly value="false"/>
  </assert>
</action>

TestScript.test.action.assert.expression

The following asserts show the use of various combinations of the NDJSON Assertion Prefix syntax items within the assert.expression element. Please refer to each assert.description element for details of expected behavior.

  1. { Evaluation-Operator 'any' } Regular-Assert 
    <action>
  <assert>
    <extension url="http://touchstone.aegis.net/touchstone/fhir/testing/StructureDefinition/testscript-assert-stopTestOnFail">
      <valueBoolean value="false"/>
    </extension>
    <description value="Assert passes as long as any resource in the NDJSON contents contains a Patient.name.family value in 'Allen,Gracia'"/>
    <direction value="response"/>
    <expression value="{any}Patient.name.family"/>
    <operator value="in"/>
    <value value="Allen,Gracia"/>
    <warningOnly value="false"/>
  </assert>
</action>
  2. { Evaluation-Operator 'all' } Regular-Assert 
    <action>
  <assert>
    <extension url="http://touchstone.aegis.net/touchstone/fhir/testing/StructureDefinition/testscript-assert-stopTestOnFail">
      <valueBoolean value="false"/>
    </extension>
    <description value="Assert passes as long as all resource in the NDJSON contents contain a Patient.name.family value equal to 'Allen'"/>
    <direction value="response"/>
    <expression value="{all}Patient.name.family"/>
    <operator value="equals"/>
    <value value="Allen"/>
    <warningOnly value="false"/>
  </assert>
</action>
  3. { Filter-Index-Range } Regular-Assert 
    <action>
  <assert>
    <extension url="http://touchstone.aegis.net/touchstone/fhir/testing/StructureDefinition/testscript-assert-stopTestOnFail">
      <valueBoolean value="false"/>
    </extension>
    <description value="Assert passes as long as the first three (3) resources in the NDJSON contents contain a Patient.name.family value in 'McKay,Gracia,Allen'"/>
    <direction value="response"/>
    <expression value="{1-3}Patient.name.family"/>
    <operator value="in"/>
    <value value="McKay,Gracia,Allen"/>
    <warningOnly value="false"/>
  </assert>
</action>
  4. { Filter-Path } Regular-Assert 
    <action>
  <assert>
    <extension url="http://touchstone.aegis.net/touchstone/fhir/testing/StructureDefinition/testscript-assert-stopTestOnFail">
      <valueBoolean value="false"/>
    </extension>
    <description value="Assert passes as long as the resources in the NDJSON contents that contain a name.family value equal to 'Allen' contain a Patient.name.given value in 'Carol,G'"/>
    <direction value="response"/>
    <expression value="{.name[?(@.family=='Allen')]}Patient.name.given"/>
    <operator value="in"/>
    <value value="Carol,G"/>
    <warningOnly value="false"/>
  </assert>
</action>
  5. { Filter-Index-Range | Filter-Path } Regular-Assert 
    <action>
  <assert>
    <extension url="http://touchstone.aegis.net/touchstone/fhir/testing/StructureDefinition/testscript-assert-stopTestOnFail">
      <valueBoolean value="false"/>
    </extension>
    <description value="Assert passes as long as the first three (3) resources in the NDJSON contents that contain a name.family value equal to 'McKay' contain a Patient.name.given value equal to 'George'"/>
    <direction value="response"/>
    <expression value="{1-3 | .name[?(@.family=='McKay')]}Patient.name.given"/>
    <operator value="equals"/>
    <value value="George"/>
    <warningOnly value="false"/>
  </assert>
</action>
  6. { Evaluation-Operator 'any' | Filter-Index-Range | Filter-Path } Regular-Assert 
    <action>
  <assert>
    <extension url="http://touchstone.aegis.net/touchstone/fhir/testing/StructureDefinition/testscript-assert-stopTestOnFail">
      <valueBoolean value="false"/>
    </extension>
    <description value="Assert passes as long as the any of the first three (3) resources in the NDJSON contents that contain a name.family value equal to 'McKay' contain a Patient.name.given value equal to 'George'"/>
    <direction value="response"/>
    <expression value="{any | 1-3 | .name[?(@.family=='McKay')]}Patient.name.given"/>
    <operator value="equals"/>
    <value value="George"/>
    <warningOnly value="false"/>
  </assert>
</action>

TestScript.test.action.assert.path

The following asserts show the use of various combinations of the NDJSON Assertion Prefix syntax items within the assert.path element. Please refer to each assert.description element for details of expected behavior.

  1. { Evaluation-Operator 'any' } Regular-Assert 
    <action>
  <assert>
    <extension url="http://touchstone.aegis.net/touchstone/fhir/testing/StructureDefinition/testscript-assert-stopTestOnFail">
      <valueBoolean value="false"/>
    </extension>
    <description value="Assert passes as long as any resource in the NDJSON contents contains a generalPractitioner.reference value"/>
    <direction value="response"/>
    <operator value="equals"/>
    <path value="{any}generalPractitioner.reference"/>
    <value value="Practitioner/2"/>
    <warningOnly value="false"/>
  </assert>
</action>
  2. { Filter-Index-Range } Regular-Assert 
    <action>
  <assert>
    <extension url="http://touchstone.aegis.net/touchstone/fhir/testing/StructureDefinition/testscript-assert-stopTestOnFail">
      <valueBoolean value="false"/>
    </extension>
    <description value="Assert passes as long as the 2-5 resources in the NDJSON contents contain a generalPractitioner.reference value that contains 'Practitioner'"/>
    <direction value="response"/>
    <operator value="contains"/>
    <path value="{2-5}generalPractitioner.reference"/>
    <value value="Practitioner/"/>
    <warningOnly value="false"/>
  </assert>
</action>
  3. { Filter-Path } Regular-Assert 
    <action>
  <assert>
    <extension url="http://touchstone.aegis.net/touchstone/fhir/testing/StructureDefinition/testscript-assert-stopTestOnFail">
      <valueBoolean value="false"/>
    </extension>
    <description value="Assert passes as long as the resources in the NDJSON contents that contain a generalPractitioner.reference value equal to 'Practitioner/3' contain a name.family value equal to 'Allen'"/>
    <direction value="response"/>
    <operator value="equals"/>
    <path value="{generalPractitioner[?(@.reference=='Practitioner/3')]}name.family"/>
    <value value="Allen"/>
    <warningOnly value="false"/>
  </assert>
</action>

TestScript.test.action.assert.resouce

The following asserts show the use of various combinations of the NDJSON Assertion Prefix syntax items within the assert.resource element. Please refer to each assert.description element for details of expected behavior.

WARNING - THE USE OF THE NDJSON Assertion Prefix SYNTAX WITHIN THE assert.resource ELEMENT PREVENTS THE CONTAINING TESTSCRIPT RESOURCE FROM VALIDATING ON UPLOAD TO TOUCHSTONE.

  1. { Evaluation-Operator 'any' } Regular-Assert 
    <action>
  <assert>
    <extension url="http://touchstone.aegis.net/touchstone/fhir/testing/StructureDefinition/testscript-assert-stopTestOnFail">
      <valueBoolean value="false"/>
    </extension>
    <description value="Assert passes as long as any resource in the NDJSON contents is a FHIR Patient resource type"/>
    <direction value="response"/>
    <resource value="{any}Patient"/>
    <warningOnly value="false"/>
  </assert>
</action>
  2. { Filter-Index-Range } Regular-Assert 
    <action>
  <assert>
    <extension url="http://touchstone.aegis.net/touchstone/fhir/testing/StructureDefinition/testscript-assert-stopTestOnFail">
      <valueBoolean value="false"/>
    </extension>
    <description value="Assert passes as long as the first five (5) resources in the NDJSON contents is a FHIR Patient resource type"/>
    <direction value="response"/>
    <resource value="{1-5}Patient"/>
    <warningOnly value="false"/>
  </assert>
</action>
  3. { Filter-Path } Regular-Assert 
    <action>
  <assert>
    <extension url="http://touchstone.aegis.net/touchstone/fhir/testing/StructureDefinition/testscript-assert-stopTestOnFail">
      <valueBoolean value="false"/>
    </extension>
    <description value="Assert passes as long as all the resources in the NDJSON contents that contain a name.family value equal to 'Gracia' is a FHIR Patient resource type"/>
    <direction value="response"/>
    <resource value="{.name[?(@.family=='Gracia')]}Patient"/>
    <warningOnly value="false"/>
  </assert>
</action>