Multiple Assertions

This section covers how assertions can be performed on multiple requests and responses and reporting all of the assertion failures and warnings.

Short-circuiting when a request or response assertion fails

This is the behavior that has been covered extensively in Rule API.

To grab the last response in TestScript Execution and perform an assertion on the response code, for example, we can use the following construct:

response.assertResponseCodeEquals(200)

To perform assertion on a response in TestScript Execution other than last response (e.g. one that was received upstream in another test), we could first grab the response from responses as such:

def createResponse = responses.get('create-read-response');

The responseId ‘create-read-response’ in the example above would have been specified by the TestScript author in the TestScript operation definition and would be shown on the TestScript Execution screen:

A series of assertions could then be performed using the Rule API:

createResponse.assertResourceEquals("Bundle")
assert createResponse.header("Content-Type").contains("json") || createResponse.header("Content-Type").contains("xml"): "The actual value \""+response.header('Content-Type')?.value +"\" did not contain the expected value \"json\" or \"xml\" for 'Content-Type' header in response."
createResponse.assertResponseCodeEquals(200)
createResponse.assertBodyNotEmpty()
createResponse.assertValidWithProfile("http://hl7.org/fhir/StructureDefinition/Bundle")

The rule execution will fail on the first assertion failure. This is good in that we wouldn’t want assertValidWithProfile to be performed if assertBodyNotEmpty failed, for example.

If we wanted to perform assertions on multiple requests and responses and wanted the assertion failures to be reported on all of the requests and responses, then the above approach would not work as rule-execution would stop on the first request or response assertion failure.

Continuing rule-execution when a request or response assertion fails

If multiple assertions are performed in a rule (e.g. on all search responses) and one of the assertions fail, then subsequent assertions are not performed. To perform all the assertions, the rule author must catch the assertion failure and register the error with the rules-engine before proceeding with the other assertions. Touchstone will fail the overall rule assertion if one or more assertions fail and will consolidate all the assertion failure messages into one message.

Here’s an example that demonstrates how to perform assertions on multple responses without short-circuiting on any given response assertion failure:

responses.each() { responseEntry ->
   def responseId = responseEntry.key
   if (responseId.startsWith("search-response")) { // The prefix can be passed as a rule parameter from the test script
      try {
         def response = responseEntry.value

         logger.info("Validating Resource in "+responseId)
         response.assertResourceEquals("Bundle")

         logger.info("Validating Content-Type in "+responseId)
         assert response.header("Content-Type").contains("json") || response.header("Content-Type").contains("xml"): "The actual value \""+response.header('Content-Type')?.value +"\" did not contain the expected value \"json\" or \"xml\" for 'Content-Type' header in response."

         logger.info("Validating Response Code in "+responseId)
         response.assertResponseCodeEquals(200)

         logger.info("Validating Body in in "+responseId)
         response.assertBodyNotEmpty()

         logger.info("Validating Resource in "+responseId+"\n")
         response.assertValidWithProfile("http://hl7.org/fhir/StructureDefinition/Bundle")
      } catch (Throwable e) {
         errorsAndWarnings.registerAndContinue(e);
      }
   }
}

Notice how we’re iterating through the responses and registering any assertion errors with the Rules Engine using errorsAndWarnings.registerAndContinue(e);, and then proceeding with assertions on the next response.

The log messages will be accessible on TestScript Execution screen within the assertion Log Output link:

Note

When writing rules that act on requests and responses other than the last one in the TestScript Execution, it’s best to set sourceId to ‘none’ in the assertion rule definition. The sourceId can be left out if the rule were operating on the last request or response.