FAQ

1. The ‘% Conformance’ on the Current page is sometimes different from Result Summary page for Unpublished results of a given suite and test system. Why is that? If a suite is newly created or the version has incremented, there might still be test executions for the test scripts contained within the suite against the selected system. The Current screen takes into account all those test executions whereas Results Summary screen does not. The two screens will get synchronized upon completion of the next test execution.

   This is indicated on the Current screen when the two results are out of sync:

   

2. I’ve selected a band and don’t see any test scripts. Why is that? Make sure you’ve cleared the filter box (blue arrow below):

   

3. When I click on the Test Script link, it sometimes pops the TestScript resource definition and at other times it navigates to the Test Script Execution screen. If the test script has never been executed before (as is the case below), clicking on the Test Script link will pop the Test Script resource definition.

   

   If the test script has been executed already, then clicking on the Test Script link will take you to the Test Script Execution screen. Aftr launching the execution, you may have to wait a few seconds before the Start time appears:

   

4. I reached a higher ‘% Conformance’ and the system has knocked the % down. I was green on certain bands and now the system shows gray again. Why is that? This can happen if the conformance suite version has incremented. It would be erroneous for Touchstone to continue to show green or red for a conformace suite version that you haven’t executed yet.

   The conformance results of previous versions are maintained though and you can see them on Results Summary page by changing the Version dropdown:

   

5. The Conformance Current page is tedious. I don’t want to execute one test script at a time after I’ve reached my desired ‘% Conformance’ level. You don’t need to. Touchstone stores the variable values you entered for test scripts that require variable values. You can click on the root node (/FHIR4-0-1-Basic in this case), change the page size to 300, check the “All” checkbox and click on “Execute Selected”. This will launch all the test scripts as part of one execution.

   

6. Does my ‘% Conformance’ stay the same from one version of the suite to the next? No. It is monitored by version. The ‘% Conformance’ resets to zero when a conformance suite version increments. Changes to test groups, categorization, and other components of the suite could drastically change the interaction totals and meaning.

   Please refer to Versioning for the list of events that can cause the suite version to increment.

7. Are the Conformance results going to be different from the results that I would get by executing tests via Test Definitions screen? They are the same from execution standpoint. Test executions launched from Test Definitions screen can be viewed on Conformance screens and vice versa. Every launch of test execution on Conformance screens creates a Test Setup in the background and launches it. You can view those test setups on Test Setups screen and launch from there if you like. The difference though is that Conformance screens relay the test system’s conformance level while the test executions screens don’t.

8. Why does my test system show up in the Test System dropdown if it doesn’t even support the validator version that the suite is on? You may choose to execute test scripts of one specification version against a test system that hasn’t been declared to support that specification version yet and see how it responds. After all, this is a testing environment. For that reason, we don’t filter test system dropdown based on suite’s validator version. The dropdown though displays only test systems that’s accessible by you and not all test systems.