Server Suites

Note

This section assumes that you are already familiar with Conformance Testing. Please review the Conformance Testing documentation for additional information.


Test systems can be Server or Client test systems in Touchstone. This attribute is selected by the user on the Test System screen.

../_images/client_server_attribute_a2.png

The following steps describe how to create Conformance Suites that are geared towards testing Server test systems:

  1. Go to Conformance Suites page.


  1. Click on New Conformance Suite button:

    ../_images/new_conformance_suite_button_a1.png

  2. You will encounter the following error if you haven’t uploaded test scripts to Touchstone yet:

    ../_images/test_groups_required_a1.png

    Note

    Conformance Suites can be created only with test groups that your organization owns. Otherwise, the test group owner could inadvertently impact the Conformance results of many users in your program when a test group is modified.

To learn how to upload test groups to Touchstone, you can refer to the TestScript Authoring guide.

The user then uploads a few test groups:

../_images/test_groups_uploaded_a1.png

  1. Click on New Conformance Suite button again on the Conformance Suites page:

    ../_images/new_conformance_suite_button_a1.png

  2. Below is the New Conformance Suite page:

    ../_images/new_conf_sute_page_a1.png

    We will describe the meaning of each field as we populate them in this screen.


  1. Enter the Name of the suite:

    ../_images/new_suite_name_a2.png

    Note

    Touchstone will prepend the specification name/version to the suite name. This is done to maintain uniformity and consistent naming convention across suites so it’s easier for users to identify the specification that a suite is testing for. The type of suite (Server vs Client) is also appended by Touchstone to the name for the same reason.

The final name of the suite in this instance would be the following:

../_images/new_suite_name_final_a1.png

  1. Select the Type of the suite. For a suite that’s geared towards testing Server test systems, the type would be Server:

    ../_images/new_suite_type_a1.png

  1. Select the Anchor System of the suite. For a suite that’s geared towards testing Server test systems (the SUT of the suite), the Anchor System would be a Client/Origin test system:

    ../_images/new_suite_anchor_system_a1.png

    Conversely, if the suite is geared towards testing Client test systems (the SUT of the suite), the Anchor System would be a Server/Destination test system.

    Anchor systems are needed to achieve comparable Conformance results among SUTs. If a suite uses different Anchor Systems, then the results would not be meaningful.

    Anchor systems must be accessible by the organization that’s creating the suite.

    Note

    It is highly recommended to use reliable anchor systems for your suites. This is to avoid unintended consequences on the program’s conformance testing if the anchor system is deleted or its behavior is modified by the owning organization. Touchstone does not require the suite owner to also own the anchor system because it’s not easy to make such systems available. It’s rather common for organizations to rely on third parties for such systems. In this case, we’re using TouchstoneFHIR as the Client anchor system from which requests will be sent to the Server test systems being tested in the suite.

  2. Select the Valdiator of the suite:

    ../_images/new_suite_validator_a1.png

    Validator is the AEGIS Validator to use for validating request and response payloads during test execution.

    The list of Validators offered in the dropdown will be limited to the Validators used by the Test Groups owned by the suite author’s organization.

  1. Select the Test Group(s) of the suite:

    ../_images/new_suite_test_group_a2.png

    Test Groups will contain the test scripts that the user will execute in conformance testing. The Test Group dropdown entries will be driven by the Validator selection. If you select FHIR 3.0.2 as the Validator, for example, then the Test Group dropdown will be populated with FHIR 3.0.2 test groups that you have access to.

    Additional test groups can be selected by clicking the + sign:

    ../_images/new_suite_test_group_plus_a1.png

    We’ll cover multiple test group selection in the Categorization section of this guide.

    Note

    In order to be able to properly select test groups that were uploaded prior to Touchstone 5.0.0 release, you must re-upload them to Touchstone.


  1. Enter a Description for the suite:

../_images/new_suite_description_a1.png

The system will generate a default description if none is entered.


  1. Choose whether or not Supported checkbox is offered to users:

../_images/new_suite_supported_a2.png

If checked, users will be offered the Supported by CapabStmt checkbox on conformance results screens. They would be able to filter results by supported interactions in the capability statement in addition to the default (All):

../_images/new_suite_supported_effect_a1.png

Please refer to this section to see how this checkbox selection affects conformance results.


  1. Choose whether or not Formats dropdown is offered to users:

../_images/new_suite_formats_a2.png

If checked, users will be offered the Format dropdown on conformance results screens. They would be able to filter results by individual formats (JSON / XML) in addition to the default (All):

../_images/new_suite_formats_effect_a1.png

Please refer to this section to see how this checkbox selection affects conformance results.


  1. Choose whether or not conformance results can be Published by users:

../_images/new_suite_publishable_a1.png

If checked, users will be offered the option of publishing conformance results. Those results along with the associated test executions would become publicly accessible if published:

../_images/publishing_publish_button_a2.png

Please refer to Publishing to learn more about conformance results publishing.


  1. Select the access attributes of the suite:

../_images/new_suite_access_a1.png

This works the same way as access attributes for test systems, test definitions, etc.

If you’d like only your organization to view and execute tests within the suite, then select My Organization for Can be viewed by.


  1. Click on Create. Conformance Suite gets created:

../_images/conf_suite_created_a2.png

We selected /FHIRSandbox/Initech/FHIR3-0-2-S-Basic as the sole test group and TouchstoneFHIR as the anchor system for this suite.


  1. Click on the name of the suite:

../_images/conf_suite_name_click_a1.png

  1. You’ll be taken to the Current Conformance page where you can test the suite:

../_images/conf_current_a2.png

  • The selected Conformance Suite (red arrow) is the one we clicked on in the list of suites in prior step.

  • The suite is made up of only one test group (blue arrow) as that was the only test group we selected during creation.

  • The test system dropdown (green arrow) lists the Server test systems that you have access to. Only Server test systems are populated because this is a Server Conformance Suite.

  • The anchor system chosen for the suite (TouchstoneFHIR) is not listed in the test systems dropdown. The anchor system is the Client test system that will submit requests to the chosen Server test system in a Server conformance suite. As users select different Server test systems to run this Conformance Suite against, the same anchor system (TouchstoneFHIR in this case) will be used to submit requests to these Server test systems. This allows for uniform comparison of Server-system conformance results. TouchstoneFHIR is used to submit requests from because this was the anchor system chosen for this suite in step 8 above.

  • The Supported checkbox and the Format dropdown (purple arrows) are offered to the user because we had checked these boxes during suite creation in steps 11 and 12 above.