Adding workflow tests with Planemo


Ensuring a Tutorial has a Workflow

  1. Find a tutorial that you’re interested in, that doesn’t currently have tests.

    This tutorial has a workflow (.ga) and a test, notice the -test.yml that has the same name as the workflow .ga file.

    machinelearning/workflows/machine_learning.ga
    machinelearning/workflows/machine_learning-test.yml
    

    You want to find tutorials without the -test.yml file. The workflow file might also be missing.

  2. Check if it has a workflow (if it does, skip to step 5.)
  3. Follow the tutorial
  4. Extract a workflow from the history
  5. Run that workflow in a new history to test

Extract Tests (Online Version)

If you are on UseGalaxy.org or another server running 24.2 or later, you can use PWDK, a version of planemo running online to generate the workflow tests.

However if you are on an older version of Galaxy, or a private Galaxy server, then you’ll need to do the following:

Extract Tests (Manual Version)

  1. Obtain the workflow invocation ID, and your API key (User → Preferences → Manage API Key)

    screenshot of the workflow invocation page. The user drop down shows where to find this page, and a red box circles a field named "Invocation ID"

  2. Install the latest version of planemo

    # In a virtualenv
    pip install planemo
    
  3. Run the command to initialise a workflow test from the workflows/ subdirectory - if it doesn’t exist, you might need to create it first.

    planemo workflow_test_init --from_invocation <INVOCATION ID> --galaxy_url <GALAXY SERVER URL> --galaxy_user_key <GALAXY API KEY>
    

    This will produce a folder of files, for example from a testing workflow:

    $ tree
    .
    ├── test-data
    │   ├── input dataset(s).shapefile.shp
    │   └── shapefile.shp
    ├── testing-openlayer.ga
    └── testing-openlayer-tests.yml
    

Adding Your Tests to the GTN

  1. You will need to check the -tests.yml file, it has some automatically generated comparisons. Namely it tests that output data matches the test-data exactly, however, you might want to replace that with assertions that check for e.g. correct file size, or specific text content you expect to see.

  2. If the files in test-data are already uploaded to Zenodo, to save disk space, you should delete them from the test-data dir and use their URL in the -tests.yml file, as in this example:

    - doc: Test the M. Tuberculosis Variant Analysis workflow
      job:
         'Read 1':
            location: https://zenodo.org/record/3960260/files/004-2_1.fastq.gz
            class: File
            filetype: fastqsanger.gz
    
  3. Add tests on the outputs! Check the planemo reference if you need more detail.

    - doc: Test the M. Tuberculosis Variant Analysis workflow
      job:
         # Simple explicit Inputs
         'Read 1':
            location: https://zenodo.org/record/3960260/files/004-2_1.fastq.gz
            class: File
            filetype: fastqsanger.gz
      outputs:
        jbrowse_html:
          asserts:
            has_text:
              text: "JBrowseDefaultMainPage"
        snippy_fasta:
          asserts:
            has_line:
              line: '>Wildtype Staphylococcus aureus strain WT.'
        snippy_tabular:
          asserts:
            has_n_columns:
              n: 2
    
  4. Contribute all of those files to the GTN in a PR, adding them to the workflows/ folder of your tutorial.

Persistent URL
Resource purlPURL: https://gxy.io/GTN:F00162
Still have questions?
Gitter Chat Support
Galaxy Help Forum