WPS design

Setting up a new WPS

If you are familiar with all the upper chapters you are ready to create your own WPS. The WPS in birdhouse are named after birds, so this section is giving you a guidline of how to make your own bird. Birds are sorted thematically, so before setting up a new one, make sure it is not already covered and just missing some processes and be clear in the new thematic you would like to provide.

There is a Cookiecutter template to create a new bird (PyWPS application). It is the recommended and fastest way to create your own bird:

Writing a WPS process

In birdhouse, we are using the PyWPS implementation of a Web Processing Service. Please read the PyWPS documentation on how to implement a WPS process.

Note

To get started quickly, you can try the Emu WPS with some example processes for PyWPS.

_images/process_schema_1.png

Another point to think about when designing a process is the possibility of chaining processes together. The result of a process can be a final result or be used as an input for another process. Chaining processes is a common practice but depends on the user you are designing the service for. Technically, for the development of WPS process chaining, here are a few summary points:

  • the functional code should be modular and provide an interface/method for each single task

  • provide a wps process for each task

  • wps processes can be chained, manually or within the code, to run a complete workflow

  • wps chaining can be done manually, with workflow tools, direct wps chaining or with code scripts

  • a complete workflow chain could also be started by a wps process.

_images/wps_chain.png

Writing functions

A Process is calling several functions during the performance. Since WPS is a autonom running process several eventualities needs to be taken into account. If irregularities are occurring, it is a question of the process design if the performance should stop and return an error or continue with may be an modified result.

In practice, the functions should be encapsulated in try and except calls and appropriate information given to the logfile or shown as a status message. The logger has several options to to influence the running code and the information writing to the logfile:

_images/module_chain.png
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
# the following two line needs to be in the beginning of the *.py file.
# The ._handler will find the appropriate logfile and include timestemps
# and module information into the log.

import logging
LOGGER = logging.getLogger("PYWPS")

# set a status message
per = 5  # 5 will be 5% in the status line
response.update_status('execution started at : {}'.fromat(dt.now()), per)

try:
    response.update_status('the process is doing something: {}'.fromat(dt.now()),10)
    result = 42
    LOGGER.info('found the answer of life')
except Exception as ex:
    msg = 'This failed but is obligatory for the output. The process stops now, because: {} '.format(ex)
    LOGGER.error(msg)

try:
    response.update_status('the process is doing something else : {}'.fromat(dt.now()), 20)
    interesting = True
    LOGGER.info(' Thanks for reading the guidelines ')
    LOGGER.debug(' I need to know some details of the process: {} '.format(interesting)
except Exception as ex:
    msg = 'This failed but is not obligatory for the output. The process will continue. Reason for the failure: {} '.format(ex)
    LOGGER.exception(msg)

Writing documentation

Last but not least, a very very important point is to write a good documentation about your work! Each WPS (bird) has a docs folder for this where the documentation is written in reStructuredText and generated with Sphinx.

The documentation is automatically published to ReadTheDocs with GitHub webhooks. It is important to keep the Code Style and write explanations to your functions. There is an auto-api for documentation of functions.

Todo

explanation of enabling spinx automatic api documentation.

The main `documentation`_ (which you are reading now) is the starting point to get an overview of birdhouse. Each birdhouse component comes with its own Sphinx documentation and is referenced by the main birdhouse document. Projects using birdhouse components like PAVICS_ or `COPERNICUS Data Store`_ generally have their own documentation as well. To include documentation from external repository here, two custom made sphinx directives can be used. The gittoctree directive behaves like a normal table of content directive (toctree), but takes as an argument the URL to the git repo and refers to files inside this directory through their full path. The gitinclude directive acts like an normal include directive, but takes as a first argument the URL to the git repo this file belongs to. For example:

Note

Look at the Emu to see examples.