Developer Guidelines

Code of Conduct

Note

Before we start please be aware that contributors to this project are expected to act respectfully toward others in accordance with the OSGeo Code of Conduct.

Contribution Workflow

The Birdhouse project openly welcomes contributions (bug reports, bug fixes, code enhancements/features, etc.). This document will outline some guidelines on contributing to birdhouse. As well, the birdhouse Communication is a great place to get an idea of how to connect and participate in birdhouse community and development where everybody is welcome to rise questions and discussions.

Here are some basic guides to smoothly contribute to birdhouse:

Source code

The source code of all birdhouse components is available on GitHub. Respecting the git mechanisms you can fork, clone and pull source-code into your repositories for modification and enhancement. Once your improvement is ready, make a pull request to integrate your work into the origin birdhouse repositories.

Note

Please keep your forks close to the origin repositories and don’t forget the pull requests.

Issue tracker

To keep track on the contribution and development, please use the issue tracker on GitHub for the corresponding birdhouse component.

Please find the coding guide in the Wiki.

Writing a WPS process

In birdhouse, we are using the PyWPS implementation of a Web Processing Service. Please read the PyWPS documentation on how to implement a WPS process.

Note

To get started quickly, you can try the Emu WPS with some example processes for PyWPS.

_images/process_schema_1.png

Another point to think about when designing a process is the possibility of chaining processes together. The result of a process can be a final result or be used as an input for another process. Chaining processes is a common practice but depends on the user you are designing the service for. Technically, for the development of WPS process chaining, here are a few summary points:

  • the functional code should be modular and provide an interface/method for each single task

  • provide a wps process for each task

  • wps processes can be chained, manually or within the code, to run a complete workflow

  • wps chaining can be done manually, with workflow tools, direct wps chaining or with code scripts

  • a complete workflow chain could also be started by a wps process.

_images/wps_chain.png

Writing functions

A Process is calling several functions during the performance. Since WPS is a autonom running process several eventualities needs to be taken into account. If irregularities are occurring, it is a question of the process design if the performance should stop and return an error or continue with may be an modified result.

In practice, the functions should be encapsulated in try and except calls and appropriate information given to the logfile or shown as a status message. The logger has several options to to influence the running code and the information writing to the logfile:

_images/module_chain.png 
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
# the following two line needs to be in the beginning of the *.py file.
# The ._handler will find the appropriate logfile and include timestemps
# and module information into the log.

import logging
LOGGER = logging.getLogger("PYWPS")

# set a status message
per = 5  # 5 will be 5% in the status line
response.update_status('execution started at : {}'.fromat(dt.now()), per)

try:
    response.update_status('the process is doing something: {}'.fromat(dt.now()),10)
    result = 42
    LOGGER.info('found the answer of life')
except Exception as ex:
    msg = 'This failed but is obligatory for the output. The process stops now, because: {} '.format(ex)
    LOGGER.error(msg)

try:
    response.update_status('the process is doing something else : {}'.fromat(dt.now()), 20)
    interesting = True
    LOGGER.info(' Thanks for reading the guidelines ')
    LOGGER.debug(' I need to know some details of the process: {} '.format(interesting)
except Exception as ex:
    msg = 'This failed but is not obligatory for the output. The process will continue. Reason for the failure: {} '.format(ex)
    LOGGER.exception(msg)

Writing tests

Todo

Guideline to write tests. Look at the Emu to see examples.

Writing documentation

Last but not least, a very very important point is to write a good documentation about your work! Each WPS (bird) has a docs folder for this where the documentation is written in reStructuredText and generated with Sphinx.

The documentation is automatically published to ReadTheDocs with GitHub webhooks. It is important to keep the Code Style and write explanations to your functions. There is an auto-api for documentation of functions.

Todo

explanation of enabling spinx automatic api documentation.

The main documentation (which you are reading now) is the starting point to get an overview of birdhouse. Each birdhouse component comes with its own Sphinx documentation and is referenced by the main birdhouse document. Projects using birdhouse components like PAVICS_ or COPERNICUS Data Store generally have their own documentation as well. To include documentation from external repository here, two custom made sphinx directives can be used. The gittoctree directive behaves like a normal table of content directive (toctree), but takes as an argument the URL to the git repo and refers to files inside this directory through their full path. The gitinclude directive acts like an normal include directive, but takes as a first argument the URL to the git repo this file belongs to. For example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
Here is the text of the birdhouse main documentation. At the place where you want to integrate
a part of a remote sphinx documentation stored in a `git` repository you can fetch the docs
parts and integrated it with a table of content referring to external files:

.. gittoctree:: https://github.com/Ouranosinc/pavics-sdi.git

   docs/source/arch/backend.rst

or include an individual file:

.. gitinclude:: https://github.com/Ouranosinc/pavics-sdi.git docs/source/arch/backend.rst

The directive will clone and checkout the repository, then include these external files as if
they were part of the native documentation.

Code Style

A good start to contribute is an enhancement of existing code with better or new functions. To respect a common coding style, Birdhouse uses PEP8 checks to ensure a consistent coding style. Currently the following PEP8 rules are enabled in setup.cfg:

[flake8]
ignore=F401,E402
max-line-length=120
exclude=tests

See the flake8 documentation on how to configure further options.

To check the coding style run flake8:

$ flake8 emu   # emu is the folder with python code
# or
$ make pep8    # make calls flake8

To make it easier to write code according to the PEP8 rules enable PEP8 checking in your editor. In the following we give examples how to enable code checking for different editors.

Sublime

{
 // set vertical rulers in specified columns.
 "rulers": [79],

 // turn on word wrap for source and text
 // default value is "auto", which means off for source and on for text
 "word_wrap": true,

 // set word wrapping at this column
 // default value is 0, meaning wrapping occurs at window width
 "wrap_width": 79
 }

Todo

Add PEP8 instructions for more editors: PyCharm, Kate, Emacs, Vim, Spyder.

Environment with conda

Todo

How to create a conda package

Make your own Bird

If you are familiar with all the upper chapters you are ready to create your own WPS. The WPS in birdhouse are named after birds, so this section is giving you a guidline of how to make your own bird. Birds are sorted thematically, so before setting up a new one, make sure it is not already covered and just missing some processes and be clear in the new thematic you would like to provide.

We have now a Cookiecutter template to create a new bird (PyWPS application). It is the recommended and fastest way to create your own bird:

https://github.com/bird-house/cookiecutter-birdhouse

Note

The cookiecutter is brand-new. Please give feedback and help to improve it.

Release Notes and Versions:

The development of birdhouse is following a release cycle of around three month. Updates of modules are coordinated by the developers over the communication channels (gitter chat or Video Conference). New releases are documented in the release notes and communicated over the mailing list. A release of a birdhouse module is taged with a version number and appropriate git repository version branch.

For an orientation of when to release a new version:

  • Full version (v1.0) with scientific publication in a reviewed journal

  • subversion (v1.1) by major changes

  • subsub versions (v1.1.1) by minor changes

out of the release cycles bug fix patches can be released every time ( communication is not mandatory )

  • patch v1.1.1_patch1 bugfix