Sharing the software with other users

As soon as you have some working software that you intend to share with others, you should package your software in a standard way such that users can easily download your software, install it, improve it, and ask you to approve their improvements in new versions of the software. During recent years, the software development community has established quite firm tools and rules for how all this is done. The following subsections cover three steps in sharing software:

1. Organizing the software for public distribution.
2. Uploading the software to a cloud service (here GitHub).
3. Downloading and installing the software.

Organizing the software directory tree

We start with organizing our software as a directory tree. Our software consists of one module file, decay.py, and possibly some unit tests in a separate file located in a directory tests.

The decay.py can be used as a module or as a program. For distribution to other users who install the program decay.py in system directories, we need to insert the following line at the top of the file:

#!/usr/bin/env python

This line makes it possible to write just the filename and get the file executed by the python program (or more precisely, the first python program found in the directories in the PATH environment variable).

Distributing just a module file

Let us start out with the minimum solution alternative: distributing just the decay.py file. Then the software is just one file and all we need is a directory with this file. Users will also this directory to contain an installation script setup.py and a README file telling what the software is about, the author’s email address, a URL for downloading the software, and other useful information.

The setup.py file can be as short as

from distutils.core import setup
setup(name='decay',
      version='0.1',
      py_modules=['decay'],
      scripts=['decay.py'],
      )

The py_modules argument specifies a list of modules to be installed, while scripts specifies stand-alone programs. Our decay.py can be used either as a module or as an executable program, so we want users to have both possibilities.

Distributing a package

If the software consists of more files than one or two modules, one should make a Python package out of it. In our case we make a package decay containing one module, also called decay.

To make a package decay, create a directory decay and an empty file in it with name __init__.py. A setup.py script must now specify the directory name of the package and also an executable program (scripts=) in case we want to run decay.py as a stand-alone application:

from distutils.core import setup
import os

setup(name='decay',
      version='0.1',
      author='Hans Petter Langtangen',
      author_email='hpl@simula.no',
      url='https://github.com/hplgit/decay-package/',
      packages=['decay'],
      scripts=[os.path.join('decay', 'decay.py')]
     )

We have also added some author and download information. The reader is referred to the Distutils documentation for more information on how to write setup.py scripts.

Remark about the executable file

The executable program, decay.py, is above taken to be the complete module file decay.py. It would normally be preferred to instead write a very short script essentially importing decay and running the test block in decay.py. In this way, we distribute a module and a very short file, say decay-main.py, as an executable program:

#!/usr/bin/env python
import decay
decay.decay.experiment_compare_dt(True)
decay.decay.plt.show()

In this package example, we move the unit tests out of the decay.py module to a separate file, test_decay.py, and place this file in a directory tests. Then the nosetests and py.test programs will automatically find and execute the tests.

The complete directory structure reads

Terminal> /bin/ls -R
.:
decay  README  setup.py

./decay:
decay.py  __init__.py  tests

./decay/tests:
test_decay.py

Publishing the software at GitHub

The leading site today for publishing open source software projects is GitHub at http://github.com, provided you want your software to be open to the world. With a paid GitHub account, you can have private projects too.

Sign up for a GitHub account if you do not already have one. Go to your account settings and provide an SSH key (typically the file ~/.ssh/id_rsa.pub) such that you can communicate with GitHub without being prompted for your password. All communication between your computer and GitHub goes via the version control system Git. This may at first sight look tedious, but this is the way professionals work with software today. With Git you have full control of the history of your files, i.e., “who did what when”. The technology makes Git superior to simpler alternatives like Dropbox and Google Drive, especially when you collaborate with others. There is a reason why Git has gained the position it has, and there is no reason why you should not adopt this tool.

To create a new project, click on New repository on the main page and fill out a project name. Click on the check button Initialize this repository with a README, and click on Create repository. The next step is to clone (copy) the GitHub repo (short for repository) to your own computer(s) and fill it with files. The typical clone command is

Terminal> git clone git://github.com:username/projname.git

where username is your GitHub username and projname is the name of the repo (project). The result of git clone is a directory projname. Go to this directory and add files. As soon as the repo directory is populated with files, run

Terminal> git add .
Terminal> git commit -am 'First registration of project files'
Terminal> git push origin master

The above git commands look cryptic, but these commands plus 2-3 more are the essence of what you need in your daily work with files in small or big software projects. I strongly encourage you to learn more about version control systems and project hosting sites [Ref1].

Your project files are now stored in the cloud at https://github.com/username/projname. Anyone can get the software by the listed git clone command you used above, or by clicking on the links for zip and tar files.

Every time you update the project files, you need to register the update at GitHub by

Terminal> git commit -am 'Description of the changes you made...'
Terminal> git push origin master

The files at GitHub are now synchronized with your local ones. Similarly, every time you start working on files in this project, make sure you have the latest version: git pull origin master.

You are recommended to read a quick intro that makes you up and going with this style of professional work. And you should put all your writings and programming projects in repositories in the cloud!

Downloading and installing the software

Users of your software go to the Git repo at github.com and clone the repository. One can use either SSH or HTTP for communication. Most users will use the latter, typically

Terminal> git clone https://github.com/username/projname.git

The result is a directory projname with the files in the repo.

Installing just a module file

The software package is in the case above a directory decay with three files

Terminal> ls decay
README   decay.py   setup.py

To install the decay.py file, a user just runs setup.py:

Terminal> sudo python setup.py install

This command will install the software in system directories, so the user needs to run the command as root on Unix systems (therefore the command starts with sudo). The user can now import the module by import decay and run the program by

Terminal> decay.py

A user can easily install the software on her personal account if a system-wide installation is not desirable. We refer to the installation documentation for the many arguments that can be given to setup.py. Note that if the software is installed on a personal account, the PATH and PYTHONPATH environment variables must contain the relevant directories.

Our setup.py file specifies a module decay to be installed as well as a program decay.py. Modules are typically installed in some lib directory on the computer system, e.g., /usr/local/lib/python2.7/dist-packages, while executable programs go to /usr/local/bin.

Installing a package

When the software is organized as a Python package, the installation is done by running setup.py exactly as explained above, but the use of a module decay in a package decay requires the following syntax:

import decay
u, t = decay.decay.solver(...)

That is, the call goes like packagename.modulename.functionname.

Package import in __init__.py

One can ease the use of packages by providing a somewhat simpler import like

import decay
u, t = decay.solver(...)

# or
from decay import solver
u, t = solver(...)

This is accomplished by putting an import statement in the __init__.py file, which is always run when doing the package import import decay or from decay import. The __init__.py file must now contain

from decay import *

Obviously, it is the package developer who decides on such an __init__.py file or if it should just be empty.