.. !split .. _softeng1:prog:se:git: Sharing the software with other users ===================================== As soon as you have some working software that you intend to share with others, you should package your software in a standard way such that users can easily download your software, install it, improve it, and ask you to approve their improvements in new versions of the software. During recent years, the software development community has established quite firm tools and rules for how all this is done. The following subsections cover three steps in sharing software: 1. Organizing the software for public distribution. 2. Uploading the software to a cloud service (here GitHub). 3. Downloading and installing the software. Organizing the software directory tree -------------------------------------- We start with organizing our software as a directory tree. Our software consists of one module file, ``decay.py``, and possibly some unit tests in a separate file located in a directory ``tests``. The ``decay.py`` can be used as a module or as a program. For distribution to other users who install the program ``decay.py`` in system directories, we need to insert the following line at the top of the file: .. code-block:: python #!/usr/bin/env python This line makes it possible to write just the filename and get the file executed by the ``python`` program (or more precisely, the first ``python`` program found in the directories in the ``PATH`` environment variable). Distributing just a module file ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Let us start out with the minimum solution alternative: distributing just the ``decay.py`` file. Then the software is just one file and all we need is a directory with this file. Users will also this directory to contain an installation script ``setup.py`` and a ``README`` file telling what the software is about, the author's email address, a URL for downloading the software, and other useful information. .. index:: setup.py The ``setup.py`` file can be as short as .. code-block:: python from distutils.core import setup setup(name='decay', version='0.1', py_modules=['decay'], scripts=['decay.py'], ) The ``py_modules`` argument specifies a list of modules to be installed, while ``scripts`` specifies stand-alone programs. Our ``decay.py`` can be used either as a module or as an executable program, so we want users to have both possibilities. Distributing a package ~~~~~~~~~~~~~~~~~~~~~~ If the software consists of more files than one or two modules, one should make a Python *package* out of it. In our case we make a package ``decay`` containing one module, also called ``decay``. To make a package ``decay``, create a directory ``decay`` and an empty file in it with name ``__init__.py``. A ``setup.py`` script must now specify the directory name of the package and also an executable program (``scripts=``) in case we want to run ``decay.py`` as a stand-alone application: .. code-block:: python from distutils.core import setup import os setup(name='decay', version='0.1', author='Hans Petter Langtangen', author_email='hpl@simula.no', url='https://github.com/hplgit/decay-package/', packages=['decay'], scripts=[os.path.join('decay', 'decay.py')] ) We have also added some author and download information. The reader is referred to the `Distutils documentation `__ for more information on how to write ``setup.py`` scripts. .. index:: Distutils .. admonition:: Remark about the executable file The executable program, ``decay.py``, is above taken to be the complete module file ``decay.py``. It would normally be preferred to instead write a very short script essentially importing ``decay`` and running the test block in ``decay.py``. In this way, we distribute a module and a very short file, say ``decay-main.py``, as an executable program: .. code-block:: python #!/usr/bin/env python import decay decay.decay.experiment_compare_dt(True) decay.decay.plt.show() In this package example, we move the unit tests out of the ``decay.py`` module to a separate file, ``test_decay.py``, and place this file in a directory ``tests``. Then the ``nosetests`` and ``py.test`` programs will automatically find and execute the tests. The complete directory structure reads .. code-block:: text Terminal> /bin/ls -R .: decay README setup.py ./decay: decay.py __init__.py tests ./decay/tests: test_decay.py Publishing the software at GitHub --------------------------------- .. index:: GitHub The leading site today for publishing open source software projects is GitHub at ``_, provided you want your software to be open to the world. With a paid GitHub account, you can have private projects too. Sign up for a GitHub account if you do not already have one. Go to your account settings and provide an SSH key (typically the file ``~/.ssh/id_rsa.pub``) such that you can communicate with GitHub without being prompted for your password. All communication between your computer and GitHub goes via the version control system Git. This may at first sight look tedious, but this is the way professionals work with software today. With Git you have full control of the history of your files, i.e., "who did what when". The technology makes Git superior to simpler alternatives like Dropbox and Google Drive, especially when you collaborate with others. There is a reason why Git has gained the position it has, and there is no reason why you should not adopt this tool. To create a new project, click on *New repository* on the main page and fill out a project name. Click on the check button *Initialize this repository with a README*, and click on *Create repository*. The next step is to clone (copy) the GitHub repo (short for repository) to your own computer(s) and fill it with files. The typical clone command is .. code-block:: text Terminal> git clone git://github.com:username/projname.git where ``username`` is your GitHub username and ``projname`` is the name of the repo (project). The result of ``git clone`` is a directory ``projname``. Go to this directory and add files. As soon as the repo directory is populated with files, run .. code-block:: text Terminal> git add . Terminal> git commit -am 'First registration of project files' Terminal> git push origin master The above ``git`` commands look cryptic, but these commands plus 2-3 more are the essence of what you need in your daily work with files in small or big software projects. I strongly encourage you to learn more about `version control systems and project hosting sites `__ [Ref1]_. Your project files are now stored in the cloud at ``_. Anyone can get the software by the listed ``git clone`` command you used above, or by clicking on the links for zip and tar files. Every time you update the project files, you need to register the update at GitHub by .. code-block:: text Terminal> git commit -am 'Description of the changes you made...' Terminal> git push origin master The files at GitHub are now synchronized with your local ones. Similarly, every time you start working on files in this project, make sure you have the latest version: ``git pull origin master``. You are recommended to read `a quick intro `__ that makes you up and going with this style of professional work. And you should put all your writings and programming projects in repositories in the cloud! Downloading and installing the software --------------------------------------- Users of your software go to the Git repo at ``github.com`` and clone the repository. One can use either SSH or HTTP for communication. Most users will use the latter, typically .. code-block:: text Terminal> git clone https://github.com/username/projname.git The result is a directory ``projname`` with the files in the repo. Installing just a module file ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The software package is in the case above a directory ``decay`` with three files .. code-block:: text Terminal> ls decay README decay.py setup.py To install the ``decay.py`` file, a user just runs ``setup.py``: .. code-block:: text Terminal> sudo python setup.py install This command will install the software in system directories, so the user needs to run the command as ``root`` on Unix systems (therefore the command starts with ``sudo``). The user can now import the module by ``import decay`` and run the program by .. code-block:: text Terminal> decay.py A user can easily install the software on her personal account if a system-wide installation is not desirable. We refer to the `installation documentation `__ for the many arguments that can be given to ``setup.py``. Note that if the software is installed on a personal account, the ``PATH`` and ``PYTHONPATH`` environment variables must contain the relevant directories. Our ``setup.py`` file specifies a module ``decay`` to be installed as well as a program ``decay.py``. Modules are typically installed in some ``lib`` directory on the computer system, e.g., ``/usr/local/lib/python2.7/dist-packages``, while executable programs go to ``/usr/local/bin``. .. index:: importing modules Installing a package ~~~~~~~~~~~~~~~~~~~~ When the software is organized as a Python package, the installation is done by running ``setup.py`` exactly as explained above, but the use of a module ``decay`` in a package ``decay`` requires the following syntax: .. code-block:: text import decay u, t = decay.decay.solver(...) That is, the call goes like ``packagename.modulename.functionname``. .. admonition:: Package import in ``__init__.py`` One can ease the use of packages by providing a somewhat simpler import like .. code-block:: text import decay u, t = decay.solver(...) # or from decay import solver u, t = solver(...) This is accomplished by putting an import statement in the ``__init__.py`` file, which is always run when doing the package import ``import decay`` or ``from decay import``. The ``__init__.py`` file must now contain .. code-block:: python from decay import * Obviously, it is the package developer who decides on such an ``__init__.py`` file or if it should just be empty.