In-house Software Development

Allegro management scripts

There are a number of Allegro-developed scripts having various functions which are maintained under the version control system git. A good introduction with tutorials can be found in the git documentation.

Currently the scripts are divided between four repositories. For each of these, there is an ‘origin’ repository directory under the directory $ALLEGRO_GIT, an active or working copy, and a copy under /home/alma/git_repos/src.

Location of the active copy	$ALLEGRO_GIT repository	Bulk update script
$ALLEGRO_STAFF/src/casa/tasks/	casa/tasks.git	update_buildmytasks
$ALLEGRO_STAFF/lib/python/site-packages/allegroUtils/	scripts/allegroUtils.git	update_allegroUtils
$ALLEGRO_STAFF/bin/	scripts/project_management.git	update_allegroStaffBin
$ALLEGRO/bin/	scripts/allegroScripts.git	update_allegroBin

• The ‘origin’ copy should be considered the place of storage of the scripts. You cannot (and certainly should not) edit scripts in this location directly.
• The ‘active’ copy contains the versions of the scripts which are used. For example, the disk-useage display tool baobab_lustre.py is a member of the project_management repository. If you type which baobab_lustre.py on the command line, you will be directed to the active copy in $ALLEGRO_STAFF/bin/. You should not edit or make changes directly to the active copies either. Why not? Because we now have more than one file system, and it is desirable to synchronize all the scripts across all the file systems. That’s what the bulk update scripts do: they obtain the latest version of the repository from the $ALLEGRO_GIT area and download that to the active area on all of the file systems.
• Although you can edit the source code in your own copy of a GIT repository, then push this to $ALLEGRO_GIT, it is probably a better idea if only one person makes changes, working as user alma. In this case you can work in the areas under /home/alma/git_repos/src. (Working as user alma also makes it easier to test things, as many of the scripts will only work if run by this user.) Many of these scripts are vital for running and managing Allegro computers and projects, and should only be modified if you know what you are doing. A typical work cycle would be as follows:

1. Make changes in your local copy (or, as alma, in /home/alma/git_repos/src).
2. Test the changes.
3. git commit -am <brief commit notes>
4. git push origin master
5. As user alma, run the appropriate bulk update script.

Note finally that various documents (including the present guide) are also under git version control, and can be modified by Allegro members via the same procedure. See section Documents under version control for details.

CASA Tasks

When starting up CASA on one of the Allegro computers, it loads the self-written CASA tasks from the default installation path. However, that is not very convenient if you try to develop a new task and you have to have access to data on chaxa, helada or tebinquiche.

Therefore, you can define an environment variable CASA_MYTASKS with the full path to the mytasks.py in your development environment, which is then loaded instead of the systemwide installation. After removing the variable (or setting it to a non-existing file), the default installation will be loaded during the next startup.

So as an example, in bash you can do:

$ export CASA_MYTASKS=/home/alma/devel/casatasks/mytasks.py

and in c-shell you can do:

> setenv CASA_MYTASKS /home/alma/devel/casatasks/mytasks.py

and it will load your development version at at startup.

Working with GIT

Creating a new Project

If you have a new project that you would like to include in the local github, the following chain of commands should be executed.

1. Go into your project directory (in this example, called myAllegroProject)

   $ cd <my project dir>/myAllegroProject
   

2. Initialise git

   $ git init
   Initialized empty Git repository in <my project dir>/myAllegroProject/.git/
   

3. Note down files that should not be shared in .gitignore (e.g. if your project is python, you don’t need to store the compiled *.pyc files.):

   $ echo "*.pyc" >> .gitignore
   

4. Add all the files that you would like to include into the repository, and (optionally) check the status

   $ git add .
   $ git status
   On branch master
   
   Initial commit
   
   Changes to be committed:
   (use "git rm --cached <file>..." to reverse committal of any file)
   
       new file:   bar.py
       new file:   foo.py
   

5. Make your first commit, i.e., put these files under version control.

   $ git commit -m "Initial commit"
   [master (root-commit) 60ee998] Initial Commit
   2 files changed, 0 insertions(+), 0 deletions(-)
   create mode 100644 bar.py
   create mode 100644 foo.py
   

   If you do not specify a message with the tag -m "Message", an editor will open up and ask you to give a quick description of this commit.

6. Optionally, you can also tag any commit (which allows to quickly return to this point). Typical occasions for a tag are a version release.

   $ git tag -a InitialCommit
   

7. Clone this directory into a bare repository

   $ cd ..
   $ git clone --bare myAllegroProject myAllegroProject.git
   Cloning into bare repository 'myAllegroProject.git'...
   done.
   

8. As user alma, copy the repository into the github (taking into account the subdirectory structure of the github), and assign the group the same permission as the user

   $ cp -r myAllegroProject.git $ALLEGRO_GIT/scripts
   $ cd $ALLEGRO_GIT/scripts
   $ chmod -R g=u myAllegroProject.git
   

9. Mark the repository as shared repository (to modify the file permissions for the group so that everyone can modify things)

   $ cd myAllegroProject.git
   $ git config core.sharedRepository true
   

10. You are done! All Allegro members can now contribute to this project.

Contributing to an Existing Project

Figure 11: Schematic of the distributed workflow for Allegro projects.

The workflow for Allegro projects (see Figure 11) currently follows closely the scheme of a Centralised Workflow. Developers pull and push from the main branch of the shared repository. The system-wide library is only updated by pulling the most recent changes from the repository, so that they are available for all users on the Allegro computers.

Initialise

In order to contribute to a project, you first have to clone your local copy

$ git clone $ALLEGRO_GIT/scripts/allegroProject.git
Cloning into 'allegroProject'...
done.

This is a complete mirror of the project in the repository.

Edit

A developer pulls the latest version of the project from the repository

$ cd allegroProject
$ git pull origin master

It is then strongly recommended to open a development branch. This can have an arbitrary name

$ git checkout -b devel

You can add, delete, or modify any of the files. Afterwards, the changes must be staged and commited

$ git add foo.py bar.py
$ git commit

or if all the changed files should be committed, this command can also be used

$ git commit -a

Then one should return to the master branch and pull the latest version from the repository

$ git checkout master
$ git pull origin master

The changes in the development branch have to be merged with the main branch now

$ git merge devel

Any conflicts have to be resolved (follow the instructions). Then, the updated sofware must be pushed back into the shared repository

$ git push origin master

Warning

Any conflicts with the master branch in the repository (if, e.g., another developer has also pushed changes) must be resolved locally first, before the final version can be pushed to the repository.

After the pushing, the system-wide installation has to be updated. For this, dedicated scripts are available (see Table in Allegro management scripts).