Advertisements

Steve Bennett blogs

…about maps, open data, Git, and other tech.

Category Archives: System administration

Git: what they didn’t tell you

Credit:Tim Strater from Rotterdam, Nederland CC-BY-SA

Of all the well-documented difficulties I’ve had working with Git over the years, a few conceptual difficulties really stand out. They’re quirks in the Git architecture that took me far too long to realise, far too long to believe, or far too long to really grasp. And maybe you have the same problem without realising it.

Branch names are completely arbitrary


git branch -d master
git checkout develop -b master

There, I did it. I’m now calling the develop branch master. What you call this branch, and what I call it, and what your Github repo calls it, and what my Github repo calls it just don’t matter. Four different names? No problem. There are some flimsy conventions that Git half-heartedly follows to link two branches with the same name, but it gives up pretty easily.

Remote branches are local

I have very frequently fallen into this trap:

$ git diff origin/master
$

No differences, so my branch must be in sync with Origin, right? Wrong. What is true is my branch is in sync with the local copy of Origin. If you don’t run git fetch, then Git will never even update its local copies.

Technically, Git has always been upfront about this. The Git book opens the section on remote branches:

Remote branches are references to the state of branches on your remote repositories.

But it’s counterintuitive, and so I keep messing it up. I keep hoping (and assuming) that Git will one day include an auto-fetch option, where it constantly synchronises remote branches

‘Detached HEAD’ mode is fine

Here’s the message that we have seen many times:

Note: checking out ‘origin/develop’.

You are in ‘detached HEAD’ state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

git checkout -b new_branch_name

This scary looking message threw me off for a long time, despite the fact that it’s actually one of Git’s most helpful messages – it tells you everything you need to know.

It boils down to this: you can’t make a commit if you’re not at the top of a branch. The two most common situations that cause this are:

    1. Checking out a remote branch. You should do this:

      git checkout origin/develop -b mydevelop
      or even, if you want to abandon your branch completely:
      git branch -d develop
      git checkout origin/develop -b develop
  1. Checking out a commit in the middle of a branch, like:

    git checkout a8e6b18

    Usually in this case you just want to look at it, so you can just ignore the message.

There’s nothing special about ‘git clone’

For a long time, I thought that ‘git init’, ‘git pull’ and ‘git clone’ somehow created repositories that were different, even if they ended up with the same commits in them. It’s hard to recreate my state of mind, but I spent a long time trying to salvage certain directories on disk when I should have just abandoned them.

Similarly, there is no difference between:

  1. git clone http://github.com/stevage/myrepo
  2. git init
    git remote add origin http://github.com/stevage/myrepo
    git pull origin master

Well, in the second case Git’s a bit confused about which local branches map onto which remote branches, so you have to be more explicit or fix it with some configuration option.

Don’t call any remote ‘origin’

Credit: Chevassu (GFDL)

For some reason, Git encourages you to call the source of the first clone “origin”. I have found this very confusing and ultimately very unhelpful. Let’s say you’re working on a project called widget, and you fork it in Github so you can work on it. You will want both remotes accessible locally, so you will probably do one of these:

  1. git clone http://github.com/stevage/widget
    git remote add widget http://github.com/widget/widget
  2. git clone http://github.com/widget/widget
    git remote add mine http://github.com/stevage/widget

So you either have remotes called “origin” and “widget”, or remotes called “origin” and “mine”. But on the next project, you might make the opposite choice, and soon you really don’t remember what “origin” means.

My tip: never name any remote “origin” ever. Name them all after their Github username.

  1. git init
    git remote add widget http://github.com/widget/widget
    git remote add stevage http://github.com/stevage/widget
    git fetch --all
    get checkout stevage -b master

You will never understand “git reset”‘s options

The difference between “git reset”, “git reset –soft” and “git reset –hard” is beyond your comprehension. And you probably wanted “git checkout” anyway.

Rule of thumb:

  • If your directory is FUBAR: git reset –hard
  • If you just want to throw away changes to one file: git checkout file
  • In all other cases, google it.
Advertisements

One week of Salt: frustrations and reflections.

 

Reflecting on Salt. (Credit: Psyberartist, CC-BY 2.0)

Salt, or SaltStack, is an automatic deployment tool for clusters of machines, in the same category as Chef, Puppet and Ansible. I’ve previously used Chef, found the experience frustrating, and thought I’d give Salt a go. The task: converting my tilemill-server bash scripts, which set up a complete TileMill server stack, with OpenStreetMap data, Nginx and a couple of other goodies.

The goals in this kind of automation for me are threefold:

  1. So I can quickly build new servers for Mapmaking for Academics workshops.
  2. So I can manage other, similar (but slightly different) servers, such as cycletour.org and gis.researchmaps.net. “Manage” here means documenting and controlling the slight configuration differences, being able to roll out further changes (such as web passwords) without having to log in.
  3. So other people can do this too. (That is, the solution has to be simple enough to explain that other people can do it).

The product of my labours: https://github.com/stevage/saltymill

Disclaimers and apologies

A week is too soon to feel the real benefits of automatic deployment and configuration management, but plenty long enough for frustrations and annoyances. There are lots of good things about Salt, but those will go in another post.

I’ve intentionally written this post as I go, to document things that are surprising, counter-intuitive or bothersome to the newcomer. Once you become familiar with a system, it’s much harder to remember what it was like as a newbie, and easier to make allowances for it. So experienced Salt users will probably rankle at the wilful ignorance displayed herein.

A few things I quickly appreciated, compared to Chef:

  • Two parties, not three. Chef has the client (your laptop), the Chef Server (which I never used because it was too complicated) and the target machine itself. Salt doesn’t have the client, so you do everything either on the master (“salt ‘*’ state.highstate) or on the target machine in masterless mode (“salt-call –local state.highstate)
  • No need for “knife upload”: the Salt Master just looks for your files in /srv/salt, so you can use any method you like for getting them there. I always hated having to upload recipes, environments etc individually – you always forgot something.
  • YAML rather than Ruby.
  • A nifty command line tool that lets you try states out on the fly, or run commands remotely on several servers at once. (Chef’s shell, formerly ‘shef’, never really worked out for me.)

Mediocre documentation

Automatic deployment systems are complicated, and you’d love some clear mental concepts to hang on to. Unfortunately, Salt’s chosen metaphor is pretty clumsy (salt grains, pillars, mines…meh), and they’re not well explained. Apparently the concept of a “state” is pretty important to Salt, but even after reading much documentation and quite a reasonable amount of hacking away, I still have only the vaguest notion of what a Salt state is. What’s a “high state”? A “low state”? How can I query the “state” of a minion? No idea. It’s hard to tell if “states” are an implementation magic that makes the whole thing work, or if I, the user am supposed to know about them.

(After more digging around, I think “state” has several meanings, of which one is just one of the actions defined in a SLS file.)

And then, strangely, how do you refer to the most important objects that you work with, the actual code that specifies what you want done? “SLS files”. Or “state files”. Or “SLS[es]”. Or “Salt States”. Or “Salt state files”. Those terms are all used in the docs. (I thought they were also called “formulas“, but those are apparently only pre-canned state files.)

YAML headaches

It turns out YAML has some weird indenting and formatting rules that occasionally ruin your day. Initially it’s especially tedious having to learn this extra language and its subtleties (> vs |, multi-line hyphenated lists versus single line square-bracketed lists) when you just want to get started – it steepens the learning curve.

IDs, names

The behaviour of statement IDs is cute at first, but not well explained, and becomes pretty tiresome. The idea is you can write this:

httpd:
  pkg.installed

Succinct, huh? That’s actually shorthand for this:

install_apache_please:
  pkg:
    - name: httpd
    - installed

That is, the “id” (httpd) is also the default value for “name”. Secondly, you can compress one item of the following list  (“installed”) onto the function name (“pkg”) with a dot.

So far, so good. But it quickly leads to this kind of silliness:

"{{ pillar.installdir}}/myfile.txt":
  file.managed:
    - unless: ...
another_command:  
cmd.wait: - watch: [ "{{ pillar.installdir}}/myfile.txt" ]

Free-text strings don’t make great IDs. On the whole, I’d prefer a syntax which is robustly readable and predictable, rather than one which is very succinct in special cases, but mostly less so.

The next problem is that all those IDs have to be unique across all your .sls files. That’s tedious when you have repetitious little actions that you can’t be bothered naming properly.

Another annoyance is that Salt’s data structure sometimes requires lists, and sometimes requires dicts, and it’s hard to remember which is which. For example:

finishlog:
  cmd.run:
    - name: |
        echo "All done! Enjoy your new server.<br/>" >> /var/log/salt/buildlog.html
  file.managed:
    - name: /var/log/salt/index.html
    - source: salt://initindex.html
    - template: jinja
    - context:
       buildtitle: "Your server is ready!"
       buildsubtitle: "Get in there and make something."
       buildtitlecolor: "hsl(130,70%,70%)"

Notice how the first level of indentation doesn’t have hyphens, the second level does, then the third level doesn’t? I’m not sure what the philosophy is here – it looks to me like each level is just a list of key/value pairs.

Jinja templating logic

Jinja is a fantastic templating language. But Salt uses this template language as its core programming logic. That’s like writing a C program using macros all over the place. It’s sort of ok as a way to access data values:

update_data:
 cmd.run:
 - cwd: {{ pillar['tm_dir'] }}

Here, the {{ … }} bit is a Jinja substitution, so the actual YAML code that gets parsed and executed looks like this:

update_data:
 cmd.run:
 - cwd: /mnt/saltymill

As most people who have dealt with macro preprocessors know, they rapidly get complex and very hard to debug. The error you see is a YAML parse error, but is the problem in your YAML code, or in what got substituted by Jinja?

It’s certainly possible to use the full range of Jinja functions (that is, {% … %} territory) inside Salt formulas, but for sanity I’m trying to avoid it. On the flip side, certain things that I expected to be easy, like defining a variable in a state {% set foo = blah %} and then being able to access it from inside any other template, turn out to not work. You need to explicitly pass context variables around, or import state files, and it becomes quite cumbersome.

Help may be at hand.  Evan Borgstrom also noticed that “simple readability of YAML starts to get lost in the noise of the Jinja syntax” and is working on a new, pure Python syntax called NaCl.

Many ordering options

There are about 4 competing systems simultaneously operating to determine which order actions get executed in, ranked here from most important to least important.

  1. Requisite order: The “preferred”, declarative system, where if X requires Y and Z requires X, then Salt just figures out that Y has to happen first.
  2. Explicit order: you can set an “order flag” on an action, like “order: 1” or “order: last”. Basically, if you’re tearing your hair out with desperation, I think. (I tried it once, and it seemed to have no effect.)
  3. Definition order: things happen in the order they’re declared in. (By far my preferred option). This doesn’t always seem to work, in the top.sls, for reasons I don’t understand.
  4. Lexicographical order (yes, really): actions are executed in alphabetical order of their state name. (Apparently this was a good thing because at least it was consistent across platforms.)

The requisite ordering system sounds good, but in practice it has two shortcomings:

  1. Repetitiveness: let’s say you have psql 10 commands that all strictly depend on Postgresql being installed. It’s very repetitive to state that dependency explicitly, and much more efficient to simply install Postgresql at the top of the formula, and assume its existence thereafter.
  2. Scope: Requiring actions in other files seems to prevent the running of just this file. That is, if in load_db.sls you do “require: [ cmd: install_postgres ]”, then this command line no longer works: salt ‘*’ state.sls load_db

I’m yet to see any advantages to a declarative style. I like the certainty of knowing that a given sequence of steps works. The declarative model implies that one day the steps might be rearranged based only on my statement of what the dependencies are.

Arbitrary rules

There’s a rule that says you can’t have the same kind of state multiple times in a statement. This is invalid:

/var/log/salt/buildlog.html:
 file.managed:
 - source: salt://initlog.html 
 file.append:
 - text: Commencing build...

Why? I don’t know. The rules dictate that you have to do this:

/var/log/salt/buildlog.html:
 file.managed:
 - source: salt://initlog.html 

append_to_that_file:
 file.append:
 - name: /var/log/salt/buildlog.html
 - text: Commencing build...

Notice the cascade of consequences as this fragile “readability” tower crashes down:

  1. You can’t have two states of the same type in a statement, so we move the second state to a new statement; but:
  2. You can’t have two statements with the same ID, so we give the second statement a new ID; meaning:
  3. We now have to explicitly define the “name” of the file.append function.

Clumsy bootstrapping

I miss Chef’s elegant “knife bootstrap” though. In one command line command, Chef:

  1. Defines properties about the node, like its environment and role.
  2. Connects to the node
  3. Installs Chef
  4. Registers the node with the Chef server, or your client, or both.
  5. Starts a deployment, as required.

With Salt, these all seem to be manual steps:

  1. I SSH to the node
  2. I download and install Salt (using one-liner bootstrap script)
  3. I write ‘grains’ to the newly created /etc/salt/grains file (yes, I think I’m doing this wrong – roles should be defined through pillars maybe)
  4. The node then attempts to register itself with the Salt Master, but because the connection is insecure:
  5. I have to go to the Salt Master and accept the pending key: yes | salt-key -A [and the docs suggest you’re supposed to manually verify the keys!]
  6. Now I can launch a deployment: salt-master <nodename> state.highstate

When I rebuild a server, the above are preceded by these steps:

  1. Log in to OpenStack Dashboard, click ‘rebuild’ on the instance.
  2. Edit my ~/.ssh/known_hosts to remove the old SSH key.
  3. On the Salt Master, delete the old key: yes | salt-key <node> -d

There are probably ways of streamlining this process. I hope so, anyway, because it’s pretty clumsy. I don’t know yet whether the SaltMaster can automatically launch deployments when new nodes register.

A TileMill server with all the trimmings

Recently, I set up a server for a series of #datahack workshops. We used TileMill to make creative maps with OpenStreetMap and other available data.

The major pieces required are:

  • TileMill, which comes with its own installer, and is totally self-sufficient: web application server, Mapnik, etc.
  • Postgres, the database which will hold the OSM data
  • PostGIS, the extension which allow Postgres to do that
  • Nginx, a reverse proxy, so we can have some basic security (TileMill comes with none)
  • OSM2PGSQL, a tool for loading OSM data into PostGIS

I’ve captured all those bits, and their configuration in this script. You’ll probably want to change the password – search for “htpasswd”.

The script below is out of date, contains errors, and is not maintained. Go to:

http://github.com/stevage/saltymill

# This script installs TileMill, PostGIS, nginx, and does some basic configuration.
# The set up it creates has basic security: port 20009 can only be accessed through port 80, which has password auth.

# The Postgres database tuning assumes 32 Gb RAM.

# Author: Steve Bennett

wget https://github.com/downloads/mapbox/tilemill/install-tilemill.tar.gz
tar -xzvf install-tilemill.tar.gz

sudo apt-get install -y policykit-1

#As per https://github.com/gravitystorm/openstreetmap-carto

sudo bash install-tilemill.sh

#And hence here: http://www.postgis.org/documentation/manual-2.0/postgis_installation.html
#? 
sudo apt-get install -y postgresql libpq-dev postgis

# Install OSM2pgsql

sudo apt-get install -y software-properties-common git unzip
sudo add-apt-repository ppa:kakrueger/openstreetmap
sudo apt-get update
sudo apt-get install -y osm2pgsql

#(leave all defaults)

#Install TileMill

sudo add-apt-repository ppa:developmentseed/mapbox
sudo apt-get update

sudo apt-get install -y tilemill

# less /etc/tilemill/tilemill.config
# Verify that server: true

sudo start tilemill

# To tunnel to the machine, if needed:
# ssh -CA nectar-maps -L 21009:localhost:20009 -L 21008:localhost:20008
# Then access it at localhost:21009

# Configure Postgres

echo "CREATE ROLE ubuntu WITH LOGIN CREATEDB UNENCRYPTED PASSWORD 'ubuntu'" | sudo -su postgres psql
# sudo -su postgres bash -c 'createuser -d -a -P ubuntu'

#(password 'ubuntu') (blank doesn't work well...)

# === Unsecuring TileMill

export IP=`curl http://ifconfig.me`

cat > tilemill.config <<FOF
{
  "files": "/usr/share/mapbox",
  "coreUrl": "$IP:20009",
  "tileUrl": "$IP:20008",
  "listenHost": "0.0.0.0",
  "server": true
}
FOF
sudo cp tilemill.config /etc/tilemill/tilemill.config

# ======== Postgres performance tuning
sudo bash
cat >> /etc/postgresql/9.1/main/postgresql.conf <<FOF
# Steve's settings
shared_buffers = 8GB
autovaccuum = on
effective_cache_size = 8GB
work_mem = 128MB
maintenance_work_mem = 64MB
wal_buffers = 1MB

FOF
exit

# ==== Automatic start 
cat > rc.local <<FOF
#!/bin/sh -e
sysctl -w kernel.shmmax=8000000000
service postgresql start
start tilemill
service nginx start
exit 0
FOF

sudo cp rc.local /etc/rc.local

# === Securing with nginx
sudo apt-get -y install nginx

cd /etc/nginx
sudo bash
printf "maps:$(openssl passwd -crypt 'incorrect cow cell pin')\n" >> htpasswd
chown root:www-data htpasswd
chmod 640 htpasswd
exit

cat > sites-enabled-default <<FOF

server {
   listen 80;
   server_name localhost;
   location / {
        proxy_set_header Host \$http_host;
        proxy_pass http://127.0.0.1:20009;
        auth_basic "Restricted";
        auth_basic_user_file htpasswd;
    }
}

server {
   listen $IP:20008;
   server_name localhost;
   location / {
        proxy_set_header Host $http_host;
        proxy_pass http://127.0.0.1:20008;
        auth_basic "Restricted";
        auth_basic_user_file htpasswd;
    }
}

FOF

sudo cp sites-enabled-default /etc/nginx/sites-enabled/default
sudo service nginx restart

echo "Australia/Melbourne" | sudo tee /etc/timezone
sudo dpkg-reconfigure --frontend noninteractive tzdata

Forget trying to remember your servers’ names!

I run lots of Linux servers. I create them, install some stuff, mess around with them, forget them, come back to them…and forget my credentials. My life used to look like this:

$ ssh -i ~/steveko.pem steveb@115.146.83.268
Permission denied (publickey).
$ ssh -i ~/steveko.pem sbennett@115.146.83.268
Permission denied (publickey).
$ ssh -i ~/stevebennett.pem steveb@115.146.83.268
Permission denied (publickey).
$ ssh -i ~/steveko.pem ubuntu@115.146.83.268
Welcome to Ubuntu 12.10 (GNU/Linux 3.5.0-26-generic x86_64)
...

This got really tedious. You can’t memorise many IP addresses, so you’re constantly referring to emails, post-its or even SMSes. Then you rebuild your server, the IP address changes, and you’re lost again.

And because some of the servers are administered by other people, I can’t always choose my own user name, so more faffing around.

So, here’s my solution:

A naming convention for servers

Give each server a name. Ignore the actual hostname of the server, or what everyone else calls it. My convention goes like this:

<infrastructure><project>[<purpose>]

Examples:

  • nectar-tugg-dev: A development server for the TUGG project, running on NeCTAR Research Cloud infrastructure.
  • rmit-microtardis-prod: A development server for MicroTardis, running on RMIT infrastructure.
  • nectar-tunnelator: A side project “tunnelator” running on NeCTAR Research Cloud infrastructure. Small projects only have one server.

The key here is minimising what you need to remember. If I’m doing some work on a project, I’ll always know the project name and whether I want to work on the prod or dev server. Indeed, it’s an advantage to have to specifically type “-prod” when working on a production machine.

Put all IP addresses in /etc/hosts.

When I create a server, or someone tells me an IP, I immediately give it a name, and store it in /etc/hosts. The file looks like this:

##
# Host Database
##
127.0.0.1 localhost
255.255.255.255 broadcasthost
::1 localhost 
fe80::1%lo0 localhost
115.146.46.269 nectar-tunnelator
136.186.3.299 swin-bpsyc-dev
...

This has the huge advantage that you can also put those names in the browser address bar: http://nectar-tunnelator

If a server moves location, just update the entry in /etc/hosts, and forget about it again.

Put all access information in ~/.ssh/config

The SSH configuration file can radically simplify your life. You have one entry per server, like this:

Host latrobe-vesiclepedia-dev
 User steveb
 IdentityFile /Users/stevebennett/Dropbox/VeRSI/NeCTAR/nectar.pem
 Port 9022

Notice how we don’t need to spell out the IP address again. And by storing the access details here, we can connect like this:

$ ssh latrobe-vesiclepedia-dev

So much less to remember. And because it’s so easy to connect, suddenly tools like SCP, and SSH tunnelling become much more attractive.

$ scp myfile.txt latrobe-vesiclepedia-dev
$ ssh latrobe-vesiclepedia-dev sudo cp myfile.txt /var/www

In reality, it gets even simpler. Most of my NeCTAR boxes are Ubuntu, with a login name of “ubuntu”. The “nectar-” naming convention proves valuable:

Host nectar-*
 IdentityFile /Users/stevebennett/Dropbox/VeRSI/NeCTAR/nectar.pem
 User ubuntu

That means that any NeCTAR box using that key and username doesn’t even need its own entry in .ssh/config:

$ ssh nectar-someserver

Getting started with Chef on the NeCTAR Research Cloud

Opscode Chef is a powerful tool for automating the configuration of new servers, and indeed, entire clouds, multi-server architectures etc. We now use it to deploy MyTardis. It’s quite daunting at first, so here’s a quick guide to the setup I use. You’ll probably need to refer to more complete tutorials to cover gaps (eg  OpsCode’s Fast Start Guide, the MyTardis chef documentation and this random other one.) We’ll use installing MyTardis as the goal.

  1. Sign up to Hosted Chef. That gives you a place where Chef cookbooks, environments, roles etc will live.
  2. Install Chef (client) locally.
  3. Get the MyTardis Chef cookbook:
    cd ~/mytardis
    git clone https://github.com/mytardis/mytardis-chef
  4. Make a directory, say ~/chef, and download the “…-validator.pem” and knife.rb files that Opscode gives you to there. The knife.rb file contains settings for Knife, like directories. Modify it so it points to ~/mytardis/mytardis-chef. Mine looks like this
    current_dir = File.dirname(__FILE__)
    log_level :info
    log_location STDOUT
    node_name "stevage"
    client_key "#{current_dir}/stevage.pem"
    validation_client_name "versi-validator"
    validation_key "#{current_dir}/versi-validator.pem"
    chef_server_url "https://api.opscode.com/organizations/versi"
    cache_type 'BasicFile'
    cache_options( :path => "#{ENV['HOME']}/.chef/checksums" )
    cookbook_path ["#{current_dir}/../mytardis/mytardis-chef/site-cookbooks", "#{current_dir}/../mytardis/mytardis-chef/cookbooks"]
  5. You now need to upload these cookbooks and assorted stuff to Chef Server. This part of Chef is really dumb. (Why isn’t there a single command to do this – or why can’t knife automatically synchronise either a local directory tree or git repo with the Chef server. Why is one command ‘upload’ and another ‘from file’…? Why are cookbooks looked for in the ‘cookbooks path’ but roles are sought under ./roles/?)
    knife cookbook upload -a
    knife role from file ../mytardis/mytardis-chef/roles/mytardis.json
  6. Get a NeCTAR Research Cloud VM. Launch an instance of “Ubuntu 12.04 LTS (Precise) amd64 UEC”. Call it “mytardisdemo”, give it your NeCTAR public key, and do what you have to (security groups…) to open port 80.
  7. Download your NeCTAR private key to your ~/chef directory.
  8. Now, to make your life a bit easier, here are three scripts that I’ve made to simplify working with Knife:bootstrap.sh:
    #!/bin/bash -x
    # Bootstraps Chef on the remote host, then runs its runlist.
    source ./settings$1.sh
    knife bootstrap $CHEF_IP -x $CHEF_ACCOUNT --no-host-key-verify -i nectar.pem --sudo $CHEF_BOOTSTRAP_ARGS

    update.sh:

    #!/bin/bash -x
    # Runs Chef-client on the remote host, to run the latest version of any applicable cookbooks.
    source settingsi$1.sh
    knife ssh name:$NECTAR_HOST -a ipaddress -x $NECTAR_ACCOUNT -i nectar.pem -o "VerifyHostKeyDNS no" "sudo chef-client"

    connect.sh:

    #!/bin/bash -x
    # Opens an SSH session to the remote host.
    source ./settings$1.sh
    ssh $CHEF_ACCOUNT@$CHEF_IP -i nectar.pem -o "VerifyHostKeyDNS no"

    (We disable host key checking because otherwise SSH baulks every time you allocate a new VM with an IP that it’s seen before.)With these, you can have several “settings…sh” files, each one describing a different remote host. (Or just a single settings.sh). So, create a “settings-mytardisdemo.sh” file, like this:

    # Settings for Ubuntu Precise NeCTAR VM running mytardis as a demo
    export CHEF_HOST=mytardisdemo
    export CHEF_IP=115.146.94.53
    export CHEF_ACCOUNT=ubuntu
    export CHEF_BOOTSTRAP_ARGS="-d ubuntu12.04-gems -r 'role[mytardis]'"
  9. Now you have it all in place: A chef server with the cookbook you’re going to run, a server to run it on, and the local tools to do it with. So, bootstrap your instance:
    ./bootstrap.sh -mytardisdemo

    If all goes well (it won’t), that will churn away for 40 minutes or so, then MyTardis will be up and running.

  10. If you change a cookbook or something (don’t forget to re-upload) and want to run chef again:
    ./update.sh -mytardisdemo
  11. And to SSH into your box:
    ./connect.sh -mytardisdemo

Improving on the “administration rights required” workflow

Consider an action like creating a new “space” in Confluence. Because of the visibility of this action, people want it restricted. Controlled. Managed. So Atlassian makes it only available to people with administrator rights. Which seems ok, until you realise that the workflow of the non-administrator ends up looking like this:

  1. …doing stuff…
  2. Decide to create new space
  3. Attempt to create space, discover that you need an administrator to do it
  4. Find out who the administrator is
  5. Ask them to do it
  6. Wait
And for the administrator it looks like this:
  1. …doing stuff…
  2. Receive request to make a new Confluence space.
  3. Confluence? What? I’m busy configuring a new VM here…oh, fine.
  4. Go log into Confluence, remember how to create a space
  5. Get back to work
And probably there will be some miscommunication about exactly what is required. Confluence spaces are a bit of a trivial example. In other instances, you need a lot of information to perform the administrative action, and if there’s a mistake, it again requires the administrator to fix it. Eventually the administrator gets sick of being bothered with such trivia, and the non-administrator gets sick of hassling them, finding an alternative, or living with a misconfigured thing.
For the user: frustration, blockage, helplessness
For the administrator: disruption, menial tasks

Alternative workflow #1: approval only

For the non-administrator:

  1. …doing stuff
  2. Decide to create a new Confluence space
  3. Enter the form, fill out all the details, press Ok.
  4. (Confluence sends a “Is this ok?” email to administrator)
  5. Wait
For the administrator:
  1. …doing stuff
  2. Receive approval request.
  3. Since it’s from a fairly trustworthy user, and seems to make sense, click the link.
  4. Get back to work.
It’s better. The administrator now doesn’t need to know anything about the request. This workflow exists in some kinds of systems (CMSes particularly), but could be a lot more widespread. Ironically, although Jira is a workflow tool, it doesn’t actually have any workflows built in for administration.
For the user: blockage
For the administrator: disruption

Alternative workflow #2: private pending approval

Since creating a space has very limited potential for destruction if it’s hidden, how about this:

For the administrator:

  1. …doing stuff
  2. Decide to create Confluence space
  3. Fill out form
  4. Press Ok
  5. (Confluence sends email to administrator, creates space in “private” mode)
  6. Start working in new space. Do almost anything except collaborate with others in this space.
The benefit is clear: there is now no “waiting” step.
For the administrator, it’s the same as before:
  1. …doing stuff
  2. Receive approval request.
  3. Since it’s from a fairly trustworthy user, and seems to make sense, click the link.
    1. If more information is required, look at what the user has done in the new space, for a bit of context.
  4. Get back to work.
There is a mild benefit here, too: the administrator is no longer under as much pressure to immediately approve the thing, and can even see what has been done with the space.
For the user: flow maintained
For the administrator: very mild disruption

Alternative workflow #3: public until reverted

Most users aren’t destructive. And especially in professional environments, virtually all users can be trusted to act in good faith. (Managers strangely predict “chaos” if many users are authorised). So, let them do it:

  1. Decide to create Confluence space
  2. Create Confluence space
  3. Work in new space
For the administrator:
  1. Receive notification that a space has been created
  2. If it looks wrong, strange, inappropriate etc, discuss with the user, and possibly remove it.
  3. Otherwise, keep working
For the user: productivity, responsibility
For the administrator: productivity, trust