Sage on Ubuntu

This is a work in progress.  I have been using http://wiki.sagemath.org/SageServer and have got stuck with sage0 logins.  It is supposed to setup passwordless ssh keys but doesn’t seem to have worked.

Resetting the Product key in Windows 8

I have a Microsoft MSDN Pro subscription that lets me try out software pretty much as it is released. Microsoft's MSDN website apppeared to be struggling yesterday with the delivery of Windows 8. I installed a new version and struggled with the conversion from Windows consumer preview to the Release To Manufacturing (RTM). In the progress the Windows activation didn't work and today I couldn't activate it.

A quick websearch got this website with the key command in it:

"For some users who're using Windows Vista Business Edition or other volume licensing (VL) editions meant for activation via KMS method, the system may erroneously search for KMS host for activation, even if the user uses MAK product key with less than 25 machines on the network. The solution to volume licensing customers for 0X8007232B is also the same - reenter the product key again, using step above, or alternatively, run the following commands in an elevated command prompt:

slmgr.vbs -ipk <Windows activation product key or MAK key>"

I used this and all was well with Windows 8. I didn't even need to activate it again.

Gmail and the iPhone

I have a client who is having some trouble with the connection of Gmail and the iPhone.  They are reporting that by reading emails on the iPhone they are being deleted.  I have never noticed a problem for myself.

The variables

There are a number of things that can vary:

• Subscription level Free (Google Apps) or paid for (Google Apps for Business) (The education  level
• Imap or Pop 3 connectivity
• Iphone setup as Gmail or Outlook
• Transitioned account form single login
• Enable SSL (enforce SSL for Gmail connections)

I have three Google accounts.  I logged into each one and copied the settings

• A has no Google Apps, forwards a copy of each email to my hosted Outlook account.
• B has the free Google Apps
• C has paid for Google Apps for business. 

One thing that I was not clear on before is that you can log into your Google account but you need to seperately login to your Apps for Business account  (which you can get to by https://www.google.co.uk/a) (unless you click on a Manage Domain button in an email from Google).

Settings for account A:

Settings for accounts B and C:

Creating a new GIT repository on Synology

These are the tasks I need to go through to create a new git repository on my Synology server.  I already have the Git server running and a putty client connection setup.

I login to the server using the root user. Go to the directory \volume1\git which was a share I had create earlier.

mkdir newproj

chown me:users newproj

cd newproj

git init –bare

chown –R me:users *

Then logging to new project directory on the development machine and right click and using TortoiseGit leaving the bare option unclicked.  You now have a local repository.  Now push it to the arbitrary URL using the master branch ssh://me@192.168.1.1/volume1/git/newproj

You will then be prompted for the password as the Putty key has not been set up.

Configuring Django on Vagrant not including chef

In my last post the formatting was going odd so have started again.  My next job is to get Django installed on the UAT rather than a plain Ubuntu box.  You will see at the end I deferred using chef and just got the machine up and running.

I am going to try and follow Simon William’s post. I have a working server.  I also looked at this video. git was not installed on my base machine so I logged in and installed it, made a new directory and got my test software:

sudo apt-get install git
git clone https://github.com/simeonwillbanks/vagrant-up-django-app-server.git

The syntax of the command in the Vagrant file had changed from

config.vm.forward_port “http”, 80, 8080

to

config.vm.forward_port 80, 8080

Then I needed to install chef which I hadn’t done. So trying the bootstrap knife on my local machine, and having the usual problem about verifying hosts:

C:\Users\me\chef-repo\.chef>knife bootstrap 192.168.1.7 –x me -P mypassword --sudo --no-host-key-verify

It didn’t work.  Unsure why I just installed directly on my base station using the apt-get process which worked fine and I tested it had worked with:

chef-client –-version

However I had forgotten the /etc/chef/validation.pem file which I moved with a fabric script:

def base_key():
    put(r'c:\chef\validation.pem','/home/me/validation.pem')
    run('sudo mv /home/me/validation.pem /etc/chef/validation.pem')

Then chef-client worked (don’t forget the sudo)

sudo chef-client

I had to reedit the Vagrantfile.  Vagrant up now nearly worked.  However I would get the message “The file /etc/chef/validation.pem does not contain a correctly formatted private key.”  It looked fine but I must have edited it with notepad as it had CR (Ctrl-M) at the end of each line.

Now it booted ok but it complained that it had already been registered (which it had in the last debug cycle).  At the end of the cycle I just destroyed the machine leaving the node registered on the chef server.  I need to bring the machine offline gracefully.

This command is still failing:

chef-client -c /etc/chef/client.rb -j /etc/chef/dna.json

One of my problems mois that I am building Ubuntu lucid boxes which have chef 0.09 and I am running it from a base build of Ubuntu Oneiric 11.10 with chef 0.10.  So I am going to use veewee to make a new base box.  I install veewee:

sudo gem install veewee

It didn’t go well veewee installation didn’t complete and it corrupted some gemspec files that I had to hand edit to fix.  I rebooted and my test vagrant box is working again.  I learnt one useful thing in that the Vagrantfile directory (on my base machine ~/uatbox32) is mounted as /vagrant in the vagrant virtual machine.

I mounted the VBoxGuestAdditions.iso in the client box and that eliminated one problem.  I still need to work on the chef upgrade from 0.09 to 0.10 which I did on the virtual box with:

sudo gem install chef

I hadn’t realised that chef had both clients and nodes and I needed to delete both from the chef server page after a vagrant up.  On a reboot the guest additions reverted – I should have done halt rather than destroy.  Luckily I had packaged the build so can use that as a base.

I am going back to Fabric for deployment.  A  historical chanes that might catch you out (it caught me out).  In 0.9 and later versions of fabric (I am using 1.4) Fabric uses the ‘env’ environment dictionary which used to be called config.  So any examples with config.x = are no obsolete.

I have found some nice guidance on django best practices:

http://lincolnloop.com/django-best-practices/projects.html

After a lot of mucking about I now have a working site.  The next stage is to simplify and to roll out for the dev,test,uat and prod environments.

dev is on my development machine and might do anything.

test is on production type hardware and is for testing.

uat is on production type harward and a recent backup of the production database for final acceptance.

prod is then the delivered product.

Long post now shorter

I had a long post about the tribulations of installing Ubuntu on a local server and then getting virtual box.  However a combination of Windows update and Windows live writer managed to edit it down to nothing so I am starting again.

Installing the OS, OpenSSH, virtual box and Vagrant

I used Ubuntu 32 bit 11.10 (as I needed RubyGems >= 1.3.6) to install virtual box and vagrant.  I should have selected OpenSSH as a default package (I selected nothing) so had to install that later:

sudo apt-get install openssh-server openssh-client

In order to login with out a username and password I wanted to used the SSH public key.  I modified the network interfaces to change the IP adresss and after a reboot changed the SSH control file to read the atuhorisation file.  Then added the authorisation key to the public key from that generated by Putty.  Note you have to edit the framework and end of lines, see here.

Another lifesaver which slowed me down all afternoon:

“--disable-known-hosts stops Fabric from getting confused when you generate the same VM repeatedly”

Vagrant up for the first time

The aim here is to get the first box so that I can power it up and down – then aim to make that happen using fabric.  I thought I might have to this several times so firstly I decided to download the box to my local webserver from opscode and then install from there.

 vagrant box add lucid32 http://files.mysite.net/ubuntu10.04-gems.box

That turned out to be a good plan as the update then only took a few seconds. Then to start the machine and vagrant up:

mkdir devbox32
cd devbox32
vagrant init lucid32
vagrant up

The machine paused and after some messages I had a new machine in about 2min.

I can login to it either using putty from my machine or:

vagrant ssh

vagrant ssh from the host machine.  The username and password were vagrant.  Then I shut it down with:

vagrant halt

vagrant destroy -f

You don’t need the halt but it is a bit cleaner.  Next time you start up you don’t need to download the box.  The vagrant up command is now in fabric.  The complete code is in the appendix 1, here is the guts of it and the command lines:

def uat_up():

run('cd uatbox32 && vagrant up')

and the command line on my windows development box is :

fab --disable-known-hosts base uat_up

and I can login with Putty.

Appendix 1 Initial fabric file to do vagrant up remotely

Complete fabfile.py to get you started you will need to edit the me to your user:

import os.path

import subprocess

from  fabric.api import *

from fabric.contrib.console import confirm

# Constants

KEY_DIR = r'C:\Users\me\Library\keys'


PUTTY_DIR = r'C:\Program Files (x86)\PuTTY'

# These private python functions do not get exposed on the command line via the

# 'fab' tool. They will not show up when the user runs 'fab --list'


def _start_pageant_and_store_key(keyfile_name):


    """


    Starts the Pageant key agent, and loads the private key in to it, prompting


    for the passphrase if needed. Also sets the Fabric env.keyfile_name


    property.


    """


    keyfile_path = os.path.join(KEY_DIR, keyfile_name)

    # Start pageant. Note that this works with the PuTTY key format (*.ppk), not

    # the OpenSSH format.


    path_to_pageant = os.path.join(PUTTY_DIR, 'pageant.exe')


    result = subprocess.call([path_to_pageant, keyfile_path])


    if result > 0:


        raise OSError, "Bad result %s" % result

    # Store the path to the key file in fabric's global env dictionary

    env.key_filename = keyfile_path

def base():

    """


    Set up for connection to the base server for provisioning of


    """


    # Once DNS is set up for these hosts, I can use hostname instead of ip address


    env.user = 'me'


    env.hosts = ['me@10.0.0.7']


    _start_pageant_and_store_key('UAT_PrivateKey2.ppk')

def uat_up():

    run('cd uatbox32 && vagrant up')

def uat_destroy():

    run('cd uatbox32 && vagrant destroy -f')

Installing chef on a TurnkeyLinux instance

I thought this would be easier than it turned out to be.  I needed to do a number of things :

• install sudo eg  

apt-get install sudo

• copied all the infstall files from my orignal instalataion eg knife.rb, Mame.pem, myOrg_validator.pem to /etc/chef
• make sure directory is mapped

cd ~ | ln –s /etc/chef ./.chef

Then my command knife note create django00

knife node create django00

I started using the hosted chef model and after this I had a single node. Exciting stuff.  It would be nice project to get the bootstrap code to work for the TurnkeyLinux instance.

Installing Dia python plug in on Windows 7 x64

I tried to run the intro tutorial for Dia but it didn’t work.  Assuming that this might be something do with the different directory structure I  tried putting my code here

C:\Program Files (x86)\Dia

which is not very nice but does work.

SVN creating a new repository

Since I am now running my own SVN server there is only a slight cost in time to creating a new repository.  The top level directory in an SVN server is the repository and has to be created on the server rather than creating lower level directories which can be done by Tortoise SVN.  Getting round running at the adminstrator level is handy about svnSync and also about the SVN hosting services such as Codesion  and SlikSVN.

So on synology SVN host I logged in as root, there may be better ways of doing this but this worked for me.  Created my new site:

svnadmin create /volume1/svn/new_repository

This now inherits the wrong file permissions (root) so the directory listing looks like this:

so I moved to the directory and changed the owner and group:

cd /volume1/svn/new_repository
chown –R svnowner:users .

(don’t forget the trailing dot) and now it looks like this:

Next you need to add your username and password to the configuration file new_repository/conf/passwd (I am using vi for the editor) eg

Then in the same directory you need to change the following entries for the configuration file (svnserve.conf)

Make anonymous access have no access and authorised access write access:

Turn on using the local password database:

I also changed the realm to my URL but that is optional.

Now you can import your new directory using the method I talked about a while back.

Configuring VOIP phone and PBX passwords

I have been using the voipfone service as a virtual PBX.  It is so far working well although I had not been using any of the VOIP features just the call diversion service.  I tried out RingCentral but didn’t like the fact that my calls appeared to come from the US.  I like the fact that you can easily apply the recording of calls to meet compliance.

I had a problem configuring my VOIP SPA941 phone yesterday as I had not realised that my voipfone account has a master password but each extension has its own password.  When setting the password you need to give the extension password.  Everything now works fine.

Using Boto and Python to connect to a specific region

Here is my slightly inelegant but working code to connect to specific region without setting up a configuration file or altering the default in the library (the access keys you can find under the tab security credentials on :

aaaa = your AWS Access key
sssss = your Secret Amazon key

from boto.ec2.connection import EC2Connection
default_conn = EC2Connection('aaaaa','sssss')

regions = default_conn.get_all_regions()

for r in regions:
    print r.name
    if r.name == 'eu-west-1':
       conn = EC2Connection('aaaaa',’sssss',region =r)

print regions

rs = conn.get_all_security_groups()
print rs

Installing Python Pyjamas and Pyjamas desktop for Windows 7

I am testing out python pyjamas as a gui front end which I can scale from a desktop app to web app to cloud deployable scalable app with very little change.  I have tried a number of different GUI stacks, the last was pytgtk which only worked with 32 bit Python which I had to install on top of my 64 bit python.  To start I have gone back to 64 bit python.

Going to the pyjamas wiki getting started  I started using the windows help.  I noticed it was a little out of date so decided to write my experience here.  I have already got a working version of Python 2.72 64 bit on Windows.

Next I installed tortoiseGIT as I have used TortoiseSVN a lot.  I also needed to install the Windows GIT client which I should have done first.  Then I create a pyjamas directory in my download directory and use right click Git clone to create the directory.

You need to change the URL to git://pyjs.org/git/pyjamas.git and press OK.

The first time I did it it failed so I deleted everything and tried again and this time it worked.

I then installed comtypes which was version 0.6.2.  I had two versions of Python installed so I selected Python 2.7.  It then had an error at then end:

Carrying on, assuming that GTK may not be needed as Pyjamas desktop is using MSHTML and for my desktop I am at the moment only worried about desktop Windows apps. (I really don’t want to go back to Python 32bit again) I copied the pyjamas directory to my C drive so that I had the following directory C:\pyjamas\pyjd

I updated the system path  ControlPanel->System->Advanced system settings->(Advanced Tab)Environment Variables->Edit path  I added C:\python27;C:\pyjamas\bin. Note the pyjamas bin directory is added in the next step:

Open up a command window and then run which will create the bin directory:

cd \pyjamas
python bootstrap.py

I then created a build.bat in the helloworld directory:

pyjsbuild.bat --print-statements Hello

and also a run file:

pyjd Hello.py

Pyjamas works by combining files from the public directory eg files and template html and your code.  In order to use either the output or the desktop you must run the build process first.  Then you can use the run file first the app just opened up  in a new window and worked:

If I opened the browser with output\hello.html I just got the static text but need to change to the output directory and run the html file (so the default location of other files works) then I get the application in a browser window.  I was using by default IE and didn’t work on getting the others to work.  This from start to end took me about 3 hours including writing my notes as I went.

SVN sync

SVN has a very nice sync feature.  I have used at the end of a project to hand over by coping the version control information to a client repository.

I normally use TortoiseSVN and AnkhSVN to access the SVN repository but for this you you need an SVN client.  I used the the SlikSVN client with the typical configuration since they are selling hosting they are likely to make sure it works well (I used CVSDude way back now its unclear if it is called Codesion or Collabnet).  This has the svnsync command in it.

I got started with a post by Paul Querna.  Since I was doing it on a Windows machine and had a couple of problems I have written about my experience here.

Instead of defining environment variables I just created a number of batch files.   The first to make sure my SVN environment is working:

"C:\Program Files\SLIKSVN\bin\svn" --version

This command should return the version and a whole lot more info.  Then I tested access to the destination and source repositories eg:

"C:\Program Files\SLIKSVN\bin\svn" list https://myAcct.svn.cvsdude.com/myProj –username secretUser --password

secretPW

or

"C:\Program Files\SLIKSVN\bin\svn" list svn://192.168.0.0.1/myProj –-username secretUser -–password secretPW

This will return nothing for the destination repository as nothing in it and the  list of directories in the source repository.  If the hook properties need to be changed then the following code should be executed in the root directory of the source repository:

$ echo "#!/bin/sh" > hooks/pre-revprop-change

$ chmod 755 hooks/pre-revprop-change

Now you are ready to initialise syncserve:

"C:\Program Files\SLIKSVN\bin\svnsync" init svn://192.168.0.1/myProj --sync-username desSecretUser --sync-password destSecretPassword https://myAcct.svn.cvsdude.com/myProj --source-username sourceSecretUser --source-password sourceSecretPassword --non-interactive

You may get a warning about no supporting atomic revision and upgrading to 1.7.  For 1.6 this was fine.

svnsync: warning: W200007: Target server does not support atomic revision property edits; consider upgrading it to 1.7 or using an external locking program
Copied properties for revision 0.

And now you should be ready to start the sync (just change init to sync):

"C:\Program Files\SLIKSVN\bin\svnsync" sync svn://192.168.0.1/myProj --sync-username desSecretUser --sync-password destSecretPassword https://myAcct.svn.cvsdude.com/myProj --source-username sourceSecretUser --source-password sourceSecretPassword --non-interactive

You will then get a list of changes copied over (the . after Transmitted file data refer to the amount of data transferred):

Copied properties for revision 9.

Transmitting file data …..

Commited revision 9.