Category Archives: Linux

VirtualBox – moving virtual disks

Having installed VirtualBox I created a few virtual machines, with 20GB virtual disks each. After all, I’ve got a 1TB hard disk in my computer.

When Linux started complaining about running out of disk space I realised that my main drive is a 256GB Solid State Drive, and that is where VirtualBox was storing the virtual disks. Time to move the virtual disks. Please note that it is a bit of a manual process and, whilst it worked fine for me, if you’ve already invested a lot of time setting up your virtual machines, I suggest that you test this out by moving one or two VMs first.

  1. Stop all VMs, stop VirtualBox
  2. Find a folder called “VirtualBox VMs” (probably in your home folder), and move it to the new location
    1. Or move just a few files first, to make sure it works
  3. Restart VirtualBox
    1. In File -> Preferences -> General, point the Default Machine Folder to the new location
    2. All your existing VMs are now broken. Delete them
    3. One at a time: Machine -> Add
      1. Select the .vbox file from the new folder

Virtual machines using VirtualBox

Tapping into all the wonderful tools and languages that make our lives as developers so interesting can throw up some fun (?) challenges.

You start installing the latest linter or library, and before you know it you’re in dependency hell. Required libraries require even more libraries, with difference versions numbers and clashing with your existing setup.  If you’re really unlucky you break some essential software along the way.

A safer approach is to use virtual machines, for experimentation, and to keep different (and clashing) environments separate.

For more detail see the step by step instructions at Everyday Linux User.

These instructions are for installing VirtualBox on Linux Mint (17.2), and then creating a Linux Mint virtual machine

Installation and setup

I used mostly used the default settings. This may not be right for everyone

  1. Use a package manager to install VirtualBox.
    1. I used Synaptic, and selected virtualbox-5.0
  2. When I started VirtualBox and tried to create my first virtual machine (VM), it only let me create 32 bit VMs, even though my computer is fully 64 bit
    1. Fix this by going into the BIOS and enabling (Intel) Virtualisation
  3. If like me you have multiple drives (SSDs and/or HDDs) or multiple partitions, specify where VirtualBox stores the virtual drives (files):
    1. File -> Preferences -> General -> Default Machine Folder

Your first virtual machine

  1. Download a copy of the relevant linux Distro
  2. Start VirtualBox: Menu -> Administrate -> Oracle VM VirtualBox
  3. Click on “New”
    1. Name: E.g. “Linux Mint 1”
    2. Type: “Linux”
    3. Version: Linux 2.6/3.x (64 bit) or Other Linux (64 bit)
    4. Memory: Recommended size or higher (note: you can always change it later)
      1. During my first attemp I used the default of 256MB. When trying to boot Linux Mint off the install ‘disk’, the virtual machine ground down to a halt. Increasing this to 1GB fixed this)
      2. 512MB minimum, 1GB+ is probably better
  4. Hard drive: Create a virtual drive now
    1. Note: VirtualBox creates a file and pretends that it is a whole hard disk (i.e. a “virtual drive”)
    2. Type: leave as is (VDI)
    3. Size: Dynamically allocated
    4. Limit: leave as is or increase
    5. Create
  5. The machine has been created, but isn’t running yet. Click on the “New” button
    1. Click on the folder with green ‘arrow’ icon
    2. Select the previously download distro
    3. Click Start
      1. This will start the VM, which will boot off the downloaded distro
    4. Follow the instructions to install the new OS
    5. Suggestion: Use a different password from your main password. This is generally good security practice, but in this case it may stop you doing something on your actual Linux install whilst thinking you’re working on a VM. I realised this when I tried to stripdown all s/w on a VM to only that relevant to development work, and nearly removed some software of the host operating system
  6. If you get a large error message “Running in software rendering mode”:
    1. With VM stopped, go into settings -> Display -> Enable 3D accelleration

I suggest you clone this VM before you start experimenting, to save you having to reinstall the OS should you need another (clean) VM

Python, CherryPy and Websockets – hosted

To set up websockets on a host can take a few extra steps. The host needs to know what to do with the websocket traffic.

It took a few attempts to get a Python/websockets sample up and running. The first websocket library I used (for Python/Bottle) worked very well on my local machine, but the host didn’t have the required libraries installed and didn’t give me enough access rights to fix this. I switched to a different framework and library, which worked much better.

  1. Create a new (WebFaction) application, wstest, custom websocket
    1. Save it, attach it to a domain/website, and make a note of the port
  2. ssh into the host
  3. Set up the requirements:
    1. pip install cherrypy
    2. pip install ws4py
    3. pip install argparse
  4. Switch to the application’s folder (cd webapps/wstest)
  5. Create index.html and site.py as created by Sylvain Hellegouarch, but with following changes:
    ….
    return index_page % {‘username’: “User%d” % random.randint(50, 1000),
    ‘ws_addr’: ‘ws://<url>/ws’}
    ….
    cherrypy.config.update({
    ‘server.socket_host’: ‘127.0.0.1’,
    ‘server.socket_port’: <the application’s port – see above>
    })

    1. Important (this took me a while to figure out): the original websocket url (first/second line, above) in the sample code is “ws://localhost:9000/ws”. Having tried a bunch of different ports (80, 9000, the application’s port, etc), the correct solution is to leave the port off completely, so just use “ws://<url>/ws” instead
  6. Start CherryPy: python2.7 site.py
    1. Since you’re running this still attached to your terminal session, when you close the (ssh) terminal the process will stop as well.
  7. Now open the url in two separate browsers. You should be able to enter messages in either browser and see it appearing in both

Managing the repositories in Linux

Much of Linux’s software is stored in repositories. Your Linux installation has a list of repositories, with the most stable recent packages for your distro and possibly some others.

To view, add or delete the repos

In Synaptic, go to Settings -> Repositories

  1. Official Repositories
  2. Upgrade to the latest version – PPAs – Personal Package Archives
    • The official repo contains a fully tested set of packages – both tested individually and together. So you can be confident that all packages work together, that they don’t clash. Many of the packages in the official repo are one or more versions behind the latest version. PPAs contain other versions, and may include the latest version. Installing a package from a PPA increases the risk of packages clashing
    • For more information on PPAs see this article on How To Geek
  • Additional Repositories – for software that isn’t part of the official repo. Often the software’s website will give you step-by-step instructions on how to install it, including the command line for adding the repo. Or you can use Synaptic – Click on “Add a new repository” and enter the name and the URL

Linux package management – some common operations

For my own benefit (and maybe others), here are some of the common package management operations and how to do it (in Linux Mint and other Linux distros)

  • Update all install software:
    • apt-get update # refresh repos
    • apt-get upgrade # upgrade all software
    • (note: do both of the above, in that order)
    • Note: In Linux Mint you’ll see a shield with an exclamation mark in the bottom right hand corner when you need to update your software. Click on it and follow the instructions
  • Install a new package, if you know the name: apt-get install <package name>
  • Search for a package: use Synaptic (Menu -> Administration -> Synaptic), enter (partial) name or other keyword (e.g. “game”) in the search box at the top of the screen
  • View installed packages: In Synaptic, bottom left hand corner, click on “Status”, then (top left hand corner) click on “Installed”
  • Get more information about an installed package
    • Use Synaptic, find the package, click on the Properties button (top of the screen)
    • Common: Installed version, status (e.g. installed)
    • Dependencies – what other packages this package needs to work
    • Installed files – what files is this package made up of
    • Versions – what versions are available
    • Description

Linux package management: apt-get, aptitude, synaptic, wajig

When you set up your Linux system, you typically grab an image, burn it to DVD, boot the machine off the DVD and follow the instructions. The image will contain the most recent (stable) version of the core Linux distro files and of the packages which are bundled with it. The package managers let you keep your packages up to date, add new packages, remove packages, etc. They also deal with the dependencies; when installing a package, all packages it relies on are automatically installed as well

The package managers do this by accessing repositories (repos) of packages. Your distro came configured with a set of repos which contain packages which work on your repo. If you want to install some software which is not in your standard repos you may need to add a repo to the list

At the basic level, the system uses apt and dpkg to manage the software. The following are some front-ends on top of apt and dpkg:

  • apt-get: command line only
  • aptitude: very similar to apt-get, plus a text-based “gui”
  • synaptic: gui
  • wajig: slightly friendlier front-end to both apt-get and aptitude. Brings together the power of both

For more information have a look at this article

Linux Mint – family tree

Having used Fedora, CentOS and Debian (and a little of Raspbian and Unix) a couple of years ago I swapped over to Linux Mint. Since it is based on the popular Debian and Ubuntu it has an excellent eco-system –  packages, community and support

See Wikipedia for some more information about Linux Mint. This family tree shows you the relationship between many of the main Linux distros.

DevOps – intro

If I understand it correctly (it looks like the jury is still out), a DevOp is someone who helps developers be more productive by giving them the tools they need.

As a freelance developer, from time to time I do some DevOp work, such as installing a library or some new software. Things can get quite complicated, but understanding how it all fits together, and knowing some tools and commands, can really speed things up.

Since I mainly use Linux Mint and Python, many of the DevOp articles on this website are about Linux Mint and Python. A lot of it will be relevant to other Linux distributions (distros) and languages.