Flashing CyanogenMod onto Galaxy SII via Ubuntu

Pre-Requsites

Install Heimdall

Visit the Heimdall home page, select “Linux”, and download the package for your distribution.  On Ubuntu variants you want the latest ‘.deb’ package.  e.g.:

ubuntu13.04-heimdall_1.4.0-0_amd64.deb

Note that the “Frontend” packages are not required.  Everything can be done easily from the command line.

Install Heimdall using dpkg:

sudo dpkg -i ubuntu13.04-heimdall_1.4.0-0_amd64.deb

Install adb

Install adb from the Ubuntu repositories:

sudo apt-get install android-tools-adb

sudo apt-get install android-tools-fastboot

Download ClockworkMod Recovery

Download ClockWordMod Recovery for the Galaxy SII (i9100).  I got it from here.

Download CyanogenMod

Download a compatible version of CyanogenMod for your handset.  You can get it here.

My personal preference is to stick with CM11 (Android 4.x) on the Galaxy SII.  CM12 (Android 5.x) is available too but it’s an old handset now so you need to consider if it’s ideally suited to the newer versions of Android.  CM11 runs really fast on it, so I prefer that.

Download GAPPS

Normally the best place to get GAPPS (Google Applications; Play Store etc) is via OpenGAPPS.  But in my experience the packages provided by OpenGapps are too large for the Galaxy SII.  If you took my advice to install CM11, then you need GAPPS version gapps-kk-20140105-signed.zip, provided here.  This one is small enough to install successfully.

Flash Custom Recovery onto the Galaxy SII

With the handset turned OFF, place it into “Download Mode” by holding VOLUME-DOWN, HOME & POWER.
You should see a screen which looks like this:

IMG_20160717_092817

Press ‘VOLUME-UP’ to confirm and then you should see the Download Mode screen:

IMG_20160717_092840

Connect the handset to your computer via USB.

Now, on your computer, browse to the location where you downloaded the CWM package (see Pr-Requisites section).  Extract it.  It should come out as a filename called “zImage”.  If what you get is named something a bit different don’t worry, just take account for that in the terminal command you’re going to enter next…

Open a terminal and ‘cd’ to the directory where the CWM package was extracted.  Then enter the following command:

sudo heimdall flash --KERNEL zImage --no-reboot

You should see a blue transfer bar appear on the handset download screen.  The transfer is complete when the blue bar has filled, as shown in the photo below.

Downloading

IMPORTANT – Boot into Recovery Next

This next bit is a little important; you need to boot into the custom recovery you just installed next, before you do anything else.  If you don’t, the stock recovery might overwrite CWM.  The consequences of this are minimal; you’ll just have to start again! But to avoid this, boot into recovery as follows:

  • Disconnect your USB cable from the handset.
  • Turn the handset OFF by holding the POWER button.
  • Boot into CWM by holding VOLUME-UP, HOME & POWER.

You should see a screen like this:

IMG_20160717_095032

  • Now, using the VOLUME buttons select “wipe data/factory reset”
  • Press POWER to enter.
  • Select “Yes – Wipe all factory user data”
  • Press POWER to enter.

If factory wipe is successful you will be returned to the main menu.

Flash CyanogenMod

Now for the cool bits.  Connect your handset back to the computer via USB.
Open a terminal and ‘cd’ to the location where you downloaded the CyanogenMod zip file.  Don’t extract the zip! We’re going to transfer it as-is to the handset and then the handset will extract it and install.

At the handset, use the VOLUME buttons to select “install zip” and then press POWER to enter.
Select “install zip from sideload” and select.

At the handset you should see a message confirming that sideload has started.  Using adb, you need to transfer the CyanogenMod zip file using a terminal command as follows:

adb sideload cm-11-20141115-SNAPSHOT-M12-i9100.zip

Adjust the command to suit the filename of the ClockworkMod you downloaded.

You should see some information in the terminal to confirm that the file is being transferred.  When it’s done, the handset will attempt to install the update, and you’ll see a screen similar to below.

IMG_20160717_095948

Unfortunately you don’t get any obvious confirmation that it completed at the handset.  But one way to tell is to try the menu selection; if you have control of the menu using the VOLUME keys, it has finished.

Install GAPPS

This next bit is optional (maybe you don’t want any standard Google Apps on the handset) but without it you won’t be able to use the Play Store.

Using the same method as you used to transfer and flash CyanogenMod, ‘cd’ to the location where you downloaded the GAPPS package (see Pr-Requisites) and then enter the zip sideload function on the handset.  When ready, use adb to transfer and install GAPPS:

adb sideload gapps-kk-20140105-signed.zip

If the process fails you’ll be informed about it.  Otherwise, if you haven’t been shown any errors on the handset and you have control of the menu, it’s finished.

DONE! Reboot

That’s it! You’re done.  Use the menu to select “Reboot” and enjoy CyanogenMod on the Galaxy SII.
Note that boot-up will take longer the first time as the handset will install and set itself up.

 

 

Install Freerouting on Ubuntu 14.04 – 16.04

Introduction

Freerouting was a web application created and maintained by Alfons Wirtz at his website, here.  It allowed us to import a design file from kicad and then auto-route the project based on some design preferences.  It was basically a free auto-router for kicad.  Hence the name!

For “various reasons” the author has decided to drop the project, and it is no longer available as a web application at his site.  However, he has been kind enough to open-source the project and he encourages users to run it as a local application.

There are some wikibooks instructions for installation of the Freerouting application here.  I have recently attempted to follow these instructions, but there are some stumbling-blocks which prevented me from being able to run a working application.  I believe these same stumbling blocks will face lots of people; at least those who decide to install it on a recent Ubuntu variant.

This blog will detail all of the instruction steps necessary to run a working local copy of Freerouting.  My instructions are based on those at Wikibooks, but with some extra steps to get around the stumbling blocks.

The Pre-Requisites

You can skip this part if you already have installed a recent version of:

  1. Git
  2. Java JDK
  3. Netbeans

Otherwise, read on!

Installation of Git

Use your package manager, as follows:

sudo apt-get install git

Installation of Java JDK & Netbeans

This post used to direct you to install the JDK and Netbeans packages separately, but all of a sudden this method stopped working.  When I queried it with the Netbeans team they refused to admit any problem, but suggested the “combined package” instead.  Well, I don’t agree there isn’t a problem but the combined package does work so let’s do that instead!

First you need to locate the JDK & Netbeans combined package.  At the time of writing this post, you can get it here.

Once you’ve downloaded it, you need to make sure that you have permissions to execute it.  To do that, ‘cd’ to the directory where you downloaded the installer, and then type the following command, substituting for your version of netbeans if it’s different from mine:

chmod u+x jdk-8u111-nb-8_2-linux-x64.sh

Then execute the installer:

./jdk-8u111-nb-8_2-linux-x64.sh

You should see a screen something like this:

JDK & Netbeans Combined Installer
JDK & Netbeans Combined Installer

Walk through the installer dialogue to completion.

Installation of Freerouting Dependancies

The freerouting application depends on a couple of things to work.  It requires jh.jar and netx.jar.  To get those, install the following:

sudo apt-get install javahelp2 icedtea-netx-common

Installing FreeRouting & Running in NetBeans

Downloading FreeRouting

Now you need to download freerouting.  It’s available on github, so you just need to ‘cd’ to a directory where you want to download the project, and then enter the following command:

git clone https://github.com/nikropht/FreeRouting

Loading FreeRouting into NetBeans, Compiling and Running

Now you need to start netbeans (should be under ‘Development’ in the application launcher) and import the FreeRouting project.

  1. With netbeans open, select File -> New Project or use the keyboard shortcut CTRL-SHIFT-N.
  2. In the Categories window, select ‘Java’.  In the Projects window, select ‘Java Project with Existing Sources’.
    Select Next.
  3. Give your project a name.  e.g. FreeRouter, and choose a path.  e.g. $HOME/programming/netbeans_projects/
    Select Next.
  4. In the Source Package Folders area, select Add Folder and browse to the place where you downloaded the FreeRouting sourcecode from git.
    Select Finish.  Netbeans will create the project.
  5. Select File->Project Properties.
    In the categories area, select Libraries.  Then, with the compile tab displayed in the area on the right, select ‘Add JAR/Folder’.  Browse to and choose /usr/share/java/jh.jar.
  6. This next part is not explained in the wikibooks instructions I linked to at the beginning of the post.  But it is absolutely necessary in order for FreeRouting to work.  While still in the Libraries area, Select ‘Add JAR/Folder’ again.  Now browse to /usr/share/icedtea-web/netx.jar.
  7. Now, contrary to the instructions in the wikibooks link, we are NOT going to use Web Start:
    While still in project properties (File->Project Properties), select from the categories area, underneath Application, ‘Web Start’.   Make sure this check box is NOT enabled.  Then Select ‘Run’ from the categories area and make sure the Configuration pull down menu is set to <default config>
    Select OK to exit Project Properties.
  8. Now from the netbeans main menu, choose Run and select “clean and build project”.  If all has gone well, it will conclude with “BUILD SUCCESSFUL”.  Now you can run the application: select the green triangle or press F6 to run the project.  You should see the following:

Freerouter_004

Success!

Now you just need to learn how to use Freerouter! I got it routing a PCB pretty quickly without instructions so I think it is pretty intuitive to run.  However, it may not be so intuitive to choose all the correct options for your PCB.  That’s for another post, maybe.

Recover Data From Unfinalized DVD RW

Introduction

Sometimes you’ll end up having to deal with a DVD which has video recorded on it but the author neglected to finalize the disc. This can be a bit troublesome, because until the disc is finalized it doesn’t contain the necessary data to make it compatible with the DVD standard. Most consumer DVD players and PC based DVD drives rely upon this information to make sense of the data content on the DVD, so if it isn’t finalized most systems won’t open the disc.  Either it will open a blank explorer window, or it won’t mount this disc at all. What to do?

Well, thankfully, it is possible to recover from this situation. There are some commercial applications out there which will do it (get your wallet out) or you can make use of open source software instead. I am going to describe the process for some free open source software on Linux.

DVD+RW-Tools

First of all, you need some free open source software called DVD+RW-Tools.  This is a collection of programs for working with optical video disc media.  Most modern linux distributions come with the tools installed as standard.

Reading the disc

Once you have the tools installed, you can use them to extract useful information about the data content on your unfinished disc.  To do this, open a terminal and type:

dvd+rw-mediainfo /dev/dvd

You will get some output from the command.  Here is a sample of output from an unfinalised DVD+RW which has one track on it:

brian@kubuntu:~$ dvd+rw-mediainfo /dev/dvd
INQUIRY:                [HL-DT-ST][BD-RE  BH16NS40 ][1.03]
GET [CURRENT] CONFIGURATION:
Mounted Media:         13h, DVD-RW Restricted Overwrite
Media ID:              80SONY-W0
Current Write Speed:   2.0x1385=2770KB/s
Write Speed #0:        2.0x1385=2770KB/s
Speed Descriptor#0:    00/716191 R@6.0x1385=8310KB/s W@2.0x1385=2770KB/s
READ DVD STRUCTURE[#10h]:
Media Book Type:       00h, DVD-ROM book [revision 0]
Legacy lead-out at:    716800*2KB=1468006400
READ DVD STRUCTURE[#0h]:
Media Book Type:       32h, DVD-RW book [revision 2]
Last border-out at:    2045*2KB=4188160
READ DISC INFORMATION:
Disc status:           appendable
Number of Sessions:    1
State of Last Session: incomplete
"Next" Track:          1
Number of Tracks:      1
READ FORMAT CAPACITIES:
no media:              716192*2048=1466761216
00h(800):              716192*2048=1466761216
10h(10):               716192*2048=1466761216
15h(10):               716192*2048=1466761216
READ TRACK INFORMATION[#1]:
Track State:           incomplete incremental
Track Start Address:   0*2KB
Next Writable Address: 132208*2KB
Free Blocks:           583984*2KB
Fixed Packet Size:     16*2KB
Track Size:            716192*2KB
READ CAPACITY:          0*2048=0

Note that it is possible to have many video clips contained within a single track or you can have video clips spread across multiple tracks; it depends on how the video information was written by the original equipment.  You may also see multiple disc ‘sessions’.  In the sample case above, we have a single track within a single session and I happen to know that there are multiple video clips contained in it.

I will show you how to extract the video for the above single track, but the method is exactly repeatable for any number of tracks and any number of sessions.

The dd Command

‘dd’ is a command for copying data from some media.  To use it, you give it a block size, a start address, an end address, an input file and an output file.  To see how this applies to the above information, consider the following command:

dd bs=2048 skip=0 count=716192 if=/dev/dvd of=/$HOME/track1.vob

This will extract the single track from the disc and place it in your home.  To explain, let’s go through the parameters one at a time:

bs

This is block size.  To get the correct block size, take a look at the information from DVD+RW-mediainfo.  You will see that the data information (e.g. Start Address) is terminated with an asterisk and a number:

Track Start Address:   0*2KB

In this case we are given 2KB.  Since there are 1024 bytes in a kilobyte, the number to give for bs is 2048.

skip

This is the number of bytes to skip in order to reach the start address of the track.  In the above example the start address is 0, so there are 0 bytes to skip.  But in examples with multiple tracks and/or sessions, you will have non-zero start addresses.  Just enter the start address for the track you are interested in as the parameter for skip.

count

This is the number of bytes to extract, from the start address.  To obtain this information, simply use the track size data:

Track Size:            716192*2KB

if

This is the input file.  You need to supply the path to your DVD device.  In most cases it will be /dev/dvd

of

This is the location where the output file will be written.  $HOME works fine.

Finished – Play It!

Once you have extracted your track, you can play it in any standard media player.  You could also write it to a new disc, remembering to finalise it this time!!!

Good luck.

rarcrack review

rarcrack

I stumbled across a little program called rarcrack yesterday, which provides brute-force cracking of password protected archives.  I decided to give it a try and make some general observations about it.

An example

To set up an example, I created a .rar with a 5 character password.  The password consists of upper-case letters, lower-case letters, and numerals.  rarcrack does not know ahead of time how many characters the password is, so it has to try all possible combinations.  I set the cracker going, and left it for a while. The next day, over 24-hours later, it was still going and had not cracked the password.  Here’s a sample of the program output this Morning:

Probing: '6tgvK' [1296 pwds/sec]
Probing: '6thwV' [1305 pwds/sec]
Probing: '6tiyi' [1309 pwds/sec]
Probing: '6tjzn' [1303 pwds/sec]
Probing: '6tkAt' [1304 pwds/sec]
Probing: '6tlBu' [1302 pwds/sec]

 So it seems that it has started on 5 character passwords, having completed 4 character combinations.  But it is still working through the 5 character combinations and I can see that it hasn’t got to mine yet because it does numerals first and my password started with a letter.

Some Calculations

Okay, so given the above information, how long can we expect rarcrack to take to crack a 5 character password? The speed is ~1300 attempts per second.  That seems really low to me, but I assume it has something to do with the overhead associated with accessing the .rar file.  Otherwise I would expect a PC as powerful as mine to make a lot more attempts per second than that.

Let’s do some calculations.

How many characters?

We have 26 lower-case letters, + 26 upper-case letters, + 10 numerals =

62 possible characters.

How many combinations?

Easy.  62^5 = 916,132,832.

Best case scenario

The best case scenario is that rarcrack finds the password on its very first attempt at a 5 character word.  That scenario is 62^4 + 1 attempts = 14,776,337 attempts.

14,776,337 ÷ 1300 ÷ 3600 = 3.2 hours.

Worst case scenario

The worst case scenario is that rarcrack finds the password on its very last attempt at a 5 character word.  That scenario is 62^5 = 916,132,832 attempts.

916,132,832 ÷ 1300 ÷ 3600 = 195.7 hours, or 8.1 days!!!

It gets a lot worse very quickly from here!

Okay, so what happens if we add more characters? The situation gets a lot worse very quickly.  Here’s some worst case scenarios for some more characters:

[table id=3 /]

You can see that even an 8 character password takes over 5,000 years to crack.  Wowsers.

A little harsh?

Okay, maybe we’re being hard on rarcrack.  What if we were foolish, and we set a password consisting of only lower-case letters? The worst case for an 8 character password would then be:

26^8 ÷ 1300 ÷ 3600 = 44,621 hours.  Or 5.2 years.

Still a very, very long time.

Conclusion

rarcrack is an interesting program, but at the current speeds I can achieve with it the program is useless except for passwords consisting of 5 or less characters.

 

Adding USB devices in a Virtual Machine

Adding USB Devices in a Windows Virtual Machine with VirtualBox

One of the most confusing things for me when I first experimented with VirtualBox was the question of how to add USB devices.  I run Linux (Ubuntu) as my main OS now, but sometimes I need to dig around in Windows (e.g. work related stuff) so for those cases I run it in a virtual machine.  I recently needed to add a data acquisition device for use in the virtual machine on Windows, so I thought I’d document the procedure here.

Pre-Requisites

You need to have VirtualBox installed on your machine, and a Windows virtual machine set up.  I am going to run Windows XP in a virtual machine on Ubuntu in this tutorial.
You also need Virtual Guest Additions installed in the virtual machine to enable the USB support, amongst other things.  If you’re confused about this aspect, there is plenty of information available if you ‘google’ it.

Sorting out the Group Permissions

The only reason that USB support is confusing at all is that you need to add your user to the VirtualBox group before any USB devices will show up for use.  VirtualBox doesn’t offer any other useful information; it just complains that there are no available devices.  The problem is that VirtualBox is not allowed to share your USB devices, so the next steps will fix this.

First you need gnome-system-tools.  Run the following command in a terminal to install it:

sudo apt-get install gnome-system-tools

Now go to the dashboard and type ‘user’.  You should see an icon for Users & Groups appear.  Select it, and the following window will appear:

Users Settings_002Select ‘Manage Groups’.  The following window will appear:

GroupsSettingsScroll down to ‘vboxusers’ and then select ‘Properties’.  The following window will appear:

Group'vboxusers'PropertiesMake sure your user is ‘ticked’.  It probably won’t be by default.  Then select ‘OK’.

WinXP-Dev-SettingsNow USB should work.  But you have to log out and back in again first.  You should do that now.  When you log back in, open Virtual Box and select the settings for your virtual machine.  Select the USB settings, and make sure USB is enabled.  Now you just need to add the USB device you want to use in your machine.  Make sure it’s connected to your computer, and then select the ‘add’ button.  I’m going to add my National Instruments DAQ device:

selectniNow, when you start your virtual machine, Windows should detect new hardware and install it for you:

installingnidaqFinally, when it’s done, you can test out your hardware.  I am using a DAQ device so I started the software for it and captured an analog signal 🙂

Capture

DONE

That’s it, you’re done.  Any time you need to add more USB devices, you can add them to your virtual machine in the same way.

Good Luck.

 

Hosting a Calibre Library with Linux, COPS & Google Drive

Introduction

Free your Ebook collection!!!

Are you tired of being locked in to commercial Ebook software such as Apple iTunes (shudder), Google Books or Amazon Kindle? Would you like to take complete control of your Ebook collection and host it yourself, giving you the ability to serve whatever Ebooks you want at any time and at no extra charge? You’re not alone.

This is a short tutorial to show you how to host a Calibre library on a Linux server with the help of Google Drive and a nifty php program called COPS.
The general idea is that you will create a Calibre library on your local computer (or perhaps you already have an existing Calibre library) and then synchronise it using Google Drive (or you can use a cloud storage provider of your choice, the concept is the same).  In this way any changes you make to your Calibre library, now or in the future, will be automatically synchronised with cloud storage.  At your Linux server end, you will install a command line google drive client which will allow you to pull the synchronised Calibre library down to the server and keep up to date with any changes made on your local machine.  This synchronisation, using Google Drive, will help you to keep an up-to-date Calibre Library that has all your latest books, book covers and meta data on your Linux server.  The synchronised calibre library will be ‘served’ (made available on the web) using a program called COPS, which is able to serve a calibre library using a web based interface and also standard (ODPS) Ebook library interfaces.  This means you will be able to read your ebooks on any device with an internet browser or standard Ebook reader application.

library
COPS web based interface

 

You will be able to access your entire Ebook collection, anywhere and at any time.  Nobody will tell you which books you can host and which you cannot.  You won’t be locked in to any Ebook provider.  Your Ebook library will be free.

Careful about the ‘free’

Okay, so we will need to take a small precaution here.  Your Ebook collection is okay to be free to you, both in terms of liberty and cost, but you don’t want every man and his dog getting access to your Ebook collection.  This will be easily solved by creating a .htaccess file which you will place on your library’s web space.  This will allow you to create a simple username and password which will be required to access your library.  It just takes a few minutes and only needs to be set up once.  I will show you how.

A demonstration of what we’re trying to achieve

I often feel that it helps to be able to see and use an end product so that I can see where I’m going with something before I delve too far into it.  The user interface for our personal online Ebook collection will be provided by a little PHP program called COPS, and the author of this program provides a demonstration of this at his website, here.

A Couple of PreRequisites

In order to be able to synchronise files and host your own content you will of course need access to a few things first.  The pre-requisites for this tutorial are:

  • Some web space, hosted on a Linux platform, which you either have direct access to (your server is at located at home, for example) or you have remote SSH (root) access to – for installation of your own programs and services.  Simple web space hosted by someone else is not good enough because those services won’t let you have control of the server itself.
  • A Google Drive account.  You can just register a free one if you want; competition in cloud storage is rife these days, so even free accounts have enough storage to reliably synchronise thousands of Ebooks.

The Tutorial

Step 1 – Installing Google Drive on your local machine.

If you’re a Windows or Mac user this is easy.  Just visit https://tools.google.com/dlpage/drive and download the client for your computer.  Follow the installation instructions and choose a location on your local machine which you want to sync.  Create a folder in your local Google Drive called ‘Ebooks’.  You will put a Calibre library here later, and use Google Drive to synchronise it.  If you’re a Windows or Mac user, this step is now complete and you can skip forwards to step 2.  If you’re a Linux desktop user, read on…

Drive not Ready!
Drive not Ready!

If you’re a Linux user it’s not quite so easy.  Google have promised us an official Linux client for years, but so far it hasn’t been delivered and there is no indication as to when it will ever be delivered.  One google employee has even labelled the entire concept of automatic background synchronisation as fundamentally flawed, and I think she has a point.  She is the initial creator of a Google Drive command line client for Linux (called ‘drive’), which provides manual synchronisation capability.  This is the program you will install on the server later to pull down your Calibre library from Google Drive.  But you can also install it on your Linux desktop machines to push local changes to your Calibre library up to Google Drive.

To install the semi-official drive command line client for a Linux local machine, follow the same steps as in (3) and (4) for your server; the process is identical.

Step 2 – Installation & Setup of Calibre on your local machine

Calibre
Calibre

Calibre is a free, open-source, Ebook management tool.  Even if you do nothing else in this tutorial, this step alone will empower & transform your local Ebook collection.  You can manage the book covers, change the meta-data, sort your books using tagging or virtual libraries, etc.  If you just want to manage your Ebooks locally then Calibre is a fantastic tool.  It even has its own hosting tool to allow you to host a Calibre library online.  The tool we are going to use instead (COPS) is designed with light-weight servers in mind (suitable for a VPS), and that’s why I am not going to be covering the built in Calibre hosting here.

To install Calibre, go here and download the client that is appropriate for your operating system.  If you use Linux I still recommend that you fetch the program from the Calibre website as opposed to using apt-get.  In my experience the version in the standard repository for Ubuntu is way out of date.

I won’t cover set up of a Calibre library in any detail here.  That is a separate topic.  But you will find that it is pretty intuitive to set up a library, and there is extensive help documentation available.  During the set up of your library you will be asked to set a location for it:

Make certain that you set up your library inside the Ebooks folder you created on your local Google Drive.  This is how you will synchronise it to Google and to your server.  I called my Calibre library ‘BJH’.  Call yours whatever you like.

Once the library is set up you can add books by simply dragging them into the application.  There is a tool which can automatically fetch your book cover and meta data from the internet too – this is extremely useful.  I have one hint here; it is often better to ‘prepare’ your books for automatic meta-data collection by manually adding the ISBN into the meta-data editor yourself first.  This helps Calibre to find your book online in the first place, and it also helps it to get the cover for the correct edition.

Any changes you make to your Calibre library should be automatically synchronised with Google Drive.  Or, if you’re on Linux, you should manually run the “drive push” command described later.  You don’t have to waste all your day perfecting your Ebook collection right now if you don’t want to; just add some books in there to get started so that you can continue with the server setup.  You can perfect it as much as you like another time!

Step 3 – Installing GO on your server

If you want to run Google Drive on a Linux server (or on a local Linux machine), then you will be disappointed to learn that there is no official client for it.  There are a few options, both free and commercial, but I am only going to discuss Google’s semi-official command line client here, which is free.  I say “semi-official” because it was developed by a Google employee who has worked on a Linux client.  The command-line client is written in a programming language developed by Google called ‘GO’.  You will need to install a GO programming environment on your system so that you can fetch the source code for the drive client and build it on your system.  This is nowhere near as difficult as it sounds, and I’ll take you through all the steps:

Install GIT

You need GIT on your machine first, so that you can clone GIT repositories.  Open a command line (if installing on your server, you need to open an SSH connection to it) and type:

sudo apt-get install git

Follow the instructions.  Once done, you can check all is well by typing:

git --version

You should get back a version number for git.

Install Mecurial

sudo apt-get install git mercurial

Download GO & Extract

Visit the golang main page and access the downloads area. Select a stable version to download:

Downloading GO
Downloading GO

 

Upload the archive to your server, or you can use wget on your server to download it straight to your remote machine.  Open a terminal and ‘cd’ to the directory where you downloaded/uploaded the archive. e.g:

cd $HOME/Downloads

Then extract the go archive to /usr/local making certain to substitute “go1.4.linux-amd64” in the example below for the name of the archive you downloaded:

sudo tar -C /usr/local -xzf go1.4.linux-amd64.tar.gz

 Create a GO Workspace

Now you need to create a GO workspace.  This is a place where you will download 3rd party packages (the semi-official Google Drive program, in this example) and where you would create your own GO programs if you decided to experiment with GO yourself.  You can create the workspace anywhere you like.  In my example, I will create a workspace in $HOME/Programming/GO/ as follows:

mkdir -p $HOME/Programming/GO/src/github.com/bh4017

The reason for the [src] directory is that a GO workspace requires somewhere to put sourcecode.  The reason for the [github.com] directory underneath [src] is that you will often download the sourcecode for GO programs from github, and finally the reason for the [bh4017] directory is that I wanted a place to store my own GO programs.  bh4017 is my github.com id.  You should substitute ‘bh4017’ for your own github.com account name, or leave it out completely if you don’t intend to experiment with your own GO programs.

Inform the GO tool about your workspace

You need to tell GO where your workspace(s) are.  You can do it inside the terminal using an ‘export’ command, but the problem is that GO will forget about your workspace as soon as your terminal is closed.  To make GO ‘remember’ your workspace, you can remind it each time you log in by placing the ‘export’ commands inside your ‘.profile’ file.  This file is executed at each log-in and so GO will be automatically reminded of your workspace(s) every time you start Linux.  Open .profile in the editor of your choice.  In the following example I will use nano:

nano $HOME/.profile

Once you have opened your .profile, scroll to the bottom and add the following lines.  In the second and third ‘export’ lines, substitute my GO workspace path for the path you chose when you created your workspace.

# BEGIN GO
export PATH=$PATH:/usr/local/go/bin
export PATH=$HOME/Programming/GO/bin:$PATH
export GOPATH=$HOME/Programming/GO
# END GO

In order to make these changes go ‘live’ you need to log off and back on again so that .profile is executed.  You should do that now.

A good way to test your GO installation is to write the ubiquitous ‘hello world’ program.  There’s a tutorial available for this on the GO website.  If you don’t want to bother with that for now, then as a minimum you should open a terminal and type ‘go version’.  If successful, GO should return a version number to you.

Step 4 – Installing ‘drive’

Assuming you’re happy with your installation of GO, the following command should fetch the latest ‘drive’ source code and build it on your system:

go get -u github.com/odeke-em/drive/cmd/drive

No news is good news, so if your GO installation is working properly the terminal will return after a short while without any messages.  GO should have fetched the source code from github.com and built it for you.  To check that it did something, cd to $HOME/Programming/GO/bin (substituting for your own workspace path as created earlier).  Type ‘ls’ and if all is well you should see drive listed in there.

If your ‘export’ commands have been entered correctly you should be able to open a terminal at any time and simple type ‘drive’ to execute a google drive command.  Entering ‘drive’ on its own will bring up a usage response:

drive usage response
drive usage response

Step 5 – Synchronising your Ebook collection to webspace

First you need some webspace.  Instructions for this are outside the scope of this tutorial; presumably if you’re managing a linux server then you already know how to set up some web space.  For my own Ebook collection I used a subdomain called ‘library’, so my Ebook collection is now hosted at http://library.brianhoskins.uk.

Once you’ve got some web space, you need to find out where the space is located on your server.  For example, it could be located at /var/www/vhosts/yourdomain/sub-domain/ but the actual location will vary depending on your setup.  Once you know where your web space is located on your server, ‘cd’ to the location in a terminal and create a Google Drive folder using the ‘mkdir’ command.  You can name it whatever you want – I literally named mine “GoogleDrive”.

Once you have created your folder, you need to initialise it.  In your terminal, type the following:

drive init your-folder-name

If all goes well your terminal should respond with a url to obtain an authentication code.  This is to allow the drive client access to your Google Drive.  You need to use a browser to visit the provided link, and then you will be given an authentication code.  Paste this authentication code back into the terminal and press ‘enter’.

Once your folder has been initialised as a folder to manage using ‘drive’, ‘cd’ into it and type:

drive pull your-folder-name/Ebooks/your-library-name

drive should spend a few minutes resolving (comparing cloud to local) and then it should respond with a confirmation showing what it intends to download fresh from the cloud and, if this is not the first time you’ve pulled from Google (you are updating your Ebook collection) the files already downloaded which it intends to modify.  You are asked to accept, and then drive will pull down your Calibre library from the cloud to your server.  You now have a fully synchronised Calibre library on your Linux server!  In future, when you make changes to your library using Calibre on your local machine, you will simply allow Google Drive to synchronise them and then SSH into your server to run the ‘drive pull Ebooks’ command on your server again.  In this way, your server’s calibre library will be kept continually up to date with the books you add, delete, and maintain (covers, tags, meta data and all) on your local machine.

Note that if you are also using the linux drive client to synchronise a local calibre library up to the cloud, then you should use the ‘drive push’ command to synchronise local changes made to your Calibre library.

Step 6 – Installing & Configuring COPS on your server

Now that you have a way of maintaining a Calibre library on your local machine(s), synchronising them with Google Drive, and then pulling down synchronised changes to your server, you need a way of ‘serving’ your calibre library online.  This is what COPS does for us.  Fortunately, it’s extremely easy to install and set up!

Pre-Requisites

First of all, you need PHP 5.3/5.4 on your server and SQLite3.  My server is Ubuntu 12.04 and it wasn’t necessary for me to install or change anything on my system.  If yours is different, you should check which versions of these applications your server is running and update if necessary.

Downloading COPS

You can get COPS from github or you can get a zip archive from the author’s website.  I won’t link to the actual zip download because it may change depending on future updates to the software.  Visit the website I linked to, scroll to the very bottom, and under the heading “Download” you will find the latest zip file.

Installing COPS

Once you’ve downloaded the zip, installation is very straightforward.  Simply extract the contents of the zip to the root of your chosen webspace.  Once this is done, you should be able to browse your website and see a COPS interface.  It’ll report a bunch of errors because it can’t find your Calibre library – we’ll set that up in the next step.

Configuring COPS

First of all, enter the following commands:

mv config_local.php.example config_local.php
nano config_local.php

This should copy a sample setup into the COPS config file, and then open it for editing.

Find the line $config[‘calibre_directory’] =

Here you must enter the location of your Calibre directory on your server.  It is the directory underneath which the metadata.db file is to be found.  As an example, if you called your library ‘myBooks’, and placed it inside a folder structure called GoogleDrive/Ebooks/, then the config line would be:

$config['calibre_directory'] = './GoogleDrive/Ebooks/myBooks/';

Make CERTAIN that you include the final ‘/’, or COPS will not be able to find your Calibre metadata.db file and it won’t be able to serve your books!

Next, find the line $config[‘cops_title_default’]

This line simply sets the title bar.  Call it whatever you want – I changed mine to say library.brianhoskins.uk.

Once you’re happy, save the file and exit.

DONE!

That’s it, you should be done! Browse to your library webspace and COPS should be serving your Ebooks.  You can get to them via any modern browser.  You can also use an Ebook reader to view the books.  There are some dedicated Ebook readers for mobile platforms too – I use Moon+ reader pro on Android.  You can set up access to your library in these applications.

Setting Password Access

.htaccess

Okay, so now your library is working but every body on the planet can access it.  We’ll solve this problem in the next steps.

Open up a terminal and ‘cd’ to the root of your library website, where you installed COPS.  In there you will find a .htaccess file.  Open it up in a text editor.  Scroll to the bottom and find the following text:

#<FilesMatch "\.php$">
#AuthUserFile /path/to/file
#AuthGroupFile /dev/null
#AuthName "Acces securise"
#AuthType Basic
#Require valid-user
#</FilesMatch>

These are the entries which control password access.  At the moment they’re commented out, so password access is disabled.  Uncomment them by removing the ‘#‘ characters.

Now you need to point the .htaccess file to a password file called .htpasswd.  You haven’t created this file yet, but we’ll create it in a minute.  You need to place it somewhere where access is generally restricted (usually the root of webspace is).  Here’s a sample setup for you – tailor it you your specific requirements:

<FilesMatch "(index|feed)\.php">
AuthUserFile /var/www/vhosts/mydomain/sub-domain/.htpasswd
AuthGroupFile /dev/null
AuthName "SPEAK FRIEND AND ENTER"
AuthType Basic
Require valid-user
</FilesMatch>

.htpasswd

Now you need to actually create your .htpasswd file at the location where you told .htaccess it would be.  Create it and open it up in a text editor.

All you need to do now is add a user and password.  The password, though, should be encrypted.  The easiest way to go about this is to use an online tool, like this one.  Enter in your desired username and password.  The tool will then create the text you need to enter into your .htpasswd file.

FINISHED

Hopefully you’re done! If you now browse to your online library you should be prompted to enter a username and password.  Enter the plain text user and pass you set up earlier; the decryption is taken care of for you.

It’s a lot of fun managing your own online ebook repository.  Any questions, leave them in the comments and I’ll try my best to help!

 

Installing daily build of KiCAD [Ubuntu]

I noticed, recently, that KiCAD footprint libraries now carry the .pretty extension, and that these files cannot be read by the latest stable release of KiCAD.  In order to make use of these libraries it is necessary to install a recent build of KiCAD, which has undergone a significant overhaul of its footprint library support.

As a new Ubuntu user, it was not clear to me how one would go about installing a daily build of KiCAD.  The instructions on the KiCAD website state:

Old stable should be in the official Ubuntu repo. Daily builds are available in js-reynaud’s PPA.

But how can you go about adding this PPA? And, once the PPA has been added, how do you then use it to install a new-build of KiCAD?

Here are the steps you need to take.

Adding the PPA

First you need to add the PPA.  You are going to run a command that will add a new PPA for KiCAD to your linux sources list, so that apt-get will use it to install future versions of KiCAD.  To add the PPA, open a terminal and type:

sudo add-apt-repository ppa:js-reynaud/ppa-kicad

Follow the on-screen instructions, and your result should look something like:

Add reynaud PPA
Add reynaud PPA

 

Install daily build

Once the PPA has been added, you first need to update apt so that it knows about the new versions of KiCAD in your PPA.  To do that, type:

sudo apt-get update

You will see a lot of output in the console.  Don’t worry; the apt service is just busy hitting all of your sources and updating itself so that it knows about all the latest software.  When it gets to the PPA you just added, it’ll update its knowledge of available KiCAD versions.

Once apt has updated, installing the daily build is then as simple as typing the following into your terminal:

sudo apt-get install kicad

Follow the instructions, agreeing where necessary, and then you should be up and running with a new version of KiCAD