Monday, 18 November 2013

Use sed to retreive links from a Firefox bookmark file

Sed is a great text manipulation tool. It allows to manipulate text through simple substitution commands.

It did occur to me that I had a few backup .json files in which Firefox bookmarks were stored. As I did not want to overwrite the existing bookmarks, but I wanted to retreive just a few links, I tried to open the file with a text editor.

The result was not pleasant:

I decided to use some CLI magic to transform this in a list of URLs which is easily readable, instead. It can be done in one line:

$ sed "s/\"uri\"\:\"/\\`echo -e '\n\r'`/g" bookmarks.json | sed "s/\"[^@]*$//" > new_bookmarks.txt

Where:   bookmarks.json -> bookmark file name
            new_bookmarks.txt -> new file name where the links will be saved


Thursday, 30 May 2013

MailMe: how to get notified automatically via email through BASH

In my job, I have to perform calculations on a remote machine to which I connect through ssh. It happens quite often that I have to run a few jobs overnight, if not for days. I could check manually for their termination, but I came up with a better way.

In a previous post I described how useful it can be to run script in background on the machine without having to be logged-in. Combining the power of screen with the utility of mail we can create a script which will automatically send an email whenever the job is finished.

The Script

## This scripts sends an automated email as soon as a job is finished
## ***   Use with screen:
## ***  
## ***   $ screen -S [job-id]               (start screen)
## ***   $ mailme [job-id]                  (run mailme in screen)
## ***   Ctrl+a d                           (detach)
## ***   Ctrl+d                             (kill screen, from session)
## ***   $ screen -ls                       (check running screens)
## ***   $ screen -r [screen-id]            (resumes session)
## ***   screen -X -S [screen-id] kill      (kill screen, from outside)
## ***  

# Need argument: job id which needs to be checked for completion
if [ -z "${1}" ] ; then
    echo "Please insert job ID as argument"
    echo "Exiting status 1"

    exit 1

# Need second argument: email
if [ -z "${2}" ] ; then
    echo "Please insert email as second argument"

    echo "Exiting status 1"
    exit 1

# Put your email here
zzFREQ=3600 #Frequency of checks, in ms

# Loop that checks job status
while "${zzEND}"

    zzSTRING=$(bhist -d | grep "${zzJOBID}")
    if [ ! -z "${zzSTRING}" ] ; then
    echo -e "Dear Sir,\n\nThis is an automatic email from grep Linux indicating that the job ID\# ${zzJOBID} has been terminated (succesfully or not).\n\nThis is the output:\n\n $(bhist -d) \n\nBest, \ngrep Linux | mail -s "Job ${zzJOBID} Completed!" "${zzEMAIL}"
    sleep "${zzFREQ}"

exit 0 

The comments should provide a satisfactory explanation, but it could be used straight away for jobs submitted through the routine bsub.

You would just need to run the script from a screen and the game is on. If you do not remember how to do this, I have put some reminders in the header section of the script, or you can check my guide to screen.

You can run multiple of these scripts in multiple screen sessions if more jobs need to be tracked.


This script has been designed to check job progress and send an email  whenever the job is complete. Through an easy modification of the condition to be checked in the variable zzSTRING in the loop, anything can trigger the delivery of an automated email, potentially making the scope of this script very broad.

Have a play with it and let me know how you use/modify it!


Wednesday, 15 May 2013

Motion: use your webcam as a security camera!

Last month, my landlord told me he had to send an engineer to fix the heating in my room. I could not be at home at the time they would come, but they would have access anyway.

This is one of the many occasions in which you wish you had a hidden security camera at hand (together with when you want to know if you roommate is really stealing your chocolate and when you suspect there are undesired animals in your room!).

But the solution is just in front of you, right now! Your computer's webcam is a hidden and usually unsuspectable object and it can be easily turned into a monitoring device.

And Linux helps us doing it very easily with a small program called motion.

Motion saves snapshots from your webcam whenever movement is detected by the webcam itself. Increasing the capture rate, it can produce decent videos as well.


Motion needs to be configured before it can be used. Configuration allows for a big list of customizations, as frame capture rate, image resolution of the frames, motion sensibility, webcam properties and many others. It is very easy as most of it is already configured.

First of all, copy the config file in your user account. From a terminal:

$ cd
$ mkdir .motion
$ sudo cp /etc/motion/motion.conf ./.motion

Then change the ownership of the file to your user (it was copied from root)

$ sudo chown [username] ./.motion/motion.conf

 And open it with your favourite text editor (which is emacs, of course):

$ emacs ./.motion.conf

In the ideal case, you would not need to modify anything. But it would be safe to skim through the file to see if there are things you want to change. Comments (line starting with #) are very exhaustive and it is difficult to get lost. The most important would be:

videodevice /dev/video0

which selects the webcam device used, or the directory where the snapshots are saved:

target_dir /home/user/your_folder


Funnily enough, using the program is easier than configuring it. Just open a terminal and run the command:

$ motion

You will see a lot of gibberish-looking lines and if you are in front of your webcam, you can test it straight away, as it will be saving pictures already, if you are moving.

To close the program when you are finished just press


and you will not lose anything as the pictures will be saved in the folder you chose during the configuration stage.


As an addition for extra security, you could connect the folder to the internet over cloud sharing services as dropbox or copy, and look for movements in your room real time!