Gaming Google Adwords the Experian Way

There has been a lot of talk in the SEO community recently about the hypocrisy of Google’s aparent 2-tier model when it comes to link building – if you’re a brand then its good PR, if you’re a small business then its spam. However its great to see a big brand that is truly rolling with Google’s developing attitude. If you were a small site would you get away with this?

These ads look alarmingly similar:

And they go to strangely similar looking URLs:

But of course the pages are completely different brands. That’s ok then:

Great user experience!

How will Twitter change search and the mainstream media?

Since the tragic passing of Michael Jackson, the internet has been rife with stories proclaiming how social media broke the news first, and how traditional news outlets and search engines lagged behind. This social media back-patting has resulted in plenty of insults being thrown at everyone who isn’t Twitter.

I’m a Twitter user and a Twitter fan, and lets think for a minute about what it has added to our online lives:

• Its an alerts service – when news breaks, we often see it on Twitter before we see it in RSS feeds, on news sites, on other social sites or on search engines.
• Its a guage of community feeling – we don’t just get facts from Twitter, we get emotions from a large number of people, showing the general spirit of feeling around an event.
• Its about personal experiences – we get to hear the personal stories that we wouldn’t hear from the news sites.

Those three points are impressive, and something Twitter and its community can be extremely proud of. But do these make Twitter a competitor to existing services like search and traditional media, or are these benefits merely complimentary? I believe its somewhere in the middle.

The Mainstream Media

Lets look first at the mainstream media. What do they have to fear from Twitter?

• Will people stop reading news? Absolutely not. I’m not sure I know anyone who, on seeing the Michael Jackson story break on Twitter, didn’t go to their favourite news site(s) and try to verify the information. When they found it was yet unconfirmed, I think most of us still held on to a glimmer of hope that maybe it wasn’t true – it was only when the mainstream news confirmed Michael Jackson’s death that we all finally believed it. And many of us spent plenty of time today browsing obituaries or photo archives on the mainstream media sites. Twitter is an unverified source of information, and although the mainstream media makes plenty of mistakes, its still considered a more reputable source of information than @someguysomewhereelse.
• Will stop people reading news commentary? Possibly, a little. I think that as we get more information on the “general zeitgeist” from social services like Twitter we’ll have less and less need to rely on some columnists opinion to reaffirm and develop our own ideas.
• Will and should mainstream media change? Hopefully mainstream media will become less about entertainment and commentary, and much more about verifiable facts and information, which is what it was for originally. Now that we have the tools, lets leave the gossiping to the people, and turn our news organisations into something we can truly trust. In some ways, to achieve this goal I’d rather our news organisations were less worried about being first to break the news, but more about being the most accurate and trustworthy.

The Search Engines

So what about search? What do search engines have to fear from the mighty Twitter?

• Will people stop searching on search engines? Absolutely not. Unless Twitter uses its database of links to build a Google-like search engine powered by a “social pagerank”, Twitter in its current form will not have any impact on the core search terms in the search engines.
• Will people stop searching for breaking news on search engines? The key question here is, have they ever? I don’t think that people automatically go to Google to find out the latest news. So I don’t think Google will lose share there.
• Will the search engines have to change? Yes, but I think that Twitter is an opportunity for search engines, rather than a threat. Twitter provides a highly indexable, highly relevant and fairly easy-to-interpret flow of information. I can imagine a day in the near future where searches on Google for Twitter’s trending topics appear with a link at the top saying “45,534 social mentions in the last hour – view all.”. Clicking on the link will take users through to a nice search results page amalgamating social results from a whole load of social sources such as Twitter, with Adwords ads down the side. The result? Google just monetised Twitter AND people start searching Google for breaking news: win win for Google.

In Conclusion

So yes, I think that Twitter will change the way search engines and (especially) mainstream media do business, but I don’t think it will do the damage that some are expecting. We still need both of them, and they both have a role in the world, they just need to adapt to our new expectations. Search engines will do this easily and in the very near future. Mainstream media will kick and scream like the record industry has been for the past few years until they realise that its not evil people doing this to them but their own customers. Then they’ll either adapt and survive or they’ll stagnate and die, but somewhere from that chaos an “iTunes” of the media world will rise that will revolutionise their model and make it something more suitable for the 21st Century Web 2.0 world.

Nice and easy file encryption in Linux using openssl

Providing you’ve got openssl installed then command line file encryption and decryption is a breeze on Linux. And since almost all web servers have openssl installed this is a more robust solution than relying on bcrypt or gpg.

To encrypt a file:

openssl enc -aes-256-cbc -e -in /file/to/encrypt -out /file/to/save

To decrypt a file:

openssl enc -aes-256-cbc -d -in ~/file/to/decrypt -out ~/file/to/save

It’s worth noting that openssl does not delete the original file when encrypting or decrypting, so you’ll need to make sure that you do that afterwards. It’s also worth mentioning that you can use almost any type of encryption you could dream of. Simply replace -aes-256-cbc with one of the values below:

 base64             Base 64

 bf-cbc             Blowfish in CBC mode
 bf                 Alias for bf-cbc
 bf-cfb             Blowfish in CFB mode
 bf-ecb             Blowfish in ECB mode
 bf-ofb             Blowfish in OFB mode

 cast-cbc           CAST in CBC mode
 cast               Alias for cast-cbc
 cast5-cbc          CAST5 in CBC mode
 cast5-cfb          CAST5 in CFB mode
 cast5-ecb          CAST5 in ECB mode
 cast5-ofb          CAST5 in OFB mode

 des-cbc            DES in CBC mode
 des                Alias for des-cbc
 des-cfb            DES in CBC mode
 des-ofb            DES in OFB mode
 des-ecb            DES in ECB mode

 des-ede-cbc        Two key triple DES EDE in CBC mode
 des-ede            Two key triple DES EDE in ECB mode
 des-ede-cfb        Two key triple DES EDE in CFB mode
 des-ede-ofb        Two key triple DES EDE in OFB mode

 des-ede3-cbc       Three key triple DES EDE in CBC mode
 des-ede3           Three key triple DES EDE in ECB mode
 des3               Alias for des-ede3-cbc
 des-ede3-cfb       Three key triple DES EDE CFB mode
 des-ede3-ofb       Three key triple DES EDE in OFB mode

 desx               DESX algorithm.

 gost89             GOST 28147-89 in CFB mode (provided by ccgost engine)
 gost89-cnt        `GOST 28147-89 in CNT mode (provided by ccgost engine)

 idea-cbc           IDEA algorithm in CBC mode
 idea               same as idea-cbc
 idea-cfb           IDEA in CFB mode
 idea-ecb           IDEA in ECB mode
 idea-ofb           IDEA in OFB mode

 rc2-cbc            128 bit RC2 in CBC mode
 rc2                Alias for rc2-cbc
 rc2-cfb            128 bit RC2 in CFB mode
 rc2-ecb            128 bit RC2 in ECB mode
 rc2-ofb            128 bit RC2 in OFB mode
 rc2-64-cbc         64 bit RC2 in CBC mode
 rc2-40-cbc         40 bit RC2 in CBC mode

 rc4                128 bit RC4
 rc4-64             64 bit RC4
 rc4-40             40 bit RC4

 rc5-cbc            RC5 cipher in CBC mode
 rc5                Alias for rc5-cbc
 rc5-cfb            RC5 cipher in CFB mode
 rc5-ecb            RC5 cipher in ECB mode
 rc5-ofb            RC5 cipher in OFB mode

 aes-[128|192|256]-cbc  128/192/256 bit AES in CBC mode
 aes-[128|192|256]      Alias for aes-[128|192|256]-cbc
 aes-[128|192|256]-cfb  128/192/256 bit AES in 128 bit CFB mode
 aes-[128|192|256]-cfb1 128/192/256 bit AES in 1 bit CFB mode
 aes-[128|192|256]-cfb8 128/192/256 bit AES in 8 bit CFB mode
 aes-[128|192|256]-ecb  128/192/256 bit AES in ECB mode
 aes-[128|192|256]-ofb  128/192/256 bit AES in OFB mode

Backing up your website using FTP on Linux

Linux provides a load of great command line tools for working with FTP, but by far my favourite is the lftp client – it is immensely powerful and allows you to work in a consistent way across all FTP operations. It is also very easy to use. Chances are you’ve got lftp installed already, but if not you should be able to install it using your default package manager:

apt-get install lftp

The easiest way to get started with lftp is to use it in interactive mode, typing commands into the command prompt. To connect to the remote FTP server type:

lftp -u ftpusername ftp.ftpserver.com

You can then use the help command to give you a breakdown of all of the FTP commands available to you. You’ll find many of them are very familiar: ls, cd, rm etc. To get help on any command type help commandname.

However lftp becomes most powerful when its scripted. You can create a script like the following to pretty much do whatever you want:

open -u ftpusername,ftpassword ftp.ftpserver.com
... some FTP operations here...
close

Save this file as a text file, and then run it manually or from a bash script using:

lftp -f filename

One of the most useful (and simple) scripts that you can create is to back up the entire contents of your remote FTP account to your local PC:

open -u ftpusername,ftppassword ftp.ftpserver.com
lcd /local/path/to/copy/to
cd /remote/path/to/start/from
mirror -v
close

lcd points lftp to the relevant folder on your PC. cd points to the relevant folder on the remote PC. The mirror command will then download the contents of all changed files to your local PC. Once you’re happy with the way that it works, you can add the --delete parameter to the mirror command to delete any local files that are not found on the remote server.

You can also very easily add exclusions to the task, using the exclude parameter:

mirror -v --exclude somefolder/somesubfolder

In this case, nothing in somefolder/somesubfolder will be synchronised. This is especially useful for ignoring large image directories, or logfile directories.

You can also use the -R parameter to reverse the operation, so if you wanted to work locally and then sync the contents of the local folders back to the remote server, you can use this script:

open -u ftpusername,ftppassword ftp.ftpserver.com
lcd /local/path/to/start/from
cd /remote/path/to/copy/to
mirror -R -v
close

In this case all contents from the lcd folder will be copied to the remote directory specified by cd. Again you can specify exclusions using --exclude or use the --delete parameter to get rid of deleted files.

If you’re concerned about version control (overwriting changed files when synchronising) you can add the --only-newer parameter to mirror to stop it overwriting any files on the destination that are newer than the source.

You could pretty easily get a nice development workflow going by syncing down to your local PC before you start working, and then syncing up to the remote FTP server when you’re happy with the result.

Checking if a file has changed using Bash scripting

Linux Bash scripting is a powerful way to easily automate or simplify tasks. However the solutions are not always intuitive unless you are very familiar with the syntax (which I am not). For one particular script I wanted to launch a file in a text editor and then take an action afterwards only if the user had changed the file.

This solution seems to be working brilliantly:

MD5OLD=`md5sum ~/thefile | cut -d " " -f1`
vim ~/thefile
MD5NEW=`md5sum ~/thefile | cut -d " " -f1`
if [ "$MD5OLD" != "$MD5NEW" ]; then
    echo "The file has changed - do something here"
else
    echo "The file has not changed - don't do anything"
fi

The trick is to get an MD5 hash before launching the text editor, and then get another MD5 afterwards. Comparing the two MD5 hash values will tell you if the file has changed or not.

Solved: Cygwin Speed Issue

While writing my previous post on Getting Started with Cygwin I was slightly perturbed to discover that my brand-new Cygwin installation was running pretty slowly. Even simple commands like ls or cd were taking up to 10 seconds to run.

After doing a bit of digging I came across a very simple solution in the Cygwin mailing list archives – I had a reference in my Windows PATH variable to a network drive that was unavailable.

To check this, right click on My Computer, and go to Properties. Go to Advanced, and then Environment Variables:

In System Variables find PATH:

Click on Edit, and copy and paste the entire string into your favourite text editor (in my case Notepad++, but Notepad will do, or vim):

In this case the offending entry was U:\scripts – the U:\ drive was only available part of the time. Removing this, pasting it all back into the PATH variable and then clicking OK instantly sped up Cygwin.

Getting Started with Cygwin

Cygwin is a version of the Linux bash shell for Windows. It provides a great way to automatically script operations, and if you’re a fan of the command line then its a great way to navigate your system. However the default Cygwin installation is not very friendly, and many features that you’d expect from the Linux command line are not configured by default. The aim of this guide is to describe the actions I take when installing Cygwin from scratch.

Part One – Downloading and Installing Cygwin

Installing Cygwin is very easy – simply visit the Cygwin project page and click on the Install Cygwin Now link. Download the setup.exe file to your computer and run it. The setup is a simple wizard interface, which will take you step by step through the process, including selecting the packages you want included. I usually use the defaults, adding only a couple:

• vim – the UNIX text editor of my choice. emacs and nano are also available (along with many others)
• lftp – a powerfull command line FTP client, used for scripting FTP operations.
• openssh – the Linux SSH client
• openssl – command line SSL, great for doing simple encryption tasks

You can also select to install everything, which will give you a pretty complete Linux system, but there are a lot of packages and they take ages to download. Most of them you’re unlikely to use.

You can come back and run setup.exe again later if you find you want to add or remove packages. It acts as a pretty comprehensive package management system for your Cygwin installation – it can be used to update your system with the latest binaries too.

Part Two – Starting Up

When you first start Cygwin, you might get the error:

Your group is currently "mkpasswd".  This indicates that
the /etc/passwd (and possibly /etc/group) files should be rebuilt.
See the man pages for mkpasswd and mkgroup then, for example, run
mkpasswd -l [-d] > /etc/passwd
mkgroup  -l [-d] > /etc/group
Note that the -d switch is necessary for domain users.

This means that Cygwin is not aware of your current user setup. Run the commands specified, using the -d switch if your computer is a member of a Windows domain. These commands will create /etc/passwd and /etc/group files that tell Cygwin about your user setup. Exit Cygwin and start it up again.

Part Three – Setting Your Home Directory

When you restart Cygwin you’ll probably find that your home directory is now your Windows home directory. This might be what you want, but I always prefer to move it. Moving it has the following advantages:

• Cygwin won’t get in the way of your Windows setup by adding configuration files (starting with .) to your nicely organised Windows home folders.
• You can create symbolic links to My Pictures, My Music, My Videos etc, rather than having to type “My\ Pictures” every time.

To change your home directory, edit the /etc/passwd file in your favourite text editor. If you installed Cygwin in C:\Cygwin then this file will be here:

c:\Cygwin\etc\passwd

The passwd file is a list of users registered within Cygwin, with the following fields:

username:password:uid:gid:user info:home directory:shell

Find your Windows user and change the home directory to your preferred folder. I usually just use /home. You’ll need to create this folder in c:\Cygwin\home too. You should find that when you restart Cygwin your home directory is the one you specified.

Part Four – A Better Command Line

The default Cygwin command line console is pretty naff. I much prefer to use PuttyCyg, which is a patched version of the PuTTY telnet/ssh client.

To install PuttyCyg, download it from the project downloads page, and unzip it to your Cygwin directory. When you run it, simply select Cygterm as your session type and enter “-” as your host name. Clicking Open will open up Cygwin.

There are a few minor tweaks I like to select when running PuttyCyg, which I save into a session called Cygwin:

• Window > Behaviour > System Menu Appears on ALT-Space
• Window > Behaviour > Full Screen on ALT-Enter

You can then create a Windows shortcut to the following command to run that saved session every time:

C:\cygwin\putty.exe -load Cygwin

Part Five – Getting Your Keyboard Working

If you have a bit of a play around with the Cygwin command line you might find that some things don’t work quite as you’d expect them to:

• There’s no way to paste into the PuttyCyg window (without using the mouse)
• Home and End keys might not work properly
• The Delete key might not work as it’s supposed to
• Auto-completion using the Tab key is case sensitive – fine in UNIX but hard to use with Windows

To fix those issues (and a few more) create a file called .inputrc in your home directory. The file should contain the following:

"\e[1~": beginning-of-line
"\e[7~": beginning-of-line
"\e[4~": end-of-line
"\e[8~": end-of-line
"\e[3~": delete-char
"\e[2~": paste-from-clipboard

set completion-ignore-case On
set meta-flag On
set convert-meta Off
set output-meta On

set horizontal-scroll-mode On

After restarting Cygwin you should find that the Home, End and Delete keys work nicely, and you can paste the contents of the Windows clipboard using Insert. Auto-completion will be case insensitive.

Part Six - Getting VIM Working

As per any Linux installation, VIM without a .vimrc file defaults to the default behaviour of vi, which is fairly unfriendly. Also some of the key mapping doesn't work beautifully in Cygwin. The following is my .vimrc file (to be placed in your home directory):

set expandtab
set shiftwidth=3
set tabstop=4

set backspace=indent,eol,start
set scrolloff=1
set whichwrap=b,h,l,<,>,~,[,]

nnoremap  dh

set background=light
set hidden
set showcmd
set ttyfast

set encoding=utf-8

set ignorecase
set smartcase
set incsearch
set hlsearch

syntax on
colorscheme torte

Part Seven – Setting Up Some Folders

Your Windows drives are all stored under a virtual folder called /cygdrive/ in Cygwin. So to access your C:\ drive you would type:

cd /cygdrive/c

This is probably the best way they could have set this up, but its not exactly easy-access when you want to quickly get around your system. To solve this I set up a few symbolic links. You only need to run these commands once:

ln -s /cygdrive/c /c
ln -s /cygdrive/c/Documents\ and\ Settings/Your Windows Username/My\ Documents/ ~/docs

You should now find that you can access your C:\ drive using cd /c and your documents folder by using cd ~/docs. Obviously you can set up as many of these symbolic links as you’d like – I usually set up links to My Pictures (~/pics), My Videos (~/videos) and My Music (~/music).

Part Eight – Setting Up Some Shortcuts

Personally I like to set up some shortcuts in my ~/.profile file so that I can launch applications quickly:

alias cs='cygstart'
alias firefox='cygstart firefox'
alias ff='cygstart firefox'
alias notepad='cygstart notepad++'
alias navicat='cygstart /c/Program\ Files/PremiumSoft/Navicat\ 8.0\ MySQL/navicat.exe'
alias opera='cygstart /c/Program\ Files/Opera/opera.exe'
alias iexplore='cygstart /c/Program\ Files/Internet\ Explorer/iexplore.exe'
alias gimp='cygstart /c/Program\ Files/GIMP-2.0/bin/gimp-2.6.exe'
alias filezilla='cygstart /c/Program\ Files/FileZilla\ FTP\ Client/filezilla.exe'
alias cygwin='cygstart /c/cygwin/cygwin.lnk'

Note that the commands above won’t work unless you’ve implemented my /c symbolic link above – if you haven’t done that then you’ll need to use /cygdrive/c instead. Also note that all of the commands use cygstart – cygstart is a great little command that basically launches whatever you want using Windows. So if you’re in any folder and type:

cygstart .

… then it will open the current folder in Windows Explorer. If you type:

cygstart somedoc.doc

… then somedoc.doc will open in Microsoft Office or Open Office. In my aliases above I shorten cygstart to cs for quick access.

Part Nine – What Next?

After carrying out all of those steps you should have a nice, neat and easy to use Cygwin installation. Next its down to you to leverage the power of the UNIX command line and give Windows a bit of kick.

Convert nonsense ASCII encoded characters into another encoding

Have you ever had a situation where someone from abroad has sent you some text, but because an application doesn’t support proper international encoding it comes out as a random string of nonsense? The little JavaScript applet below will let you convert that nonsense back into the original language it came from, and politely ask Google to try and translate it back to English.

Simply enter the nonsense text into the text box below, select an encoding, and click Convert. If you don’t know which encoding to use, try a few!

Hiding and showing a table row using JavaScript

Hiding a table row in JavaScript is very easy. Give your row an ID, and then call this function:

     document.getElementById("rowID").style.display = "none";

The row will disappear. But showing it again is a little more tricky. If you’re testing in Internet Explorer you’ll probably discover that this works:

     document.getElementById("rowID").style.display = "block";

… but that it doesn’t do what its supposed to in Firefox. Unfortunately this is one of the situations where browsers have gone in slightly different directions to make our jobs more fun. The standards purists blame IE for this, and they’re probably right, but I’m more interested in making the code work!

To get the row to display again in Firefox you need to use “table-row” instead of block. Naturally this won’t work in IE so you need to detect which browser the user is using:

     if (navigator.appName.indexOf('MSIE')==-1 && navigator.appName.indexOf('Internet Explorer')==-1)
         document.getElementById("rowID").style.display = "table-row";
     else
         document.getElementById("rowID").style.display = "block";

You should now get your row back in either browser.

Smarter resizing of images in PHP

Whenever I want to resize images in PHP code I use the SimpleImage.php script by Simon Jarvis. This code wraps PHP’s image functionality into a nice little class that’s very quick to implement.

One of the most helpful features of the script is its ability to resize an image to a set height or width, while maintaining the image aspect ratio:

   function resizeToHeight($height) {
      $ratio = $height / $this->getHeight();
      $width = $this->getWidth() * $ratio;
      $this->resize($width,$height);
   }
   function resizeToWidth($width) {
      $ratio = $width / $this->getWidth();
      $height = $this->getheight() * $ratio;
      $this->resize($width,$height);
    }

But the problem with this sort of resize operation is that users will always find a way to upload the longest and thinnest image they can, or the shortest and widest image they can, and mess up your nice layout.

Instead, adding the following code to the SimpleImage.php class addresses this issue:

   function resizetoMax($size) {
      if ($this->getWidth() > $this->getHeight())
		resizeToWidth($size);
	  else
		resizeToHeight($size);
   }

Calling the function resizetoMax(150); will resize the image so that neither the width nor the height are over 150px. However if it is a long, thin image then the height will take up the entire 150 pixels, and if it is a short, wide image then the width will take up the entire 150 pixels.