> root / 2010 / November / 8th

#550 in a series of Things That Have Arrived In The Post

Ah, the impossible to start first Week Notes post. I suppose a good start is to not explain what weeknotes(.com) are. Good done.

Next is the numbering system, as covered by Jemina over on the Guardian they get counted from the start of the company/thing. For the Guardian it was first published 9,890 weeks ago, so here we are, my Week Notes 9,890, which cover last week.

The first half of the week was spent tweaking the "Searcher" part of the Zeitgeist which is explained in more detail here. In essence to understand what and how people are reading and arriving at stories on the Guardian there are two systems, one that crunches our own analytics (the brain) and one that looks out to the internet (the searcher), here's a diagram...


The tweaking parts at the start of the week was messing with how I'm checking for incoming links from the Drudge Report, Huffington Post and chat around twitter. Involving various Cron Jobs and Beautiful Soup, a handy Python HTML/XML parser designed for screen-scraping (and other stuff). Topsy doesn't need Beautiful Soup as they have a great API, with this call being a great starting place...


The "Searcher" part of the system (which is running on Google App Engine) looks at these places and records when it spots new links coming in and the activity around them.

The interesting part it to see how a link on Drudge or the Huffington Post can kick off a rush of Twitter posts that then get retweeted and so on, or how someone (or a group of people) will pick something up on twitter that then gains attention and appears on Drudge or Huffington post.

Anyway, for the sake of Week Notes the fun part was almost as an aside throw that information over to a group of WordPress blogs. Emailing and posting content to WordPress is so trivial that it's almost criminal not to do it. So now there are three very quickly set up blogs that track the content as is come in from different sources ...

The styling on the blogs in minimal, and it's a bit odd when there's no image on the article as it puts the advert image as the main image on the front page but whatever, not really worth sorting out atm.

The trick behind this is that when the Searcher spots an incoming link from Drudge of the HuffPo, it grabs the article, category and tags from the Guardian API and posts it to the blog. There's no grand masterplan behind that other than general interest. The fun part about WordPress is the amount of plugins that allow you to view the data in various ways.

Here's a snapshot of the tag clouds for each blog.

Tag Clouds

Giving us a curious look at what type of articles each source tends to link too. Drudge does World news, HuffPo does Books & Culture while Twitter unsurprisingly focuses around Technology.

Obviously there are different/better(?) ways of doing this but for a quick easy way involving very little coding, using WordPress as a database with interesting views is kinda fun.

Also Zeitgeist related during the week was providing certain "feeds" of hot tags and articles via JSON for various projects going on here. Because the Zeitgeist is monitoring our own analytics and what's happening around the internet sometimes looking at what it's throwing up has uses for other projects. Now and then it's quite quick to knock up a handy endpoint for people to poke and look at. Clearly I'm being suitably vague here, that's the nature of Week Notes it seems.

Later in the week I had to compress all the thoughts I had for the Rewired State: NHS Choices/Digital Signage Developer Challenge into an actual document, interesting stuff and a fun way for us Guardian techies to get involved in "Hack" projects, something that's (rightly) encouraged. I'll post more about that as stuff develops.

A whole bunch of time was blown setting up coding environment for working on a not-public project, amazing how much time can be spent getting ssh, rsa, tunnelling, git and so on setup just-so allowing you to just work. Anyway, that all got setup.

As part of that I've been moving around how I work, considering I often Work-From-Home and even then I can be on the "road" working in other people's offices, hotel room and even the train. This week I think I've finally got the balance sorted but there's always more to do.

For quite a while now I've been keeping my code on GitHub (mostly private) which allows me to keep it synced up between my work machine, home machine and laptop. DropBox has been invaluable for moving files around. For the not-public project above I went the extra step and checked out git code into my DropBox folder ... this may be insanity but so far it seems to be working. It means I can keep certain project files in sync between my coding environment (Komodo IDE) on each machine and other setting files that don't get checked into GitHub. I think it also means I only need to bother pushing to github and not pulling (I _think, _although I'm still pushing anyway for the moment) I'll keep an eye on how that goes.

I also tried getting away with not carrying around my laptop and coping when away from my desk with just the iPod touch. You can see the bluetooth keyboard in the photo at the top of this post to see. Again DropBox has been invaluable because a number of text-editing/note taking Apps sync up with DropBox I can now also make quick adjustments to files, yes - including those that get checked into git from the iPod. When I make changes the file then gets synced to my other machines, it's all quite magic.

Oh also EverNote is also on each machine ... which is possibly overkill but it serves a different purpose. Curiously To-Do lists I still keep on paper that I carry around and in notebooks.

The last part of all that setup is LogMeIn that basically a posh version of VNC which then allows me to connect to any of my Work/Home/Laptop machines from any of my other machines ... and the iPod. Which is really handy for quickly moving files into the DropBox folder when I've realised I've left one on the wrong machine or something.

At a pinch I can also connect to a DeskTop machine from the iPod to trigger a push and deploy of the git files.

Anyway, all of that was admin/setup work which I'm glad is out of the way (is it every out of the way?).

Oh and on Friday I got to speak to Oslo again...

lcome to Oslo Airport

...sadly not in person this time but via the medium of Skype. Timezones dictated a stupidly early start to the day! To students at the Arkitektur-og designhøgskolen who were presenting their news related projects. Interesting stuff!

The second half of the week also involved, as it aways does when I'm down in London, a lot of walking around catching up and talking with people about various project and such like.

Well, that's the end of my first week of Week Notes. I suspect ones that follow on from this will be a lot shorter as there's an amount of set-up and introduction to projects here, which I'm not convinced I did terrible well, but it's a start :)

[disclaimer: I work for the Guardian and these notes are about the Guardian but they don't necessarily reflect those of the Guardian. Or whatever we're supposed to write to cover our asses about this stuff.]

## Table of Contents: November

### More

### About

This is what used to be a blog, but is now an online diary/archive sort of thing. As I'm laying off the twitter a bit and currently reclaiming all my Flickr photos this is the main place to find me, (subscribing via RSS) should still work.

I also have a podcast, with more audio experiments on SoundCloud. A smaller "scrapblog" is over on tumblr. If you need to get hold of me email hello@revdancatt.com

### Blogroll

Other blogs I track, some on the offchance they start publishing again.