[WARNING: the sound quality is awful on this video but it is all in one piece, see below for links for better quality split over 2 videos (or if you're reading this in an RSS reader)]

…this trick is split over two much better quality videos, you’re probably better of viewing them on YouTube:

Part 1: The World’s Most Expensive Card Trick, starting at 8:28
Part 2: The World’s Most Expensive Card Trick, second half

I’ll run through the trick in just a second, first a little background on my introduction to magic.

Like many people around my age in the UK, I grew up watching the Paul Daniels Magic Show, we only had three channels at the time so Saturday night entertainment TV options were limited. We ended up watching magic not because we sought it out but rather because there was nothing else on, we didn’t hunt it, it came to us.

Noel Edmonds, Tomorrow’s World, Grange Hill and Paul Daniels; entertainment, wonky technology, school life and magic was part of our growing up curriculum, it’s just something you had to sit through until you graduated to being old enough & trusted enough to go play with your friends in the street.

Thus the UK ended up with this odd magic literate generation, some hated it, some loved it and some of us had a passing interest.

I generally took magic at face value, a trick designed to delight and entertain, being happy to play the roll of the naive observer trusting what the magician said to enjoy the surprise payoff at the end. The magician isn’t really the one doing the tricking it’s the audience, successfully fooling the magician into thinking they’ve pulled off the trick. The better the audience is with this deception the greater the magician. Otherwise everyone would just sit there and go “Trap door! Slight of hand! Twins! Hidden compartment” and no-one would have a very good time. The audience make a deal with the magician(s) to go along with the deception for everyone’s benefit.

Penn & Teller changed that deal.

They were magicians who showed us how (some of) the tricks were done, this was new and interesting to me, it also changes the dynamic. Now we weren’t on two separate ends of a deal but on the same side, still complicit with each other but now just focused on enjoying the performance rather than fooling each other.

Or so I thought until the above trick.

Ok, so about the trick then

For those that don’t care to watch the trick, here’s the run down.

Penn is out in the street, explaining to us the audience in the studio and watching at home, how the trick is going to be done. How we are going to fool a couple of people on the street, using a couple of top-of-the-range tricked out computers and a perfect fanning of the cards.

Penn will get the oh-so-foolish marks to pick a card from a deck. The remaining cards are quickly fanned and shown the the camera, and then the bit I loved.

I loved it, because I wanted to believe in computers, they were magical wonderful machines. I had one (an Acorn Electron) and I could make it do, well, stuff, I typed in incantations of special words and invoked a response. I totally bought into the computer part, I was always telling people that computers would be able to do great things, and now, here, was a demonstration of that. Boy Penn & Teller got computers, I got computers, it’s almost like we could hang out and talk about computers.

In this trick, the computer would take a screen shot of the fanned cards, analysis the image recognising each of the cards in turn and, here’s the great bit, deduce the missing card, the one the “marks” in the street picked.

This, is did perfectly.

Teller then typed the card into the second computer which displayed it on a large advertising screen above and behind the people on the street, so that Penn could see it and therefore tell them what the card was.

I was so down with this, we, all of us, knew how the thing was done, together we had tricked the two people in the street.

Until the very last moment, and then this happened.

Penn had asked them their names, Teller typed it into the display, which was then shown above their heads, just like this…

….and at that moment, my perception of magic changed forever.

The big switch-a-roo

I’m sure most people already know what’s going on, you probably have it figured out already, I mean it’s so obvious really. However remember I was magic tricks naive, I took things a face value, I trusted those guys. We were on the same side, right?

That sign is old, made up of bulbs, not these super modern screens with millions of pixels, but a limited number of pixels/bulbs across and down, it’s a low resolution screen. Or in other words you can only fit so many letters on it.

And I asked myself, what are the chances that out of all the people on the street, Penn ended up with a pair that had short names?

Roger & Laura, too long.
Padmavati & Steve, too long.
Bob & Martin, too long.
Andrew & Felicity, too long.
Mary & Sue, too long.

No, none of that, but Sue and Ian. Two three letter names, what are the chances of that? Well not high enough for me. And then it unravelled, I could feel my mind playing the whole trick backwards, and then even further back to all the TV magic shows I’d seen.

If the two people had to have short names, then they couldn’t have been just anyone. If they weren’t just anyone then there’s no reason to try and fool them, all that (now apparently fake) stuff with the computer wasn’t to trick them so who was it tricking?

Us.

This wasn’t part of the deal, I bought into the whole we were going to trick them part, the inside knowledge, the collusion. I thought Penn & Teller were being straight up with me, letting me in on the trick, after all that’s what they do isn’t it, show you how things are done?

I had been tricked. The magicians had lied, not the lying that we all agreed to do together to make the trick work “the card is lost in the deck”, but out and out, bare faced, lying lying.

And this made me very happy.

I thought back even further, if that had been a lie, then what else?

A trick, of sorts I remembered from the Paul Daniels Magic Show, a master of memory trick, in which he had memorised each audience members’ name, 100s of people. He explained it as a “visualization and word association technique” by creating word pictures, the crazier the better you can remember a great number of things. The example in this case was looking at a man wearing glasses called Ash. You mentally  build up an image of the man with a volcano growing out of his head, dripping lava and billowing ash out that comes to rest along the top of his glasses. Once you have that image, you can easily remember his name. And then you do this for each person in turn. Watching it was impressive.

But as feasible as the explanation for how it was done was, it didn’t mean he wasn’t “cheating”. I’d heard a little about visualising images to help memory before, there are books on the subject and so on, but just because it’s a real thing, doesn’t mean that’s how it was done, but enough for me to buy into at the time.

I then applied this thinking to other tricks I remembered, is the trick itself a trick? At which point I discovered a new joy in magic, a new dimension to explore, an extra layer on-top of the layers that I thought I knew about. The meta-trick, a trick wrapped inside a trick.

A good few years later along came Derren Brown, wonderful fantastic entertainment. Many of the tricks explained to us upfront with perfectly reasonable psychological explanations. But were we being let in on the trick, or being tricked?

I certainly had no better understanding of how the magic tricks were being done, no magic insight to the moves, slights, gaffs or trick construction. But instead a greater appreciation of the art and the setup than I had before.

And it was this one trick by Penn & Teller, the World’s Most Expensive Card Trick, shown on Channel 4. That very evening was the moment that shifted everything about magic for me, made me fall in love with it more, made me think about it a little more, and made me once more enjoy being tricked by charlatans, cheats and liars.

Update: The US version & an interview extract

@hugovk uploaded the US version of this trick to vimeo.

Penn & Teller: The World’s Most Expensive Card Trick from Hugo on Vimeo.

And also pointed me to this interview on WiReD from back in the day: A completely unredeeming profile of Penn Jillette, wired magician, Umaphile, little girl trapped in a fat man’s body. Which includes this extract…

Teller quotes Arthur C. Clarke, who noted, “A sufficiently advanced technology is indistinguishable from magic.”

Eight years ago, Penn & Teller devised a trick that illustrated Clarke’s Law, Teller says. You might recall this number – the World’s Most Expensive Card Trick – done before millions of people on Saturday Night Live.

While the Saturday Night audience watched Teller sit on stage surrounded by computer equipment, Penn was out in Times Square, “by a newsstand,” Teller recalls. “Penn had two passersby select one card from the pack and remove it. He then fanned the cards so the TV camera could get a glimpse of the 51 others. I apparently captured the fan on my computer scan, and took a still frame to determine which card was missing. I turned to a second terminal and typed in Four of Diamonds. The terminal transferred it to the giant Spectacolor screen in Times Square: Penn just looked over the bystanders’ shoulders, and there it was.”

“All that elaborate rigamarole could have just as easily been done by a standard palming of the card,” he adds. In fact, the trick was done like this: The “bystanders” were really two actors. And the answer was always going to be the Four of Diamonds; that was the card that the actors had been instructed to select. But what everyone saw was an incredible stunt done by powerful, mystical computers.

“The subject of that trick was how close to magic being able to manipulate a lot of technology is,” Teller says.

Which still feels true today.

The title says it all – after two-and-a-half years I’m going to be leaving the Guardian.

I was just going to continue to copy & paste Martin’s leaving blog post from the other week, but decided that was just a little bit too cheeky ;) instead…

I shall miss the Guardian terribly, the people, projects and even the building despite it giving me electric shocks when I’m least expecting it. Highlights have included sitting down with various editorial teams, especially the technology peeps and joining in the recording of the Tech Weekly podcast. The wide range of guest speakers that come into “Morning Conference” each week and talk off the record. Getting words I’ve written printed in the actual paper, I know, old school!

Also, the space to push forwards the connectedTV agenda, a whole bunch of number crunching data dashboards and widgets to help us understand what various measurement metrics actually mean. Bits & bobs that integrate with Tumblr, twitterbots tweeting, an “ebookifier” and a “liveblog-o-matic”. A weekly Kindle digest that never saw the light of day and a weekly print version in the form of The Long Good Read that did.

Actually it’s probably easier just to point at this blog’s guardian tag and be done with it.

* * *

Anyway, enough words.

I don’t have a next thing lined up, just the great wilderness of the unknown future with nothing but some vague ideas and my pockets stuffed full of Guardian API keys to pass the time. So I expect there’ll be a few more Guardian hacks and projects coming out, just from the outside this time rather then the inside :)

Farewell the Guardian, it was… well, actually, that’s once I’ve worked through the notice period, and tied up loose ends, fleshed out some documentation, made sure all my projects are in git, ticked off the knowledge sharing, fixed my assigned bugs…

I’ve written up about the GAHR  before, but now that it’s finished (well for the moment anyway) I wanted to put it all down in one place, sorry this is a long one, you may want to skip to the part you’re interested in…

• THE VISION
• SOUND DESIGN
• TEXT TO SPEECH
• GRAPHIC DESIGN
• TECHNOLOGY
• THINGS I LEARNT ALONG THE WAY

If you’re on Chrome or Firefox (the only things I’ve tested it on) you can listen/see the project here: http://revdancatt.github.com/GuardianAmbientHeadlineRadio/. It should play seamless looping sounds and read headlines at you.

The source code is here: https://github.com/revdancatt/GuardianAmbientHeadlineRadio

The Vision

It started just over a year ago, before the last SxSW (2011), the Guardian was going & internally we were asked for suggestions for what we could do there, that demonstrated our internet/data/hacking/open/api skillz. I think we had a room to show our stuff, one reaction I normally have when faced with a space to fill, is to fill it with speakers. The bigger and more fuck-off the better.

A room, with a a couple of laptops running Ableton Live with various pad/soundscapes running through them. Also, have it wired up to the Guardian API picking up the latest articles, videos and podcasts. Then, converting those articles into speech, auto-tune it and then drop it into the mix, changing the levels on the soundscapes to match the “mood” of the article, or the section it was from. So “sporty” background sounds for sport stories, “filmy” background sounds for film stories and so on. While cleverly and automatically chopping up the podcasts to drop soundbites and loops into the mix to.

Meanwhile our reporters would go round interviewing people and bands (from the music bit) and general ambient field recordings; out in the street, in the crowd and so on. Which they could also download to another laptop in the room, which would again feed bits and pieces into the mix.

All this would then be broadcast on the internet 24 hours a day for the full duration of the SxSW festival(s). An evolving news soundscape, that could happily sit in the background but still keep you informed of roughly what was going on if you paid attention to it.

* * *

This browser-based end result is sort of that.

Oh of course, I had no idea how to use Ableton Live, and so began the year of learning.

In-between then and now a couple of things happened

One was a hack-day at work where I threw some of the code running live through Ableton, and the other was leaving it running over a weekend. Here’s a video from the hack-day, where I point at various things…

Guardian Ambient Radio from RevDanCatt on Vimeo.

And the audio on SoundCloud so you can hear how it compares to the javascript version.

SOUND DESIGN

Actually, I expect some influence for the project comes from way, way back, specifically these two tracks…

Vangelis: Albedo 0.39

Vangelis: Intergalactic Radio Station

…for those without embeds, here are the links…

• Vangelis: Albedo 0.39
• Vangelis: Intergalactic Radio Station

…which I kind of fell in love with when I was a teenager. This idea of future radio, soundscape, spoken word over the top and so on.

So I wanted an evolving soundscape that mutated based on the current headline’s section, or several soundscapes that I could blend between. But for this I have just the one made up of three layers, all produced by Native Instruments’ SpaceDrone [WARNING: auto plays sound] in REAKTOR.

SpaceDrone is one of the three factory defaults that ships with the free version of the REAKTOR player (you can grab it here)

The layers are…

A background ambient drone, the “A Planet Called Earth” Snapshot.

The rain/”white noise”, unsurprisingly “PouringRain” Snapshot.

And bird “tweets”, again unsurprisingly “Birds” default.

Just to take the edge of things volume of the rain and birds is controlled by two out of sync stretched sine waves, making them flow in and out of the main ambient sound.

And the code controlling that basically performs a sin calculation based on the number of seconds since the start of 1970 (the birth of time for computers)…

setInterval(function() {
var d = new Date();

var v = Math.sin(d.getTime()/32127)/2+0.5;
$('#birds').get(0).volume=v;

v = Math.sin(d.getTime()/25000)/2+0.5;
$('#rain').get(0).volume=v*0.8;
}, 1000);

The results of a sin give us a value between -1 and 1. Obviously we only want positive numbers, negative volume is no good to us. To shift everything up into positive numbers we dividing by 2 (giving us -0.5 to 0.5) and finally add 0.5 moves everything up.

In the above code I’m knocking back the rain a bit by multiplying the volume by 0.8

And lastly because the main background ambient sound is being recorded directly from SpaceDrone and is evolving with sweeping pads rather than a simple looping sample I had to make it loop (the rain and birds loop just fine without any work).

Here’s the waveform of the background as recorded into Ableton Live. As you can see it’s recorded in stereo and some bits are louder than others. To increase the chances of getting a good loop starting position I recorded until it looked like the volume of each channel roughly matched the starting volume…

To make the loop I could have just stuck a reversed copy onto the end, so it’d play forwards and then backward, which would make the whole thing loop. But I didn’t want to double the length of the sample or the size of the file.

Instead I took it into Audacity…

Converted it from Stereo to Mono…

Duplicated the track and reversed it…

And then finally mixed it down to a single track that would have a seemless loop, which *is* mirrored, but the original length…

Speaking of loops…

OGG v MP3 and seamless loops

Here are the three soundscape components in <audio> tags on the page.

<audio id="drone" width="300" height="32" class="player" src="snd/drone.ogg" loop="loop" autoplay="autoplay"></audio>
<audio id="rain" width="300" height="32" class="player" src="snd/rain.ogg" loop="loop" autoplay="autoplay"></audio>
<audio id="birds" width="300" height="32" class="player" src="snd/birds.ogg" loop="loop" autoplay="autoplay"></audio>

As you can see they are all set to loop & autoplay. I found with MP3s there were gaps in the looping and reading up various bug reports about this suggested three things…

1. because of the way MP3s are decoded its quite tricky for the browser to seamlessly loop them (although I’m not clear why this should be).
2. Write various event checking hacks to try and force a better loop or bounce between two samples, starting one when the other finished.
3. Use OGG instead.

Because I really want to get behind the <audio> & <video> tags and believe that they should (and one day will) just work without any hacks or work arounds I’ve stubbornly used OGG and nothing else to make sure the user gets a good audio experience.

In my experience the looping is better, but gaps start to appear if you have other things going on the computer or in many other tabs. One day all this will magically just work!

The last two sound elements

With all the background sound out of the way there are two more (well three I guess) audio elements.

Because this is supposed to be ambient and just running the the background, your mind isn’t really paying attention to it. Meaning that when a new headlines arrived if it just launched into speaking you miss most of what was actually said. This isn’t helped by the Text-to-Speech being pretty hard to understand anyway.

I tried to fix this by taking a couple of seconds to fade out the ambient background before speaking the headlines. The fading gives your brain a cue that something is about to happen, as well as making it easier to hear the speaking over the top of the background sound.

Turns out it still wasn’t enough to prepare you to hear the headline.

So I added two more things. An alert before the headline to let you know one was about to appear, and the Text-To-Speech to start with the section that the headline was coming from, ie. ‘Sport – some footballer just did something’. Not because I particularly cared for the section, but rather to have one or two words at the start that the brain could throw away while tuning in and it not to matter too much.

But because this is supposed to be ambient, the alert wasn’t to be too alerty, so you could just ignore if (and the headline) if you wanted to. Basically a not very alerting alert.

For no better reason than just to amuse myself I took the three tone xylophone Bing-Dee-Bing noise from the 80s BBC comedy program Hi-De-Hi. This is very much like the Ding-Dong sounds you get before airport and train station announcements, but with more knowing irony.

For anyone my age in the UK who wasn’t allowed to watch ITV because of the adverts the “Bing-Dee-Bing, Hello Campers” (as seen at the start of this YouTube clip right after the credits) represents the age at which our uncomfortable understanding of things that are supposed to be funny being deeply unfunny and yet unable to escape it, begins to develop.

“Bing-Dee-Bing” at 30s

Last but not least, sometimes it can be a long time between headlines. This is the point at which I begin to wonder if something has gone wrong with my code and it has stopped updating or goodness knows what. To put my mind at rest, if after 7 minutes there are no new headlines it plays a little jingle, and gives a “You’re listening to Guardian Ambient Headline Radio” ident.

Instead of the alert, I opted for a “jingle” kind of sound, something newsy and more upbeat than the ambient sound. Hopefully it wouldn’t play too often, because if it did it meant that the Guardian itself was broken :)

The ambience fades, the jingle kicks in, ident, and then back to ambience.

I think that’s all for the sound. Probably need to add a volume control at some point too.

TEXT TO SPEECH

The text to speech is provided by speak.js which is a javascript implementation of eSpeak the open source speech synthesizer. The file itself though is 2.3MB, not really something you want to throw at a project just because having some extra functionality would be nice.

In this case it was the main premise for the project so no big deal.

I did have a quick look round at web services that’d take a chunk of text and turn it into a better quality mp3 file but nothing really took my fancy.

And, in theory I could recompile the speak.js to use a different, slightly more defined voice but following the instructions made my head hurt :)

Anyway, the implementation is really easy and I just had to add one hook to detect when the audio had finished playing (it creates an <audio> tag on the fly, so I just added a line directly afterwards attaching the “end” event).

GRAPHIC DESIGN

I’ve already covered the design in more detail over here: The pxl effect with javascript and canvas (and maths), the gist of it though is this…

Via the Guardian API, without an API key, you only have access to the thumbnails for each story, which are tiny 140x86px. I wanted to do something that could blow-up the image to a full screen in an interesting way, but didn’t overshadow main objective of the project being about sound.

And so it turns the thumbnail into a somewhat abstract background, which you can still kind of tell what it is, but not always.

Then I figured it could also have a second function of giving you a general idea of how “fresh” a headline was, by adding some blur/glow filters over time. If the image is nice and crisp with sharp edges then it’s new. If it’s blurred then it’s been around a while.

Yet another background signal of information, it looks a bit like this…

I’m using the Pixastic js library for all the heavy blur lifting (or at least until we get it a bit more native into webkit)

Once a second after two minutes it applies the following filters. The values are pretty small because otherwise the whole thing goes to mush in less than a minute :)

ct=$('#targetCanvas')[0];
Pixastic.process(ct, "glow", {amount:0.003,radius:3.0});
ct=$('#targetCanvas')[0];
Pixastic.process(ct, "noise", {mono:true,amount:0.01,strength:0.05});
ct=$('#targetCanvas')[0];
Pixastic.process(ct, "blurfast", {amount:0.04});

The end result is often very pleasing and a contrast to the original sharp image.

TECHNOLOGY

And there we have it, background image, ambient audio and text to speech all running on the client in javascript and modern technologies, i.e. HTML5, CSS3 when we get the image filters.

Running in the client is a lot more limiting but at the same time I don’t need to tie up two laptops constantly running with 100s of pounds worth of software “broadcasting” out to the internet 24hours a day.

Although that does have a certain appeal. I’m going to see if I can get that down to one laptop.

THINGS I LEARNT ALONG THE WAY

The final implimentation turned out to be pretty quick and easy, it’s just the year thinking about it and hunting out libraries that took the time :)

And during that time I’ve had a good look around Ableton Live, it no-longer make no-sense-at-all, which is nice. Same for Native Instruments stuff, lots of gorgeous synths I’ve not had the chance to dig into too much.

I also watched a whole bunch of videos over at MacProVideo, specifically:

• Music Theory
• Ableton Live; 101, 102, 202 & 405
• Native Instruments; 102, 203 & 104

I grabbed a years subscription, not sure I’d want to buy each lession individually.

Open Sound Control is another fun thing I looked at, along with TouchOSC and the Lemur OSC controller mainly because I like the idea of controling stuff with futuristic looking touch surfaces…

Image from h e x l e r

Processing + MIDI & OSC, I’ve used processing for a while now but it was nice to drop in the oscP5 and proMidi libraries to have a quick play at sending messages back and forth. I didn’t get to use them for this project, but knowing I can make Lemur running on the iPad talk to processing makes me very happy indeed. Now I just have to shoe horn it into a project.

I guess the one thing that I saw a hint of, but didn’t really dig into was generating the ambient soundscapes with javascript. I’ve seen some interesting pieces here & there about generating WAVs on the fly (which is how the text-to-speech works), and bits & bobs around the audioData API. So at some point I may try and have a go at doing something with that.

Anyway, The END.

Ok, so last mention of the pxl effect thing that I posted about a short while back and then we can all move on with the next stage of the Guardian Ambient Headline Radio thing.

I decided I quite liked the effect and rather than just applying it to thumbnails of Guardian news stories I rather wanted to add it to any Flickr photo I was looking at. This was a great excuse to combine a few things.

1. Make the code universal for any sized image, rather then the 140px*84px images from the Guardian.
2. Have a go at creating a Chrome Extension that would appear on a specific site, in this case Flickr, it’s here btw.
3. Play with the new automatic GitHub pages generator.

Universal Image Size

First thing I did was remove all the size specific code, and clean up the code a bit making it shorter, tighter and faster, oh and better comments. Probably the best place to find it is over here: github.com/revdancatt/…/blob/master/js/aesthetic.js

Scott took the code, node.js-ified it and used it to process some video, the effect is good [Flash video]…

Chrome Extensions & Accessing the Canvas Object

Then I turned the whole thing into a Chrome Extension that you can install over here: http://revdancatt.github.com/Flambient/

I kept hearing that Chrome Extensions were easy (like Greasemonkey scripts) and could be pretty full featured, to the extent of full on web Apps that you can download from the Chrome Appstore thingy. But had never really had an excuse to do so.

Turns out that Google’s documentation is both 100% correct and 100% unhelpful. Maybe how my brain works just doesn’t grok their way of writing.

Anyhow, I thought it was all going to be easy, you create a bunch of files that are going to be your Extension, some of which “Content Scripts” which run in the context of the page you’re looking at. Brilliant I thought, I’ll put all my code to grab the image and put it onto the canvas there, that way I won’t have any trouble with same-domain security policy stuff.

Wrong!

As soon as I copied the photo on the Flickr page to a canvas object I tried to get the pixel data and the security error was thrown. Hummmmmmmm.

With extensions you also have “background” html or scripts. A “background” html page is basically a hidden page that you can use as a scratch pad. But the background page (and any javascript on it) don’t have access to the page the extension is running on, Grrrrrrrrrrr.

Finally there’s a messaging system that allows the content scripts (running in the context of the page) to talk to the background html, and of course the other way. So in the end I had the content script find the URL of the image, pass it to the background html in a message, load it into a canvas object, do the pxl effect there which oddly doesn’t throw a same domain error (I’m sure it makes sense why, but you know, whatever). Encode it as base64 and send it back.

Frankly far more effort than I thought it was going to be. But at least once I’d gotten through it all I ended up with a pretty good understanding and framework for any other projects I foolishly decide to do.

If you’re interested the code now sits over here: https://github.com/revdancatt/Flambient/tree/master/source

Automatic GitHub Pages

And lastly, the actual hosting of the extension.

Luckily for us, and unsurprisingly because GitHub are awesome, they have all the mime types setup on their servers to correctly serve a Chrome Extension (and you can even point the extension’s update URL to the update file on GitHub which is nice).

I really like GitHub Pages, for hosting project that run purely in HTML5/CSS/javascript they’re a great way of doing it. It tightly couples the code you’re writing to how people can view it up and running.

But this project was slightly different as it’s a thing not a webpage/site, which gave me a chance to try out the Automated Pages Creator. The end result is here…

http://revdancatt.github.com/Flambient/

…which looks pretty good. You kick off the whole page generator thing, enter a few settings, pick a theme and then click build. It gives you a “gh-pages” branch which you then checkout and tweak to your hearts delight.

In this case I chucked in the nice Install Now button and the graphic up in the header.

Closing Words

So there we go, code to turn any sized image into a pxl image, a Chrome Extension built and a natty little GitHub pages page, wasn’t bad for a nights work.

Now however, back to getting the Ambient Headline Radio to actually do all the reading out headlines & sound bit rather than banging on about the graphics :)

…to make that stupid Dell infographic not the most recent thing on my blog :(

So there’s a nice photo I took on my morning commute to London instead.

ONE

I quite like Bauhaus (both the school/movement and the band)

TWO

“To choose order over disorder, or disorder over order, is to accept a trip composed of both the creative and the destructive. But to choose the creative over the destructive is an all-creative trip composed of both order and disorder” - Malaclypse the Younger, K.S.C.

THREE

Disregard anything said by anyone older than you, pay attention to those younger.

The thing about the #newaesthetic I don’t completely get is why it’s so heavily influenced by the colors and 8 bit graphics of the 1980s.

— Tom Coates (@tomcoates) April 6, 2012

I mean, things being pixellated – digital in the real world – if anything, pixellation is on its way *out* with retina screens et al…

— Tom Coates (@tomcoates) April 6, 2012

Pixellation, tessellation, patterns and artefacting seem to me to be already retro. We’re going HD, algorithmically generated, retina screen

— Tom Coates (@tomcoates) April 6, 2012

The answer is this (and I’ll expand later) images, fashion and so on in the 80s was inspired by the computer graphics of the time, and ideas around what computer graphics would look like. So lots of 8bit stuff, neon and Tron like faff.

The New Aesthetics, or at least the aspect I’m looking at, is inspired by computer vision. And computer vision is at the point now that computer graphics was at 30 years ago. The New Aesthetics isn’t concerned with retro 8bit graphics of the past, but the 8bit graphics designed for machines of the now.

Maybe this image will help, before I go and justify my above statement…

I’ll come back to Vision vs Graphics in a moment.

But if, for example a fashion designer took inspiration from the motion capture outfits above, and made clothes featuring large 1bit “pixels” I’d count that as New Aesthetic. But how are new pixels different from old pixels?

The Robot Readable World

A lot of the New Aesthetic is directly related to (or descended from) the Robot Readable World (RRW). You can read way more about that over at the BERG: The Robot-Readable World. And even earlier in Matt’s slides from 2007: The Vernacular of the Spectacular – Playing with Bits of The City, in which you can see obvious precursors to what we’re now called NA (slides 25, 30, 34 & 45).

See also, Robot Readable World - Timo Arnall, THE NEW ROBOT DOMESTICITY - bldgblog and With Robots - Diego Trujillo Pisanty. Here’s an image from the With Robots project…

Image: From With Robots by Diego Trujillo-Pisanty

Because machine/computer vision isn’t very advanced, to exist with machines in the real world we need to mark up the world to help them see. The above image is imagining how this could work, but below is it occurring in the real world…

Robots navigate via a grid of QR codes – Kellan

…QR codes (or something somewhat similar) are placed on a factory floor in a grid solely to aid robots, there is nothing here for us humans to use. And yet the passage of the robots create a pattern, tracks from the wheels and circles where the robots turn and turn and turn. This is computer vision spilling out into the real world.

It’s been there for a while now, bar codes have been around for years, but we can expect to see more of it. The testing and slow introduction of computer driven cars will most likely see special markings on roads & signs giving the cars instructions. All throughout shops, malls, streets and cities markings for machines are appearing.

The first part of NA I’ve been paying attention to has been the examples of this spilling out, the bits of the city not meant for us. The second part of NA I paid attention to was what happened when artists began to deconstruct and respond to that encroachment, how can designing for robots influence our own design?

8bit retro computer vision

So how do I know that computer vision is lagging behind computer graphics? Well firstly I know I can easily buy two consumer grade specifically design graphics cards with GPUs on running in parallel to install in my desktop PC. I don’t seem to be able to buy specifically design computer vision cards to stick in my machine. In the consumer space one is definitely leading the other.

Secondly, games can create all sorts of worlds, spaceships, fantasy lands, renditions of hell, cities from the 80s and so on with their fancy graphics, reflections, fog, shaders, realistic light sources and particle effects all adding to the effect.

Computer vision can recognise faces (badly), blocky Augmented Reality (AR) surfaces and very specifically registered AR surfaces.

Consumer desktops can produce pretty good environments in real-time  (heck the new iPad can) and render photorealistic images given a bit longer. Meanwhile they can see bugger all, unless we’ve specifically designed it for them.

Here’s a quick list of the progression of computer game graphics, and a screenshot from Sentinel because I like it…

Sentinel screenshot from sci-fi-o-rama.com

…actually now’s a good time to jump over to The Virtual Art of 80′s Game Worlds for a quick read, and then come back.

That list, in sort-of kinda easily disputable chronological order (fit Alone in the Dark in where you like)

• 2D graphics
• Parallax
• Isometric
• Wireframe 3D
• Solid 3D
• Textured 3D/Gradient filled 3D
• Particle effects
• Voxels
• Level of Details 3D models & shaders
• Dynamic lighting
• Procedurally generated environments
• Retina displays/3D displays
• Augmented Reality (AR)
• Photo realistic realtime AR

That seems like a not unreasonable list. And getting back to the tweet posted by Coates above, if New Aesthetics was aping early computer graphics, the 2D, Isometric, Solid 3D, 8Bit colours stuff, then indeed it’s being retro for no good reason, when it could take its cue from photo realistic AR & retina displays, moving beyond the pixel.

Now a list of computer vision, in similar not-quite but almost correct order…

• Motion detection
• Edge detection
• “Blob” detection
• Shape detection
• Face detection
• Depth & Joint detection (XBox Kinetic, motion capture)
• Colour detection
• Gait detection
• Gradient/Shade detection (ie. understanding colour in different lighting conditions)
• Recognising humanoid forms against real world backgrounds
• Recognising individuals against real world backgrounds
• Recognising specific items against real world background
• Understanding all nearby items within a real world environment
• Skynet

3D Wireframes were around 30 years ago, solid & textured 3D shortly after and still all done in software. 20 years ago some of these calculations moved onto GPUs on dedicated 3D graphics cards.

Computer vision it’s all still done in software, and we’re roughly up-to depth, joints, colour & shading detection, if the evolution was on par with graphics we’d start to see the first few dedicated vision cards appearing on the market for consumer use.

Or put another way, current computer vision can probably “see” computer graphics from around 20-30 years ago.

Which in turn means to design for machine eyes we need to be at the level of computer graphics from the 8bit era, and so we have QR codes all over the place.

Chris Heathcote wrote about NA fashion: a new fashion aesthetic…

“Sometimes it’s just the right colours, or the cut. It’s more gradient fill than pixels. It’s things that couldn’t be made 5 years ago. Supersymmetry and asymmetry. It’s not about the ‘machine vision’ that the New Aesthetic references, but it’s hard to see how that will not be appropriated and re-emerge into fashion as something not necessarily technically correct but aesthetically interesting.”

I’m going to disagree with the “not about the ‘machine vision’” part, and wrap it up with markings for machines spilling out into the real world I mentioned above. If the markings machines can understand are currently at the level of edge detection, outline/shape, depth, joint & colour shading/gradient and that’s out there in the world, being all odd and not for us, but inspiring and guiding designs, art and designers, then we get the gradient fills, sharp cuts & polygones.

And I can see how that’s easy to confuse with retro 8bit 80s style, but as I mentioned before NA is about what machines can see now, rather than what they could produce back then, it’s just that vision is 20-30 years behind creation and so there are many similarities.

New Aesthetic is about the polygones and edges and pixels of now vision, not the polygones and edges and pixels of back then creation.

 A few last words

I wanted to write this because I’ve seen a number of occasions where the question “But isn’t NA just retro?” has come up, and I can see why it comes up and where it comes from. I just wanted to explain that the NA pixels do come from some other place, the RRW. It isn’t all just glitch, but actually a response to new markings appearing in our world.

There are also other angles to the NA that do involve retro 8bit, such as this pixel Whale…

Whale – Flickmor

…which is a representation of pixels in the real world. And as Coates says, as we’re moving to retina displays, pixels as presentation is something that’s going away, making this image distinctly about nostalgia and yet still feels like NA.

And then on-top of that there’s art/design inspired by what’s been collected together so far, which gives us…

1. Found art that’s actually stuff for machines
2. Art based on found art that’s actually stuff for machines
3. Art based on the art that’s based on found art that’s actually stuff for machines

Number 1 is the markings on the factory floor for robots, number 2 is a lot of the New Aesthetic that’s been gathered that’s taken it’s inspiration from the factory floor markings.

I’ve partaken of number 3, I wrote some code to turn images into patterns, rather like this…

I probably wouldn’t have made these design decisions if I hadn’t recently been immersed in the thinking about exactly what NA is (*spoiler* I still don’t really know). The look and feel is supposed to be that of NA, but I’m not sure this makes what I’ve done NA.

But what I’ve done here is no longer about machine vision and looks far closers to a variation on pixels and so reinforcing the retro confusion. It’s supposed to be modern not retro.

New pixels, not old pixels.

Drones

I should probably mention drones, abet quickly, and my understanding of drones.

When I think of a drone I think of an autonomous agent, not something that’s being controlled remotely by people in a bunker with laptops (or whatever). Something with GPS, altimeter, navigation, satellite images, vision.

Something that left to its own devices can operate itself, sensing, watching, reacting. It is machine vision actualised.

As part of a bigger project I wanted to generate an abstract background image/texture. There’s obviously a huge number of ways to do this but due to a few constraints I put on myself the method I ended up with (described below) seemed like a good solution. First the constraints…

1) The project involves grabbing the latest news headline from the Guardian API, and because I want to run this in javascript from a github hosted page I don’t want to use an API key that everyone will be able see. Which in turn means I don’t have access to any of the images (you need an API key to get the media assets related to a story) other than the thumbnail.

2) The thumbnail is tiny 140px x 84px.

3) Telling the browser to scale up the image without smoothing (see snippet below) gives it a nice mosaic effect, but the iPad ignores that and makes it look all blurred and shit anyway.

{
image-rendering:-moz-crisp-edges;
image-rendering: -o-crisp-edges;
image-rendering:-webkit-optimize-contrast;
-ms-interpolation-mode:nearest-neighbor;
}

4) It has to be done in javascript, so no server-side image processing (although as you’ll see later it does involve a small amount of server side stuff, but for a totally different reason).

Recently the pxl app for iPhone has filtered through my friends. It allows you to apply a variety of abstract compositions to your photos, a popular one turns an image into a collection of triangles, rather like this…

…a general effect that should work well, and allow us to create a large background image out of a relatively small source image.

This seemed like a good solution, by doing something very similar I can now turn a thumbnail that actually relates to the latest headline (which is the thing we’re actually after) into a fairly abstract and pleasing, but not too distracting background. All I needed to do was chop it up into triangles.

Method

No idea if this was intentional, but the thumbnail from the guardian is divisible by 7 in both width & height, giving us a grid of 20 x 12 tiles, each tile 7 x7 pixels.

The next step was to look at each “tile”, work out which way to split the tile, diagonally from top-left to bottom-right or top-right to bottom-left and the colours to use for each split.

I decided the best way to do this was to divide the square up into quarters, top, left, right and bottom. Average the rgb values in each quarter and then work out if the top quarter was closer in colour to either the left or right quarter. If it was closer to the one on the left we’d split the tile that way, combining the top average colour with the left one, and the opposite if it was closer to the right quarter. A picture will probably explain it better.

You can see that each quarter contains 9 pixels, I didn’t bother counting the pixels on the diagonals. So anyway, averaging the rgb values of the 9 pixels in each quarter, and then comparing them is what I did. In the above example the green is closer to the blue than the red (and the yellow is closer to the red than the blue) so we’d split top-left to bottom-right.

The code: which is here on GitHub, is a little bit specific to the 140 x 84 pixel size, and grabs the pixel values long-hand. At some point I’ll probably make it generic so that it can work with any sized image divided down into any number of tiles. But just to get it working it seemed easier to hardcode some of the values.

The end result is something like this…

You can see the whole thing running over on GitHub…

http://revdancatt.github.com/GuardianAmbientHeadlineRadio/

I also dropped in some meta-tags to make the whole thing work a bit better on the iPad/iPhone and GoogleTV, just because I wanted some nice abstract art in my living room :)

<meta name="gtv-autozoom" content="off" />
<meta name=apple-mobile-web-app-capable content=yes />
<meta name=apple-mobile-web-app-status-bar-style content=black-translucent />
<meta name =viewport content=initial-scale=1.0 />
<meta name=viewport content=width=device-width />
<meta name=viewport content=user-scalable=no />

Weirdly the project is really supposed to be about text-to-speech and music, with the image background as just an extra bit of colourfulness. But now it’s here I may fork the code to keep the image part as a snapshot for other things, or at least so it doesn’t get bogged down in all the other stuff.

Notes about loading images into canvas from remote servers

Again, if I was running this on something other than GitHub hosted pages I could do a bunch of this stuff on the server side and not worry about it, but instead I have to deal with loading an image from a remote server.

In an ideal world the guardian would be using CORS and allow canvas to access the image data from the image, but boo, not yet. And so a proxy server of sorts has to be used to deliver the image to javascript. We can’t put anything on GitHub to pull the image to the server so we still have to use something on a remote server.

I put this up on Google AppEngine (which is my general weapon of choice and experimented with python 2.7 on there at the same time)…

import json
import urllib2
import cStringIO
import base64
import webapp2

class MainPage(webapp2.RequestHandler):
    def get(self):

        remote = urllib2.urlopen(urllib2.unquote('http://static.guim.co.uk/sys-images/%s') % self.request.get('img'))
        img = cStringIO.StringIO(remote.read())
        rtnimg = base64.b64encode(img.getvalue())
        type_prefix = "data:image/jpg;base64,"
        data = {
                            "data": type_prefix + rtnimg
                        }

        self.response.headers['Content-Type'] = 'application/json'
        if 'callback' in self.request.arguments():
                data = '%s(%s)' % (self.request.get('callback'), json.dumps(data))
                self.response.out.write(data)
        else:
                self.response.out.write(json.dumps(data))

app = webapp2.WSGIApplication([('/img_to_json', MainPage)], debug=True)

…which  reads in an image from the guardian, converts it to base64 and chucks it out in a json response.

Meanwhile, on the client side, jquery calls the endpoint passing over the second half of the image URL, gets the json back and sets the source of an image to the data:image/jpg;base64 value. Which now doesn’t suffer from cross-domain security and we can get access to the image data for shoving onto a canvas object.

It’s a bit of a hack, abet a fun one, but hopefully greater implementation of CORS will remove the need for any server side stuff, at which point I can get rid of that part out of the code altogether.

Yay, I got to launch the Guardian GoogleTV project that I’ve been tinkering with over the last few months. I’ve already written about it over on the Guardian Developers Blog and there’s a more general overview on Inside Guardian so there’s not much point repeating that here.

But I did want to tie it in with a couple of things bouncing around in my head, let’s start with…

Every so often I have a mild panic attack on behalf of journalism in the face of speed of innovation elsewhere. Like now #sxswi

— emily bell (@emilybell) March 12, 2012

And the Hacking is Important post from Michael, which I shall now quote extensively from…

‘[Facebook is worried about] the growth paradox, which goes something like this: The end result of successful hacking is product, and that product needs to grow by building more things. The more you grow, the more things you have, and the more you need people whose job is simply to coordinate the increasingly interdependent building activities. These people, called managers, don’t create product, they create process.

 

‘Hackers are allergic to process not because they don’t understand the value; they’re allergic to it because it violates their core values. These values are well documented in Zuckerberg’s letter: “Done is better than perfect”, “Code wins arguments”, and that “Hacker culture is extremely open and meritocratic”. The folks who create process care about control, and they use politics to shape that control and to influence communications, and if there is ever a sentence that would cause a hacker to stand up and throw his or her keyboard at the screen, it’s the first half of this one.

 

‘The growth paradox is that the chaotic means by which you found success might become distasteful to those you hire to maintain and build on that success. Once they’ve established themselves, they will point at the hacking and ask important sounding questions like, “What is it they are building?” or “How does this poorly defined thing fit into our overall strategy?” They will label these hackers “disruptors” and they are 100% correct.

 

‘Hacking is disruptive, and whether you code software, write books, or film movies, I believe bringing anything new into the world is a disruptive act. By being novel and compelling, the new is likely to replace something else and that something else isn’t being replaced without a fight.’

Fortunately the Guardian (as has facebook) found ways to work around the growth paradox. For the Guardian the growth in this case isn’t your sudden start-up to internet infrastructure over 6 years growth, but the been-around-for-a-very-long-time growth.

The 1st way was to build the Guardian Open Platform & Content API. At a time when other news organisations are putting up paywalls, the Guardian sticks all its content into APIs for developers to build on and incorporate into tools.

The 2nd is to actually use its own Content API on a daily basis to quickly prototype projects, hold internal & external hack-days and then giving developers enough rope with which to hang themselves :)

Obviously I’m coming at this from the developer angle and this is a technology project rather than the journalism that’s giving Emily mild panic attacks. Which is why we also have Open Journalism (and #OpenNews) and ways of linking these two together internally.

With those two prongs performing a pincer movement on “Open”, along with the API and internal support for “hackers” we have a pretty good ecosystem for moving quickly(ish) on the digital. We don’t have an R&D “lab” (unless it’s very will hidden under Guardian Towers) but I’m slightly wary of shuffling off innovation into a separate unit anyway, it should just be core to business as normal. From which fertile ground sprung forth the GoogleTV project.

Back to the TV

Anyway, as mentioned in the Dev Blog post, this project started as a “what if?”, in this case, “we’re looking at desktop, tablets & mobile a lot, what if ConnectedTVs really start take off?”

There are just over 300 million people in the USA, and around 270 million TVs, there’s actually more TVs per household than people (2.7 TVs vs 2.5 people). You couldn’t move for ConnectedTVs at CES this year, although no-one can quite tell me exactly what a Connected TV is. In years past the invention of the radio was going to kill newspapers and then video was going to kill the radio, the internet in turn was set to finish off the television and all of them had it in for books. Instead what happened is that nothing really died but stole ideas and characteristics from the newer shiny toy. Books can now be ebooks with links to twitter, newspapers now have podcasts which are like the radio but on the internet, the internet has TV programs on it and the telly can now show webpages.

Clay Shirky in his book Cognative Surplus attempted to quantify “spare time”, the premise being that we no longer work all our waking hours and have some time left over and that time is often spent sat infront of the television. A trillionly billionty hours, and if we did something with that “surplus” time great things could be achieved.

The measure he ended up with was this: if everyone spent just one advert break per week updating Wikipedia, we could have a new Wikipedias every few weeks. Admittedly they’d most likely be terrible, terrible Wikipedias but I think the point still stands. As much as some people claim that other people spend too long on the internet, as a society it’s nothing compared to that spent infront of the TV.

“We can do a back-of-the-envelope calculation, for example, using Wikipedia, to see how far we still have to go. All the articles, edits, and arguments about articles and edits represent around 100 million hours of human labor. That’s a lot of time. But remember: Americans watch about 200 billion hours of TV every year.”

- Clay Shirky, Wired: Cognitive Surplus: The Great Spare-Time Revolution.

There’s all sorts of things we could take away from this. But for me, at the moment it’s this… people spend a fuck load of time infront of the TV.

And now, TV manufactures want people to interact with the TV, various brands have their own web browsers and services built in, Apple have their Apple TV “puck” that you can attach and Google have GoogleTV that comes either built in or as a unit you can plug in.

It seems reasonable that we (the Guardian) would want our news, and images and video and podcasts, on all those millions of devices.

One Million GoogleTVs

Of course it’s still very early in the ConnectedTVs world, there’s currently only around 1 million GoogleTVs out there in the wild, but then there aren’t that many of the other breeds of “connected” TVs either, Apple has no browser on their puck, some gaming consoles have browsers but of different abilities and so on. But you have to start somewhere and if you want to quickly sketch out an idea then GoogleTV is a pretty good place to start.

It’s basically Chrome with HTML5 and CSS3-ish, and that right there should tell you that you already have all the tools you need to knock something together that’ll work on a TV.

Our plan was to quickly sketch out ideas, throw them onto the TV to see what worked and the big screen and what didn’t. It also meant that we could iterate quickly and push changes as often as we liked, which is nice.

That all new TVs in a couple of years will have internet browsers and access is all but inevitable, and as they get into more and more homes you suddenly have a new audience, people who like to just sit down and be informed/entertained by the nice, bright, large screen in the corner (or mounted on the wall). And now seemed like a better time to start thinking about that than trying to play catchup later.

So this experiment isn’t so much about the now but rather what may be in a couple of years time.

Well that and an excuse to buy a new big telly ;)