electronic museum

[ Do you know what - I'm a bit nervous about this blog post. The reason I'm nervous is that I'm writing about something I really don't understand too well. I've tried - I really, really have - I've watched videos and slideshows, looked at diagrams, read explanations. But I still don't really understand how OpenID works. And for a long while that put me off writing this. I know that OpenID has a lot of people gunning for it. And I know that support is gaining, at least in numbers of service providers. But in the end, it comes down - as always - to the user - and the experience I have had has been as that user. And I simply can't, won't - and don't use OpenId. Because it's rotten, and broken, and failing. So I went ahead and wrote this anyway..I'm sure you'll let me know what you think ]

The geek world has been getting excited for a fair while about OpenID. You’re probably all familiar with it and I’ll leave it up to Wikipedia to describe the service in detail, but in short the notion is that managing multiple identities online is increasingly problematic, and that some kind of way of managing these identities in one trusted, decentralised place is what is needed to make life better.

OpenID is based around the use of a uri as the unique identifier for an individual, not an email address, as is so common today with most sites.

All well and good, you’d have thought. The only thing is there’s an enormous, hulking great elephant in the room: OpenID doesn’t work.

I should clarify. In a technical sense, OpenID works. But from a usability perspective, it’s absolutely horrible.

Let’s examine the user flow for someone signing up to a.n.other site using the “traditional” method: they arrive, they click “register”. They put in their details, including email address. They go to their email account and click on the “validate” link. Done. The purists all shift uncomfortably in their seats - the users’ identity has been propogated to yet another site (eek, duplication) and there is also a reliance on the email provider (eek, single point of failure / “evil” company fear, etc).

Now let’s have a look with OpenID. And let’s consider it in the best possible case scenario - user has not only already created an OpenID but knows the address AND is signed in (i.e has a currently active session/cookie) to that providers’ service.

So..user arrives at site and is asked for their OpenID. They put in the address and push go. The site then redirects them to their OpenID provider. User clicks to allow access to data, and selects a persona. Provider site then redirects back to the original site. Original site then (inevitably, in my experience) asks user to fill in additional “persona” data for their service as well as what they already entered. User enters site.

That’s at least a couple more steps, and remember that’s if they’re signed in or even have an OpenID account. If they’re not signed in (but have an account) then they still have to sign in on the OpenID providers’ site. Using a username and password…If they don’t have an OpenID, just add at least 3 more steps. If they forget their OpenID then the process to get it back has to be done on the provider site and not on the site they’re wanting to access.

There are several thing that are really badly wrong with the OpenID / user landscape. Here’s how I see them:

1. Users don’t understand the use of a URI as identifier
This is about education, but it’s an important point. People see URI’s as “web addresses”, not as personal identifiers. They don’t get it, and aren’t being encouraged to get it, either.

2. Users don’t like redirects
Actually, users don’t care about redirects - what they do care about is maintenance of trust and brand. A user mid-basket on Amazon is not going to be happy about a jump away to another site unless they’re very clear that there is brand association between the sites.

3. Users won’t remember OpenID’s
Not only are OpenID’s longer and more complex, they’re also a dog to get back once forgotten. With email/pwd, you just click the “forgotten pwd” link. Email, click, done. With OpenID you have to go back to your provider site and do it from there, not on the site you’re trying to access.

4. There is no paradigm
Apart from password remembering within the browser, there isn’t a “central persona management” paradigm. This doesn’t mean there shouldn’t be one, but it makes the job of invisibile tech that much harder.

I’ve left what I see as the single biggest issue until last:

5. There isn’t a problem that needs solving
As I’ve indicated before, we (tech savvy geek types) are not the normality. I may have a sign-up obsession and belong to hundreds of sites, but normal people just don’t. By some gentle “finger in the air” reckoning, I’d suggest that most people have - what - ten sites they sign in to? That’s hardly shouting out for a distributed, decentralised, persona-based solution, is it? What’s actually wrong with a “remind me of my password” link, anyway? And using email as identity is secure enough for pretty much any application. We geeks are making assumptions based on our experiences of the web. It’s us, not Joe Normal who has 400 passwords in our heads, surely?

So on the one hand we’ve got an elegant, beautiful, technically “good” solution that is almost completely unusable. On the other is something ugly and flawed - but something that works well for most people: something that isn’t actually broken, and - frankly - doesn’t need fixing.

OpenID feels like it could and should be better, but the current scenario whereby hundreds and thousands of sites are becoming providers (AOL, Orange, Yahoo!, etc) and very little effort is being put into fixing the flawed user flow - or user education for that matter - is just a road to nowhere. Some sites (LiquidID, ClickPass, Vidoop as examples) are just starting in the usability direction, but it’s nowhere near enough. And right now, I - like most people I know - are just fine sticking with the original email/pwd alternative.

On June 18th 2008 (the day before UK Museums on the Web conference) a bunch of us met in a room at Leicester University to do some museum mashing. Our aim was:

” …to give ourselves an environment free from political or monetary constraints. The focus of the day is not IPR, copyright, funding or museum politics. Our energies will be channeled into embracing the “new web”: envisaging, demonstrating and (hopefully) building some lightweight distributed applications. “

I thought I’d borrow Matt’s N95 and do a quick “interview” (loose term) of everyone that wanted to say something about what they’d done. Finally last night I got round to chucking this into Window’s Movie Maker and doing some editing and have uploaded the result to http://blip.tv/file/1029060. It’s around 12 minutes long, so grab a cuppa and a comfy chair…

Finished?

The following day I did a presentation at the conference itself. This is now on Slideshare (and embedded below)

I’d just like to say thanks loads to everyone who attended - I know giving up a day is always problematic (even if it involves beer at the end). I hope you had fun. I know I did. Also enormous thanks to Ross Parry and the MCG for giving us the opportunity to do this.

Several people who attended have written about / linked to the things we built:

• Jim O’Donnell did some nice stuff with Yahoo! and RSS - see his demos of a tagged RSS feed, browsing by tag and browsing by object
• Brian Kelly played about with PicLens - see his coverage on the UK Web Focus site
• Daniel Pett blogged about what he got up to over here
• Steve Pope used his OpenCalais helper to mashup some hoard.it data with Flickr and Amazon
• Frankie Roberto did some very cool stuff with Freebase (my coverage here) and the MIT Simile Timeline
• Fiona Romeo used the awesome Many Eyes visualisation tool on some data she had…
• ..and I built an SMS thing in the morning (but gave up as we had no mobile coverage..) and a very simple mystery object game (again based on hoard.it data) in the afternoon. (deliberate error: the right answer is always the first one…damn…)

(I’ll also be continuing to update www.mashedmuseum.org.uk with future museummashingmoments…)

The message? Well, Lee Iverson from the Univerity of British Columbia used a phrase during his presentation the following day to beautifully encapsulate what I’ve posted about so, so often - and the one thing that makes any mashups possible:

“If you expose data, you lose control but give it life“

And that pretty much sums it up.

Now is a hugely exciting time to be involved in the web. I believe we’ll look back at the early 2000’s with a sense of awe at the rate and extent of technological change. Personally, I believe it’s faster and more engaging than it ever has been before. The 1990’s were exciting in a different kind of way - in a crazy, rollercoaster, unsustainable, first toe-dipping way. It was good for jobs, too - I remember a period of a few months where I literally - and I exaggerate not - got 2/3 offers a week; and I wasn’t really that much cop, either. That doesn’t happen any more. Or maybe Google lost my number

The good ‘ole days were fraught with tension and frustration, too. There was so, so much we couldn’t do - so many hacked together approaches, so many nastinesses in browser renderings, so much uncertainty about frames, browser-dependent tags and dialup speeds. Even saying that makes me chuckle now - hey, we used to worry about whether a page was 10Kb or 20Kb - imagine…

Today, things are markedly different. The challenges are - at least to someone like me - actually much more satisfying. They’re not generally about whether system A can talk to system B, they’re about the more human challenges: will person X understand what I mean when I say Z; will they use what I’m building, and how? How can I hit this niche? What’s the best marketing strategy? Can I ever, ever make money doing X?

More and more, we’re also asking “where will we use it?”.

Everyware - the notion that the internet is all around us - is, as you will probably know, a notion that I believe forms an absolutely central part to where this all goes from here. And core to Everyware are real questions about convergence and re-purposing (remember those phrases..?).

A while back, I took a quick look at the blurring line between desktop and web apps. Back then I commented that two major player were entering the “Rich Internet App” (RIA) space - Microsoft with Silverlight/WPF and Adobe with what had been Adobe Apollo and which became Adobe AIR.

Microsoft is - obviously - never a player to be underestimated. They’ve got the desktop sown up, an unfathonable quantity of cash, and some serious technologists who know their shit. On the other hand you’ve got Adobe, previously somewhat irritating to “serious” web developers who didn’t want to learn Flex or Flash, and considered them “something for them weird designer types”.

Microsoft have done some nice stuff with Silverlight and WPF. But…it seems terribly, horribly cumbersome - it’s all a bit .Net, a bit codey, a bit deep and impenetrable. Frankly, a bit ‘fn dull. Over on the other side, Adobe chose a lightweight “if you can do it in xhtml, know a bit of AJAX and can cope with expanding your javascript a smidgin” approach.

You know what? Adobe are winning, hands-down, with absolutely no doubt whatsoever: AIR is getting huge numbers of column inches from the major blog writers compared to Silverlight. The AIR applications being built are deep, beautiful and useful, even for the Enterprise. They install easily, they’re rich and quick and well designed. They make the transition from desktop to web seamless.

And as usual, it’s the easy, lightweight, familiar approach which is beating the heavy, technical and possibly “more feature rich” one. Once again, people like me - semi-serious web types - can play with technologies we understand; we can see results almost immediately; we can view source and copy what other people have done. And we are writing desktop apps which you can - pretty much as-is - also deploy to the web. From a reality perspective, this is an exciting way to think of applications: true develop once, deploy to many channels.

This is holy grail land. If you haven’t played with AIR, go do it now…

Most of you will probably have seen this already, but I’m running another museum mashup day in Leicester on 18th June 2008, the day before the annual UK Museums on the Web conference. It’ll be a developer/techie focussed day working with various bits of museum data to see what we can mash together in just one day.

We’re having to be very strict with the number of people allowed and the list is starting to look pretty full…but if you’re up for it, visit www.mashedmuseum.org.uk to find out more, or jump straight to the “I’m interested” form.

On a related note, I knocked out a quick application (the “hoard.it collector”) over the weekend based on data from the hoard.it application which Dan and I unveiled last week. It’s a terribly quick (read: a bit flaky) prototype which lets you “collect” museum object records and then share them so others can see what you’ve saved. Think Listmania but for museum stuff. The main purpose? Not to build something that’s never been done before, but (as ever) to demonstrate that it is possible to do interesting stuff without funding, the usual museum treaclewading or worrying too much about perfection. I’ll hone and polish the idea as time goes on - any ideas, stick ‘em in the comments…

Dan and I are planning for the hoard.it data to be made available for the Mashed Museum day, but if you fancy a play before that, or can’t come along, you can read the documentation, do some building, and then let me or Dan know so we can chuck something in the hoard.it application gallery.

What seems like a looong time ago I came up with an idea for “bootstrapping” the Non API Web (NAW), particularly around extracting un-structured content from (museum) collections pages.

The idea of scraping pages when there’s a lack of data access API isn’t new: Dapper launched a couple of years ago with a model for mapping and extracting from ‘ordinary’ html into a more programmatically useful format like RSS, JSON or XML. Before that there have been numerous projects that did the same (PiggyBank, Solvent, etc); Dapper is about the friendliest web2y interface so far, but it still fails IMHO in a number of ways.

Of course, there’s always the alternative approach, which Frankie Roberto outlined in his paper at Museums and the Web this year: don’t worry about the technology; instead approach the institution for data via an FOI request…

The original prototype I developed was based around a bookmarklet: the idea was that a user would navigate to an object page (although any templated “collection” or “catalogue” page is essentially the treated the same). If they wanted to “collect” the object on that page they’d click the bookmarklet, a script would look for data “shapes” against a pre-defined store, and then extract the data. Here’s some screen grabs of the process (click for bigger)

	An object page on the Science Museum website
	User clicks on the bookmarklet and a popup tells them that this page has been “collected” before. Data is separated by the template and “structured”
	Here, the object hasn’t been collected but the tech spots that the template is the same, so knows how to deal with the “data shape”
	The hoard.it interface, showing how the fields are defined

I got talking to Dan Zambonini a while ago and showed him this first-pass prototype and he got excited about the potential straight away. Since then we’ve met a couple of times and exchanged ideas about what to do with the system, which we code-named “hoard.it”.

One of the ideas we pushed about early on was the concept of building web spidering into the system: instead of primarily having end-users as the “data triggers”, it should - we reasoned - be reasonably straightforward to define templates and then send a spider off to do the scraping instead.

The hoard.it spider

Dan has taken that idea and run with it. He built a spider in PHP, gave it a set of rules for templates and link-navigation and set it going. A couple of days ago he sent me a link to the data he’s collected - at time of writing, over 44,000 museum objects from 7 museums.

Dan has put together a REST-like querying method for getting at this data. Queries are passed in via URL and constructed in the form attribute/value - the query can be as long as you like, allowing fine-grained data access.

Data is returned as XML - there isn’t a schema right now, but that can follow in further prototypes. Dan has done quite a lot of munging to normalise dates and locations and then squeezed results into a simplified Dublin Core format.

Here’s an example query (click to see results - opens new window):

http://feeds.boxuk.com/museums/xmlfeed/location.made/Japan/

So this means “show me everything where location.made=Japan’”

Getting more fine-grained:

http://feeds.boxuk.com/museums/xmlfeed/location.made/Japan/dc.subject/weapons,entertainment

Yes, you guessed it - this is “things where location.made=Japan and dc.subject=weapons or entertainment”

Dan has done some lovely first-pass displays of ways in which this data could be used:

• A randomly selected gallery of 100 objects
• “100 years of British Culture”
• my favourite: a world map showing object locations

Also, any query can be appended with “/format/html” to show a simple html rendering of the request:

http://feeds.boxuk.com/museums/xmlfeed/location.made/exeter/format/html

What does this all mean?

The exposing of museum data in “machine-useful” form is a topic about which you’ll have noticed I’m pretty passionate. It’s a hard call, though (and one I’m working on with a number of other museum enthusiasts) - to get museums to understand the value of exposing data in this way.

The hoard.it method is a lovely workaround for those who don’t have, can’t afford or don’t understand why machine-accessible object data is important. On the one hand, it’s a hack - screenscraping is by definition a “dirty” method for getting at data. We’d all much prefer it if there was a better way - preferably, that all museums everywhere did this anyway. But the reality is very, very different. Most museums are still in the land of the NAW. I should also add that some (including the initial 7 museums spidered for the purposes of this prototype) have some API’s that they haven’t exposed. Hoard.it can help those who have already done the work of digitising but haven’t exposed the data in a machine-readable format.

Now that we’ve got this kind of data returned, we can of course parse it and deliver back…pretty much anything, from mobile-formatted results to ecards to kiosk to…well, use your imagination…

What next?

I’m running another mashed museum day the day before the annual Museums Computer Group conference in Leicester, and this data will be made available to anyone who wants to use it to build applications, visualisations or whatever around museum objects. Dan has got a bunch of ideas about how to extend the application, as have I - but I guess the main thing is that now it’s exposed, you can get into it and start playing!

How can I find out more?

We’re just in the process of putting together a simple series of wiki pages with some FAQ’s. Please use that, or the comments on this post to get in touch. Look forward to hearing from you!

As I mentioned in an earlier post, because I haven’t got enough on outside of work (OneTag+, various blogs, a novel, music composing, Stufflinker, museum mashup day, 2 kids) - and really need to fill that 25th and 26th hours of the day - I’m working with a bunch of people to put together a Bar Camp in Bath (BathCamp…!) later this summer.

Frankie, Mia and I thought it up while we were at Museums and the Web after 400 pints of Rickard’s Red and a plate of finest poutine, and have been shaping it since then.

It was all seeming a bit overwhelming, and I’m not the first person I’d call on to organise anything, but luckily I met a fine geezer called Tim Beadle (who does tech stuff in Bristol) and a fine geezerette called Laura Francis (who does tech stuff in Bath), both of whom were involved in setting up a Bristol Bar Camp last year. We’ve agreed it’d be sensible to combine the two events and bring Bristol Bar Camp to Bath this summer.

Anyway (Get to the point, man!): Tim Beadle and I have booked a venue and date for this summers’ BathCamp. It’ll be from Saturday 13th - Sunday 14th September 2008 at Invention Studios in Bath. Further details including Upcoming link, map, Flickr shots etc are on the BathCamp blog over here. I’ll drop the occasional post on the Electronic Museum site, but if you want to watch progress, the BathCamp blog is the thing to subscribe to.

Brian has written a short post about universities actively trying to stop promotional material (yes - promotional material) finding freedom on the web. How funny is that?

On a related note, Sarah Perez from ReadWriteWeb did a post a couple of days ago about hidden image resources in the so called “deep web”. The list of links is great - I particularly like Calisphere and this collection of the 1906 SF earthquake. Lovely.

A couple of things though - first, surely Perez is wrong to suggest that these images are “the deep web”? I did a couple of tests looking for images via Google and it all seemed to be spidered ok. This one for instance was found via a Google search for the image title. It also appears on Google Image Search. Granted, you’d likely not find it given the quantity of other stuff, but it is definitely being spidered, so to me that means it’s not Deep Web. I may have missed something..

The finer point is more interesting, which is about what these institutions have done (or not) to promote these exceptionally fine collections. I haven’t looked into it any further in these cases but it’s familiar territory (you know, the whole open content, CC licensing, Flickr-usage, watermarking, marketing gubbins).

That’s where it comes back to Brian’s post - the content is great, the hard work has been done: the digitisation, the cataloguing, the site design. Then at the last hurdle, fear seems to strike. Better hide the content, you know, in case someone - like - uses it.

Go figure.

A lot of rumbling about the noise created by the (social) web has been reaching our ears recently. I’m not in this instance talking about the management of “outgoing” social media but more about how people deal with the sheer quantity of stuff which is arriving through various channels. The news feeds, tweets, emails, IM - all are part of the incoming stream. Then of course there are conversations with people in the real world (gasp!), paper-based print, TV and so on.

Fundamental, of course, to any conversation about technology is that you are ultimately destined to fail, if you’re hoping to know everything. I’ve been following the conversation at a sprint for more than ten years now and like to think that I’ve got a reasonably good grasp of the web technologies out there, but it doesn’t take a genius to recognise that the speed of change is so intense that we’re all going to get left behind sooner or later. Those who have tried particularly hard to keep up have suffered because of it - Om Malik’s heart attack and the death of Russell Shaw are pretty well publicised. While much of the media are swinging off in what is obviously ridiculous “blogging kills you” type directions, there are still some lessons. We’re all getting older (goddamit) and sooner or later we’ll be that “back in the days of X command-line interface, when the world was rosy” IT bod in meetings. Get used to it. I’m almost there already - remember the…NO, STOP..

There’s a tendency I’ve noticed when some are faced with this craziness: ostrich the problem. The argument is articulated like this: “With so much noise, maybe we’d be better off just not doing anything“. It’s either a conscious decision, or a rabbit-stuck-in-headlights paralysis. Either way, to me it’s always been the most spurious of positions to take. To steal and adapt (CC-styley!) a well-known phrase:

“where there is noise, there is signal”

Choosing to actively run away from the noise - to “not do the social web because it’s too noisy” is a hugely perverse argument. Yes, there is noise and hype…no, Twitter probably won’t last..no, you shouldn’t be on Facebook just because you can…but the point as far as I see it is this: the social web has signal far above the hype: signal far stronger than the noise, provided you can take a step backwards and look at the direction of travel rather than the individual paths being walked. The social web is important because it lets us connect, not because it lets us tweet.

There’s no doubt that the noise is intense - unfiltered, it is way more than most of us can cope with. Here’s a (probably incomplete) list of my current inputs. Every one of them is a stream of information but also a potential distraction, red herring, attention-grabber, too:

email (Outlook), email (Gmail), twitter (via twhirl), IM (Google Talk), IM (MSN), IM (Skype), phone (mobile), phone (desk), phone (skype), feeds (google reader), “the web”, …not forgetting conversations with real people…

I may be in the upper quartile of “wiredness” but I’ll bet most of you are exposed to these, and some possibly more.

As many commentators have pointed out, as the noise continues to grow (which it will), the signal to noise ratio drops and the need for us to find mediated experiences will become ever more important. My good friend Dan Zambonini pointed me to this excellent blog post by Kevin Kelly. Here’s a quote:

“I have tried to temper my celebration of the bottom with my belief that the bottom is not enough for what we really want. To get to the best we need some top down intelligence, too. I have always claimed that nuanced view. And now that crowd-sourcing and social webs are all the rage, it’s worth repeating: the bottom is not enough. You need a bit of top-down as well.”

He’s right of course - the lesson that we all take away is that although the technologies get more “intelligent” (dare I say, “Semantic”..?), the noise is probably increasing at a far greater rate. Net result - at least a cancelling-out of the “filtered benefit” and more likely - just more and more noise.

The human author - the topdown influence in Kelly’s post - is the conduit by which everything is managed. This role isn’t going anywhere, but it’s easy to forget this when we’re all getting excited about the machine -processable web, the API, Twitter and so on.

The human element is always going to be the single most important thing in the equation, which is exactly why the social web is so important, and can’t - or won’t - be ostriched.

I’m off to the 2008 Foundation Symposium tomorrow, a day of asking: “what do current Web trends tell us about the future of ICT provision for learners and researchers?…”

The programme and speakers look pretty good - Larry Johnson from NMC and Jem Stone from the Beeb to mention but a couple of the better known names.

Andy, Pete and Ed from the Foundation are trying out a bunch of interesting stuff to support the conference backchannel. For starters they’ve set up a Ning iteration at http://efsym2008.ning.com/ which, as Andy and I point out in the comments, is (at the least) pretty useful in this day and age to put a face to the virtual contacts we’ve all made via Twitter, IM and blogging. Ning for me has really risen in importance as a social media platform to be taken seriously since they solidified their API and developer network…

Next off, they’ll be streaming live video from the event, so if you can’t make it you can catch our ugly mugs on the Eduserv website. On the same page there is also a live chat facility as provided by CoverItLive. I’ve been roped in to pick questions and shout them out during the QA’s after each session. So be nice and I might just let you ask something

Last but not least, OneTag will be in operation with a slideshow and mobile version of everything tagged efsym2008.

All in all, it’s going to be fun. And an early start, so I’m off to bed. See you there (or not) tomorrow.

I just sat through an excellent session by Steph Mastoris from the National Waterfront Museum in Swansea.

They’ve got a 35 million pound museum here, and it’s been built with new media at its core. Some of the interactives here are very cool (wave your arms and spin a virtual object); others are a bit more pedestrian and a couple are, frankly, slightly baffling.

The point - as illustrated by the extraordinary statistic that the museum spends £27k a year (wait for it) on projector bulbs (!) - is that new media is a vital component to interpretation here.

So far, so good. But Steph then pulled up a slide about the issues: changing those lightbulbs, for one - but also the notion of identifying critical paths. And the one example he used (hence the title to this post) was that the cooling system for the museum relies wholly on waterflow from the Swansea marina. So when “15 condoms and a dead dog” (Steph’s words..) get stuck in the water intake, the cooling fails, the servers crash, the museum interactives go down.

The point (apart from a great post title): the critical path for your IT systems may be somewhere you least expect it, and that path might be nothing whatsoever to do with technology…