Russ Garrett

Snowy Great Britain

A few days ago, a beautiful satellite photo of snow-covered Great Britain got quite a lot of press coverage (and garnered me a couple of hundred retweets). The image was taken with the snappily-named MODIS camera which flies aboard two NASA Earth Observation System satellites: Terra and Aqua.

Turns out these are pretty neat pieces of kit — they actually record images in 36 frequency bands ranging from blue to thermal infra-red. The two satellites are in a sun-synchronous polar orbit which means they each record an image of the whole earth every day: Terra in the morning and Aqua in the afternoon.

All the image data is collated and released into the public domain by NASA on several sites, but the most interesting outlet for the data is the semi-realtime site here.

It struck me that it would be pretty cool to have the most recent image on your desktop, updated twice a day. So I wrote a slightly hacky little script to do it. NASA provide georectified true-colour images for a selection of regions, including most of Great Britain, so I’m just co-opting that. It does mean I don’t have a full image of GB, but my screens aren’t really the right aspect ratio for that anyway.

Here’s an example of the image it produces (for my dual 19″ monitor setup at home).

Here’s the script.

I’ve just learned that the MODIS satellites constantly downlink their imagery in the clear, so I think the next step is to build a receiver and grab the data directly ;).

Tags: modis, nasa.

A couple of weeks ago, I dropped by the Open Hardware Conference in my capacity as hackerspace founder and general interested hardware hacker. I’m a bit disappointed that I didn’t stay longer — I had to split my time with ScaleCamp — since there was a really interesting mix of people in attendance.

One of the companies present, and indeed sponsoring the event, was 40Fires, who are taking on the audacious task of building an open source car. There was a lot of discussion about how to get more people involved with this project, especially considering that the CAD software used to design the car costs £20,000 per seat.

There was discussion about the best way to organize a community to build an open-source car. The Apache project was mentioned several times as an example of a project to emulate. The point was raised that most open source developers are “scratching their own itch” (which I don’t believe to be true — I suspect most open-source code written today is by people employed to do so).

The point which everyone missed (myself included) is that if you want to emulate the success of the open source software community, you can’t just decide how things will work in a top-down fashion. And these guys weren’t just talking about designing the hardware top-down, they were talking about designing the community. Open source hardware is sufficiently — radically — different from open source software that we can’t just copy the community patterns wholesale and hope they work — that’s not even a great idea for a software project. We have no idea of the best way to build an open hardware project of that size. It’s never been done before.

(Of course this isn’t even considering the engineering quality benefits that bottom-up design brings. See Richard Feynman’s brilliantly prescient appendix to the Challenger disaster report, comparing the design of the Space Shuttle’s software to the design of its hardware.)

Linus Torvalds didn’t write Linux in order to build an enterprise, server-class operating system. He started it as a toy project and the community built up around it. Open source software has evolved organically, with the tools used to build the software evolving with it. Crucially, as Richard Stallman would repeatedly remind you, Linux would have not even been born if it wasn’t for the open-source GNU toolchain used to build it. The toolchain completes the feedback loop which is essential for open source to thrive.

Attempting to build an open source car when we don’t have a capable open hardware toolchain is jumping the gun. That doesn’t just mean decent CAD software, although that’s certainly got to be a priority, but also open source manufacturing tools. The RepRap and associated commercial versions are a great indication that this is really possible — now let’s see an open source milling machine.

So, let’s start building what we can build with our open source tools today. That feedback loop which fuels open source software, and the RepRap wants to bring to the world of hardware, is just starting to kick in. Our Makerbot Cupcake is in the post, so come join us. It looks to be an exciting new decade.

Tags: 40fires, openhardware.

Back in February, Jonty and I started the Hackspace Foundation to provide a legal structure for our efforts to create a hacker space in London. I’m going to try and document this process to make it a little less daunting for other organizations (and because people keep asking me). This is naturally very UK-specific.

Types of Organization

The Hackspace Foundation is a membership association — the company is controlled and run by its members. The easiest way to set one of these up is to start an unincorporated association, which is basically just a group of people who have registered a business name, with which you can create a bank account. This is the easiest way to go about creating an organizational structure (and One Click Orgs is making it even easier), but unincorporated associations can’t enter into contracts, and therefore can’t sign a lease or get a loan.

In order to be able to do that, you need an actual legal structure — a limited company. There are two types of private limited company: Limited by Share Capital and Limited by Guarantee. Limited by Share Capital is the structure most profit-making companies have. Companies Limited by Guarantee (CLG) are the other option, and that’s what we used with the Hackspace Foundation. Most non-profit companies, including charities, are CLGs. Instead of having shareholders, a CLG has members, all of whom are liable to contribute a nominal amount (£5 in our case) if the company goes under.

In addition to incorporating as a CLG, there are certain other actions that can be taken to make sure people won’t profit from a company. Doing this will not only reassure prospective members, but may also help with grants and taxes.

A quick word about objects

The objects of a company are the purpose under which it trades, and are recorded in the company’s Memorandum of Association. Trading outside of the objects of a company is illegal. Starting in October 2009, a company no longer needs to have objects, however restricting the objects of a company to educational non-profit aims may help with getting grants and reducing tax.

Section 30 Companies

A Section 30 company refers to a company incorporated in accordance with section 30 of the Companies Act 1985. This rather odd section refers to the ability to omit the word “Limited” from the company name. However, it also adds restrictions on the objects you can use:

(3) Those requirements are that—
(a) the objects of the company are (or, in the case of a company about to be registered, are to be) the promotion of commerce, art, science, education, religion, charity or any profession, and anything incidental or conducive to any of those objects; and
(b) the company’s memorandum or articles—
(i) require its profits (if any) or other income to be applied in promoting its objects,
(ii) prohibit the payment of dividends to its members, and
(iii) require all the assets which would otherwise be available to its members generally to be transferred on its winding up either to another body with objects similar to its own or to another body the objects of which are the promotion of charity and anything incidental or conducive thereto (whether or not the body is a member of the company).

Section 30 companies are also exempt from sending details of their members to Companies House, and so are ideal for membership associations. The Hackspace Foundation is incorporated as a Section 30 company.

Community Interest Companies

Community Interest Companies (CIC) are a relatively new innovation which takes the Section 30 idea a bit further. Converting a CLG into a CIC is a one-way process which adds a statutory “asset lock” to the company’s assets. The company can only transfer its assets to another body for less than their market value if that body is also a CIC. This differs from Section 30 because a company can convert themselves back to a standard CLG from a Section 30 company (although this still requires a vote of all the members).

We haven’t gone down the CIC route with the Hackspace Foundation because we’re not ready to make that much commitment to our community-only business model just yet. We were also concerned about finding a relevant CIC to donate any remaining proceeds to if the company was wound down.

Charities

For completeness, I’ll just mention a few things about charities. Obviously being a charity is a bonus because donations are tax-deductible. However, charities are required to have a public benefit, and we don’t believe that hacker spaces necessarily pass that test. (The rules are quite complex.) Additionally, charities are required to submit more complex audited accounts, which are more costly.

Hopefully this is helpful to someone. We registered the Hackspace Foundation with UKPLC — they are cheap and very helpful, so I would definitely recommend them.

.

Yesterday morning I went to Help us Free London’s Data on the top floor of City Hall. Organized by the GLA, it was intended to get feedback from the development community about their upcoming Datastore project, which is slated for launch in January 2010.

The people from the GLA admitted that they don’t produce that much interesting data, and have limited influence over the authorities who do (like TfL and the borough councils), so their plan is to set a precedent. I don’t agree that all the GLA’s data is boring — I think that transparency and accountability for our elected representatives is important, even if it isn’t as alluring to everyone as the elusive transport data.

Discussion was mainly about the mundane details of how to go about publishing the data. Thankfully pragmatism prevailed and the anticipated hour-long semantic web ontology discussion/flamewar didn’t materialize.

I was impressed by the amount of grassroots support open data appears to have within the GLA — there were representatives of several departments in attendance — and they definitely seem to be taking the right route by publishing data when they can. I’m really interested to see what they come up with in January.

My biggest annoyance was that they started at 10am on a Saturday. Hackers don’t do mornings, guys.

.

This is the letter I’ve sent to my MP, Emily Thornberry, about the government’s current plans to disconnect file-sharers. You should write to your MP too. It’s important.

I’ve put it up here because sometimes it’s nice to have something to base your letter on when contacting your MP. That said, please don’t copy and paste this, MPs want to hear that you’re a real person and not some mindless sheep. So write your own damn letter. This is mine.

Dear Mrs. Thornberry,

I’m writing to you to express my concern about the new Digital Britain proposals to disconnect internet users who are accused of violating copyright.

These plans amount to taking disproportionate action against internet users for a crime they have not been found guilty of committing. Internet access is a vital utility, and disconnecting users due to alleged copyright violation would also cut off access to many other services, contrary to the government’s plans for universal access to broadband.

The recording industry is backing these heavy-handed tactics in preference to making their licensing rates more reasonable for small music streaming and download businesses. I believe the only viable way to effectively reduce music copyright infringement is to make music available on demand as a service. Services such as Spotify are doing this, but they are struggling to negotiate sensible rates from the music industry.

I’d appreciate it if you indicated your support for Tom Watson’s Early Day Motion 1997 which concerns these file-sharing plans.

Yours sincerely,

Russell Garrett

.

Lots of people ask me what the goal is.

The goal is for you to be able to play any song, anywhere you are, in CD quality, for less per month than the price of a cd. If you’re too cheap to pony up, then you can listen to ads instead, but no more than 10m worth for every hour you listen. A light DRM in place is fine provided the technology exists to stream this anyplace you are. Who wants or needs downloads if you can just stream it whenever you want. Disks are so overrated. If the tech isn’t there do make that happen, then screw the DRM and let those that will pirate pirate and those that will pay pay. You’ll never get them to behave otherwise anyway.

That’s the goal. First person to make it happen wins everything.

This quote was written by Tom Pepper at Nullsoft about a decade ago. Today it became a reality (give or take the ads part, in certain territories, terms and conditions apply…).

Now let’s see if they do actually win everything. They’ve got my tenner a month.

Tags: music.

We use arrays of booleans – a non-standard Postgres feature – quite frequently for storing permissions and similar data. It’s an elegant way to denormalize potentially scary schemas, as long as your client libraries support them.

We needed a function to count the number of true values across a whole table’s worth of boolean arrays – basically a histogram of how many true values in each array field. I thought it would be useful to share because I use PL/PgSQL so occasionally I always forget how to write it, and examples are useful.

Here’s the code:

CREATE OR REPLACE FUNCTION boolean_array_count(INTEGER[], BOOLEAN[]) 
RETURNS integer[] AS
$BODY$
DECLARE
	r INTEGER[];
	size INTEGER;
BEGIN
	IF $2 IS NULL THEN
		RETURN $1;
	END IF;
	size := max(coalesce(array_upper($1, 1), 1), array_upper($2, 1));
	FOR i IN 1..size LOOP
		IF $2[i] = true THEN
			r[i] = coalesce($1[i], 0) + 1;
		ELSE
			r[i] = coalesce($1[i], 0);
		END IF;
	END LOOP;
	RETURN r;
END;
$BODY$ 
LANGUAGE 'plpgsql' VOLATILE;
 
CREATE AGGREGATE boolean_array_count (BOOLEAN[]) (
	SFUNC = boolean_array_count,
	STYPE = integer[]
);

You also need an implementation of the inexplicably missing max(integer, integer) function:

CREATE OR REPLACE FUNCTION max(integer, integer) RETURNS integer AS
$BODY$
BEGIN
	IF $1 > $2 THEN
		RETURN $1;
	END IF;
	RETURN $2;
END;
$BODY$ LANGUAGE 'plpgsql' IMMUTABLE STRICT SECURITY DEFINER;

.

Earlier this week, I handed in three months’ notice – I’m leaving Last.fm at the beginning of October. It’s been more than six years since I started working on a small project called Audioscrobbler with RJ, and since the founders’ leaving I’m now the longest-serving member of staff. It’s been massively fun and Last.fm is still an awesome place to work, but I think it’s time for a change.

That said, it’ll be nice not to work in a company which is constantly subject to perplexingly arbitrary rules and unrealistic royalties set by increasingly out-of-touch record labels.

It feels like a lot of my work in the last few years has been trying to prevent myself from being such a single point of failure. It’s quite a relief to be in a situation where I can finally leave.

As for what I’m doing next; I’m going to work for Smarkets heading up tech operations. I think it’ll be quite a different challenge to Last.fm, and it’ll be great to work for a proper startup again. It should be fun.

(If you’re interested in coming to work for Last.fm, we have a couple of vacancies in our ops department on a variety of levels.)

Tags: lastfm, personal, smarkets.

I’ve been reading Larry Lessig’s book Remix over the last few days, and one thing which struck me is his comparison of sharing economies against commercial economies. A sharing economy is something such as friendship, love, Wikipedia, or Linux, where no money is exchanged; A commercial economy is where money is exchanged.

Lessig notes, with lots of examples, that even attempting to compare a sharing economy with a commercial economy will end up in trouble. In sharing economies, “price is poisonous”:

You have, or have had, or will have, lovers. That relationship exists within a complex, sharing economy. The statement “Wow, that was great. Have $500″ isn’t gratitude in such relationships. It might be perversion, but if not matched by perversion on the other side, it will likely be terminal to the relationship. [...] Prostitution is sex within a commercial economy. Both sides seek the simplicity of cash.

At this point, something rang true for me. Way back in 2004, RJ and I were working on Audioscrobbler, while Martin and Felix were working on Last.fm.

Audioscrobbler was naturally a sharing economy: you contributed your listening data, and in return you received recommendations for new music and people who shared your taste. Last.fm’s premise was to make a personalized radio station. This unashamedly exists in a commercial economy, even if the price was free for several years.

In 2005 we merged these two sites — both sites were doing almost the same thing — and last month the shit hit the fan. We had created some kind of hybrid commercial/sharing monster, supported for the time being by investment and ads. Now nobody has any money to throw at anything, so we have to ask for money in an economical way: only asking for money in places where it makes up for our ads shortfall.

Sure enough, when we went from apparently being a sharing economy to charging for the bits which cost us the money, all hell broke loose. Despite this, in the countries we are charging, we’re running the equivalent of the good old Audioscrobbler service, plus a music streaming service with an identical, country-specific, business model to Spotify.

I wonder how many other startups which seem like sharing economies are going to have the same problem as Last.fm: staying true to their users’ impressions of the service and going under, or being forced to charge and potentially losing their users’ trust.

You can’t win, sometimes.

.

Although I work in the Operations room at Last.fm which has its fair share of monitoring screens, sometimes you’ll see me wishing for the good old days: when men were men and control rooms had hundreds of gauges, dials, and lights.

A while ago I decided to take a small step towards that reality, so I hit up eBay and picked up a variety of old-school Bakelite panel gauges (many of which didn’t work). Throw in an Arduino, a maze of wires and some resistors, and I had myself a plan.

I’ve now had this setup for a number of months, and I have to say it’s massively useful. The gauges show a variety of key metrics on how Last.fm’s architecture is performing within a couple of seconds of realtime. That’s a lot quicker than our main monitoring systems which tend to take between three and five minutes to notice something going awry. The gauges are easy to read at a glance, and I generally notice if something’s going horribly wrong because I can see the gauges twitch out of the corner of my eye.

If you’re interested, you can get hold of the (Ruby and Arduino)  code which powers the gauges here on Github, complete with a lame ASCII art diagram. When I get round to it, I’d like to get them mounted onto a nice wooden panel, possibly with some warning lights.

I’ve got a while to go before I can rival Battersea Control Room A, though…

Tags: arduino, monitoring, operations, sysadmin.