The best things in life were free

• • 33 Comments

[Cross-posted at Cliopatria.]

The Royal Historical Society has for some years maintained an online bibliography of British and Irish history, updated three times a year. It currently has over 460,000 records. It's a fantastic resource for scholars interested in any aspect of the history of the British Isles, not least because it's free. But from 1 January 2010 it won't be: it will be rebranded as the Bibliography of British and Irish History which will be sold by Brepols, with subscriptions available for institutions and individuals.

This is a shame, of course. A resource which was freely available to anyone with an internet connection will now only be open to those who can afford to pay. Presumably that includes big universities and libraries (although even librarians at Yale, of all places, are complaining that digital resources are getting to expensive, according to this H-Albion post), but what about smaller universities, local libraries, schools, independent researchers? There is the individual subscription, but there's no information about pricing yet and it seems unlikely to be cheap.

The reason for this move is the end of government funding for the bibliography. That's understandable; the money has to come from somewhere. The fact that it has been funded by British taxpayers does raise the question of why a commercial entity should be allowed to profit from that expenditure. But as I'm not a British taxpayer it could equally well be asked why I should benefit from that expenditure. So I don't really have a basis for moral outrage here. It's just ... a shame.

But it seems to me that must be some other way to do this -- crowdsourcing, scraping, some combination of both? There are some sites which show the potential of crowdsourcing by way of people uploading and updating their own bibliographies, such as Librarything, or in a more academic context, CiteULike and Mendeley. Given a critical mass of users, a crowdsourced bibliography would be close to up to date. Scraping could be used to automatically feed in journal articles via RSS (books would be harder -- though maybe not). There are many difficulties inherent in such an approach, but I'd rather see something like this be the future than an ever-increasing array of paywalls.

CC BY-NC-ND 4.0 This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License. Permissions beyond the scope of this license may be available at http://airminded.org/copyright/.

Related Posts:

33 thoughts on “The best things in life were free”

  1. Jakob

    I can't remember where I saw this - I think it was in a discussion of open access publishing - but I seem to remember a number on the order of £5 million was being bandied around for online access charges for one of the larger uni libraries (Senate House?)

    I certainly would like to see open-access publication be a requirement for all government-funded (which in practice means all universtity?) research, at least for journal articles.

    Is the RHS bibliography downloadable in its entirety?

  3.

    I think Brepols and their ilk are the spawn of evil.

    Crowdsourcing the updates to this are the sort of work my MA class in Digital History would be made to do as part of their learning.

  4. Chris Williams

    Like pretty much every UK historian, I have to register everything I write on my university's open access 'aren't we cool?' register of research. This is a Good Thing. Mine is here:

    http://oro.open.ac.uk/view/person/caw322.html

    Given 100 scrapers ( I have no real idea what a scraper is but I can make a pretty good guess. When I was 30 I was on top of all this e-thingy, you know) you could put every publication onto one big list, then crowdsource the editing (is this history? yes/no) to a volunteer collective, and create yr own free-to-air bibliography. It would be really rather easy. Stuff published outside the UK would be harder to find.

    PS - went to the IHR for a briefing from the National Archives today. 10% cuts, charging for car parking (the horror!), closing on Mondays, cutting out the publishing operation almost entirely, and 35 redundancies. Digital services won't be affected. Personally I think that they've made the best of a duff hand, poor sods.

  6. Erik Lund

    10 million CAD a pop? This is the best idea since Encarta! Seriously, way to price yourselves out of the market in the Wiki-age.

    Now, gotta go edit the "Dimensions (Kenneth Bulmer series)" entry.

  7.

    I was going to post about this but you've saved me the trouble. I'm not very happy about it either. This is such a basic and widely useful resource that things must be very bad if the AHRC can't continue funding it. It's useful to anyone working on the history of Britain, Ireland, or any part of the British Empire or Commonwealth. As well as excluding non-academics, independent researchers, and less well funded UK institutions, it could also have a bad impact on overseas institutions.

    A couple of weeks ago I was lucky enough to meet Sandra Swart from Stellenbosch (Google her - she's amazing!), and she was telling me that South African historians just don't get the kind of funding that the ESRC gives to big projects like the one I work for. The SA government can't afford to throw that much money at history because they have higher priorities, like funding AIDS clinics. There are plenty of African countries that are poorer than SA and have bigger problems but whose history is intimately tied up with Britain. Despite current economic problems, Britain is still a very wealthy and privileged country. Why shouldn't we give something back by providing free history resources to the rest of the world? So I don't think "But as I’m not a British taxpayer it could equally well be asked why I should benefit from that expenditure." would be a valid criticism. You work on British history, and this bibliography is relevant to Australian history too. Therefore you have a legitimate interest in it. Putting this resource behind a pay wall will make history more exclusive and elitist. Capitalism and elitism go together, and don't let any right-wing libertarians tell you otherwise.

    But I also agree that crowdsourcing, social networking and folksonomies could render the idea of a central database under strict editorial control with hierarchical categories obsolete. When Zotero 2.0 comes out of beta it should make a big difference as it offer lots of possibilities for sharing, classifying, importing and exporting bibliographic data. So maybe in a few years Brepol will find that they've backed a loser. It would just be nice if the AHRC could keep funding the free site until it actually is obsolete.

  8. Chris Williams

    This isn't going to hit me or my students because my university will subscribe to it. It is going to hit scholars in other countries in a big way.

    Brett, is there a society for Historians of Britain in Oz? There's the British Studies lot in the US, and I already know some of the French who study the UK? That lot, all pulling together, could either:
    1) get the RHS to change its mind (difficult because the UK public sector is about to get hammered)
    2) set up a crowdsourced alternative, coming online in Jan 2010, using Zotero.

    Writers have a strong incentive to get their output indexed in as many places as possible: some software that would scrape exising indices would do the trick. This kind of stuff is not something that I can do myself, but I know some people who know some people...

  9. Chris Williams

    By the way, the RHS' terms and conditions strongly imply that downloading the whole thing would be counter to them. That's fair enough, in my book.

    I have just emailed a number of people involved in British Studies, speculating about the Zotero/OpenURL solution. Watch this space.

  10. Jakob

    I was at the IHR today and spoke to one of their publications people, who said that the hosting costs of the site were such that they couldn't afford to keep it up.

    I do wonder though - how much server horsepower does the site's search engine need? I wouldn't have thought the bandwidth requirement was excessive. And of course the commercial charges will no doubt bear little resemblance to the site's actual running costs.

  11. Pingback:

  12. Chris Williams

    It's always been famously slow, but... I really can't see how bandwidth charges are the problem here, to the extent that they have to start PAYING PEOPLE WHOSE JOB IT WILL BE TO COLLECT RENT. They can provide this via the SAS's existing infrastructure, after all. Perhaps our world-wide opposition to the change might include offering up some mirror sites for them, hosted in universities in NA, Oz and SA, and over Europe?

    I have to say that if the IHR stops providing this for free, one of their major reasons for existing - and being funded to do so by the state - will have taken a remarkably large hit.

  13.  Post author

    Surely commercial hosting could be found for something like relatively cheaply. According to Netcraft they're only running a Windows 2000 server with IIS 5 (though if you click around it also seems they may have just moved to a new server). It's hardly impressive computing power.

    Chris, there's something called the Australasian Modern British History Association, but as far as I know it doesn't exist as an actual organisation, it just hosts a conference every couple of years. I'll email the AHA (the Australian one, not the American one) and see if they can put an announcement out. As Gavin says, it's not just historians of Britain who are affected.

    I agree that we couldn't just download the RHS bibliography and use that as the basis for an open version. But we could reverse engineer it, to make sure it's getting most of the same items as the RHS. The problem with scraping (as I see it, not actually having done any) is the wide diversity of data sources needed -- different web pages at different universities/publishers with different formats ... some standard metadata formats (like OpenURL) might help to a degree, but at some level there's going to have to be ugly, ugly handtweaking to be done. Which is why I'd lean towards crowdsourcing but you need enough people to make that work, and I don't know that there would be enough historians for that.

    The situation in the UK does sound dire. Privatisation seems to be a reflexive response, not a reflective one, however.

  15.

    Maybe we don't need to do much, because the whole point of social networking and folksonomies is that they grow naturally from the bottom up. People will make friends with people who have similar interests, and pick up references from friends and friends of friends. This is what Zotero 2.0 is going to be all about: users can share their collections with other users and form groups for people with similar interests. Zotero already has scrapers for lots of different sites, more are always being added, and anyone is free to develop their own and contribute it to the project (and it's only Javascript, so not that hard). Although no-one has the whole RHS database, many people will have already picked up lots of references from it and have them in their Zotero collections.

  16. Kay Woll

    Because of the pressure exerted by comments on the web, especially H-Albion, the Royal Historical Society will be putting out a detailed explanation of its decision about the bibliography very shortly. As I hear it, the RHS was told about 18 months ago by the Chief Executive of the Arts and Humanities Research Council in the U.K. that, after ten years of funding, there would be no further financial support for the bibliography. The RHS approached a number of British and American universities for help unsuccessfully. That left the RHS with a choice between (a) putting the bibliography on a university website but making no additions to it (b) just like (a) but with some small-scale additions being made or (c) going into partnership with a commercial organisation to keep the service going and to develop it. The RHS chose the last course. So now universities and individuals will be asked to pay to use the bibliography, although Fellows of the RHS will get a reduced rate. We shall see whether this is right and where the RHS decides to comment.

  17.

    Right, I'm exploring the possibilities of scraping the whole thing. It has an option to return data in XML, which makes it easier to process automatically, but the main problem is creating the search requests to get everything. The database primary key seems to be a unique identifier consisting of 9 digits, of which the first 4 may be the year of publication, and presumably the remaining 5 are a serial number unique within that year. However, checking shows that the first 4 digits don't necessarily match the year of publication; perhaps the year it was added? They do seem to be sorted by default by year of publication.

    The crude option would be to generate all possible positive 9-digit ints and query for them all, handling the not found responses, but I can't help thinking it's a bit barbaric and inefficient.

  18.

    Mind you, it might be possible to do something weird with the OAI protocol interface as it supports requests for all records from or until a specified time.

  20. Jakob

    Alex: Would it then be theoretically possible to scrape the entries directly into a database that would give the functionality of the bibliography search locally?

  21. Pingback:

  22. It would, if you want to maintain a half a million record DB locally and keep updating it.

    A big record in it is 1.4KB; if we assume the average is, say, 0.8KB, it's 350.95MB on disk, counting the XML angle bracket tax and not counting the DB overhead.

  23. Jakob

    Well, the updating isn't as critical, as what's most offensive about this is not that the publishing deal is needed pay for new updates - if it costs money, it costs money - but that the old, publically-funded information disappears behind the paywall.

  24. Chris Williams

    I think that the updating is critical. It's also the thing that we can do something about ourselves, or threaten to. All we can do about the old info is ask them not to privatise it. I suggest that we do that as well.

    Of course, if we can crowdsource such a resource into existence, it will lower the demand from Brepols' service to such an extent that they are likely to pull out of the deal.

  25.  Post author

    Gavin:

    Zotero 2.0 does sound tasty. Though I must admit I've never gotten into the mindset of doing a bibliography in a browser extension ...

    Kay:

    Thanks for the update -- I look forward to the response from the RHS!

    Alex and Jakob:

    IANAL, but I'd be very careful about scraping it and turning the result into something usable. It's against the TOS and I doubt Brepols would look kindly upon it. But thanks for looking into the technical side of things, Alex!

    Chris:

    We could ask them not to privatise it, but presumably that's a big part of the attraction for Brepols. And as it appears that this is a done deal, it might be hard for the RHS to change the terms at this late date. But as you said, we can try ...

  26. Kay Woll

    According to Sarah Richardson of Warwick University on H-Albion, an individual subscription to Brepol's Bibliography of British and Irish History will cost £110 plus VAT for the year from January, 2010. That is £129.25p in pounds sterling. Depending on the exchange rate - say at $1.60 to the pound - that would be over $200 U.S. Sarah Richardson thinks few, if any, individuals will be able or willing to pay a charge at this level.

  27. Matt

    There are three different prices (info from H-Albion):

    125 Euros ($175) - An individual license

    700 Euros ($972)- A Standard license, which includes IP access for 3
    simultaneous users and remote access

    1000 Euros ($1,389) - A Campus wide license includes unlimited IP
    access and remote access.

  28. Chris Williams

    So, assuming that 80 History Deartments, at the very least, pay for the Campus version, that's £70,000 heading from UK public institutions towards Brepols - in addition to the money that the RHS and the AHRC are paying towards the project. Looks like a nice little earner, and the majority of the cash would still come from UK taxes.

  29.  Post author

    Yeah, if my institution doesn't pay up at those prices I doubt I would either. Jonathan Dresner's suggestion of a small fee per search would be a good alternative, and it could presumably be offered as well as the other options, even now.

  30. Kay Woll

    H-Albion now has a long explanation from the Literary Director of the Royal Historical Society (Ian Archer), the President of the RHS (Colin Jones) and the Director of the Institute of Historical Research (Miles Taylor) about the decision to transform the RHS's free bibliography into the Bibliography of British and Irish History to be run on a fee-paying basis by Brepols Publishing. It fills out the picture on the funding problems faced by the project and on the considerations taken into account before the IHR and RHS took their recent decisions. However, I do not think either would have been so forthcoming without the coverage on H-Albion or here or on other blogs. I shall be interested to learn what other people think of this explanation and whether any alternative is viable.

  31. Chris Williams

    The explanation is fine as far as it goes, but it doesn't answer the question I wanted it to: which was did they talk to JISC? And if so, what did JISC say?

    They are right to note that we need 'a debate' (ie a change of policy) about the way that there is currently money for development but not for sustainability. We might also check out the value-added (even measured in crude REF terms) of a pound spent on this resource compared to a pound spent on the other things that the AHRC funds.

  32.  Post author

    Thanks, Kay -- here's a link for those not subscribed to H-Albion. It's clear that the RHS carefully considered its options, and maybe commercialisation was the best choice open to it. But a more open consultation process before the decision would have been desirable.

  33. Pingback:

Leave a Reply

Your email address will not be published. Required fields are marked *