Tools

[Cross-posted at Revise and Dissent.]

I stumbled across this by accident: a pilot digitisation of Hansard, funded and operated by Parliament. What an excellent thing! It’s functional, but based only on a subset of 20th-century Hansard material:

What’s on this site? This site is generated from a sample of information from Hansard, the Official Report of Parliament. It is not a complete nor an official record. Material from this site should not be used as a reference to or cited as Hansard. The material on this site cannot be held to be authoritative.

This warning should be heeded — it’s only a prototype and should not be relied upon for any purpose. It’s easy to find omissions, such as Baldwin’s ‘the bomber will always get through’ speech, even though there’s quite a number of entries for the day in question. The text itself appears remarkably uncorrupt, given the volume of data that’s been OCRed: I’ve only found a few errors (most amusing one: the Marquees of Londonderry — I guess it must rain there a lot). There are certainly a few minor problems — for example, once I managed to get the search engine to tell me that a debate in 1958 happened earlier than one in 1944. At present there’s no disambiguation between different people with the same name — so the earliest utterance recorded for Mr. Winston Churchill is on 19 March 1941, and the latest on 11 March 1997 — nor combinations between (possibly) the same person with different names — such as Churchill, Mr. Churchill, Mr. Churchill (by private notice), Mr. Churchill (Stretford) and so on. It’s all experimental at this stage, so these issues will presumably be addressed in future. (LibraryThing lets its users do a lot of the work for similar problems, but I doubt a HansardThing would ever reach the critical mass needed for that to work.)
Read the rest of this entry »

For the last couple of weeks I’ve been using the free preview of Things, a task management application for OS X. I’ve just entered the final year of my PhD — or rather the final year of my PhD scholarship, which may not be the same thing — and so keeping track of everything I need to do is going to be critical. I’ve been looking for something like Things for ages, actually. Nearly all of this type of software seem to be based on Getting Things Done (GTD), a system for task management which is hugely popular, at least among techie types. But I’ve never been able to wrap my head around it, it seems too strict and hierarchical. The applications designed to help you follow it seem just as bad — you’re forced to fill in a bunch of text boxes or select from drop menus or whatever, and it’s all just too annoying for me.

That’s why I like Things, so far — you can fill out as little or as much info as you want for each task. The organisation of tasks is logical (at least to me), the interface is polished but unobtrusive and the program lightweight. It just gets out of the way and lets you get on with things. Apparently it does actually conform to GTD principles, but doesn’t force you to follow it if you don’t want to. The data is stored in an XML file so you can retrieve it if something happens. Tags are used throughout, which is a nice touch. Tasks can be organised by time priority (eg ‘Today’, ‘Next’) or as part of a larger project. When you’ve completed a task, you tick a box on its pane and it will eventually vanish out of sight into a log of completed tasks. It’s probably not the place for detailed notes (I use VoodooPad for that) but works well for jotting down things you need to do, when you think of them.

Things is only a time-crippled beta at the moment, but I’ve found it to be completely stable (there are features which aren’t implemented yet, however, such as collaboration with other Things users). I’ll almost certainly be buying the full version when it’s released; but I have to say the price seems a little steep at US$49 for what, after all, is not a huge program. Being able to get things done is probably worth that; but I’d rather pay US$39, which is the price you can get it for if you sign up to their newsletter before 31 January (which I did a while back and haven’t received a single email yet). Hopefully this doesn’t sound like an ad (NB: I am not connected with Cultured Code in any way), but perhaps there are some Mac users out there who need task management as much as I do right now — if so, Things is worth looking at.

I’ve just found the solution to a little LaTeX problem that has been bugging me for a while. To format my bibliography, I’m using the jox.bst (i.e. Oxford) style of the jurabib package. For the most part, this does exactly what I need it to do. But there are a few glitches. The most annoying one is when I have a BibTeX entry with a corporate author, for example War Office. Jurabib treats this as a personal name and so when it comes to alphabetically sorting the bibliography entries, it sorts on ‘Office’ and not on ‘War’. This puts ‘War Office’ after ‘Noel Baker’ in the bibliography instead of after ‘Turner’, which is where it should be. (Yes, this is the sort of trivia you have to worry about when writing a thesis!)

Actually, that’s not really the problem, or at least, it’s one that all BibTeX styles share. There’s a standard solution, though: put the author name in braces in the BibTeX entry: {War Office} instead of War Office. This tells BibTeX not to break the author name, to treat it as a single token. And jurabib does generally understand this — but not if you use the jox.bst style! If you try to do this with jox.bst, you get an error like this:

! Argument of \jb@lbibitem has an extra }.
<inserted text>
                \par
l.1461 \bibitem[{{W}r Office}\jbdy {1922}}
                                          %
?
Runaway argument?

While it does eventually compile, it does so by mangling the bibliography, so that’s not very useful. It would seem to be a bug in jurabib, or at least jox.bst — and as of April 2007, jurabib is no longer under development.¹ So it’s not going to be fixed. Periodically, I’ve looked for a workaround (as have others), but nothing has worked for me² — until now.

The answer: enclose the spaces between the words of the corporate name in braces! So, War{ }Office instead of War Office. That’s all there is to it, and it works perfectly. I don’t understand why, but I don’t much care either! My thanks go to Carsten Ziegert who posted this solution on the jurabib list.

1. Its developer suggests biblatex as an alternative, though it seems that it’s not yet stable. It does look powerful though; and I see that one historian is already using it.
2. Double quotation marks are also supposed to work, but don’t.

While I’m on the topic of Things to Come, I should correct a mistake I made in the talk I gave at the summer school. I said that Things to Come didn’t do particularly well at the box office. I still haven’t found any actual figures for that, but I’ve found what may be better, a ranking of its popularity out of all films shown in Britain in 1936. It turns out it was the 9th most popular film that year, out of over a hundred shown, so obviously it should actually be counted as a success. (Given that it was also an expensive film to make, it may not have turned much of a profit, if any, and that may have been what I was thinking of.)

This information comes from a very interesting exercise in quantitative history, John Sedgwick’s Popular Filmgoing in 1930s Britain: A Choice of Pleasures (Exeter: University of Exeter Press, 2000). What Sedgwick did was take a sample of cinemas and go through their programmes to see how many weeks each feature film was shown for, and whether it had first or second billing, to be used as a weight. He also came up with a weighting for each cinema, based on its capacity to earn revenue (more seats and/or higher ticket prices means more weight). The number of weeks a film was shown for at a given cinema is then multiplied by the billing weight and the cinema weight, and this number was summed across all cinemas the film was shown at, to arrive at a popularity statistic, POPSTAT, for the film. Just in case that explanation failed to confuse you, here’s the equation defining POPSTAT, from p. 71 of his book:¹

To the extent that POPSTAT actually means something, I suppose it is the potential total earnings of a film, and this in turn reflects the judgement of cinema managers as to whether cinema patrons would actually come to see the film, which in its turn would have been based upon how well the film was actually doing (ie, is it worth keeping it on for another week?) So in the end, assuming that cinema managers were responding to market forces, POPSTAT does indirectly measure something of a film’s popularity.² For the record, Things to Come has a POPSTAT of 40.65, just behind Fred Astaire and Ginger Rogers in Swing Time (40.95 — so close as makes no difference) but comfortably ahead of the Dickens adaptation, A Tale of Two Cities (34.18). The most popular film of the year was Charlie Chaplin’s Modern Times (83.26). Most films in the top 100 had POPSTATs in the teens. (The results for 1934-6 are actually online as an appendix to a seminar given by Sedgwick.)

And if you don’t trust all that number-crunching, then here’s one data point Sedgwick mentions, relating specifically to Things to Come: its run at the Leicester Square Theatre (where it premiered, as it happens) was 9 weeks, with the longest run for that cinema in 1932-7 being 11 weeks. So, I think it can safely be said that it wasn’t a flop (contra me). I stand by my other point, however, which was that Things to Come is actually very singular, at least in British feature films: there are very few depictions of a city being turned to rubble by air attack, as in the clip in the previous post. In fact, I don’t know of any. So however successful Things to Come actually was — and it should be remembered that this may have been due more to the visually stunning scenes set in 2036 than the more depressing scenes set in 1940 — it’s not something film producers rushed out to emulate.

1. You can create your own using a LaTeX-based generator. Try it, it’s fun!
2. The exact numbers should be taken with a grain of salt — I doubt four significant figures can be meaningful with such a dataset. One important caveat is the cinema sample. Not every cinema in Britain is used but only a selection of West End and first-run provincial cinemas. But unless films were markedly more popular in their second runs, I don’t think this would matter too much.

New Popular Edition Maps is an attempt to produce a copyright-free database of British postcodes. It does this by asking people to hunt around on a clickable, zoomable map of the UK for places for which they know the postcode (e.g. their home), and then enter that postcode at that spot. It’s a bit like a stripped-down Google Maps; and you can search the map by placename or postcode. But what’s interesting about this is that the maps used are out-of-copyright Ordnance Survey maps (1 mile to the inch) from the 1940s and early 1950s, which could be useful for historians or teachers, though these are obviously not the intended audience. Unfortunately Northern Ireland and most of Scotland is missing. (The National Library of Scotland has the OS maps of Scotland from the 1920s.)

Finding this inspired me to do a bit of a search for other online historical maps of Britain which similarly attempt to cover the whole country. (There’s a useful list of out-of-copyright maps here.) Old-maps.co.uk has been around a while and uses OS maps from the late 19th century. Vision of Britain (which site has lots of historical statistics which you can slice various ways, and which I must explore more thoroughly one day) is more sophisticated, and has a neat trick of switching between different maps depending upon the zoom level: for example going from a 1921 large-scale map to a 1904 OS one to a NPE map. It also has 19th-century maps and a 1930s land utilisation map. But possibly the most interesting is Old Ordnance Survey Maps, which is based upon OS maps from the 1910s, 1920s, 1930s and 1940s. The coverage is very much incomplete; but it uses the Google Maps API, which means that it has a familiar interface for users, and could be used for mashups. It already overlays the regular Google Maps satellite and street maps. There are also handy links to take you to the same location at old-maps.co.uk and Vision of Britain. I can think of some improvements (for example, printing the publication date on each map) but this approach has tremendous potential.

Last year I was playing with a plotting program for Mac OS X, which was pretty good, but not quite satisfactory. I’ve found a better one, Plot, which is free (as in beer), fairly easy to use, and very customisable. It has its own idiosyncrasies, but I like it a lot. Here’s an example plot, showing how the top speed of British combat increased up to the end of the Second World War.

The data are drawn from John W. R. Taylor, Combat Aircraft of the World From 1909 to the Present (New York: Paragon, 1979). This excludes aircraft which never saw service as well as those not intended for combat (though not all actually saw combat). The year is that in which it entered service (usually with the RAF), or if this wasn’t given, the year when the prototype first flew. (Some aircraft unfortunately had neither, and so were omitted.) The maximum speeds, in miles per hour, are not necessarily comparable, because they were often obtained at different heights; also, they may not have been sustainable under normal conditions. But they should be broadly indicative of real-world maximums. I’ve classified each aircraft as either fighters (red) or bombers (blue), based upon their actual use. However, that’s fairly arbitrary for the period up to 1915, which is when aircraft adapted for specialised roles began to appear. I haven’t included seaplanes but I have included carrier-borne aircraft. Generally, I have only included data for the most representative version (eg not for each of the innumerable marks of Spitfire). Because of these caveats and inconsistencies, the plot should not be taken too seriously — it’s just for illustrative purposes.

Read the rest of this entry »

My laptop is my primary workhorse, and I’ve just upgraded — a very exciting time in any computer geek’s life! On the left, my old 12″ 1.0 Ghz G4 Powerbook, “zeppelin”; on the right, my new 13″ 2.0 GHz Core Duo MacBook, “hendon”. Zeppelin has been a rock-solid little machine for me these last couple of years, but it was starting to lose pace with my needs. Switching over to hendon been a very smooth process (other than getting Instiki to work again), and it’s just so nice and fast — it’s better in every way (except for the size, I prefer the smaller formfactor). It should see me through me through the rest of the PhD in style.

A useful site about digitising your trip to the archives: Electronic Researcher. It was mentioned in a H-ALBION thread about which digital cameras are best for use in archives, and which archives allow them (British Library no, National Archives yes). I wish I’d found this earlier, as I have already bought a camera for this purpose, but I think it will be OK.

A few months back, I posted about my decision to use LaTeX for writing my thesis, in preference to Word or something of that ilk. I seem to get a few Google hits from other people interested in using LaTeX in the humanities, so I will occasionally post useful things I’ve gleaned, even though it will be of no interest to most readers …

So here’s one. In theses (and monographs), historians generally separate their bibliographies into different sections for the different types of sources — for example, “Primary sources” and “Secondary sources”. It wasn’t obvious to me how to do this in standard LaTeX/BibTeX, which just puts all of your references into a single bibliography.¹ So, last night while procrastinating, I went looking for the answer and found it. There are several options listed there, but the only one I tried was the multibib package, and it works just fine for me.

It works like this: in the preamble, after calling the package,² specify the name of each bibliography you need, along with a unique (and preferably, short) identifying key. For example, to make separate bibliographies for primary and for secondary sources, you might do the following:

\usepackage{multibib}
\newcites{pri}{Primary sources}
\newcites{sec}{Secondary sources}

The \newcites command takes the existing citation commands (eg \cite) and redefines them so that there is an equivalent for each of your bibliographies (in this case, \citepri and \citesec). You then use these instead of the standard citation commands:

This is a sentence about a primary source.\footcitepri{aston:1914} And this one refers to some specific pages in a secondary source.\footcitesec[1-5]{bialer:1980}

\newcites does the same thing for the bibliography commands, so at the end of the document (or wherever you want to place them) you would have something like this:

\bibliographystylepri{jox.bst}
\bibliographypri{all.bib}
\bibliographystylesec{jox.bst}
\bibliographysec{all.bib}

Then you run bibtex, as you would normally do, but now you have to run it once for each bibliography, eg:

% bibtex pri
% bibtex sec

Then latex it up again a couple of times to get the references right (again, as you normally would) and voila:

Shiny.

With a standard bibliography TeXShop can bibtex it for you, but it appears not to know about multibib, so you have to do it from the command line (not a big deal for me as I always have several Terminal windows open anyway). Apparently iTeXmac does do multibib, and a lot more besides, but for the moment I am happy with TeXShop so I haven’t tried this yet.

More information about multibib can be found here.

1. For that matter, I’m not sure how to do it in Word/Endnote either; I usually ended up cutting and pasting by hand. I’m sure there must be a better way!
2. Note that if you also use the jurabib package (and if you are writing in the humanities, you almost certainly are, or should be), you need to call that first, before multibib.

In a previous post I wondered whether the authors of the 1934 knock-out blow novel Invasion from the Air, Frank McIlraith and Roy Connolly, might have been left-wing, as the artist who (apparently) was supposed to illustrate the book was a communist. I hadn’t been able to turn up any biographical information about either of them in the usual places (eg Oxford DNB or Who’s Who). But thanks to the magic of the Internet I’ve tracked down Connolly, in 5 easy steps!

The first breakthrough came when I looked for other books by McIlraith or Connolly in the British Library catalogue, and I found one called Southern Saga (1940), which was published by the same company as Invasion from the Air - which is suggestive but not conclusive. Then Google led me to “New literatures” in The Year’s Work in English Studies which in turn led me to “Literary imaginings of the Bunya” from the Queensland Review, then “The Making of a Queensland Politician: Jack Duggan’s life before parliament 1910-1935″ from the Journal of Australian Studies, and finally I used the very handy AustLit (subscription only, unfortunately) to confirm that it was indeed the same Roy Connolly who wrote both Invasion from the Air and Southern Saga. So it turns out that Connolly was not British at all, but an Australian! He was the political journalist for the Queensland Labor Party’s Daily Standard in the 1930s (and so it is probably safe to assume that he was a Labor man himself). What he was doing writing air-scare literature for the British market, I have no idea, but if I can scare up a biography of him it might give me more of a clue. (I tried the Australian Dictionary of Biography today at the library, but naturally the volume I needed was not on the shelf.)

It makes me wonder how I would have found this out 15 or even 10 years ago, before masses of this sort of information became available on the Internet. Even if I’d thought to check non-British biographical dictionaries, there’s still no guarantee that I would have found Connolly, and without any clues I wouldn’t have known where else to look. In the end I probably would have given up: it’s not really all that important and there would be better things to spend my time on. But now, thanks to Google and other resources, this kind of sleuthing is both painless and fast - in fact, I spent more time writing this post than I did on the search itself!

Patahistory notes this horror story about a student having her USB drive stolen - and with it, her only copy of her nearly complete PhD thesis. Although she did manage to recover the drive, Dave suggests that this is a timely reminder to make backups. Absolutely! I work as an IT manager in an academic environment, and I’ve seen enough disasters and near-disasters to take backups very seriously. Here’s my advice on the subject:

• Back up often - at least weekly. The longer you leave it, the more work you will have to do over in order to get back to where you were.
• Get into a routine - even if you haven’t written much, back it up anyway, instead of just doing it when you think of it. (This minimises the chances of you forgetting to do it the one time you need it.)
• Backups should be easy to do - or else they will tend not to get done (unless you are more disciplined than I am!) Automate them, if possible.
• Check your backups periodically, to make sure that they are actually backing up correctly, and are not corrupting over time. There’s no point in having them if you can’t read them when you need to!
• Have a few different backup methods, for redundancy. Keep some away from your computer - emailing copies is a good idea, as Dave suggests. Or make physical copies and leave them with your parents or friends, or archive them online (eg Gmail or a service like Strongspace).
• Be paranoid! You can never have too many backups. You’ll probably never need them, but just think about how devastated you would be if the unthinkable happened, and you didn’t have any …

My personal backup regime is probably unnecessarily sophisticated - use CDs, USB drives, email, whatever works for you. I have a network-attached hard drive at home, and automatically write a backup to it from my Mac every hour (via a cron job - though I just make tar archives instead of the utilities mentioned there. When I’m travelling I will probably modify this to write smaller backups to a USB drive). Then I make a CD backup every week, which I take to work and leave in my desk drawer. When I start writing the thesis itself, which I am actually about to do, I might start uploading it to my web hosting server on a daily basis … it’s on the other side of the world, so if Australia slides beneath the waves, I can still get my PhD!

From the just-because-I-can department.

As an ex-physicist, I like to see numerical data plotted in a graph, as well as in tabular form - it’s much easier to visualise what’s going on. I don’t have any particular need for this right now, but I’ve been playing around with a few plotting packages anyway. The figure above was made with pro Fit (OS X only), which has a free trial version, limited in the number of graphs, data points, etc, that can be in use at one time. It’s easy to use and the end result is pleasing enough to the eye. The main problem I found is that the legend isn’t a separate object to the graph, so I can’t shift it to make room for a longer axis label. But I like it otherwise, so I think I will stick with it for the moment.

The data itself is taken from the tables in the back of John James, The Paladins: A Social History of the RAF up to the Outbreak of World War II (London and Sydney: Macdonald, 1990) - tables 5 (for the Air Estimates, ie the Air Ministry’s, and effectively the RAF’s, budget), 9 (UK squadrons only) and 15 (from which I derived the number of squadrons in 1939). A few remarks: the number of squadrons tracks the budget fairly closely. I would have expected there to be a year or two lag, because as James points out, men have to be trained, aircraft orders placed and land for airfields purchased well in advance of a squadron coming into being. I guess the squadrons may not have been effective initially, though. Secondly, despite the deterrence policy of Trenchard’s RAF, and the authorisation of 35 bomber to 17 fighter squadrons for the Home Defence Air Force in 1923, there were actually slightly more fighter squadrons than bombers right up to 1935. Finally, the graph shows how weak the RAF was in fighters at the time of Munich in 1938 (and just plotting raw numbers actually understates this, as Fighter Command mostly had obscolescent types at the time).

Addendum: I forgot to mention that James doesn’t say if the Air Estimate figures are in adjusted pounds or not - so I assume they are not.

Like about half the historioblogosphere,¹ I’ve been playing with LibraryThing (where I am airminded, naturally enough). Well, more than playing - I’ve added just about all my books (even the dodgy pseudoscience and pseudohistory ones - I’m a paid-up skeptic, I swear!) and made a first pass at tagging them too - everything from history and science fiction to Mars and Cornwall.

I’ve also added the blog widget to my sidebar, so that it will display an ever-changing selection of books from my collection. At my request, LibraryThing’s creator Tim Spalding added the ability to show random books from a selected tag only (in this case, history). Thanks Tim!

1. If Google is any guide, that word is original to me. I’m not proud of this.

As befits a self-respecting Unix geek, I’ve pretty much finally decided that I will write my thesis in LaTeX, and not in Word (which is what I have been using for the last few years). I am a bit nervous about this. Most historians, I’m sure, have never heard of it, and indeed the typical LaTeX user would be working in the sciences (which is where I first learned to use it, many moons ago; among other things, it’s great for equations). There’s not a lot of support for using LaTeX in the humanities.¹ The Astrophysical Journal may prefer papers to be submitted in LaTeX format, but the Journal of British Studies probably wouldn’t have any idea as to what to do with such a beast.² Since none of my colleagues will know how to use LaTeX, it’s next to useless for any collaborative work. But all that is get-around-able, because I can switch back to Word if need be. The big problem, though, is bibliographical management. EndNote can’t work with LaTeX in the same way as it does with Word. That means I either enter and format all citations by hand (urk), or use BibTeX-oriented software. Which is fine … except if I ever decide I want to go back to Word/EndNote, either temporarily or permanently, then my bibliography will be in BibTeX, which of course Word can’t handle. It is possible to convert from EndNote to BibTeX and vice versa, in theory, but in my experience this isn’t very unreliable. EndNote can export directly to BibTeX, but the resulting file isn’t readable; I had better luck exporting to RefMan (RIS) format instead. Unfortunately, for some reason this abbreviates authors’ first names to just their initial, so I will have to key those in by hand.

So much for the pain. What’s the pleasure? Well, for one thing, the result looks so much better than Word. It is very easy to produce a beautiful document in LaTeX. It’s the kerning … the justification … it’s just the vibe. More importantly, LaTeX separates form from content. When writing in Word, I find that I get hung up on how the thing looks, and distracted by trying to massage its appearance. In LaTeX, you just write, and worry about that stuff later. And when producing large and complex documents (like a PhD thesis!), LaTeX comes into its own: when you do need it, you have the power to specify exactly where to place that table on the page - whereas Word will put it wherever it thinks best and you have little say in the matter. In fact, LaTeX can be (and is) used to typeset entire books. The other main advantage as I see it is that LaTeX files are just plain text files, where Word uses a binary format. Which is stupidly easy to corrupt.³ This is the safest and most portable format around, and it helps that LaTeX is available for Windows, OS X and your various Unices and Unix-work-alikes. (For more comparisons, see here (with pictures!) and here.)

OK, but just what is LaTeX? It’s actually not strictly comparable to Word, because it’s not a word processor: it’s essentially a markup language, like HTML. So for example, in HTML the first sentence in this paragraph would be written like this:

OK, but just what <b>is</b> LaTeX?

In LaTeX, the equivalent is:

OK, but just what \textbf{is} LaTeX?

And so on. Then you run ‘latex’ on the document in order to produce the output (these days, generally a PDF file) - just as a web brower parses a HTML page. There’s a handy cheat sheet here, and a useful collection of installation and usage links here.

Frankly, LaTeX is hard to get the hang of, especially coming from the WYSIWYG world, and typing out the various commands is a bit tedious. But there are tools which make the process a lot easier (and this is the biggest improvement from my days as an astrophysics postgrad, when I used vi exclusively). I’m on OS X, and my favourite LaTeX editor is TeXShop, but there are others. To manage my bibliography, I’m using BibDesk (and for the humanities, the jurabib bibliographic package is a must - specifically the Oxford style, jox.bst, as Chicago support is poor).⁴ I’m currently going through my ex-EndNote bibliography, fixing up the first names and adding keywords (PRImary/SECondary, OWNed/LIBrary/UNSeen) as I go. This will be a good thing to finish, because I had been deferring adding new entries until I made a decision to go to LaTeX/BibTeX or stick with Word/EndNote, and instead writing them down in little text files here and there, and it was all starting to get away from me!

So is this a good idea? Come back in three years and I’ll tell you …

Update: for some reason, I’ve re-edited this entry about 10 times since posting it. The most important thing I forgot to mention is that all of the LaTeX/BibTeX tools mentioned above are free - an important consideration for postgrads! LaTeX is open source software, and pretty much all the related tools are too, though I think there are some commercial LaTeX editors.

1. Though there are in fact some users in the humanities, as the comments to this Crooked Timber post show.
2. Though actually, it seems that most history journals only accept paper manuscripts. How quaint!
3. To be fair, this seems to happen much less often than it used to.
4. MAKEBST might be another way to go.