Putting the AI into Airminded

Recently I've been playing around with AI-generated images. This is far less impressive than it may sound: there's a small community on Twitter and elsewhere doing this stuff already, many using scripts and tutorials which mean you don't need any more skill than the ability to log in to Google Colab, type in some keywords and hit execute. The particular AI model I'm using is VQGAN+CLIP. The AI doesn't 'know' anything about anything, to begin with, but (as I understand it) it trains from a huge image dataset drawn from the internet (imagenet_16384 seems to work best for me) and uses the associated text metadata to iteratively generate images which could be described by your keywords. You can also try starting from (or aiming towards) a selected image (which I haven't tried yet). I let them run for 500 iterations which seems to be enough to converge to something stable.

The results are usually almost, but not quite entirely, unlike whatever it is that you have in mind: not so much an uncanny valley as a whole uncanny landscape with uncanny hills, uncanny trees, uncanny streams, and uncanny clouds. (Actually it does very well with clouds.) I've got a thread going on Twitter of mostly aviation-related images; here are some that I find interesting.

A phantom airship

The first prompt I tried was 'a phantom airship'. And it's pretty good! Like any good phantom airship, meaning is in the eye of the beholder, but to me that looks something like an airship floating over an impressionistic grand house with trees, mountains and clouds.

A phantom airship

I like to generate four images for each prompt (see Twitter for more examples), which gives a sense of how much the results vary (and what the drivers might be). Here we can see another, slightly weirder airship, with something like 'APHANTOM' written on the side. (Sometimes the AI turns keywords into pseudotext.) And another big house. (I wonder if that's also a result of 'phantom' -- a haunted house?) And a mountain floating in the sky...

A phantom airship

No houses this time, but again there is a bit of hovering landscape, and a somewhat disturbing dragonfly-looking thing above the airship.

steampunk airship

It turns out that the AI is reasonably good at airships. This is a 'steampunk airship'. No, it's not really an airship, and for some reason the image seems to have a wooden border; but it's something I could image seeing in a steampunk movie or videogame (which no doubt is not a coincidence).

You can even get something airshipish without using the word 'airship': this is 'the hindenburg disaster'.

aeroplane

Try as I might, though, I cannot get the AI to generate anything looking like anything like an 'aeroplane'.

Supermarine Spitfire

This, for example, is a 'Supermarine Spitfire', which must be one of the most recognisable aircraft in the world. You can make out a roundel (though French rather than British) and there's maybe a fuselage and a wing and a tail. Maybe?

dieselpunk airplane

A 'dieselpunk airplane' -- which admittedly looks very cool, but looks more like a car than anything that flies.

Imperial Airways Handley Page HP.42 G-AAGX "Hannibal" photographed at Tel Aviv during the late thirties

I tried using somebody else's description of an aeroplane photo: 'Imperial Airways Handley Page HP.42 G-AAGX "Hannibal" photographed at Tel Aviv during the late thirties'. Clearly, the AI knows something about what a HP.42 looks like (as well as Tel Aviv, though its beach rather than its late 1930s airport): there's a silvery fuselage with windows, and the big and very prominent landing wheel. But where are the, er, wings?

Royal Air Force

Going lateral doesn't help very much. This is 'Royal Air Force'. A nice aerial view of a school (or Cranwell?); roundels again, something like Air Force blue, maybe a jet?

Royal Flying Corps

For 'Royal Flying Corps' it goes for browns, insignia and -- fur? Feathers?

Qantas

Let's get civil: 'Qantas'. The palette is spot on: reds and 'Australian' earth tones. There's a kind of flying kangaroo empennage, and also some non-flying but still mutant kangaroos? Or sheep?

the Royal Air Force Display at Hendon

Maybe these keywords are too simple. What about something a bit more complex, like 'the Royal Air Force Display at Hendon'? There are some spectators, perhaps, and some multicoloured smoke suggestive of aerobatics, and maybe hangars. And some black shapes in the sky which might be aircraft silhouettes, but also might not be.

an airport

Let's keep going sideways: 'an airport' yields something like an airport seen from above.

an airport rendered in Unreal Engine

There are some tricks people have discovered which can give more realistic (or at least differently unrealistic) results. For example, you can add keywords like 'like a Picasso painting' or 'in a Studio Ghibli' style and you may well get something like what you asked for. Another trick tells the AI to use a particular rendering engine, such as 'an airport rendered in Unreal Engine'. To be clear, it's not using Unreal Engine in any way whatsoever, but rather is learning from images in its dataset which were (probably) generated with Unreal Engine to make something which looks like it could have been generated with Unreal Engine. In this case it definitely looks more 3d, but it also looks like it's gone from an aerial plan view of an airport to a ground view of a terminal building?

an airport rendered in vray

This is 'an airport rendered in vray', V-ray being another 3d rendering engine. Again there's something that looks like a terminal building and maybe a carpark down below, but then there's a forest above (?). Some crosses in the sky which could be aircraft.

an aerodrome

Something semantically subtly different from airport: 'an aerodrome'. Again it goes with an aerial view, but this time it's a less complex plan which sounds about right?

an aerodrome rendered in Unreal Engine

This is 'an aerodrome rendered in Unreal Engine', and it kind of looks biplane wings on the right there, or am I imagining things?

a First World War aerodrome rendered in Unreal Engine

No, because, here is 'a First World War aerodrome rendered in Unreal Engine', and it has something like a collection of canvas-covered early-twentieth century flying machines.

The Next War in the Air Britain’s Fear of the Bomber, 1908-1941

Going laterally again, I tried putting in some classic* airpower history book titles. First up: 'The Next War in the Air Britain’s Fear of the Bomber, 1908-1941' (had to drop the colon, though presumably I could have escaped it). Doesn't look anything like my book's cover, so it's not based on that. But there's plumes of black smoke over some green and pleasant land, and a gratifyingly jarring shattered-glass pattern and then.... some weirdly hairy thing in the centre. Okay.

England and the aeroplane

Here's 'England and the aeroplane'. Well, there's a cross of St George along with a weird aeroplane kind of thing, so I guess it meets the brief?

the spectacle of flight

For 'the spectacle of flight' it seems to have gone with the inside of a giant theatre with some Lovecraftian flying things. (Best not to ask what it did with 'the winged gospel'.)

a bomber dropping bombs on a city

Eventually I gave up trying to get the AI to draw aeroplanes, and instead tried to get it to draw air raids. This is 'a bomber dropping bombs on a city'. No bomber, but a city from above with plumes of smoke. It's not a million miles from Sulaimaniyah.

a Zeppelin raid on London

Trying 'a Zeppelin raid on London' gives a different perspective with more smoke (the AI likes smoke) and some kind of Zeppelins, and a skewed but Londonesque streetscape of Victorian tenements and churches.

the London Blitz

Encouraged by this, I started experimenting with 'the London Blitz'. This rather dynamic image shows something like a church, maybe Big Ben, rubble and lots of red -- maybe even fire!

the London Blitz

Admittedly this is more Quatermass than 'the London Blitz', but I could almost believe this was taken from a Bloomsbury rooftop as something erupted in a formerly peaceful Georgian square.

the London Blitz

Again this is 'the London Blitz', and again there is what looks like ruined houses along with... American football players??? These turn up quite often, and (of course) they are due to the existence of the London Blitz (American football).

the London Blitz rendered in Unreal Engine

Trying some of the usual tricks gives some interesting results. This is 'the London Blitz rendered in Unreal Engine'. Some fairly good buildings, and excellent rubble, but there's some kind of split screen thing going on, with two ruined streets on top of each other.

the London Blitz photorealistic

Now let's try 'the London Blitz | photorealistic'. It's not what the Blitz looked like, or rather not what images of the Blitz look like; but it is nicely evocative of violence being done to a city. Clearly that's what the AI goes to when it is told to do 'the London Blitz'. It's a little surprising that it's so consistent, actually: why doesn't it try to do people huddled in a tube shelter, for example? But it depends on the dataset it's being fed, which is just as much a black box to me as the algorithm.

the London Blitz rendered in vray

Similarly, this output from 'the London Blitz rendered in vray' could almost be the Docklands on 7 September 1940. Or have I been looking at too many of these pictures? Well, probably. Is any of this useful? Does it tell us anything at all we didn't already know? Well, probably not. But it is fun.

Bonus: 'the Great Leicester Balloon Riot of 1864'!

the Great Leicester Balloon Riot of 1864

CC BY-NC-ND 4.0 This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License. Permissions beyond the scope of this license may be available at http://airminded.org/copyright/.

4 thoughts on “Putting the AI into Airminded

  1. Post author

    Yeah, there's something about them, isn't there? My favourites from my ongoing Twitter threard are: "the golden age of aviation" (so very golden); "Oh! I have slipped the surly bonds of Earth And danced the skies on laughter-silvered wings" (actually quite evocative, if rather abstract); "Nobody... nobody trusts anybody now, and we're all very tired" (so very tired); and "If you want a picture of the future, imagine a boot stamping on a human face — forever" (so very... fleshy).

  2. Chris Williams

    Really interesting. But I can confirm that the 'balloon riot' image doesn't have anything on it that's recognisably Leicester, which is a shame. Was hoping for perhaps the AI interpreting it as wyvern attack.

Leave a Reply

Your email address will not be published. Required fields are marked *