One of my best long time friends, being a big fan, introduced me to the music of Billy Childish many years ago. We have been in several bands together and covered several BC songs that I think are outstanding. In "Hippie Repellent" we have played "The Man With Eyes Like Little Fishes" and "Want Me - Win Me" both from Thee Headcoats - In Tweed We Trust, The Spoken Voice (Pop Rivets), Pussy Whipped (Thee Mighty Caesars - Caesars Remains) and probably a few others that I've forgotten about.
I'm also lucky enough to be playing rhythm guitar for the Armitage Shanks (all good friends of Billy) on their upcoming brief American tour. They have a few of Billy's classics in their set list to rotate: Kray Twins (Pop Rivets), It's You (Milkshakes), and Every Day (Thee Headcoats - Beached Earls).
The Milkshakes - 19th Nervous Shakedown is a good primer.