Early Wire and Pere Ubu stuff is essential.
I would start with
Wire - Pink Flag, 154, Document and Eyewitness
Pere Ubu - you should maybe just get the 'Datapanik in the Year Zero' box set, but if you aren't that sure about it the logical choice would be 'The Modern Dance', their first LP which has some of the same songs as their early singles.
Actually my favorite Ubu material is all from before their first album, I have some live tapes of their earlier lineups that are great, and the early singles on Hearthan are all pretty much classics.
It has long been said that they were an influence on JD, and it certainly sounds that way comparing, say, 'Heart of Darkness' to 'Shadowplay'. The bass sound - that's right, Hooky's trademark open string plus fretted notes on another string, pretty high up on the neck of the bass - is the same on the early Ubu material with Tim Wright (who would later play in the great No Wave band DNA) or, later, Tony Maimone on bass.