I used to get as many Factory releases as I could from our local record shop in Corby (Discovery), which wasn't always easy.
Favourites outside of the JD/NO domination (they were whole dimensions away from anyone else!) were Stockholm Monsters first album (saw them support NO in 1984 and was completely puzzled by their stuff but much better on Vinyl).
I also played the first Wake album to death. Couldn't get too far with Always Now by Section 25. I had a listen to it recently and it made me feel ill.
Cabaret Voltaire's yashar is amazing- sort of pre figures the whole Coldcut cut-up stuff.
52nd Street (going to get slated for that probably). Oh, and Pigs and Battleships by Quando Quango- Dire! Although their peel session and 12" of Love Tempo and Go Exciting are class...
And one paticular favourite is Act on Instinct by Streetlife- Just brilliant.
I recently got a copy (from eBay) of five early factory videos. They all feature NO but only in passing and were intended to showcase factory's other output. Worth getting hold of for live stuff and early videos of, among others, NO- very strange early video for Blue Monday, the wake, ACR, Stockholm Monsters, james, 52nd St, QQ, railway Children, Swamp Children etc. I think that the guy who copied it sells it quite often- it's only about 8 quid and really well done. Worth it, I'd say (and no it isn't me)
And How can anyone forget "Temple of the 13th Tribe" by Royal Family and the Poor? "Restrained in a moment" is, dare I say it, absolutely beautiful and "Radio Egypt" is a really scary song. Produced by one Peter Hook, who also features in the backing duo The Soul Sisters. I'm pretty sure that you can buy re-issues of Royal family stuff on CD and someone earlier referred to the label they're on.
Hope this helps:)