I'm not suprised he's not want to go for a start. It's not easy for man, I assure you, going to the swinger party as the novice. Pressure to be "worthy" of the company is damn strong and the stress can be overwhelming. I experienced this in my old swing/open relationship days and I've seen it with others too. He can simply afraid that he couldn't make it. It's pretty normal – remember that in biology world there is no prizes for second place and it can be difficult feeling to fight with.
It can be good idea to start friendship (not necessarily with benefits) with some other couple. You may introduce him someone like that. After some while you can propose him small home party with wine, small talks, watching some classy pr0ns (without actors having cocks bigger than Catherina's Great favourite stallions ^_- but something funny and sexy) and then maybe swapping you for this other wife. Or may he even would make this first step? The most important thing for him is to understand, feel and accept, that he HASN'T to compare himself with other man or men (at least not in this devastating way - "I'm good for nothing useless prick, my wife deseves better and even now she get's better from this guy").
I'm sure that you'll cope with all of this. After all it was he who proven his high value during that night when he first fucked you and your two friends and later turned your pussy inside out in one night, isn't it? Admit it, you didn't expecting that from him, did you? ;)
