I know someone similar to you... swears he's not gay, but very into shemales. Never known him to get with a "man", but is very regularly flirting, making out, and fucking (both ways) men who act like women. I don't consider him gay, but it's a pretty fine line.
pro-tip: book a trip to Thailand ASAP