I have dealt with what you described pretty much all my life. Really takes me aback hearing someone else articulate it. After 30 years and asking many people I got many pieces but I had to ask the gay men before it became clearer.
Regular men and women offered some things but one word sums it up "metro sexual". You put allot into your appearance because you care about maintaining a higher level image, because you notice when someone is interested in you and you want to be attractive.
Some guys look a certain way, and built a certain way and women fall all over them and it seems like they never have to try to find a partner. Your persona is what ultimately attracts someone special and if they are looking at you for what you can do for them money wise then buy a prostitute instead, at least you know what you are paying for. Many materialistic women out there today and still they grow unhappy because they loved money not you the most.
The ugly or handsome no matter what attractiveness level are still not your physical choice so it all the same in that area. You are probably decent looking, friendly, and unassuming which everyone would like to have but why is it women dont respond to that like the creepy old guys ?
Once you accept you might seem distant or tense in social settings you may understand a bit better , but guys we know our gender and no social anxiety approaching one of our own but put the opposite sex to approach then fear of rejection sets in.
The dirty old guys approach you because they aint got nothing to loose and they are not afraid of rejection just note their appearance and talking to good looking people for them is more pleasurable.
Now in my forties I see who I was then always looking for a better looking woman and no matter how much better the next one appeals to your eyes that will fade as well. Sounds so cliche but it is indeed true. The good looking ones were all about their appearance and high maintenance most times. The ones that are innately fine , never are single long enough to meet.
A plain Jane who knows how to love will help you understand that there is nothing that remains that lasts for very long except that because without real love, a real friend you can tell your heart to you get roped in, and never can know the joy of what too many never found.
I found that then didnt know it and it never came back in that way again in another and every relationship ended badly. Now after three decades it cant change the past but I'm done thinking that I'm less valuable or I need to be looking for the next date.
After all you want her to see and meet (you) the person inside and like him, and if everyone is looking to take the best they can find they are in a way, but when a women checks you out she is sizing you up for what you may be able to do for her, then she will smile and ask you questions like she is interested in you, but most times they are carrying out their program to see if you can support the life and family you will have the honor of supplying the needs for to keep them happy and maybe once a month she might fuck you and act like you are so lucky to have that since they get lazy usually after a year. One that loves you will never close the pussy store on you.
That is my life sum wisdom and for you and I'm late for work now but if I did not sincerely want to help you I would not have responded. Dont take it personal getting hit on. In closing it's because you have something that attracts people. Don't let the dirty old men give you what you are worth. After getting these the last questions answered I no longer felt threatened by getting hit on I take it as a compliment and kindly refuse them as my mother would want me to do.
It happens to women all the time too dude they just dont communicate it to men.
peace