For as long as I can remember (not that far back) I have known I was "gay". But as time has gone on and I started to look at who I was as a person, I noticed as time goes on it is harder for me to say I am gay. I am in no way what so ever aroused by a vagina. but when it comes to a romantic or sexual partner, I have no preference other than they need to have a penis. if they are trans it in no way affects my attraction to them so long as they have a penis. I am not really all into labels and boxes, but I like to know which label wants me as it helps me to know who I am, but with this part of me, I have no idea what to call it.
It just sounds like your romantic attraction is a little bit different than your sexual attraction. Sexually, you're probably gay. Romantically, that's up to you. I'd guess you were panromantic, since you really don't care what gender they are.
The majority of people you find sexual attraction to will most likely be men (since the majority of people who have penises are men) and it would be fairly rare for you to find someone of a different gender who you are sexually attracted to, since they would need to be AMAB, trans (female or NB) and pre-op or AFAB, NB and post-op in order for you to feel that attraction. That pretty much fits the definition of homoflexible - someone who feels attraction for the same gender the majority of the time, and only rarely feels attraction for other genders. Many homoflexible people simplify their sexuality to just gay, though, so I wouldn't say it was wrong for you to identify as gay still. Some homoflexible people will also consider themselves bisexual/pansexual with a preference, so you could also use this label if you liked. Alternatively... you could just say you like penises. Sometimes labels are overrated, and they can mean different things to different people, so just go with what feels correct and accurate for you.