I don't consider all judgments harmful either. But I do consider inaccurate and misinformed assumptions to be harmful and that's more what this is about for me. For me, disassociating from the umbrella term because of the assumptions attached to it means I'm tacitly saying that there is truth to the assumptions made about it when that's not the case. For me, there's more integrity in sticking to the truth of the term rather than yield to the assumptions attached to the term.
I was following, but now I think you lost me. I guess I have to make it personal for me to understand.
I am polyamorous in that I romantically love three people. However, I feel very strongly that I would not enjoy casual sex, and in the past have failed in attempts to have and enjoy casual sex. While I believe that people should be allowed to enjoy sex the way they want to provided it harms none, I do not believe in casual sex with no emotional involvement for myself and do not want that label attached to me. If the word polyamory becomes attached to casual sex with no emotional involvement, then using the word polyamory for myself not only feels erroneous, but also as a betrayal of who I feel myself to be. In addition, adding things like poly-sex parties to the definition of polyamory feels like it actually takes away from the ideal of "many loves", which I interpret as emotional involvement. Is there a happy medium?