Neighbor speaks only to hubby should hubby reciprocate?

The neighbor lady speaks only to my husband and he talks sweetly to her. He will also initiate conversation with her but not her husband. I don't see my husband being loyal to me since she behaves as if I am not there or as if she is better than I am. She will engage him conversation and went I walk out the talking stops. I say she is flirting and by continuing to be friendly he is encouraging her and being disloyal to me. Are my feelings substantial?