Yeah, it does depending on which one it's closest to. Fortunately instead of doing the math myself, I take both colors, set them on different layers and make the opacity of the top layer ~50%, then copy the merged image and use the hue it generated. :>

But yes you're exactly right!