about 20 years ago a friend and I were walking along the edge of a snow covered field bordering a woods, an area we hunted year round and knew well. A shot rang out, and we both found ourselves buried in the snow, not really knowing what just happened. When we stood up we found a gentleman getting out of a tree less than 30 yards away (no orange). Upon asking if he shot at something, he pointed to a cornfield 50 yards away, angling away from him and us, stating he shot at a doe. We were still baffled, so offered to help him check for blood. When we next turned around, he was running for all he was worth. We then realized he shot at us, and the round went between us. We were dressed hat through bibs in safety orange. I have no faith in it since then. BUT if he was, we probably would have seen him, and helloed him. In the southern tier, I wear orange. In the northern tier, I seldom do. I generally know when and if someone else is in my area due to limited access, and put more faith in my seeing someone else than I do in them seeing me.
If hunting with others though, I think it is smart to wear orange.