In the second example, the use of "by" could indicate physical proximity to a hat, whereas "with" suggests the man has the hat. In cases where there is a possible physical interpretation, it is better to use "with" (assuming you do not intend to convey that the subject is near an object). In cases where there isn't a possible physical meaning to confuse, "by" works to identify something. "By" as an indication of identity is particularly clear when you specify cases such as example one, where existence of a name is referred to. This fits into implicit phrases like "goes by" or "known by" as suggested in the comments.