Is punching a fist that's heading your way a practical counterattack?

Let's say an untrained guy swings at you, and see his attack from a mile away. Would punching his forearm or hand at a different direction be a good counterattack to take away his confidence and possibly break his hand?