If nose punches killed, there'd be a hell of a lot more dead boxers or ringfighters.
Seriously, neutralizing someone the fastest is either a temple hit, gut punch, or smash him directly in the solar plexus. A blow there would cause the diaphragm to spasm and make it hard to breathe, not to mention extremely painful. Some people might be big, but they arent worth anything if they cant breathe.
If you feel desperate, you can always go for the groin strike.
There are too many hard bones in the skull, and you stand a better chance of breaking your hand than the guy's jaw if you strike anywhere around it. A broken hand is serious pain once it sets in.
Anyhow, congrats on defending yourself, though i would watch my back for revenge attempts.