Fight fire with fire

While Shakespeare penned this phrase in 1595 he isn't credited with the phrase becoming a known one, firefighting is. As a way to ensure large fires wouldn't cause damage firefighters would set, on purpose, smaller control fires to mitigate danger. So they actually fought fire with fire. 

Today we use it in a figurative sense to indicate someone responding to an attack by using a similar method as one's attacker.