The White House fired a new interim U.S. attorney in New York's Northern District hours after a panel of federal judges installed him in the position.