As best I can tell, the only way to prevent Apple Music automatically opening when connecting headphones is to delete Music.app.

Bravo, excellent own goal there Apple.