You have probably downloaded a G'MIC Windows installer

You have probably downloaded a G'MIC Windows installer which is intended to automatically install the plug-in and is not the plug-in it self.
Close GIMP and open the same file to install the plug-in. Don't forget to remove the file from GIMP directory you mentioned so that the problem when starting GIMP doesn't happen again.
If the installer does not work download a zip version and install it manually.