Hi Eric,
Thanks for asking this question here 
Yes, we would generally advise completely removing the Super for treatment, as the honey will be too affected by the chemicals to be guaranteed fit for consumption.
There’s also this post on the forum about it in the case someone did leave the Super on, which we’d be fairly well agreeing with AussieMike’s response to.
In the event you have left the Super on for the treatment, we would advise would advise to harvest & throw away that honey or feed it back to that same hive.
Leave frames on the hive after harvest in the open position for 2-3 days (no more) so the bees can clean up the rest of that honey but don’t start filling the frames back up.
Then, pull them off & do a thorough cleaning of the frames with tubs of hot water or a gurney (not too close with the gurney as it can damage frames sometimes).
Leave in the open position while cleaning and then dry out of the sun. The aim of cleaning is to get rid of all the wax that may have traces of chemical.
In case you haven’t come across them yet, please also check out these pages from us for a comprehensive over view on varroa and treatments.
Varroa mites - what do I need to do?
Do let us know how you get on and best of luck,
Danika (Flow Team)