Not sure if i understand this correctly. When you edit content it will always open in a Popup Window (If not disabled in the DNN Portal Settings or in the Module itself), this has nothing to do with the Editor. Or do you want open the Editor inside an extra PopUp Window?
A screenshot of the problem might help.