If you previously have been denied a visa (no matter the category), have been denied an entry to the United States or have been deported/removed from the U.S., your ESTA application will most probably be rejected.
Receiving a visa denial more often than not means the applicant does not comply with the general eligibility requirements, which are valid for both visa and ESTA applications.
Take note that you must be completely honest, regarding any previous visa rejections. There will be much harder consequences if you lie during the application process and the department later discovers the truth. Such actions will be considered a fraud and legal action will be taken.
If you have been refused a visa while being citizen of one country and later have changed your citizenship to a country that enters in the Visa Waiver Program, you must answer “Yes” to the “Have you been denied a visa before?” question. This information does not refer to the citizenship you currently have, it refers to your overall visa application process made with any passport you have held and from any country you have applied.