In case of a vacancy in the office of State's Attorney, or of his removal from the county or city in which he shall have been elected, or on his conviction as herein specified, the Judge or Judges resident in the county or, if there be no resident Judge, the Judge or Judges having jurisdiction in the Circuit Court of the county in which the vacancy occurs, or by the Supreme Bench of Baltimore City for a vacancy occurring in Baltimore City, shall appoint a person to fill the vacancy for the residue of the term.