Trump also announced that Gina Haspel would replace Pompeo as CIA director - making her the first woman to ever head the country's top spy agency.
Tillerson's ouster was reported minutes before Trump's tweet by the Washington Post, though tensions between the two have simmered almost since Tillerson took the job in 2017.
Trump's statement did not mention Tillerson, with whom he has been at odds on issues ranging from North Korea to steel and aluminum tariffs throughout the first 14 months of his administration.
Trump had not offered an explanation for the change at the time of the announcement.
Tillerson had just returned from a shortened trip to Africa hours before Trump's announcement.
Gina Haspel will succeed Pompeo at the Central Intelligence Agency, the first woman in that role. "His leadership has made America safer and I look forward to representing him and the American people to the rest of the world to further America's prosperity". The secretary has generally been cut out of the loop on major foreign policy moves, like Trump's impulsive decision to meet with Kim Jong-un.
Rex Tillerson is out as secretary of state. Reporters who cover the State Department had been guessing how long into 2018 that Tillerson would last.