The defeat left the Green Falcons with four points from four games in Group Two and in serious danger of missing out on the World Cup for the first time since 1990.
The Saudi Football Federation (SAFF) announced the news on Saturday, saying in a statement that it was Al Johar, who has been national team coach three times, who made the decision without any pressure from above.
The 65 year-old will continue working for the team in a role as technical advisor.
In his nine months in the job since replacing Helio Cesar dos Anjos, Al Johar led the team to the Gulf Cup final and defeat at the hands of Oman.
The new man’s task will be to take the team to Tehran to face Iran on March 28.
Defeat in that match would almost certainly spell the end of Saudi hopes of automatic qualification.