In Hebrew, east is 'qedem' and literally means 'the direction of the rising sun'. The sun as symbol of the celestial/Heavenly Father is not unique to the restored church today. This is also how ancient Israelites saw it. That which came from the east came from God. Usually this was something positive, see e.g. Ezekiel 43:1-4. But it could also be the "east wind" sending destruction over the wicked but was nonetheless from God. This is also an expression we find in the Book of Mormon
Mosiah 12:6
And it shall come to pass that I will send forth among them, and it shall smite them; and they shall also be smitten with the
Mosiah 7:31
If my people shall sow filthiness they shall reap the , which bringeth immediate destruction.
It's not hard to imagine the 'east wind' being associated with something negative when you think about the Old World geography where this expression originated. It's the hot, unpleasant desert wind. The wind from the west, on the other hand, is a cool, humid wind from the Mediterranean.
However, when it comes to movement, going west is always the desirable direction and going east is something negative. This is because the sun moves westwards. If you go east, you go the opposite direction of the symbol of God. Going west you follow the light, so to speak. And most importantly, you go back to the Garden of Eden, back to the tree of life. Being thrown out of the garden was a movement eastwards, going back is a movement westwards. Somewhere in between you need to repent (the word for repent in Hebrew, 'shub', literally means to turn back or return).