Sefer Otzros Hatorah quotes the following story from Sefer Yalkut Sipurim in the name of Rav Nissim Gaon:
Rav Yehoshua ben Levi asked Hashem to show him His ways allow him to understand why bad things happen to good people and good things happen to bad people. After his prayer, Eliyahu Hanavi was revealed to him. Rav Yehoshua asked his permission to travel with him on his journeys and to see what he does, hoping that this would help him understand Hashem’s ways. Eliyahu agreed, but only on condition that he doesn’t ask any questions. Rav Yehoshua accepted this condition and they set off.
Towards evening, they arrived at a small, wooden hut, with one cow tied up outside. As they approached, an elderly man and woman came out to greet them. They did all they could to provide them with hospitality, despite their poverty. The next morning, before leaving the hut, Eliyahu davened for them. Rav Yehoshua was shocked when he heard him davening for the old couple’s only cow to die. And so it was. As they left the house, they heard the old lady crying that her cow had died.
That evening, they arrived at the mansion of a wealthy man. They knocked on the door but no one answered. Eliyahu said, “Perhaps no one heard us, let’s just walk in.” They went in and found the man and his wife sitting at the table, which was set with fancy dishes and all types of delicacies and their servants were hurrying to serve them. When he saw the guests, the rich man did not say hello or invite them to his meal. Instead, he angrily said to his wife, “Who let these stragglers in? Why wasn’t the door locked to make sure that no guests are allowed to enter?”
This man was always miserly and angry, but he was in an especially bad mood that day because one wall in his house had fallen down. He had hired workers to fix it but they had not shown up. When the guests asked if they could stay overnight, he reluctantly agreed. He took them to a stone bench in his backyard and told them to sleep there, without giving them anything to eat or drink.
The next morning, Rav Yehoshua urged Eliyahu to leave as soon as possible. He agreed but said that he first had to daven. This time, Rav Yehoshua heard him asking Hashem to perform a miracle in order to fix the broken wall immediately. And so it was, the wall miraculously fixed itself. Although he was puzzled, Rav Yehoshua kept his end of the deal and did not ask any questions.
They traveled all day until they came to a big city. Eliyahu said that they should go to the shul where one of the residents might invite them to stay in his home. They went to the shul and saw that it was spacious and fancy. It was evident that the residents were quite wealthy. Every individual had his own assigned seat, and they all strictly made sure that no one sat in their place. The two guests crammed into a corner and waited there until the end of Maariv. One of the members noticed them and said to his friend, “Oh no! More shnorrers came to town. Where will we send them to eat this time?”
His friend replied, “No one has to invite them to their home. Just send a little bread and salt to the shul for them.” None of the other residents invited them either. They were all happy to hand them a few pennies to buy some bread. After sleeping in the shul that night, the two guests got up and left the city. Before leaving, Eliyahu blessed the residents and said, “May they all be leaders.” Rav Yehoshua ben Levi was very upset about this and he wanted an explanation, but he had promised not to ask any questions.
That evening, they came to another city. The residents were happy to see them and provided them with nice accommodations. They gave them good food and drink and sat with them as they ate. The next morning, Eliyahu blessed them by saying, “May Hashem not make any of them into a leader.”
Rav Yehoshua couldn’t hold himself back any more. He couldn’t understand anything Eliyahu was doing and needed an explanation. After they left the city, he said, “Whenever people treat us well, you daven for bad things to happen to them. Whenever people treat us badly, you daven for good things to happen to them. Why are you doing this?”
Eliyahu replied, “You have broken our agreement. We agreed that you wouldn’t ask any questions but you are asking me a question. Therefore, you cannot travel with me anymore. But before we part ways, I will explain myself to you.
“I knew that the poor man who we stayed by the first night was destined to die the next day. Therefore, I davened that his cow should die instead of him. Yes, the cow’s death caused them a lot of pain, but it allowed him to remain alive. Furthermore, my prayer effected that they will soon become wealthy and will have no need for that one cow.
“The miserly rich man we stayed by on the second night was unaware that a huge treasure was hidden under the broken wall in his house. If the workers had come to fix it, they would have found it and given it to him. Now that the wall was miraculously fixed as a result of my prayer, he will not get the treasure. Furthermore, he will become poor soon and when the wall falls again, he won’t be able to afford to hire workers to fix it.
“When I davened that the people in the third town should all be leaders, my intent was that they would all seek leadership, which would lead to many fights and arguments. Since they would constantly be fighting, they would be unable to enjoy life. When I davened that none of the people of the fourth town should be leaders, I meant that they should all live in peace, with none of them trying to lead the others.
“You should now realize that Hashem’s ways are hidden and many things are not as they seem. Things that seem good are often bad and things that seem bad are often good.”
