The Medrash (Tanchuma 69) states: “Yosef only had to serve 12 years in jail. Why were two additional years added? Hashem said: You trusted in the minister of drink and asked him twice to remember you. Therefore, I will add two years.”
It is related that Rav Chaim of Brisk zt”l once asked his talmid, Rav Shimon Schkop zt”l: If Yosef had only asked the minister of drink to remember his one time, what would his punishment have been?
Rav Shimon answered that if he was punished with two years of imprisonment for saying it twice, it seems he would have been punished with one year for saying it once.
Rav Chaim said, “No. If he had asked once, he would not have been punished at all. A person has to do hishtadlus to save himself, and Yosef’s obligation was to ask him to remember him once. But since he asked twice, it became evident that both requests were improper for him, on his level. Therefore, he was punished for both of them.”