Mordechai was mainly called a Yehudi (i.e., from Shevet Yehuda) and not a Yemini (from Shevet Binyamin). His mother descended from Shevet Yehuda, and because Dovid (from Shevet Yehuda) saved his Zeide’s life (Shim'i ben Gera), he owed his existence to Yehuda, and therefore he is called “Ish Yehudi”.
At the end of the Megila it says that Mordechai was: יוּצָרְו ויָחֶא בֹרְל ג י רתסא liked by MOST of his fellow Jews, and the: ַמ ַה ִר ָק י" ָרא says that there were some Yidden who had Taynos on Mordechai for not bowing down to Haman and causing the whole Tzara. Why did Mordechai do this? The answer is that Mordechai HaYehudi was in a different world. He was on the level of “his entire name is the name of Hashem”. All his actions are totally Hashem, and perhaps this is why he is constantly called “HaYehudi”. A “Yehudi” means a Baal Bitachon, who only wants to impress Hashem, and isn’t involved in public opinion.
R' Yonasan Blau Shlita told me that he remembers how R' Yehuda Leib Nekritz Zatzal and R' Shlomo Figer Zatzal would have Vaadim for years with only 2-3 people in attendance, and it didn’t bother them in the least! They also didn’t care if their Divrei Torah were appreciated or not. They were happy and were real.
They understood how Redifas HaKavod dominates people in every move they make. And despite the possible temporary pleasure of getting Kavod, in the end, it can hurt and even destroy the person, Chas V'shalom, since he is hopelessly dependent upon people’s opinions (and you can never always make everyone happy!).
When everyone becomes broken from Bizyonos, Novardokers are Davka B'simcha, they deal with Bizyonos like a fish in water.