Whether one gets reward or not may be a machlokes haposkim, but certainly one should do some sort of teshuva, the question is however, what exactly should one do?
The poskim bring down a number of different options. The Shu”t Teshuras Shai (2:17) writes that one should give money to tzedokah to atone for all the berachos that were made levatolah [in vain], and he should buy a mehudar [beautiful] pair of tefillin for the son of someone who is poor.
The Shu”t Shevet Kahosi (4:14) brings from R’ Aharon of Belz that one should start wearing tefillin for longer than he has been accustomed until now, and learn whilst wearing them.
R’ Elyashiv suggests that one should buy himself a very mehudar pair of tefillin, more mehudar than what he thought he had been wearing until now.
We can seemingly bring a rayah to the opinion of R’ Elyashiv. The Gemara in Bava Basra (4a) brings, that Hurdos killed a number of talmiday chachomim, and later on in life he regretted what he had done. He went to seek advice from Bava ben Buta as to what to do to atone for his mistake. Bava ben Buta told him, that by killing talmiday chachomim he had blinded the eyes of the world, and to atone for his mistake he should busy himself with eyes of the world, i.e., the Beis HaMikdash. Hurdos then went and rebuilt the Beis HaMikdash, more beautiful than it had ever been before. It was so beautiful, to the extent that Chazal said about it, “one who never saw the Beis HaMikdash that Hurdos build, never saw a nice building in all his days”. We see that the teshuva for one who blinded the eyes of the world, was to replace the eyes of the world in more beautiful way, similarly, if one chas vesholam discovers that his tefillin are pasul, a good way to do teshuva would be to start wearing more mehudar tefillin.
