In sefer Chasidim, Rabbi Yehuda HaChasid in his tzavo’ah (siman 45) writes many things that his children need to be careful about. One of the things he writes is:אילן העושה פירות אין לקצוץ אותו – “Not to cut down a fruit bearing tree”. Many poskim are very worried about this and they say, if one transgresses it, he puts himself in risk (see Shu”t She'ilos Ya'avetz 1:76 and Shu”t Chaim B’Yad, siman 24.
The Shivim Temorim (siman 52) understands that the above applies, even when according to halachah there is a heter to cut down the fruit tree. The reason for the sakonah is because of Rashi in Pesachim (50b) who says: “(When one cuts down fruit trees) everyone looks and speaks bad about the person, which creates an ayin horah...”
We mentioned above the Da'as Kedoshim in his sefer Milei Dechasidosa who explains the tzavo’ah of Rabbi Yehuda HaChasid: That souls are reincarnated in trees and when they are cut the souls forfeit their correction. The sefer Me’ah She’arim from the Rikanti (Shaar 88) writes: “The maloch appointed to be in charge of the tree, screams and demands din for the one who cut it down before it’s time.”
We also mentioned earlier the Pirkei D'Rabbi Eliezer (34) who explains that when a fruit tree is cut, it cries out so loudly that the sound is heard from one end of the world to the other, although it is not captured by the human ear.
The Minchas Elozor (3:13) writes: I have a kabbalah from my ancestors, to avoid issuing any halachic rulings regarding cutting down fruit trees, even when it is halachically allowed, as one should be worried for the aspect of sakonah. When there is a need I prefer to send the shailah to a different rav or beis din.
The Shu”t Avnei Tzedek (Yoreh Deah 45) writes: “If they would listen to me, I would say to avoid cutting down fruit trees altogether, and one who is worried about his soul should distance himself from this. The pasuk says, כי האדם עץ השדה, which means, what a person does to the tree, may be done to himself, chas vesholam
Based on these reasons we can understand why even if there is a halachic heter, R’ Yehuda HaChasid is still very against it. As כי האדם עץ השדה, what one does to the tree may be done to him; there are gilgulim in the trees; the trees let out a big scream which goes from one end of the world to the other; the maloch appointed over the tree demands din, and all this is even when it is permissible to cut down the tree.
Tzavo'as Rabbi Yehuda HaChasid
There was once a contractor who signed a deal to go ahead with a certain building project. During the building process he noticed that there were some olive trees that were in the way, and the contractor refused to continue with the job, as he was too scared to cut down the trees, due to the sakonah involved, like it’s clear from the tzavo’ah of Rabbi Yehuda HaChasid.
When R’ Shlomah Zalman Auerbach heard this, he said, “It’s a shame that Rabbi Yehuda HaChasid never wrote in his tzavo’ah לא תגזל – “Do not steal” Meaning: Of course one should be careful with the tzavo’ah of Rabbi Yehuda HaChasid, however, one should be worried about what it says in the Torah more than what it says in Rabbi Yehuda HaChasid’s tzavo’ah which may have just been for his children, and is not even prohibited on a rabbinical level.