From the “wine of Torah” in Rashi’s commentary:
This concept that Rashi introduces — that “there we saw the Nephilim” refers to angels “who fell from the heavens” — aligns with the gist of the spies’ claim as explained in the deeper parts of Torah.
There is a well known teaching from the Alter Rebbe regarding the nature of the spies’ sin. He explains that the spies didn’t want to enter the Land of Israel because they wanted to remain in the wilderness, where they were free from concerns about financial matters (and more generally, from dealing with the physical aspects of the world). This way, they could fully devote themselves to their spiritual service of Hashem. However, {they believed that} entering the Land of Israel, “a settled land,” and beginning the cycle of “six years you shall sow your field...,” would tear them away from their spiritual service.
Their mistake was rooted in their broader perception, that spirituality and physicality are mutually exclusive. They believed that it is impossible to both be involved in physical matters and remain connected with the spiritual.
This is what they meant by describing the Land as “a land that devours its inhabitants”: If we “settle” there, and get involved with earthly matters, we will be “devoured” and submerged into material concerns — “devours its inhabitants.”
We can now understand the following statement (by the spies), “there we saw the Nephilim....” As Rashi explains, this refers to “the sons of Shamchazai and Azael, who fell from the heavens in the days of the Generation of Enosh”: As discussed at length by our Sages, these angels initially descended to this world with pure intentions, for the sake of Heaven, but {their downfall was that} they were unable to withstand the materiality and trials that this world presents, and they fell from their spiritual level. Despite their great excellence in the spiritual realms, their descent below not only prevented them from maintaining their previous level, but actually caused them to fall {even further}.
The spies made a similar claim regarding the Jewish people: Although the Jewish people are on a very elevated level in the wilderness, completely removed from worldly matters (like angels), the spies claimed that their entry into the Land of Israel will cause them to fall from their current spiritual state.