The Rishonim write that on Rosh Hashanah we eat honey and all types of sweet foods. See the Ohr Zorua (Hilchos Rosh Hashanah, siman 257), Mordechai (Beginning of Yoma), Rokeach (siman 201), Ravyah (Rosh Hashanah, siman 547) and Shibolay HaLeket (siman 283, erech Mesikah) who all bring this.
The Levush (siman 583:2) when discussing the laws of the simanim on the night of Rosh Hashanah writes: The Ashkenazic minhag is to dip a piece of the hamotzi into honey, and to say: תהא השנה הבאה עלינו מתוקה וטובה לנו – The upcoming year should be sweet and good for us. The Kaf HaChaim (583, ois 4) writes: “The Ashkenazic minhag is to dip a piece of the hamotzi into honey, and to say:תהא השנה הבאה עלינו מתוקה וטובה לנו, and now even amongst Sephardim this has become the minhag.”
The Likutei Maharich brings support to this from the Perek of Tehillim (Perek 81) which is the Shir Shel Yom for Rosh Hashanah, in which we say: ויאכילהו מחלב חיטה ומצור דבש אשביעך. He explains that, ויאכילהו מחלב חיטה, refers to the piece of hamotzi, and, ומצור דבש אשביעך, refers to the honey, and they are together as we dip the hamotzi into honey. The sefer Imrei Noam adds, that the words דבש אשביעך have the same gematria as חדש השביעי, which is the month of Tishrei.
