Answer

In talking to בני ישראל at this point, משה wished to emphasize that Hashem had wanted to bring them into ארץ ישראל immediately after מתן תורה. It was only because of the sin with the spies that their entry to ארץ ישראל was delayed for so many years.

Although they certainly gained a lot when they camped by הר סיני, since Hashem wanted to bring them into ארץ ישראל immediately, any additional time that they spent camped at הר סיני was considered being there for too long.