Herd immunity is essential to control severe acute respiratory syndrome coronavirus type 2 (SARS-CoV-2), especially in immunocompromised patients. Convalescent individuals should be vaccinated later due to vaccine shortage, as studies show that neutralizing antibodies generated during infection are stable for at least 6 months. Cellular immunity is also detectable for months. However, there is evidence of cross-reactivity of T cells with human endemic coronaviruses (HCoVs). Here, we show that cross-reactivity—which may prevent the specific detection of SARS-CoV-2-specific T cell responses—can be avoided if cells are stimulated with the N-terminus of the spike protein in IFN-γ ELISpot. In contrast to previous studies, we examined T-cell responses against all four known HCoVs using IFN-γ ELISpot in 19 convalescent volunteers and 10 fully vaccinated volunteers. In addition, we performed Spearman analyses to detect cross-reactivity of T cells. We observed no correlation between T-cell responses against SARS-CoV-2 and human endemic coronaviruses, either in the whole cohort or in the individual groups. The use of the respective stimuli could lead to a more accurate assessment of cellular immunity in recovered individuals. This testing procedure could help to define the best time point at which convalescents should receive SARS-CoV-2 vaccination.