"The general consensus is that math anxiety doesn't affect children much before fourth grade.” New research contests that.
Study 1: found many first grade students do experience negative feelings and worry related to math. This math anxiety negatively affects their math performance when it comes to solving math problems in standard arithmetic notation.
Study 2: found that second grade math anxiety affected second grade computations and math applications. Additionally, children with higher levels of math anxiety in second grade learned less math in third grade.
"Students are walking into classrooms at five and six-years-old saying that they aren't good at math before they've even stepped into a math classroom.”
Vukovic, R.K. et al. (in press) Mathematics Anxiety in Young Children. Journal of Experimental Education
 Mathematics anxiety in young children: Concurrent and longitudinal associations with mathematical performance. Contemporary Educational Psychology. 38(1), 1 - 10.(2013).