A Grattan Institute study has found that not only are Australia's students struggling with maths, but many teachers lack the confidence to teach it even at a Year Six level.