speed endurance problem?

Give yourself a few more races to let the specificity of race SE kick in. Sounds like you are well on your way. Just stick with it and you may surprise yourself shortly