Agreed!
I also only experienced this when I used a punch vs a guillotine.
I'm guessing since the punch restricts the draw more so than any other cut. As a result, when I do use a punch I will use the (for lack of a better description) "figure 8" punch. I guess a larger punch may also work.