Also confused

When I swipe a side that doesnt have arrows or arrows pointing in it ends the game. But sometimes it works. Is this a glitch or is there a rule Im not understanding?

JustifiedTaboo about Swiperoo