Definitely try midsize, if you believe your fingers are too close to/jamming into your thumb-pad (part of your thumb that forms your palm). But absolutely change grips in any event.
The force required to hang onto a club at full speed, is something in the 100 lb-force range. Friction from normal grip pressure, with the resistance supplied by your hand and arm can supply that, but you need a decent coefficient of friction. Which worn grips won’t provide.
I’ll give the same advice I’ve received, and with the caveat that I’ve not seen your swing: shorten your backswing. Or at least look at it on video and see if you need to be taking it back far enough that your grip ends up suffering. You likely don’t.
If mine gets too long, I start doing bad things like extending, getting fake depth by artificially throwing my hands behind my shoulders, and loosening my grip. All of that feels like it’s making me faster, but it just gets in the way.