I got many suggestions for the same problem, at the end the cotton “balls” (more like a thin wrapping/stuffing) worked for me, even if it can make the bottom-out a bit mushy (the right amount is tricky to get consistently). Still beats rattle in my book.
Another set of stabs (supposedly Cherry) that I later used for another build doesn’t have this problem at all. So it does come down to the product, like @Rob27shred said.