Generally this is part of the trade-off with dampening, but there are some partial exceptions.
TL;DR:
-
Cleanest stock silent tactile I know is the TTC Silent Bluish White
-
TTC Frozen Silent (linear) is even more clean, but they’re light and may need shim material to grip caps properly
-
In general, you can lube the leaf and stem legs to reduce or sometimes eliminate swish and scratch, but this comes at the price of reduced or nearly-eliminated tactility
Verbosity:
With tactile silents, there’s always going to be at least a tiny bit of that scratch - depending on the build and use environment you might not hear it, but get close enough and you will. I think so far the least swishy / scratchy stock silent tactile I know is the TTC Silent Bluish White. (Oddly enough this does not hold true at all for the TTC Silent Brown V2 which is ostensibly similar; it needs a full tuning to sound as clean as the SBWs do from the bag.)
It’s as you said - there’s little else to compete with the movement sound when top and bottom-out are silenced - and with dampened tactiles, it’s often the case that the friction of the stem against the leaf is the loudest part of the action. You can lube the leaf and stem legs which will reduce this sound, but that will also reduce the tactility of the switch. (As you may have picked-up on by now, it’s more about finding your preferred balance of factors than identifying the “best” product or tuning technique.)
The same trade-off is often true of linear silents as well, but a few manage to stand above the rest of the field in terms of clean sound, representing the closest thing to a best of all worlds example in terms of quiet, crisp, and clean.
It isn’t the most quiet thanks to its firm dampening pads, but I think the TTC Frozen Silent might be the most clean-sounding stock silent I’ve tried. A very close second would be the Kailh Deep Sea, which performs similarly with a touch less smoothness. These are both pretty light linears - so if moderate to firm tactiles are what you’re used to, you’ll likely make a lot of errors with both of these until you get good and used to them.
(The Frozen Silents don’t escape the trade-off game - they have a loose grip on keycaps, and there’s a good chance you’ll need some PTFE shower-head tape to make a good fit.)
Other silent linears might be swishy / scratchy in stock form, but those sounds can be tuned-out with lubing and filming - Bobagums, for example. They’re pretty darn quiet stock, but the sound they do make is swishy. Lubed, though, they make almost no sound at all - the loudest part might be your fingers hitting the keys. That said, I find lubed Bobagums pretty tough to use for efficient touch-typing because I’m accustomed to some kind of feedback.