Oh, thanks for the reply !
Please have a look & contribute to this bug report : https://bugs.kde.org/show_bug.cgi?id=262282
I definitely got some attention from the Dolphin developer, but he didn't manage to reproduce the issue. What nvidia board do you have ? The issue might be due to KDE, QT or the Nvidia driver and does affect icon views in general (Dolphin or KDE file selectors).
Please note the bug report was first about a specific slowdown case and then turned into something more general (that is to say : what I described in this thread).