I upgraded from a GTX970 OC to an MSI GTX 1060 and the difference in VR was very big. Mind you, I did upgrade the cpu at the same time, from an i5 2500k Sandy Bridge to a 5690k Haswell, though the overclocked cpu speed is the same, at 4.3GHz.
I was quite concerned, because I wanted a 1070, but it was too much, with upgrading the cpu and therefore mobo as well, so I got the 1060 with 6GB VRAM and it blew my socks off.
I wouldn’t be concerned.