Doing this will force Windows to use the defaults and re-build this key using group policy (or local group policy).
From an Administrative PowerShell Prompt on the affected client
Remove-Item 'HKLM:\Software\Policies\Microsoft\Windows\WindowsUpdate' -Force -Recurse gpupdate /force Restart-Service -name "Windows Update"
One thing to note is by unlinking an existing WSUS GPO, or just deleting it, or just setting the values back to “Not Configured” is NOT going to remove the entries from the registry. These registry keys don’t change and don’t remove by just stopping the policy from applying. You COULD set the WSUS Server Location GPO to Disabled, wait for GPO propagation and then set it to “Not Configured”, but with all of the systems that are not receiving GPO updates at the time you change it (remote users, offline systems, etc), it’s entirely possible that you’ll have some system that will only apply the latest state of ‘Not Configured’ leaving them with the registry entries configuring the system to go to WSUS.