After the etcd encryption is enabled - it is needed to recreate the secrets, so they will be stored encrypted in etcd:
kubectl get secrets -A -o yaml | kubectl replace -f -