Records suggest that your ancestor who was in Kentucky in the 1830s and after was in Amherst County, Virginia, before that. Don’t just grab the first guy you find in Amherst County, Virginia, in 1820 and assume you’ve got the right one.
There could be more than one guy in Amherst County with that name–see if there are guys with that name still living in Amherst County in 1830 after your guy has left. It could be that your guy wasn’t really in Amherst County at all, but lived near the border and actually appears in adjacent counties in those records.
And it could be that your guy had already left Virginia by 1820.