It’s easy to see how stories told by a family member are reliant upon the accuracy of their memory. After all, the family member is telling you those stories. But most of what is contained in other records comes from someone memory as well. The person’s memory of what they were told may be accurate (or it may not). What they were told originally may have been accurate or it may have not.
Information on census records, marriage applications, birth and death certificates, and a variety of other documents comes from someone’s memory. No “proof” was necessary and the clerk or census taker simply wrote the information down and went on.
It’s not just Grandma’s memory of personal stories that a genealogist relies up. It’s the memories of many people that were used to create the records we use.