Your ancestor may not have arrived at the US port that you think they did. Not everyone came through New York. Your ancestor’s original destination may not be where he settled and that destination may have impacted where he originally landed. Some immigrants to the United States originally settled in Canada and their “port of entry” into the United States may have been a land-locked one. Just because Grandma insists her Grandpa landed in New York City when he arrived as a young boy does not mean that he did.
If he arrived at that age, his granddaughter was not there to witness it.