Why did Joseph Smith’s family believe his story? Wouldn’t they be the first to see through the lies?

Wouldn’t the faith of his family count as strong evidence for the truthfulness of the restoration? In speaking of Joseph’s father, M. Russell Ballard said:

In speaking of Joseph’s father, M. Russell Ballard said:

Joseph Smith, Sr., was in tune with the Spirit of the Lord. He knew that his young son spoke the truth. He not only believed the boy’s words but encouraged him in the work he had been called to do.
Joseph, Sr., endured ridicule and persecution because of his prophet son’s experiences and claims. Yet, he was unwavering in his loving support and defended his son.
He saw and handled the plates of gold from which the Book of Mormon was translated and testified throughout his life to the truthfulness of that sacred book. His name remains firmly affixed, with those of the other witnesses to the Book of Mormon, in the front pages of that second witness of Jesus Christ. On one occasion he was imprisoned and told he would be released if he would deny the Book of Mormon. Not only did he not deny it, but he converted two persons during his thirty-day confinement.
– M. Russell Ballard – The Family of the Prophet Joseph Smith

And of Joseph’s brother Hyrum it was said:

“I thank God that I felt a determination to die, rather than deny the things which my eyes had seen, which my hands had handled [the plates from which the Book of Mormon was translated], and which I had borne testimony to, wherever my lot had been cast; and I can assure my beloved brethren that I was enabled to bear as strong a testimony, when nothing but death presented itself, as ever I did in my life.”
– Times and Seasons, Dec. 1839, p. 23.

Wouldn’t Joseph’s family be in a great position to know if it all was a fraud?

See:

M. Russell Ballard – The Family of the Prophet Joseph Smith

Add