Good observation Cathy, and I agree with you.
In the book the limited information provided by the distant family members who remembered him suggests he may have been autistic. Until not so long ago, autism was considered a form of deep psychosis, called infantile schizophrenia, and people were often place in institution for life.
From what we know now, I believe those eyes could well be what Deeds really saw when we looked at people. After all when you see all the attention to details given in his building (drawing every bricks perfectly) and his steam boats, I see no reason to think that his portraits were not as accurate.
What is also sad is that he was treated as a child, given a few pieces of discarded paper and crayons (limited palette fro what is observed)... what would he had done with a few tubes of paint and canvases?
What if van Gogh's brother Theo had only given him few children crayons instead of real supplies?