Ruthven family history

Hi - my grandfather was Frederick Hugh Ruthven, only son of Adolphus and Elizabeth Ruthven in Windsor. Adolphus was a son of Hugh and Emily Ruthven, namesake for Ruthven, Ontario. Hugh was the first postmaster.   I have much of their history, but not a lot about my grandfather, who was apparently "disowned" by the family for marrying my grandmother Jessie Chalmers. Her family may have been known to be troubled.  They eventually moved to Toronto but I am not sure of the year. My mother was born in 1919 in Michigan. They moved back and forth over the boarder a few times and my grandfather spent a year in Florida working as a mechanic.  I have seen a Windsor Bell directory with a Fred Ruthven listing in 1944, but I was not aware that he had ever returned to Windsor. He and my grandmother raised 5 kids and lived in Niagara-on-the-Lake in later years and then she moved back to Toronto after his death. He did work as a guard at the Campbell Soup Plant in Toronto. Any information would be welcome. I am writing a history of my parents' 54 year marriage and including a look at their ancestry as part of it. My mother never knew or talked about it.  Thanks for any help you can provide... Kim Denstedt