Gen'ichiro was shown to be the most obviously hateful to Kumi for no other reason then he did not approve of her, despite the fact that Kumi was a kind and loving woman. When Kaoru was upset over what happened with his mother, Gen'ichiro's response was to slap Kaoru telling him he is a Hanabishi.
Also, Gen'ichiro subjected Kaoru to horrific physical and emotional abuse throughout Kaoru's time with the Hanabishi Family and after Kaoru's mother died, Gen'ichiro instead of comforting his grandson, burns all mementos of Kumi much to Kaoru's horror and anger, he saves a charm with his umbilical cord in it. Gen'ichiro responded by beating him with a bamboo cane screaming at him he is a traitor and 'repaid years of kindness with contempt' (though it is plain Gen'ichiro has no real concept of what love and kindness really are). Kaoru refuses and Gen'ichiro continues to torture Kaoru with a bamboo cane but Kaoru did not break.
After this, Kaoru leaves the Hanabishi for good and Gen'ichiro is an minor but still important character because of all the Hanabishi family he is the most responsible for Kaoru's hatred of them and as events progressed he set in motion events more as the manga progressed.