According to Jet, the Rough Rhinos attacked the village where he use to live and killed his parents. This caused Jet's lifelong hatred of the Fire Nation.
When Aang and his friends were sleeping, the Rough Rhinos surrounded them and attempted to capture them. However, they all managed to hop on Appa's back and escape the Rough Rhinos. Later on, while Aang was placed on trial for what Avatar Kyoshi, one of his past lives did to their "great" leader Chin the Conqueror, the Rough Rhinos arrived to destroy the village, for the glory of Fire Lord Ozai. Despite this, all five of them were quickly defeated by Aang, Sokka, and Kitara.
Some time after this, the Rough Rhinos encountered Zuko and Iroh, while they were fugitives of the Fire Nation. They planned to capture the two, so they could bring them to claim the bounty, but Zuko and Iroh managed to fend them off, and escape. It's unknown what happened to the Rough Rhinos afterwards.