That's a good question, and I think it depends on the terrain available where you live. At one point in time, I had 7. I didn't get to utilize all of them, so it was too many. Right now I have four and a half.
1. FJ09 for long distance rides
2. SXV550 for hooliganry
3. GSXR600 trackbike
4. CRF150f for dirt track practice
4.5 GSXR600 project
There's good dirtbiking in the area so I've considered adding a full size dirtbike to the stable. But thus far I haven't. I've also considered selling the FJ because I really don't do long distance rides anymore. If I take a weekend to ride, it's generally at the racetrack.