My wife and I have 4 kids, youngest is 2.5 yrs old. All but the oldest want to ride, and I take them around the neighborhood (not the 2 year old, of course).
My wife started riding late in the summer of last year, and she is still struggling with the concept of taking a ride for fun, rather than going on a mission (grocery store, for example). It is really difficult for us to find the time to ride together, but I sometimes offer to watch the kids so she can go for a quick ride. Sure, I'm a good husband (patting myself on the back), but also realize how much better she feels when she comes back. Besides, I get my riding experience by riding to work (70 mile roundtrip), and I want her to earn her own riding miles so we can eventually ride together better.
We worry about the risk of both of us getting hurt (or worse), what effect that would have on our children. My reasoning is that we always face risks on the road, even in the family minivan, and any crash would have terrible conscequences. We can't burrow into the ground, and I really hope we set a good example for our kids about enjoying life to its fullest extent, having passion for things, and making effort to fulfil our dreams.