It could work if you get a female and get her as a puppy. Of course you will have to supervise her.
My experience is that a male dog, even if he intends no harm, may not have the instinct to be careful. Some do, but a German Shepherd would be iffy.
I have had a lot of dogs and rodents, so I have some experience this way.
Dogs generally pick up on your attitude. If these rodents are part of the family, the dog will see them as such. Also, if your rodents are calm and show no fear, they are more likely to be treated as puppies. Your dog might still go after wild rodents. Or she might not.
Keep in mind though, if your German Shepherd even steps on a rodent, it might kill it.
A rabbit might be in less danger than you think, though, I had a rabbit that played with my dogs and never met a hostile predator in his life, until one day the neighbor's dog got into the yard and went after him. Quicker than I can tell it, the rabbit turned to tell the dog he didn't want to play; realized the dog was in earnest; took off running at speed toward my brother's wagon, made a right turn, and the dog crashed into the wagon and ran back to his owner. This is a technique that wild rabbits use; he didn't have time to think.
Well, you still have to think about it. Will you be able to put in the care and training to make sure your dog sees the little animals as puppies/babies?
How upset will you be if the dog does kill one or more? Will it ruin your relationship with the dog? If so, you had better not risk it.