Yes, mouse urine stinks. Male mouse urine is even worse. It's not the poop, it's the urine.
Rat urine and hamster urine are much less stinky.
You can get special bedding that neutralizes the odor to an extent. Or set an air purifier next to the cage - I used to do that.
One thing I didn't think to try was odor-neutralizing kitty litter as a substrate; you would still need to provide something softer for bedding. Now that I have a cat, I find that the crystal litter, though more expensive than clay, is excellent at eliminating the odor; and cats also have very smelly urine. \
Avoid using litter with a scent to it, because caged mice can't get away from it and it may be harmful.
To clean the cage, use white vinegar [or any vinegar I guess] to neutralize the urine odor as well as disinfect the cage, then rinse with water with a little baking soda in it to neutralize the vinegar smell.
If you use cloth re-usable bedding, you might also want to soak it with vinegar or diluted vinegar before washing it.