Hemorrhoids are a pretty common condition, almost everybody will deal with them at least once in a life time. Statistically they are more common in men, but women get them too especially during pregnancy or during menstrual periods. Most of the times they go away by themselves in a matter of days or weeks, but sometimes they persist and they require adequate treatment.
What you are describing sounds indeed like a prolapsed hemorrhoid, which are internal hemorrhoids that are pushed outside the anal opening mostly during normal bowel movement. But in order to be sure about that, you have to go to a specialist doctor for a check up.
Usually, in order to place a diagnosis, doctors do a visual checkup first, and if needed an internal lower rectum inspection using and anoscope or colonspoe. In some cases the doctor may decide on an thorough colon check in order to rule out other diseases which have similar symptoms.
Considering that hemorrhoids are not life threatening, before going to the doctor you could also try some home remedies and see if any of them work for your condition, but if you see no improvement you should definitely consult a specialist.