Cost of a horse? Depending on the breed, age, and experience level of the horse, and the current market climate, you could be spending from $500-$20,000.
Cost of keeping a horse? Depending on your living situation, do you need to board the horse somewhere or do you have land? If your board- do you need the farm to look after him all the time or are you able to be out every day? It is very location dependant but for example, in my area of the US, full care boarding generally runs about $450-900 a month depending on the amenities of the farm, and partial care places run from $100-350. Most partial care places also require that you provide feed for your own horse. I have partial care for my horse and I also provide his feed, I average about $325/month in board/feed. Factor in also routine vet visits (shots, coggins, teeth), could be around $200-500/year. Factor in farrier, barefoot or shod? Barefoot, around $25-45/trim every 4-8 weeks, shoes might be around $60-150 every 6-8 weeks (maybe more if your horse requires special shoes). You should also have some emergency money saved up - horse's like to injure themselves at the worst times. I always have at least $500 cash ready, and I have an emergency credit card with a $5,000 limit just in case.
You need to start by learning how to stand up at the walk. My students are always totally competent at standing before we ever trot. Stand up in your stirrups and learn to find your center of balance over your horse. You should not touch the horse at all with your hands - it all comes from your leg and core. Your leg will learn the correct position. If your leg slides either forward or backward you will end up sitting down again. You have to find the correct sweet spot and straight line of heel to hip. Work on standing at the walk and try and hold for a few strides, then a circle, then the short side of the arena, then the long side, until you have built your way to standing an entire lap of the arena.
An un-mounted exercise - trot yourself around a room and try and stand a little taller every other step. A trot is 2 beats. When a human runs it is 2 beats. Next ride you have, while doing a sitting trot COUNT the beats out loud. 1 - 2 -1 - 2 - 1- 2- 1 - 2, over and over. Try and stand on the "1" beat and sit on the "2". It will take a little time to master it- but once you figure out the rhythm it will just click.