Getting started is difficult and can be very costly. When I started on here, many of the good 'ole' (referring to experience, not age) magicians gave me the advise to by books, used when possible. Most of these books provide the newbies with many good tricks you can learn at home for very little money (try to find them used on eBay or Amazon. This alone gets one started learning many of the basic moves one must learn to have a good show....below are some books I really have learned from. Some really good tricks to learn early on, that are not terribly hard are also noted below....have fun, don't spend a ton of money buying 'tricks' for a while...learn basic good tricks from books and save lots of cash for later.....I avoid looking at U-Tube as much of the 'Magic shown' does not use good techniques, and learning tricks using a tried and true technique helps one build on the magic you learn. If you learn the basics wrong, it will haunt you for a long time as you more to more complex applications....I have been there, done that!
Books: Mark Wilson's Complete Course in Magic (I found used on eBay, My best first book, covers most types of magic), The illustrated Compendium of Magic Tricks by Nicholas Einhorn (Another good starter with good illustrations for 375 tricks), Card College Volume one by Robert Giobbi's (a great detailed starter on card handling and card magic)
Good Starter Tricks
Sponge Balls (learn how in the books, sponge balls are low cost props)
Self Working Card Tricks and Card Handling (From above books)
Rope Tricks (Professors Nightmare, and Cut and Restored Rope)
Cups and Balls
Chop Cup
Good luck