There are different rules, advantages, and drawbacks depending on which type of IRA you have. Whether you are a first-time homebuyer or a seasoned one, there are many factors to consider before taking money out of your IRA to buy a house.