My aunt just picked up a new '06 325xi about six months ago. I was helping her out with stuff today so she let me drive the car around all day.
First impressions: the car has plenty of room, but the seating position is awkward for a bigger guy. If I had the steering wheel where it should be my knees were in the dash. If I sat farther back for leg room I felt like the steering wheel was too far away. Also, I had to adjust the seat pretty low to not hit my head on the top, but when I did that I had no where to rest my left arm. It made the window ledge too high.
When driving it I noticed two things. The brakes are extremely Extremely touchy and the car is definitely underpowered. The brakes I could get used to (it's like having air brakes in your car), but if I ever bought one I would have to get one with a bigger engine. The ride is a little rough at slow speeds, but not punishing (like say, an STi). Also, it handles bumps very well (I went over train tracks expecting to loose my teeth, but it took them like a champ). The handling is excellent (though I didn't get to take it around the circle, which is like the local benchmark for handling) and the steering is tight, (but the ratio could be a little bit quicker IMO). It liked to understeer a bit when messing around, but I'm not sure if that is caused by the AWD or if they set it up that way on purpose. It had the automatic with the tap shift, which is very nice and shifts very smooth (you don't even feel it most of the time), but with the lack of power I think a manual trans would have served it better.
All in all, I like it. If you are above 6' I would make sure you can find a comfortable seating position. Also, like I said it is very underpowered down low. You have to gun it just to keep up with people taking off at traffic lights and such. I really think my Envoy and my mothers RX330 could beat it in a drag race.

While I don't think it's the ultimate car that everyone hails it to be, it's a nice choice for someone looking for a sporty sedan without being punishing.
TL;DR: BMW 325xi has touchy brakes, not enough power, and uncomfortable seating position for big guys. But, has a great suspension and would make a fine sporty daily driver.
By the way, I've driven a 530i on a few occasions, and while I haven't had the driving time in it I had in the 325xi, I really think I enjoyed the 530i more.