Dressing Up To Dress Down?

I'm obsessed with sophisticated dress. Okay, I don't know if it can be called an obsession, but I love it. A shirt and tie with a nice pair of pants has to be one of my most favorite things to wear. But, honestly, I only wear it once, maybe two times a month. I have a hard time "dressing up" when no one else around me will. Shane and I were talking the other day, as i was wearing a really cute sweater with a new polo, and I told him that it bothers me that I am the only one who tries to present myself better, not bothers me as much as it makes me uncomfortable looking spiffy when no one else does. His response was very general, and it got me thinking, he said "You always say that you like to dress like that, but then you say it is uncomfortable, when in all actuality it is everyone else that your making 'uncomfortable' because we feel like we have to present ourselves better." And, I know that doesn't say don't dress how you want, in fact that tells me to do it more. And not because I want to make anyone uncomfortable, but if I like it as much as I say I do, then why does it matter how it makes anyone else feel, if it makes me feel better about myself, and gives me a bit more of a backbone, then go for it. 

I am not sure if that last sentence really makes sense, but if you can't tell from reading this, which as of right now I think there are maybe like 2 people, I have self esteem issues. I'm working on them, well trying at least, and things have been getting better;  I just feel so good about myself when I "dress up" even though I wouldn't really call it 100% dressing up. 

Here is where I start, black button down, cute purple tie,  and nice jeans.

now to get a cardigan, and a new black sweater.

here is my ideal outfit: