What do you work on? I find there's a huge difference between languages and projects. I don't mind writing Haskell or Python in emacs, but I couldn't live without VS for our fairly large C++ project.
Debugging without the IDE is a huge pain. Finding what functions call a given function, when many functions have the same name (so forget grep), is also something we use very very often to check what impact a change would have. That and "go to definition".
Well, I do develop C++ with Emacs, and I don't miss VS a single day.
And you can 'Go to definition', 'find callers', etc, with other editors (Emacs, Sublime, SourceInsight, etc) ...
Debugging without the IDE is a huge pain. Finding what functions call a given function, when many functions have the same name (so forget grep), is also something we use very very often to check what impact a change would have. That and "go to definition".