The new graphics are the best improvement, as they are better detailed while completely keeping the retro feel of the old game.
It's unfortunate that the new music isn't as good as the original and that the original music option's volume is ridiculously low comparatively, but the new music isn't terrible and not worth crying over, let alone condemning the game.
As for the translation... who the Hell pays attention to this stuff? The only part that jumped out to me as blatantly different was Balrog's "OH YEAH!", which I think is more in line with his character than the fan translation anyways. I cracked up when he burst through the wall shouting that.
If you liked Cave Story enough to have spent money on it, pick this one up. Pixel's even on board, so it's not going to get much better than this.