Load times really refer to two different things. When you load in a sampled instrument, your sampler won't load all the samples by default. There'll be an initial load, and then, as the samples are used, the remaining parts are streamed from disk on demand. Keeping initial load times low is nice, but won't really affect your work that much other than when you first load new samples or open the project. I wouldn't consider a hard drive based on disk speed alone.
Beyond that, there's a tradeoff between amount of RAM and disk speed. The faster your disk is, the more you can stream on demand, and thus the less you need to load into RAM to begin with (which of course shortens load times as a secondary benefit). If you have slower drives, they'll be able to stream less and you'll need more preloaded into RAM.
There are other tricks too, like only having a few sampled instruments loaded into memory and bouncing your other tracks to audio (which also helps reduce CPU load; it's much easier to play back an audio track than it is to trigger a virtual instrument and the effects assigned to it).
The bottom line is this: if you have enough RAM and a fast enough CPU to work with a small number of virtual instruments plus the remaining audio tracks for your project, any upgrades are more for convenience or to improve your workflow; this is why big-name orchestral composers have many networked machines, all playing back various orchestral samples at the highest quality.
There are two important questions though that I haven't seen you answer yet:
1) What are your needs in terms of sampled instruments? Are you wanting to add strings to a rock track? Are you wanting to get into orchestral writing where you need a full orchestra's worth of instruments in one song? If you're a hobbyist, you don't need to spend a ton of cash to try and meet the optimal specs for your sample libraries; you can often make do with less.
2) Have you run into any actual problems with your current rig so far? If you're spending money to fix a definite problem, especially if that problem hampers your creativity (eg. if you are being forced to bounce a lot of tracks and the inconvenience of doing so is hurting your musicianship), by all means upgrade, but if you're mostly just concerned about wanting a better rig or being worried about minimum and recommended specs for sample libraries without seeing how those stack up with how you'll actually use the libraries, it might be better to wait. Even waiting 6 months on an upgrade will mean you can spend the same money and get something better than you would if you upgraded now.
About minimum and recommended specs: I'd check around for any libraries you're thinking of purchasing and try and get an idea of how your specs will measure up, the way you'd actually use the library. If you're using a few articulations for a solo flute, for example, you can definitely get away with less than the recommended specs. Worst case, if you have the money for both your desired library and a computer upgrade, is you can always buy the library, see how it performs, and if it's not where you want it to be, upgrade then, and if it performs well enough, hold off.
There's all kinds of tradeoffs here; make the choice that's right for you based on your needs.