That is a great point, if you don't expect to be doing any interstate travel (which you shouldn't if you are just learning anyway), a street legal dirt bike will get you around just as well as any other bike. In fact, I've been thinking about getting one for a while.
Like I said though, interstate travel is a no-no on lighter bikes. If you attempt traveling on the interstate a with lighter bike, there is a good chance you'll end up in the ditch the first time a semi truck passes you. I would say that you at least need a 600 cc bike for interstate travel.
A lot of what you pay depends on the make, engine size, year, mileage, and whether or not there is any damage (no surprises there, right?). I think a reasonable range for someone's first bike is between $1000-$5000. And really, even $5000 might be a bit too much for a first bike. Unfortunately, this is probably one of the worst times of year to buy a bike since a lot of people are doing the same thing.
And by the way, I am by no means an expert on this sort of thing. I have two buddies that are starting to race competitively, and I am frequently humbled by how much they know with respect to motorcycles.