If by "ghetto" people are describing a cultural enclave, principally composed of poor-ish ethnic minorities/immigrants (Asians, Blacks, Caribbeans, Latinos, etc), and thus a strange but possibly very interesting place within a city, fine with me. The historical references, especially relating to Jews, are pretty horrible, but it's at least a reasonable use of the term.
I don't think that's what they mean when they say it. My impression is that it usually means "part of the city that looks like shit, has too many "brown people" in it, and thus is where people should avoid because they, personally, are afraid of it" (or alternatively, people who look like they would "belong" to said locale). Which might be true that it would have more crime than the rest of the city, but unless you actually lived there, then I don't see what danger you are actually in as a traveler or tourist passing through it or even visiting it intentionally.
For example, a good portion of the murders in urban environments are of low level drug dealers (because we've criminalised drugs and created a huge and lucrative black market worth killing others to protect or to gain access to in impoverished neighbourhoods). Most people using this sort of terminology are not low level drug dealers and thus unlikely to attract bullets or even hostile attention. Alertness in a higher crime area is certainly sensible, particularly at night (when alertness generally is useful anywhere). Genuine fear and even hostility toward its residents is not.