1 : the point directly above, below, or at which a nuclear explosion occurs
2 : the center or origin of rapid, intense, or violent activity or change; broadly : CENTER 2a (the party town that served as ground zero for those corporate...bashes -- Rich Eisen)
3 : the very beginning : SQUARE ONE
The World Trade Center was not bombed (at least, not in 2001), and no nuclear explosion occurred there. Two buildings within a seven-building complex were struck by airplanes, at two different times. And the wholesale destruction of the WTC, and several other nearby buildings, was caused by the collapse of those buildings, again at two separate times. Referring to a single "ground zero" simplifies what happened at the site to the point of callousness, and also implies some significant disrespect to the sites in Shanksville, PA, and Washington, DC, that were also hit that morning as part of the same attack.
Beyond that, "ground zero" as used to refer to the WTC does not refer to any specific spot within the area, because there is no single spot where the destruction began. Instead it generally refers to the entire site (what I would call "the pit"), which already had a name, a name that's still used on subway signs and maps. Referring to it as "ground zero" erases the history of the place and what it was to New Yorkers before 9/11.
The only place at the World Trade Center that might be appropriately termed "ground zero" was in the parking garage, where the bomb that killed seven people in 1993 went off. The memorial marker above that spot was destroyed on 9/11.