The folks at Bostonography are using the concept of crowdsourcing to define neighborhood boundaries. For the North End, there is not much dispute until you get to the areas south of Christopher Columbus Park and west toward the Bulfinch Triangle. In the map below, the darker area represents greater consensus about what is the “North End.”

North End (n=151). Along with Beacon Hill this is one of the best-defined neighborhoods, being easily recognized by residents and outsiders alike. The only uncertainties seem to be parts of the Bullfinch Triangle and Christopher Columbus Park (where, it’s worth pointing out, there is a sign welcoming you to the North End), and all the piers. But even those places are mostly above 50%.
The crowdsourced North End map does a good job of also showing the “waterfront” being the harborside edge of the neighborhood with a high (over 50%) association with the North End, but not as much as the interior.
Downtown gets lumped in with the Financial District by the Bostonography crew. As they explain, the consensus seems to define blocks as Downtown when they are not considered part of any other residential neighborhood.

Downtown/Financial District (n=118). “Downtown” is fun to try to define, but we’ve wrapped it up here, perhaps not entirely appropriately, with the Financial District. Besides the easier-to-locate Financial District, to some degree this looks like a process-of-elimination neighborhood: if it isn’t Chinatown, the North End, etc., it must be downtown. Maybe that’s because it’s least residential neighborhood of the bunch, so there’s not really a strong identity.
In case you were wondering, there is no mention of the Wharf District.
See the rest of the crowdsourced neighborhoods at Bostonography.com, a site filled with creative maps including the North End’s contribution to “Boozetown.” You can also be part of the “crowd” and submit your own map here.