Native plants are known to be a more sustainable choice for landscaping because they are indigenous to the area, and therefore should be well-adapted to the local climate conditions. This means that they shouldn't need much (or any) additional water, because they can survive with the area's typical weather patterns. They also require little or no fertilizer, because they should be able to grow with the nutrients found in local soil.
In fact, because they are water efficient and better for the local habitat, LEED, a popular green building rating system created by the US Green Building Council, awards points for native plants. In LEED for New Construction 2009, SS Credit 5.1: Site Development—Protect or Restore Habitat awards 1 point for restoring 50% of the site (or 20% if it includes the building footprint) with native or adapted vegetation. USGBC defines these plants as "indigenous to a locality or cultivars of native plants that are adapted to the local climate and are not considered invasive species or noxious weeds".
Therefore, it's a good idea to choose native (or adapted) plants for your garden, because they should save you time and money from watering and fertilizing. This will be especially important as droughts sweep many parts of the United States, especially in the West and Midwestern United States, which will require super water efficient landscaping. States like Texas are suing other states to get more water, and even the Great Lakes are reporting low water levels. In times like these, it's important not to waste precious potable water on your garden - instead, plant with adapted species that can handle your climate without additional water or nutrients.
How is "Native" Defined?
As David Robson pointed out in his article in the State Journal Register, "When Is a Native Plant a Native Plant?", the definition of a native plant varies, because there is no standard for them. Some consider plants that have been in North America for 200 years to be considered "native" or "adapted", but others would still consider them invasive.
Also, there's a wide range of geography involved... while some may consider a plant that is indigenous to their state as "native", it may not truly be adapted to that area. However, limiting native plants to a small area, like your neighborhood, could be extreme.
Plus, as Robson explains, the landscape and climate is changing in many areas (thanks to development and climate change), so a "native" plant may not be suited to your backyard, which most likely looks nothing like it did 500 years ago. In fact, you should not take a plant from a wild area near you and replant in your garden - you should always go to a nursery or other purveyor of native plants. Plants taken from the "wild" will most likely not adapt well to your garden.
Native Plant Databases
While the definition of "native" certainly has gray area, there are some reputable databases that can help guide you. A good way to check if a plant that you're interested in will function as a native plant in your garden is to see if other neighbors have planted it too. If it's doing well, it will likely adapt well to your garden. You can also ask the staff at the local nursery, because they might have a good idea of which plants will do best in your area.
The University of Texas at Austin has an excellent native plant database for North America. You can sort by a wide variety of characteristics, including state, size, type of plant, color, and when and how often they bloom. You can also search a specific plant and find out whether or not it is native to your area. The university even has a list of recommended native plants by state, which is helpful to gardeners.
If you are worried that you have planted or are about to plant an invasive species, you can use the US Department of Agriculture's plant database. This includes a list of invasive, noxious or introduced plants, which will help you to avoid plants that will likely require a lot of water, fertilizer, and/or herbicide (if its a weed).