No trip to New England is complete if you manage to miss the coast of Rhode Island, down in the towns of Galilee and Point Judith, where you catch the ferry to Block Island. The Block Island Ferry ride is a really raw oceangoing experience, not for the queezy landlubber. Eat at any number of fine local establishments, but after the return trip, please!
Not to discount the quaintness of smaller coastal towns such as Watch Hill (just east of the Connecticut border), and the scenic drive along Brenton Point (just south of Newport).
Click on an image below for an 800x600 view of the image, suitable for Windows Wallpaper.