Preheat oven to 350 degrees. Bake sweet potatoes until fork tender, about 30 minutes. Remove from oven and allow to cool slightly.
Scoop out centers into a small bowl, leaving a little sweet potato around edges to help skins keep their shape. Mash potato with a teaspoon of butter, then season with salt, pepper and a pinch of nutmeg or cinnamon.
Divide potatoes evenly between the skins and place in a shallow baking pan.
For brown sugar & pecan topping...
Combine flour, brown sugar, and chopped pecans in a small bowl. Drizzle with melted butter. Mixture should be crumbly, not soupy. Divide topping over four potatoes.