Heat oven to 350°F.
Divide 2 tablespoons of the butter among 4 (6-ounce) ramekins. Melt the remaining 1 tablespoon butter and set aside. Evenly divide the brown sugar among the ramekins, sprinkling it into the bottoms of the dishes.
Cut each peach into 12 to 14 slices. Lay the peaches on top of the sugar, evenly dividing them among the dishes and sprinkle with the ginger, if using. Set aside.
In a medium mixing bowl, whisk together the flour, baking powder, baking soda, and salt. In a separate bowl, whisk together the sugar, buttermilk, vanilla, and melted butter. Add the wet mixture to the dry mixture and stir until just combined.
Pour the batter over the peaches, dividing the mixture evenly among the dishes. Bake on the middle rack of the oven until the cake reaches an internal temperature of 190°F, 20 to 25 minutes
Remove from the oven and cool on a rack for 5 minutes. Run a knife around the edge of each dish and turn it upside down onto a plate. Serve immediately with whipped cream or ice cream, if desired.