Preheat oven to 350°F (175°C). Line a 12-cup muffin tin with paper liners.

In a large mixing bowl, beat butter and sugar together on medium speed until light and creamy, about 2–3 minutes.

Add the eggs one at a time, mixing well after each addition. Mix in the lemon extract.

In a separate bowl, whisk together the flour and baking powder. Add to the butter mixture and mix on low speed until just combined — don't overmix.

Pour in the milk and stir until the batter is smooth and uniform.

Divide the batter evenly among the prepared liners, filling each about three-quarters full.

Bake on the center rack for 22–25 minutes, until the tops are lightly golden and a toothpick inserted in the center comes out clean.

Transfer cupcakes to a wire rack and cool completely before frosting, at least 30 minutes.

In a large bowl, beat softened butter on medium speed until fluffy, about 2 minutes.

Mix in the lemon extract. Gradually add the sifted powdered sugar, one cup at a time, mixing on low speed and scraping down the bowl between additions.

Add the milk and beat on medium-high speed until the frosting is smooth, fluffy, and pipeable.

Divide the frosting into three bowls. Leave one white. Add red gel food coloring to one bowl and blue to another, starting with a small amount and adding more to reach your desired depth of color.

Fill a piping bag fitted with a large round or star tip with the blue frosting, and another with the red frosting. If using a multi-color coupler, attach both to pipe a swirl. Otherwise, pipe them on separately.

Pipe the red and blue frosting onto the cooled cupcakes. Then fill a third piping bag with the white frosting and pipe a small star on top of each cupcake to finish.