In a small bowl, whisk together the flour, baking powder, and salt, then set aside.
In a large bowl, beat the butter and sugar on high with an electric mixer or stand mixer for 1-2 minutes, until fluffy. Add the egg and vanilla and beat again until fully combined.
Add the flour to the butter mixture 1/2 cup at a time, mixing constantly, until the flour is fully incorporated and a soft dough has formed. Cover the bowl and refrigerate for one hour, up to overnight.*
Preheat the oven to 350 F and line a baking sheet with parchment paper.
Sprinkle flour over your counter, or over a large piece of parchment paper, then take half of the dough and place it on the floured surface. Sprinkle flour over the dough and use a rolling pin to roll it out to even 1/4-inch thickness. If the dough sticks at any time, you can add more flour to prevent it, or place another piece of parchment paper over top if you are still experiencing sticking. If the dough cracks around the edges, simply pinch it back together and roll over it again.
Once the dough is rolled out, cut your shapes, taking care to cut them as closely together as possible to utilize all of the dough. Once you have cut out all of the shapes, carefully place the cookies one inch apart on the cookie sheet. At this point you can roll the extra dough into a ball then roll and cut again. Repeat this process for remaining dough.
Bake the cookies for 8-10 minutes, until the edges are browned, then remove from the oven and let cool.
To make the frosting:
Whisk the powdered sugar and milk until the icing is thin enough to spread, but not so thin it will run off the cookies.
Spoon a small amount of icing into a piping (or plastic) bag to outline the cookies. This step makes filling them in with colored icing much easier later.
Once you're finished outlining the cookies, divide the rest into individual bowls and add food coloring to your liking. Add the food coloring to your piping bag, or Ziploc bag with the tip cut off, and use it to outline, then fill in your cookies.
Decorate the cookies with sprinkles, then set aside to let the icing harden slightly, about 30 minutes. Store in an airtight container for up to 5 days, or eat immediately. Enjoy!
If you'd like an icing that fully hardens so that you can stack the cookies on top of each other, boil 1 1/2 tablespoons of sugar and 1 tablespoon of water over medium heat for about 30 seconds, until the sugar is dissolved, then whisk in the powdered sugar and milk until desired consistency is acheived.
If you are having trouble with the dough sticking, add more flour to both sides. It can also be helpful to sandwich the dough between two pieces of parchment paper (dusted with flour) while rolling out the cookies!