Put liners in a cupcake pan and set to the side.
Prepare the cake batter according to the box directions.
Fill the cupcake liners about 1/3 full and bake for 10-15 minutes.
Remove the ice cream from the freezer to allow it to soften.
Allow the cupcakes to cool completely.
Fill the remainder of the cupcake liner with ice cream pressing down to remove any trapped air.
Place the cupcake pan in the freezer to harden for at least 1 hour.
In a large bowl, cream the butter and powdered sugar.
Beat in the milk, mixing until fluffy.
Transfer the icing to the corner of a medium sized zipper bag, try to minimize the trapped air and close. Clip a small piece off the corner to pipe.
Pipe the icing around the top of each cupcake and return to the freezer until ready to serve.
Remove the cupcake liner before serving.