Preheat the oven to 375°F. Line a 12-cup muffin pan with paper liners and prepare all ingredients before mixing.
Whisk together flour, baking powder, baking soda, cinnamon, pumpkin pie spice, and salt in a medium bowl.
In another bowl, combine pumpkin puree, sugars, eggs, melted butter, milk, and vanilla extract until smooth.
Add the dry ingredients into the wet mixture. Stir gently with a spatula until combined. Avoid overmixing because it can make the muffins dense.
Mix brown sugar, cinnamon, and melted butter until a thick cinnamon mixture forms.
Add pumpkin batter into each liner, sprinkle with cinnamon mixture, then cover with more batter until each cup is about ¾ full.
Bake for 18–22 minutes until the tops are golden and a toothpick inserted into the center comes out clean.
Allow muffins to cool in the pan for 10 minutes before adding the drizzle.
Beat cream cheese, powdered sugar, vanilla, and milk until smooth and pourable.
Drizzle over cooled muffins and serve fresh.
