Preheat oven to 350 degrees.
In a small saucepan add all four of the pumpkin sauce ingredients and heat over med low, stirring continuously.
Stir the sauce until creamy and just about to boil. Remove from heat.
Remove the top rind of the brie by carefully cutting away the top layer of skin/paper.
Place the brie in a small oven safe skillet (I like cast iron) or small baking dish.
Spoon the pumpkin sauce all over the top of the brie.
Place the skillet in the oven for 15 minutes until warm and gooey.
Meanwhile, add brown sugar and almonds to a small skillet and heat over medium low.
Stir constantly until toasted and coated with the melted sugar. Remove from heat.
Spoon the almonds on top of the warm brie.
Add sliced fruit and bread on a platter and serve with the brie for dipping.