Line an 8x8-inch square pan with parchment paper, overlapping the sides of the pan slightly. Lightly butter parchment paper. Set pan aside.

Place nuts into a saucepan in a single layer. Over low heat, toast for 1 to 2 minutes. Set aside to cool.

Over low heat, in a heavy bottomed pan, melt butter. Add brown sugar and evaporated milk. Stir continuously until mixture boils and reaches the soft ball stage (240°F on a candy thermometer). Remove from heat.

Add vanilla and confectioners' sugar. Mix using a wooden spoon until vanilla and sugar are incorporated and the mixture begins to have a dull finish, approximately 3 to 5 minutes. Stir in the toasted nuts.

Pour fudge mixture into prepared pan. Cool in refrigerator 30 minutes or until solid. Remove from pan by lifting out the parchment paper.