Coat ice cube tray with nonstick cooking spray.

Place chopped chocolate in a large heat-safe bowl and set it over a pot of simmering water. Let chocolate melt, stirring occasionally. Remove from heat and stir in vanilla extract.

In another bowl, whisk together cocoa powder and confectioners' sugar. Add the dry ingredients to the melted chocolate, a third at a time and mixing between additions, until all of the dry ingredients are incorporated into the mixture.

Press chocolate into the prepared tray, dividing evenly among eight cubes and pressing into an even layer. Insert a wooden popsicle or lollipop stick into each cube and let it harden at room temperature (or in the refrigerator).

Once the chocolate cubes are hardened, remove them from the ice cube tray. Store in an airtight container for up to a week at room temperature, or two weeks in the refrigerator.

To serve, place a hot chocolate cube into a cup of hot milk. Allow it to sit and soften for a minute, then stir until it is dissolved.