I see these occasionally in the forest. I suspect a spider may be involved but have never seen an arthropod of any kind on or near them. The last little blob usually has a visible hole or slit in the top and I've wondered if they are some sort of refuge or a place in which eggs are deposited.
The brown stuff is a mature slime mold and I believe the tiny insect is a Collembola (springtail). About 2.5mm long not counting headgear.
I believe these were built by a potter wasp and contained eggs. The wasp probably provisioned them with insects that it paralyzed and stuffed into the chambers before sealing them.
