And before someone corrects me, I'm color blind, but these look similar in color to the ones the OP posted. The shape etc however looks spot on.
Eyes in the front, antenna not bent and pointing forward, legs not spread on the side, but more underneath, body cylindrical body with no distinct variation in size or width, wingless, distinct segments, 6 legs, green, very small, numerous.
The head and forelegs of the one to the far right bottom in my pic resembles the same one at the top in the second OP pic. Springtails are pretty much everywhere.
The egg crate I found at Home Depot measures around 13mm across in the egg crate "squares." Using the "square" as a comparison, these appear to be in the 1.5-3mm size.
Just saying the size is pretty close. Anyway, I'm no entomologist, just tossing it out there.
Anyway, if these are those, they like moisture, but don't live in the water. They live off of algae and decaying plant matter, so cleaning the egg crate and the area along the top of the tank should eliminate them.