Simulating synthetic image data is crucial for the design and validation of bio-imaging pipelines. Most of the existing frameworks assume objects in pixel space, whereas most of the spatio-temporal models of biological processes are formulated in object space. We show that the key to a physically-principled synthetic image data simulation engine is to model carefully the mapping between objects and pixels. We present a sound mathematical and computational framework for our simulation engine: the virtual microscope. A careful measure-theoretic formulation of the object-pixel mapping allows us to handle the simulation of image data arising from complex spatio-temporal dynamics. Computationally, we show that we can generally approximate the object-pixel mapping by a linear combination of shifted/scaled point spread functions that can be evaluated efficiently. We demonstrate the ability of our framework to handle real-world, complex spatio-temporal dynamics.