Super-resolution is found an ambiguous term in the disciplines of informatics, physics, optics, astronomical imaging and spectroscopy (Bauer, 2011). It is ambiguous in the sense, that super-resolution has different meanings: (1) Resolution beyond the optical diffraction-limit, or (2) resolution below the barrier of the dimension of a pixel of a modern imaging sensor. In other words, super-resolution tries to reconstruct information between pixels, and possibly breaks the optical limit to resolution.

Den Dekker & Van den Bos (1997) provided a review on resolution and concluded any limits to resolution are arbitrary with no fundamental limit to resolution. In astronomy domain limits to resolution depend on the dimension of a modern imaging sensor and the aperture of the optics. Aperture of the telescope defines a point spread function that limits separation of two close stars at certain distance that is closer than the diameter of the Airy disc. This is called the diffraction limit to resolution. Ground-based astronomy suffers from atmospheric blur. Starting with an aperture diameter of 10-15 cm, stars start to dance and "explode" in the focal image plane due to atmospheric turbulence. In addition a typical astrophotography image scale is the result of the focal length of the telescope and detector resolution. Often such adaptation does not consider resolution of the telescope at the diffraction limit. These situations may be solved by super-resolution: (1) improving atmospheric blur (eg. from seeing) e.g. by sophisticated deconvolution techniques, (2) resolution beyond detector pixel dimension and (3) eventually undergoing diffraction limited resolution.

Figure 1: The technique of super-resolution applied to a set of images of Messier 57. Click to enlarge.

The above image of Messier 57, the famous ring nebula, demonstrates well one approach to super-resolution. The original raw image (left) is a sample of 15 images taken with an 8" Cassegrain. A super-resolved image was processed by an appropriate method and shown at the right. Image resolution of the raw input frames is limited by the pixel dimension of the detector. The super-resolution output presents a more detailed image of Messier 57 at a finer pixel grid and better resolution. Spatial resolution is increased by a factor of four with no additional optical aid or magnification, like adaptive optics or the use of a Barlow lens.

The sampling theorem considers optimal adaptation of the detector to the optical aperture. From Fourier optics follows: Detector pixel dimension must be equal or lower than half the diameter of the airy disc. Practical applications in astronomy typically ignore the Airy disc diameter, which is often much smaller than the pixel dimension of the detector at the given focal length. Adapting a camera for a larger field under-samples the Airy disk, resulting in images that do not provide diffraction limited resolution. In best case blurred stars look like point-light sources even in case of bad seeing (>3 arcsec). 

Example: A telescope of  20 cm aperture diameter shall provide a field of view of roughly 1° using a DSLR. Given a sensor width of 22 mm, we can compute a required focal length of about 1300 mm. This is true for my 8-inch telescope using a focal reducer with f/6.4 and a Canon EOS 60D-a. Computing the relation between pixel dimension and theoretical diffraction limited resolution of the telescope, there is a mismatch between pixel dimension and the diameter of the airy disc. The APS-C sensor of the camera provides 2592 pixels along its width at approx. 1° field of view. This covers a sky portion of 1.4 arcsec per pixel. However, theoretical resolution according Rayleighs formula yields 0.69 arcsec resolution for this telescope. If we want to perfectly adapt the sensor to the diffraction limited resolution, two pixel are required to cover the Airy disc diameter. This yields a field of view, that is just a quarter of the field of view using my focal reducer setup. Hence, the images taken with the focal reducer are under-sampled by the factor of four compared to the diffraction limited resolution of the telescope.

Figure 2: Experimental approach to achieve sub-pixel resolution from a series of images of the globular cluster Messier 13. The raw detector resolution (left) at magnification 4x demonstrates the limited detector resolution from spatial pixel dimension. Using different synthetic point spread functions and sub-sampling one can achieve a finer grid resolution at sub-pixel scale beyond diffraction limited resolution (middle, right).

Puschmann & Kneer (2005) presented remarks of having found no gain in resolution from techniques applied to achieve super-resolved images. However, the reasons they provided probably have to be interpreted in a different way. Den Dekker & Van den Bos (1997) presented a review on resolution from physics and information theory. They concluded any limitations claimed - like the theories of Rayleigh or Dawes - were the result of having defined arbitrary limits for special cases, like human vision. Careful reading of the original work of Rayleigh, for example, indeed presents considerations about human vision and contrast. This assumptions define a special use case, but does not mean to have found any fundamental physical limit to resolution.

According to Max Planck, we should consider the image formation process on any imaging detector as a statistical process of detecting individual energy events (photon events) on the detector surface. It is interesting to note: Noise and statistical image motion are essential requirements for the task of super-resolution to work. Thinking further, resolution limits are more driven by the natural Poisson noise in the light detection process. The more light, the more resolution. Image noise can be successfully overcome by powerful noise reduction techniques, like the introduction of wavelet denoising using Anscombe transformation into the Richardson-Lucy image restoration algorithms by Murtagh & Starck (1995). Recent efforts in microscopy showed, super resolution beyond diffraction limited resolution is possible with low-light level applications, like fluorescence microscopy. Wolf et al (2019)  presented images of microscopy resolution at 60 nm, which is four times better, than the theoretical diffraction limit of a large aperture microscope objective (200 nm resolution).

There still remains a paradox to be discussed, that is introduced by considerations from Fourier optics (Goodman, 2005). There is a limit given by sampling at the outer frequency boundary in the frequency domain. This is called the bandwidth limit in the Fourier spectrum. This boundary is the result of the finite (telescope) aperture of the optical imaging system. Theoretically and practically there will are no frequencies beyond this barrier that correspond to spatial details in the image domain. In microscopy this is the Abbe limit. In astronomy it is reflected by the diameter of the Airy disc, which is close to the criteria of Rayleigh or Dawes. Sementilli et al. (1993) tried to discuss this problem for discrete point sources as the ability to perfectly fit the shape of the unresolved spatial frequencies in the frequency domain. They claim to have shown that "for discrete-image models it may be possible to achieve substantial accurate bandwidth extrapolation, depending on the relationship among sampling rate, optical cutoff frequency, and object extent". The last words haven't been proven or brought to paper on this topic yet, however. One thing seems clear from the many experimental studies done by amateurs: Planetary imaging already might have shown, super-resolution works well for extended light sources.

CCD or CMOS imaging sensor pixel do not provide full coverage of the light sensitive sensor surface. Electrical wiring on the chip may shadow certain portions of the sensitive area. RGB color sensors using Bayer color matrices, like astronomical color cameras or DSLRs, have gaps between different color pixels. Such gaps on the light sensitive surface in and between pixels introduce some complexity to the theoretical discussion. Also, on should expect systematic position deviations from fitting stellar profiles (Bauer, 2009). This discussion and the level of details are not in scope of this article, however.

Discussion 

Low resolution input images of the famous ring nebula Messier 57 have been taken with an amateur telescope of 20 cm aperture (f/6.4) at a scale of 1.4 arcsec per pixel. A photographer might think of the telescope as a 1300 mm teleoptics. At extreme magnification, the raw low-resolution image of the nebula looks blocky. While an optical binary star below the nebula appears blocky and smeared, separated only 3 pix in the raw input frame (left), it is well separated in the super-resolved image (right). The fine details reconstructed from the noisy images of the nebula are seen as real features, when compared to an image of the Hubble Space Telescope! Therefore, this method of super-resolution provides an excellent image fidelity and works for both, single point sources and extended light sources. The super-resolved image (right) demonstrates well to have broken the pixel barrier by a factor of four and reflects an image scale of only 0.3 arcsec per pixel. The diffraction limit of the telescope computes to 0.65 arcsec at 550 nm wavelength.

Conclusion and Remarks 

Super-resolution refers to two different meanings to overcome certain resolution limits of an image, like atmospheric blur, limited pixel dimension and eventually diffraction limited resolution. Super-resolution also will help to remove atmospheric blur from ground-based imaging and provides better spectral resolution with a spectrograph. The outer frequency limit of the Fourier spectrum is given by the telescope aperture and sets a maximum frequency of the spatial resolution. When I studied speckle interferometry,  this also was claimed a limit to maximum possible resolution, like Abbe's resolution limit in microscopy. Practical applications have shown, however, super-resolution is possible. 

Observational data 

References

1. Bauer, T., 2011. Super-Resolution Imaging: The Use Case of Optical Astronomy. Proceedings of the IADIS International Conference Computer Graphics, Visualization, Computer Vision and Image Processing, CGVCVIP 2011, p. 49-59, Rome, Italy, 2011, ISBN 978-972-8939-48-9
2. Bauer, Thilo. Improving the Accuracy of Position Detection of Point Light Sources on Digital Images. Proceedings of the IADIS Multiconference, Computer Graphics, Visualization, Computer Vision and Image Processing, Algarve, Portugal June 20-22, 2009, p. 3-15, edited by Y. Xiao, T. Amon and P. Kommers. ISBN: 978-972-8924-84-3.
3. Den Dekker, AJ and Van den Bos, A., 1997. Resolution: a survey. Journal of the Optical Society of America A, vol. 14, no.3, p.547-557. 
4. Goodman, J. W., 2005. Introduction to Fourier Optics. 3rd edition. Roberts & Company Publishers.Sementilli, P. J. et al., 1993. Analysis of the Limit to Superresolution in Incoherent Imaging. Journal of the Optical Society of America, vol. 10, no. 11.
5. Murtagh, F., Starck, J. L., & Bijaoui, A. (1995). Image restoration with noise suppression using a multiresolution support. Astronomy and Astrophysics Supplement, v. 112, p. 179, 112, 179.
6. Puschmann, K. G. and Kneer, F. 2005. On super-resolution in astronomical imaging. Astronomy & Astrophysics, Vol. 436, p. 373–378.
7. Wolf, D. M., Segawa, M., Kondadi, A. K., Anand, R., Bailey, S. T., Reichert, A. S., ... & Shirihai, O. S. (2019). Individual cristae within the same mitochondrion display different membrane potentials and are functionally independent. The EMBO journal, 38(22), e101056.