It has the benefit of being computationally efficient since most of the computational energy is used during the one-time training process. Instead of building a specific model and finding the optimal algorithm-specific parameters heuristically (as in sparsity-based optimization approaches), it relies on large amounts of data to automatically learn tasks by finding the optimal parameters in each layer of a neural network. Furthermore, the optimal algorithm-specific parameters in the sparsity-based optimization framework must be heuristically determined.ĭeep learning (DL) is an emerging computational imaging approach dramatically improving the state-of-the-art in fast image reconstruction. Therefore, it cannot achieve real-time imaging for many applications which require pipelined data acquisition and image reconstruction. Depending on the scale and scope of the problem, an image reconstruction task can require minutes to even hours of computation. Although sparsity-based optimization has advantages in image reconstruction, the primary drawback to this approach is that it is iterative and time consuming.
Researchers have successfully applied sparsity-based optimization in a variety of imaging fields ranging from compressed ultrafast photography to holographic video to biomedical imaging. This concept is based on the knowledge that most natural images are sparse (i.e., only a few nonzero values exist) when transformed into a specific domain.
One of the most common methods in computational imaging is sparsity-based optimization which seeks to reconstruct images from incomplete measurements or an ill-posed inverse problem. Therefore, additional information about the scene or the object must be incorporated in the computational process for accurate reconstruction. However, the inverse problem is often ill-posed by design or due to the imperfect physical measurement, meaning multiple solutions exist for a given measurement. These methods rely on a measured or assumed forward operator of the imaging system to create a mapping from the image to the measurement. It relaxes the hardware requirements of imaging systems by relying on (typically iterative) computational techniques to recover the lost information, that is, solving an inverse imaging problem computationally. Overall, this TST-DL framework is a flexible approach for image reconstruction without physics priors, applicable to diverse computational imaging systems.Ĭomputational Imaging is a powerful tool in the application of image reconstruction. Furthermore, TST-DL yields better results than end-to-end training while suffering from less overfitting. The TST-DL approach is shown to perform comparable to approaches which incorporate perfect knowledge of the imaging model, to be robust to noise and model ill-posedness, and to be more robust to model mismatch than approaches which incorporate imperfect knowledge of the imaging model. The results are quantitatively compared with those from other frameworks. We demonstrate this framework using a linear single-pixel camera imaging model. Furthermore, the TST-DL approach mitigates network over-parameterization by separately training the FCL and U-Net. This approach has the advantage that does not rely on an accurate representation of the imaging physics since the first-step training directly learns the inverse model. Then, this pre-trained FCL is fixed and concatenated with an un-trained deep convolutional network with a U-Net architecture for a second-step training to optimize the output image.
First, a single fully-connected layer (FCL) is trained to directly learn the inverse model with the raw measurement data as the inputs and the images as the outputs. To solve these challenges, a two-step training DL (TST-DL) framework is proposed for computational imaging without physics priors. This can lead to suboptimal training and poorer image reconstruction results. Another main challenge is that many imaging inverse problems are ill-posed and the networks are over-parameterized DL networks have flexibility to extract features from the data that are not directly related to the imaging model. Model mismatches degrade the quality of the preliminary image and therefore affect the DL predictions. One outstanding challenge, however, is errors that arise from imperfections in the assumed model. Usually, the preprocessor incorporates knowledge of the physics priors in the imaging model. A common strategy is to use a preprocessor to reconstruct a preliminary image as the input to a neural network to achieve an optimized image. Deep learning (DL) is a powerful tool in computational imaging for many applications.
0 kommentar(er)
