Edge-Aware Image Restoration Using Hybrid Cnn-Transformer Networks

Abstract

Image restoration is a basic problem in computer vision where the aim is to generate good image from poor observations corrupted by noise and blur, compression, low-resolution and other image degradations. Although traditional CNN-based methods have shown impressive results for image restoration by capturing local spatial patterns, they lack the ability to recover fine structures and long-range dependencies, which is attributed to their small receptive fields. In recent years, global attention mechanisms have been incorporated into Transformer architectures, which are able to capture long-range context, but the enormous computation and lack of local inductive bias make them inefficient for high-resolution restoration tasks. In this study, a novel Edge-Aware Hybrid CNN-Transformer Network (EAHCT-Net) for image restoration is proposed to combine the advantage of CNN and Transformer architectures. CNN-based local feature extraction is utilized to acquire texture information, while self-attention modules based on Transformer are utilized to model the global dependencies within the image. An edge-aware attention mechanism is integrated to explicitly maintain object boundaries, contours and high frequencies during reconstruction. The proposed architecture comprises of four key components, namely, shallow convolutional feature extraction, hierarchical Transformer representation learning, edge-guided feature refinement, and adaptive image reconstruction.