Image Deblocking Scheme for JPEG Compressed Images Using an Adaptive-Weighted Bilateral Filter

Article information

Abstract

1. Introduction

Block-based discrete cosine transform (BDCT) coding has been widely adopted by the current mainstream image/video compression standards and technologies because of its excellent properties of energy compaction and decorrelation. For example, JPEG [1] and H.265/HEVC [2] are the most common international standards for still images and video compression, respectively. In an image compression based on BDCT, the image degradation would become visible when the compression ratio exceeds certain levels. It leads to the discontinuity of intensities between adjacent blocks due to the fact that images are segmented into rigid blocks and the correlations between blocks are neglected, namely blocking artifacts. In particular, the BDCT coding images will exhibit severe blocking artifacts when the bit rates are very low. Additionally, image-blocking artifacts not only become visually annoying, but also have a substantial effect on further image processing. Therefore, image deblocking is necessary for highly compressed images.

In order to reduce the shortcomings of BDCT, many researchers have proposed numerous algorithms. The algorithm that processes images only at the decoder side is called a post-processing algorithm. A post-processing algorithm is more practical because it does not affect the protocol contents of the encoder side and the protocol doesn’t need to be changed. These algorithms mainly include spatial filtering [3], transform domain filtering [4], projection iterative method [5], and so on.

To interact with the outside world effectively and rapidly, the human vision system (HVS) must extract and process a large variety of visual information from the captured images. Rather than adopting a uniform method to process these extracted structures from the captured images, HVS should be able to adapt to the different characters of these different extracted structures. Therefore, the maximum amount of information can be extracted from the extracted structures for the subsequent recognition stage and for image processing. It is this point that an adaptive and effective image processing system should be designed in order to make the machines have the adaptive capabilities of biological vision systems. A lot of works and researches have been done for adaptive image deblocking. For suppressing blocking artifacts and preserving image salient features, Foi et al. [6] proposed an adaptive image filtering method using the shape-adaptive DCT (SA-DCT), which empirical Wiener filtering and hard-thresholding technology were adopted in the SA-DCT domain. In addition, Zhai et al. [7] proposed one kind of a comprehensive image-deblocking algorithm, including three processing stages. In the second stages, block-wise shape adaptive filtering was proposed to further ameliorate edge features. To avoid possible over-smoothing, a quantization constraint set is also adopted in the third stage. Furthermore, the proposed method has relatively low computational complexity. According to the process analysis of image coding and the characteristics of a BDCT compressed image, we can formulate image deblocking clearly as optimization problems that solve the objective function using a set of convex functions [8]. Wang et al. [9] proposed an adaptive spatial post-processing method that is composed of three steps: thresholding, model classification, and a deblocking filter. Compared with these proposed adaptive image-deblocking methods, the most straightforward way is to conduct low pass filtering around the block boundaries. However, low pass filtering filters out some high frequency information, which leads to image over-smoothing and blurring. To make images obtained by the optical coherence tomography valuable for clinical interpretation, Anantrasirichai et al. [10] proposed an adaptive-weighted bilateral filter. The novel filter not only offers the desirable property of removing speckles, but also preserves texture and useful information contained in the optical coherence tomography images. Hu et al. [11] designed one kind of adaptive depth map filter based on a bilateral filter for image deblocking. The parameter of this proposed adaptive depth map filter is changed adaptively according to image edges and blocking artifacts.

In this paper, an original image deblocking method using an adaptive-weighted bilateral filter is proposed. This novel adaptive-weighted bilateral filter is proposed based on local entropy and improved blind image quality assessment for the sake of eliminating blocking artifacts and avoiding over-smoothing. The rest of this paper is organized as follows: We briefly introduce the related works in Section 2. In Section 3, we present the proposed adaptive-weighted deblocking filter algorithms. In Section 4, experimental results and comparisons are presented. Finally, we give our conclusion and further research in Section 5.

2. Related Works

2.1 Bilateral Filter

The bilateral filter [12] is a tool that can smooth images while preserving image edges and texture regions. The principle of a bilateral filter is analogous to the Gaussian convolution, which is defined as the weighted average of pixel values. The difference between a Gaussian filter and bilateral filter is that the bilateral filter takes into account not only the spatial locations but also the intensity variation to preserve image edges. The principle of bilateral filtering is that two pixels in images are analogous to each other based on spatial locations and similarity in the photometric range. A new weighted average of pixel p can be obtained by Eq. (1):

(1) BF[Ip]=1ωp∑q∈ S[Gσd (‖p-q‖)Gσr (‖Ip-Iq‖)Iq],

where, S is the window space. G_σd (·) and G_σr (·) are the closeness and the similarity functions that are the Gaussian functions with the standard deviations of σ_d and σ_r, respectively. ||·||represents the Euclidean distance, and ω_p is used for normalization.

The performance of a bilateral filter is controlled by two parameters of σ_d and σ_r. σ_d is important in image spatial domain processing because it determines the level of low-pass filtering. A greater value of σ_d will combine and weigh more pixels in the neighborhood together, which leads to a greater blurring effect. σ_r determines the weights in an image intensity domain. According to the value of σ_r, pixel points in the window space S where their values are closer to each other are combined and weighted together. Examples of smoothing images by the bilateral filter with fixed parameters are shown in Fig. 1. The bilateral filter with fixed parameters cannot obtain good results in removing the blocking artifacts.

Fig. 1

Different effects of bilateral filter with fixed parameters: (a) σ_d = 5, σ_r=0.1, (b) σ_d =5, σ_r=0.3, (c) σ_d = 20, σ_r = 0.1, and (d) σ_d = 20, σ_r = 0.3.

2.2 Local Entropy

According to Shannon’s definition of entropy, the image entropy is defined as Eq. (2):

(2) E=-∑i=0L-1[Pi log Pi],

where, i is grayscale and L is the maximal grayscale. Pi=niM×N represents the probability of i in the image, n_i is the amount of pixel points whose grayscale is i, and the image size is M × N. Similar to the image entropy, the local entropy of small neighborhood Ω_k within image E(Ω_k) can be written as:

(3) E(Ωk)=-∑i=0L-1[Pj log Pj],

where, j is the grayscale. Pj=njMk×Nk is the probability of j in the small neighborhood Ω_k. n_j is the amount of pixel points whose grayscale is j in the small neighborhood Ω_k. The window size of Ω_k is M_k× N_k.

For the small neighborhood Ω_k, the local entropy E(Ω_k) is closely correlated with the variance of grayscales. From Eq. (3), the local entropy is smaller in homogeneous regions and larger in inhomogeneous regions. Hence, the pixel points in smooth regions will have smaller local entropy values than those in unsmooth (edge and texture) regions of the image. If we select an appropriate neighbor window Ω for the image, the local entropy of the image would be computed. When the neighbor window Ω is moved on a per-pixel basis within the image from top to bottom and from left to right, the local entropy value of each pixel point will be obtained, that is, we can obtain a local entropy image. Fig. 2 is the local entropy map of Lena. From Fig. 2, we can see that the edge and texture regions can be clearly distinguished from the smooth regions.

Fig. 2

The local entropy map of Lena.

3. The Proposed Scheme Using an Adaptive-Weighted Bilateral Filter

To improve the quality of JPEG compressed images, we need to protect edge and texture information when the image is to be deblocked. In other words, σ_r and σ_d must be larger to remove blocking artifacts in smooth regions. For regions with more significant details, σ_r and σ_d should be smaller to avoid blurring details. Although a bilateral filter can protect image edges while smoothing the image, the filter with fixed parameters cannot work well because it may lead to the loss of some texture information, as shown in Fig. 1. Local entropy can show the discrete degree of image grayscales. For smooth regions of the image, the local entropy is smaller. In texture and edge regions, the local entropy is larger. As such, this paper proposes a novel adaptive-weighted bilateral filter using the local entropy map g(x) as the guidance image to obtain adaptive parameters.

σ_r is calculated for a pixel located at x:

where, n_r is a minimum level of filtering to avoid σ_r = 0. g_max is the maximum value of g(x), and k_r is a constant parameter that controls the mapping process of g(x) to σ_r.

σ_d is calculated for a pixel located at x:

where, n_d is a minimum level of filtering to avoid σ_d = 0, and k_d is a constant parameter that controls the mapping process of g(x) to σ_d. The parameters k_r and k_d are similar to σ_r and σ_d, which determine the performance of the bilateral filter. The framework of our proposed adaptive-weighted filter is shown in Fig. 3.

Fig. 3

The proposed adaptive-weighted bilateral filter for image deblocking.

For images whose blocking artifacts are more serious, we need to select larger parameters ( k_r and k_d ) to smooth images as much as possible. For images with slight blocking artifacts, we should select smaller parameters ( k_r and k_d ) to avoid over-smoothing images. In this paper, we adopted three sets of data for k_r and k_d according to the severity of image blocking artifacts.

In general, it is not possible to obtain the original image on the decoder side. In this paper, we adopted and improved a kind of blind image quality assessment (BIQA) as criterion according to [13] to choose appropriate parameters ( k_r and k_d ) for different decoded images. For the test image signal x(m, n) with m ∈ [1, M] and n ∈ [1, N], and a differencing signal on each horizontal line can be calculated as:

For a 1-D horizontal signal f_m(n) = |d_h(m, n)|, we first calculated its power spectrum for m = 1,2, …, M and then averaged them together. Finally, the power spectrum estimation P_h(l), 0 ≤ l ≤ 2/N can be obtained, which is exemplified in Fig. 4. As shown in Fig. 4, the different original images have similar power spectrum. When we adopted different quantization factors (Q = 1, Q = 3, and Q = 5), the image blocking artifacts can be easily picked out based on the peaks at the feature frequencies (1/8, 2/8, 3/8, and 4/8), as shown in Fig. 5. In addition, if the image is compressed more highly, the peaks will change more significantly. According to the characteristics of the image power spectrum, the criterion to select the parameters ( k_r and k_d ) is to calculate S. This is defined as:

Fig. 4

The power spectrum curves of different images.

Fig. 5

The power spectrum curves of Lena image with different quantization factors.

The image quality scores S of three images, which are calculated by Eqs. (7)–(10), are shown in Fig. 6. The three images are Lena, Barbara, and Peppers. All of them are compressed by JPEG where the quantization factor Q varies from 1 to 10. From Fig. 6, we can see that if the quality of image is bad, S is large. The flow diagram of our proposed image-deblocking scheme is shown in Fig. 7. If S was smaller than α, then we selected (k_rs, k_ds) for (k_r, k_d); if S was larger than α and smaller than β, then (k_rm, k_dm) was used; and if S was larger than β, (k_rl, k_dl) was used. The parameters (k_rs, k_ds), (k_rm, k_dm), and (k_rl, k_dl) were determined empirically and set as follows: (k_rs, k_ds) = (1,10), (k_rm, k_dm) = (3,20), and (k_rl, k_dl) = (5,30).

Fig. 6

The image quality scores of Lena, Barbara, and Peppers images using quantization factor Q from 1 to 10.

Fig. 7

The flow diagram of the proposed scheme for image deblocking.

In order to determine the threshold values α and β (α<β), we selected three images: Lean, Barbara, and Peppers. They were compressed by JPEG where the quantization factor Q varied from 1 to 10. All of them were carried out according to the steps shown below, which are also shown in Fig. 8:

Fig. 8

The flow chart to determine the threshold values.

In this paper, the best value ranges of α and β for the Lena, Barbara, and Peppers images are shown in Table 1. According to Table 1, we selected α = 0.3000 and β = 0.9500.

Table 1

The value ranges of α and β for Lena, Barbara, and Peppers images

4. Experimental Results

To test the performance of our proposed scheme, a lot of experiments have been conducted on monochrome and color images. In this paper, the images with blocking artifacts were obtained by JPEG. For monochrome images, we used three digital images—Lena, Barbara, and Peppers—to test the effectiveness and efficiency of our proposed scheme. For color images, we adopted images from the LIVE database. The method of bilateral filtering with fixed parameters and the method proposed in [11] were performed. These results were compared with the results from using our proposed algorithm to test the performance of our proposed method. The simulation was conducted with MATLAB 2009a. Here, we computed the PSNR and SSIM between the deblocked image and the uncompressed image. PSNR is defined as shown in Eq. (11):

where, MSE is the mean squared error between the deblocked image and the uncompressed image. And SSIM is defined as shown in Eq. (12):

where, μ_x and μ_y are the average of the deblocked image x and the uncompressed image y, respectively, and σx2 and σy2 are the variance of the deblocked image x and the uncompressed image y, respectively. σ_xy is the covariance of the deblocked image x and the uncompressed image y. c₁ = (k₁L)² and c₂ = (k₂L)² are the two parameters to prevent the denominator from being 0. L is the maximal grayscale. In general, k₁ = 0.01 and k₂ = 0.03 by default. If SSIM is equal to 0, it means that there is no correlation between the deblocked image x and the uncompressed image y, and 1 means that the uncompressed image y is almost exactly the same as the deblocked image x.

In terms of PSNR and SSIM, Tables 2–4 show the experimental results of our proposed scheme compared to other methods applied to the Lena, Barbara, and Peppers images at various bit rates, respectively.

Table 2

Comparisons of different deblocking methods applied to image Lena in terms of PSNR and SSIM

Table 3

Comparisons of different deblocking methods applied to image Barbara in terms of PSNR and SSIM

Table 4

Comparisons of different deblocking methods applied to image Peppers in terms of PSNR and SSIM

From Tables 2–4, we can see that our proposed algorithm has excellent performance compared with the two mentioned methods. Our proposed scheme can provide the best results. For these images, the results are improved from a minimum of 0.20 dB up to 1.00 dB in terms of PSNR, and the SSIM is also improved compared with the compressed images. When these images are at very low bit rates, the method proposed in [11] and the method of bilateral filtering with fixed parameters can improve their quality. However, when the images are not at very low bit rates, they cannot improve the quality and may actually reduce it. The reasons for this are that the method presented in [11] only changes one parameter and the method of bilateral filtering with a fixed parameter uses a fixed parameter.

Fig. 9 shows the perceived image quality for the Monarch image at a 0.3211 DMOS score. From Fig. 9, we can see that if we used the bilateral filter with fixed parameters, the image would be over-smoothed, as shown in Fig. 9(b). If we only change one parameter of the bilateral filter (method in [11]), there is still blackness in the wings for the Monarch image, as shown in Fig. 9(c). Our method can achieve better effects for image deblocking.

Fig. 9

Comparison of the subjective quality on Monarch: (a) is the compressed image; (b–d) are deblocked images by the bilateral filtering with fixed parameters, method of [11], and our proposed scheme, respectively.

5. Conclusions

A new efficient image blocking scheme based on adaptive-weighted bilateral is proposed in this paper. We used a local entropy map as a guidance map to change two parameters (σ_r and σ_d ) in a bilateral filter. In this process, we also proposed one criterion to determine k_r and k_d, because blocking artifacts of different images after compression are different. Through these two steps, we were able to obtain good results in terms of PSNR and SSIM.

Our experimental results show that the proposed scheme can reduce the blocking artifacts at low bit rates. In this paper, we did not consider the optimal selection for the BIQA. Furthermore, good improvements can be obtained by considering human visual characteristics and GPU or the parallel method. In the future, we may focus on these issues.

Article information Continued