1 Introduction

Underwater target detection and motion estimation based on optical technology faces challenges because of low-quality underwater images. The underwater visibility range is typically within 10 m due to absorption and scattering by various particles (Hou, 2009). Many methods, including the time-resolved technique (Tong et al., 2011), range-gating technique (Kocak et al., 2008; Tan et al., 2010), and acoustic imaging (Hou et al., 2013), were developed to improve image visibility in underwater conditions. Polar-ization imaging can increase the contrast of the target and decrease the effect of the scattering (Cheng et al., 2021b). This technique is suitable for underwater use because of its advantages (Dubreuil et al., 2013). Some marine creatures can use polarization vision for navigation, predation, disguise, and communication underwater (Shashar et al., 2000; Waterman, 2006; Cartron et al., 2013). Inspired by these marine creatures, a bionic polarized optical guidance system (Cheng et al., 2020a) was developed for underwater vehicle docking and a navigation technology using underwater polarization patterns (Cheng et al., 2020b, 2020c, 2021a) was in progress.

The optical flow, which reveals the travel distance of the target through the displacement of the corresponding pixels in the sequential images, can reflect target motion and structural information (Vargas et al., 2011). Some creatures can measure the travel distance by optical flow methods (Pfeffer and Wittlinger, 2016). Fortun et al. (2015) reviewed the optical flow technology. The underwater target detection method based on optical flow is under development. Madjidi and Negahdaripour (2006) concluded that optical flow based on HSV can improve the estimation precision for underwater motion relative to other color presentations. Using the classical Horn-Schunk approach and optical imaging model, Drews Jr. et al. (2014) proposed an optical flow algorithm that realizes target detection in underwater environment. Meanwhile, Kumar et al. (2018) proposed a histogram-based method for computing optical flow based on underwater image sequences.

Traditional optical flow is hard to apply in turbid water because of the low contrast between the target and background. In theory, the accuracy of the optical flow method can be improved by adopting imaging techniques enhancing the clarity of underwater images, including polarization imaging. Guan et al.(2018, 2020) studied a polarized optical flow algorithm for polarization navigation and detection; however, turbid underwater target detection based on polarized optical flow was not reported. In this work, we utilize polarized optical flow for turbid underwater target detection. First, we establish the model of underwater polarized optical flow to estimate target motion in turbid water. We then conduct experiments to prove the effectiveness of the method and analyze its accuracy. Finally, we conclude that our method is more effective underwater compared with normal imaging due to the superiority of polarization imaging in the scattering medium. This study can promote marine resource exploration and underwater equipment development.

2 Methods 2.1 Underwater Imaging Model

Poor underwater image quality seriously affects the ability and efficiency of underwater detection. For clear underwater target imaging, the underwater imaging model was first established (Jaffe, 1990; Schechner and Karpel, 2005):

$ I(x{\text{, }}y) = D(x{\text{, }}y) + B(x{\text{, }}y) + F(x{\text{, }}y), $

(1)

where I(x, y), D(x, y), B(x, y), and F(x, y) are the intensity, target signal, backscattered light, and forward-scattered light at position (x, y), respectively. The formulas for target signal D(x, y) and backscattered light B(x, y) are as follows:

$ D(x, {\text{ }}y) = L(x{\text{, }}y)t(x{\text{, }}y), $

(2)

$ B(x{\text{, }}y) = {A_\infty }[1 -t(x{\text{, }}y)], $

(3)

where L(x, y) is the intensity of the target without being attenuated. A_∞ is the intensity of the underwater environment at infinity. t(x, y) is medium transmittance, as shown below:

$ t(x{\text{, }}y) = {{\text{e}}^{ -\beta (x{\text{, }}y)\rho (x{\text{, }}y)}}, $

(4)

where β(x, y) is the attenuation coefficient. In the homogeneous medium, β(x, y) = β₀. ρ(x, y) is the propagation distance. If the less influential forward-scattered light F(x, y) is ignored, then a clear underwater image L(x, y) can be obtained as

$ L(x{\text{, }}y) = \frac{{I(x{\text{, }}y) -{A_\infty }[1 -{{\text{e}}^{ -{\beta _0}\rho (x{\text{, }}y)}}]}}{{{{\text{e}}^{ -{\beta _0}\rho (x{\text{, }}y)}}}} . $

(5)

2.2 Polarization Imaging Principle

Polarization imaging has shown great superiority in a variety of optical tasks. In this study, the degree of polar-ization (DOP) was used for target detection. In obtaining a DOP image, the DOP of every pixel of the image was calculated, and the Stokes vector S was used to describe the polarized light beam:

$ S = {[I, {\text{ }}Q, {\text{ }}U, {\text{ }}V]^{\text{T}}}, $

(6)

where I is the total intensity of the light, Q is the fraction of linear polarization parallel to a reference plane, U is the proportion of linear polarization at 45˚ to the reference plane, and V is the fraction of right-handed circular polarization. Linear polarization was used to format the image. The polarization state of the incident light S = [I, Q, U, V]^T, which is changed by a polarizing film, can be expressed based on the Mueller matrix:

$ \left[ {\begin{array}{*{20}{c}} {I'} \\ {Q'} \\ {U'} \\ {V'} \end{array}} \right] = \frac{1}{2}\left[ {\begin{array}{*{20}{c}} 1&{\cos 2\psi }&{\sin 2\psi }&0 \\ {\cos 2\psi }&{{{\cos }^2}2\psi }&{\cos 2\psi \sin 2\psi }&0 \\ {\sin 2\psi }&{\cos 2\psi \sin 2\psi }&{{{\sin }^2}2\psi }&0 \\ 0&0&0&0 \end{array}} \right] \times \left[ {\begin{array}{*{20}{c}} I \\ Q \\ U \\ V \end{array}} \right], $

(7)

where ψ is the angle between the main optical axis and the zero reference line and S' = [I', Q', U', V']^T is the polarization state of the emergent light. Here, the first row of the Muller matrix is relevant because the light intensity can be obtained directly by the camera:

$ I'(\psi) = \frac{1}{2}(I + Q\cos 2\psi + U\sin 2\psi) . $

(8)

Therefore, if the light intensities of the emergent light at three different ψ values are known, then the I, Q, and U of the incident beam can be calculated. If ψ is set to 0˚, 45˚, and 90˚, then the following equation set is obtained:

$ \left\{ \begin{gathered} I'({0^ \circ }) = (I + Q)/2 \hfill \\ I'({45^ \circ }) = (I + U)/2 \hfill \\ I'({90^ \circ }) = (I -Q)/2 \hfill \\ \end{gathered} \right. . $

(9)

The equation set was then transformed as follows:

$ \left\{ \begin{gathered} I = I'({0^ \circ }) + I'({90^ \circ }) \hfill \\ Q = I'({0^ \circ }) -I'({90^ \circ }) \hfill \\ U = 2I'({45^ \circ }) -I'({0^ \circ }) -I'({90^ \circ }) \hfill \\ \end{gathered} \right. . $

(10)

Then, the linear DOP was calculated, and a DOP image was obtained:

$ P = \frac{{\sqrt {{Q^2} + {U^2}} }}{I} . $

(11)

2.3 Optical Flow Theory

Optical flow shows great success in motion estimation. Most optical flow methods are based on constancy in the brightness patterns in the image. According to this principle in traditional optical flow (Horn and Schunck, 1981), the variation of pixel intensity in time is:

$ \frac{{{\text{d}}I}}{{{\text{d}}t}} = \frac{{\partial I}}{{\partial x}}\frac{{{\text{d}}x}}{{{\text{d}}t}} + \frac{{\partial I}}{{\partial y}}\frac{{{\text{d}}y}}{{{\text{d}}t}} + \frac{{\partial I}}{{\partial t}} = 0, $

(12)

where I(x, y, t) is the intensity at the pixel (x, y) on the image plane at time t. Let u = dx/dt and v = dy/dt be the velocity vectors of the optical flow along the X-axis (horizontal direction) and Y-axis (vertical direction), respectively, and I_x = ∂I/∂x, I_y = ∂I/∂y, and I_t = ∂I/∂t represent the partial derivatives of I(x, y, t) in the image. Thus, Eq. (12) can be written as:

$ {I_x}u + {I_y}v + {I_t} = 0 . $

(13)

Here, we used polarization imaging technology and the DOP image was used to calculate the optical flow. The assumption is that the DOP P of a particular point in the motion is constant under a parallel light source and small displacement (Guan et al., 2020). Thus, the equations above were transformed as

$ \frac{{{\text{d}}P}}{{{\text{d}}t}} = \frac{{\partial P}}{{\partial x}}\frac{{{\text{d}}x}}{{{\text{d}}t}} + \frac{{\partial P}}{{\partial y}}\frac{{{\text{d}}y}}{{{\text{d}}t}} + \frac{{\partial P}}{{\partial t}} = 0, $

(14)

$ {P_x}u + {P_y}u + {P_t} = 0 . $

(15)

Because there are two variables (u, v) in the optical flow, the equation cannot be solved without other constraints, such as global smoothness (Horn and Schunck, 1981), spatial consistency (Lucas and Kanade, 1981), structure-texture constancy (Aujol et al., 2006), and photometric invariants of multiple color channel (Ohta, 1989). The Lucas-Kanade method (Lucas and Kanade, 1981), which has good performance under scattering environments, was used to calculate the underwater polarized optical flow field. Spatial consistency assumes that the velocity of n nearest points is the same. The least-squares method $ A\hat \beta = (-b) $ was then utilized to solve equations:

$ A\hat \beta = \left({\begin{array}{*{20}{c}} {\sum\limits_{x = 1}^n {{P_x}} }&{\sum\limits_{y = 1}^n {{P_y}} } \end{array}} \right)\left[ {\begin{array}{*{20}{c}} u \\ v \end{array}} \right] = -({n^*}{P_t}) = -(b), $

(16)

$ {A^{\text{T}}}A\hat \beta = {A^{\text{T}}}(-b) . $

(17)

The calculated optical flow field is as follows:

$ \left[ {\begin{array}{*{20}{l}} u \\ v \end{array}} \right] = {\left[ {\begin{array}{*{20}{l}} {\sum\limits_{x = 1}^n {P_x^2} }&{\sum\limits_{x = 1}^n {\sum\limits_{y = 1}^n {{P_x}{P_y}} } } \\ {\sum\limits_{x = 1}^n {\sum\limits_{y = 1}^n {{P_x}{P_y}} } }&{\sum\limits_{y = 1}^n {P_y^2} } \end{array}} \right]^{ -1}}\left[ {\begin{array}{*{20}{l}} { -{n^*}\sum\limits_{x = 1}^n {{P_x}{P_t}} } \\ { -{n^*}\sum\limits_{y = 1}^n {{P_y}{P_t}} } \end{array}} \right] . $

(18)

Finally, the two velocity values (u, v) were calculated, and the optical flow field was obtained. The displacement of an underwater target can be acquired by time integration of velocity.

2.4 Moving Target Segmentation

Moving target segmentation based on a polarized optical flow field was conducted to further realize underwater target detection in turbid water. First, the optical flow vector calculated by the polarized optical flow method was normalized to utilize the traditional region segmentation algorithm based on the gray image. An appropriate threshold was then selected for image binarization to segment moving targets and static backgrounds. Owing to the relative motion between the target and background, the trough value of the target and background in the histogram distribution can be selected as the threshold value for segmentation. Here, the Otsu algorithm (Otsu, 1979) was used to calculate global thresholds. For the optical flow gray image f(x, y), its gray level is L = 255. The assumption is that the number of pixels with a gray value of i is n_i, and the optimal threshold of segmentation is k. The proportion of background pixel number N₁ to total pixel number N is p₁, and the average gray value is m₁:

$ {p_1} = \frac{{\sum\limits_{i = 1}^k {{n_i}} }}{N}, $

(19)

$ {m_1} = \frac{{\sum\limits_{i = 1}^k {i{n_i}} }}{{{N_1}}} . $

(20)

The proportion of the target pixel number N₂ to total pixel number N is p₂, and the average gray value is m₂:

$ {p_2} = \frac{{\sum\limits_{i = k + 1}^L {{n_i}} }}{N}, $

(21)

$ {m_2} = \frac{{\sum\limits_{i = k + 1}^L {i{n_i}} }}{{{N_2}}} . $

(22)

Let the total average gray value of the image be m and the between-class variance be $ \sigma _B^2(k) $, then:

$ m = {p_1}{m_1} + {p_2}{m_2}, $

(23)

$ \sigma _B^2(k) = {p_1}{(m -{m_1})^2} + {p_2}{(m -{m_2})^2} . $

(24)

The k is from 1 to L−1, and k^* is an adaptive optimal threshold:

$ {\sigma }_{B}^{2}({k}^{\ast })=\underset{0\le k\le L-1}{\mathrm{max}}{\sigma }_{B}^{2}(k) . $

(25)

The optical flow gray image f (x, y) is divided for image binarization:

$ g(x, y) = \left\{ {\begin{array}{*{20}{c}} {1, {\text{ }}f(x, y) > {k^*}} \\ {0, {\text{ }}f(x, y) \leqslant {k^*}} \end{array}} \right. . $

(26)

After segmentation, the target region was morphologically filled to fill the cavity of the moving target and remove the error region caused by noise.

2.5 Experimental Setup and Materials

The experimental setup of underwater target detection based on polarized optical flow is shown in Fig.1. A polarized light source with a wavelength of 532 nm was selected because its small attenuation provides a great advantage in underwater imaging. A polarization camera (LUCID, PHX050S-P) with 2448 × 2048 pixels was used to obtain the polarization information of the underwater target. This camera adopts the division of focal plane imaging mode with a frame rate of 22 fps and a Sony IMX250MRZ CMOS. The light source and camera were fixed on a slider on the straight-line guide. The glass water tank (200 mm × 150 mm × 150 mm) was covered with the black curtain except for the observation window to reduce light interference. The metal and plastic target was used as the target. The milk was prepared in the water tank to simulate an underwater environment with suspended particles. The laser interferometer with linear measurement accuracy up to 10 nm was used as the calibration instrument for displacement accuracy. The linear retroreflector of the laser interferometer was fixed with the camera to measure the displacement of the camera. Data were obtained under different levels of water turbidity by adding different volumes of milk into the water to prove the effectiveness of the method. The camera was moved by the sliding block on the straight-line guide and captured the sequential images to calculate the optical flow information of the target.

3 Results and Discussion 3.1 Polarized Optical Flow Field

Our optical flow algorithm was based on the local meth-od represented by Lucas and Kanade (1981), and the local window was set to 15 × 15 pixels. The optical flow field was calculated by two adjacent frames of the filtered DOP image. The metal target detection results under different water turbidity levels are shown in Fig.2. The volume of added milk was 0, 0.5, 1, and 1.5 mL. 'I' represents the intensity image, and 'DOP' represents the polarization image. Fig.2(a) is the original image, Fig.2(b) is the optical flow vector estimated by the sequential original images, and Fig.2(c) is the color map that is equivalent to the optical flow vector. The equivalence between the vector and color map of optical flow is also shown in Fig.2. Compared with normal imaging, underwater polarization imaging can improve image quality and reveal additional information. Thus, the optical flow based on polarization imaging achieves good results. The optical flow vector in the intensity image is chaotic and irregular, leading to difficulty in target detection, especially in turbid water. In the DOP image, the vector, mostly located on the target, is consistent with real displacement. For the color map, in the intensity image, the contrast of the target and background is low, and the key is hardly detected in turbid water. However, the outline of the key in the DOP image is clear all the time. With water turbidity, the strength of polarization imaging becomes evident. Fig.3 shows the plastic target detection results under different water turbidity levels. In the color map of polarization optical flow, the contrast between the target and background is better than that in normal optical flow, leading to accurate displacement information and a good recognition effect. When the material of the target is changed, the polarization characteristics are changed theoretically, resulting in different detection results. However, the method is also effective, proving the strength of underwater polarized optical flow. The results of polarization imaging for the metal target are better than those for the plastic target because of the optical characteristics of the metal target. The results of different water turbidity levels demonstrate the robustness of the method under various water qualities.

3.2 Target Motion Estimation

There is a linear relationship between the optical flow values in the image and the actual displacement of the target in motion. To verify the precision of the method, we compared the displacement estimated by the optical flow with the precise displacement recorded by the laser interferometer (Figs.4 and 5). We used an algorithm based on Scale Invariant Feature Transformation (Lowe, 2004) to detect the feature points and averaged their optical flow values. We then utilized the imaging principle to transform the optical flow value into the displacement value. By comparing the estimated displacement value with the truth, we can obtain the displacement precision of the optical flow. In clear water, the methods based on polarization and intensity provide results close to the truth. In turbid water, the deviation between the polarized optical flow and the standard displacement is far less than the normal optical flow. With water turbidity, the tendency becomes evident. We also calculated the average displacement error of the metal and plastic target based on the DOP and intensity images under different water turbidity levels (Fig.6). In water with 1.5 mL of milk, the average displacement errors of metal and plastic targets based on polarized optical flow are reduced by 61.58% and 57.21%, respectively, compared with those in normal optical flow. The polarized optical flow algorithm can obtain accurate displacement information of the target by improving the underwater image contrast between the moving target and background. The displacement error of the metal target based on polarization imaging is always less than that of the plastic target in turbid water due to the optical property of the metal target.

3.3 Precision Evaluation

We also calculated the average angular error (AAE) and average endpoint error (AEPE), as shown in Fig.7, to further analyze the precision quantitatively of our method. AAE is the average of the angle between the calculated optical flow vector and the corresponding truth vector, and AEPE is the average of the position deviation of the endpoints of two vectors. In clear water, the AAE and AEPE of the intensity image are close to those of the DOP. With increasing water turbidity, both measures of the intensity image increase remarkably. Meanwhile, the two measures of the DOP image, which are less than those in the intensity image all the time, increase slightly and remain at the same level. In water with 1.5 mL of milk, the AAE of metal and plastic targets based on polarized optical flow is reduced by 68.95% and 41.32%, respectively, compared with those in normal optical flow, and the AEPE is reduced by 83.84% and 59.88%, respectively. The errors for the metal target are less than those for the plastic target in the DOP image because of the characteristics of polarization imaging. The polarized optical flow is more effective and precise compared with the normal method. Polarization imaging can increase estimation robustness and noise immunity, which is significant for overcoming higher underwater image noise from various sources.

3.4 Moving Target Segmentation

Moving target segmentation under different water turbidity levels was obtained using the proposed polarized optical flow method, as shown in Fig.8. Fig.8(a) represents the normalized optical flow grayscale image. The area with a large grayscale value corresponds to the area with a large optical flow value. The key outline in the background can be clearly seen in the image. Fig.8(b) is the gray optical flow distribution histogram. The image has two normal distribution ranges: one is the moving target, whose distribution average gray value is about 50, and the other is the static background, whose average is about 0. The optimal adaptive threshold is between the two peaks. Fig.8(c) shows the moving target segmentation results.

With the increase in water turbidity, the grayscale optical flow image contrast decreases, and the normal distribution of the moving target and background in the histogram moves to the right. When the milk volume is 0.5 mL, the outline of the key is clear, and the moving area is slightly enlarged. When the milk volume is 1.0 mL, the noise outside the moving area increases, and the cavity appears inside, but the general outline of the key is still maintained. When the milk volume is 1.5 mL, the upper left part of the key is missing to some extent, and the lower part is still visible. We used pixel accuracy (PA) and mean intersection over union (MIoU) to measure the accuracy of image segmentation. PA represents the ratio of correctly predicted pixels to all pixels in the image. The physical meaning of MIoU is that the overlap of the predicted and labeled regions is divided by the combination of the predicted and labeled regions. The evaluation index of moving target segmentation results with different water turbidity is shown in Table 1. The results show that the moving target segmentation based on the polarized optical flow is feasible under turbid water.

4 Conclusions

In this work, we proposed a bioinspired method that uses polarized optical flow to estimate target motion in turbid water. Using the DOP image of adjacent frames as a basis, we obtained the optical flow field of the target and realized motion estimation. The target displacement information and two error measures of the optical flow were calculated to evaluate the performance of our method. Moving target segmentation was conducted based on the polarized optical flow field, and the segmentation accuracy was measured by the objective evaluation index. The results suggested that this approach is effective and robust in underwater environments with multiple levels of turbidity. Furthermore, the method has great advantages for the underwater detection of metal targets. This study provides an idea for target detection in scattering environments. In theory, utilizing some dehazing methods or fusing AOP images that also contain abundant information with DOP images will further improve the image quality and increase accuracy. However, in this case, the assumption in Section 2.3 will not be satisfied, resulting in the failure of the method. In the future, we will further improve our model and use dehazing methods that can satisfy the assumption to enhance the practicability and robustness of the proposed method.

Acknowledgements

This work was supported by the National Natural Science Foundation of China (No. 52394252), the Postdoctoral Fellowship Program of CPSF (No. GZC20232497), the Key Research and Development Program of Shandong Province, China (No. 2021ZLGX04), the Shandong Postdoctoral Science Foundation (No. SDBX2023012), and the Qingdao Postdoctoral Program Grant (No. QDBSH2023020 2009).