孙刘杰,刘倩倩,庞茂然.基于傅里叶卷积和多特征调制的图像修复网络[J].包装工程,2023,44(21):286-293.
SUN Liu-jie,LIU Qian-qian,PANG Mao-ran.Image Inpainting Network Based on Fourier Convolution and Multi-feature Modulation[J].Packaging Engineering,2023,44(21):286-293. |
| 基于傅里叶卷积和多特征调制的图像修复网络 |
| Image Inpainting Network Based on Fourier Convolution and Multi-feature Modulation |
| 投稿时间：2023-03-07 |
| DOI：10.19554/j.cnki.1001-3563.2023.21.036 |
| 中文关键词: 图像修复 深度学习 感受野 傅里叶卷积 特征调制 |
| 英文关键词: image inpainting deep learning perceptual field Fourier convolution feature modulation |
| 基金项目:上海市科学技术委员会科研计划(18060502500);上海市自然科学基金面上项目(19ZR1435900) |
|
| 摘要点击次数: |
| 全文下载次数: |
| 中文摘要: |
| 目的 解决大面积破损难以修复且修复过程中感受野、特征空间信息利用不足，导致修复后的孔洞区域与背景之间出现结构、纹理、风格不一致的问题。方法 基于傅里叶卷积和多特征调制的修复网络FFC-MFMGAN，傅里叶卷积在网络的浅层便具有较大的感受野，尤其是在宽掩码时能够跳过掩码区域，捕获到有效特征，多特征调制生成网络能够分别利用完整区域的信息和随机样式操纵，增强与未受损区域的语义连贯性，以及大空洞率下修复的多样性。结果 在Place 2数据集上，将文中方法与其他图像修复方法进行了对比实验，经过测试，各类指标均得到明显改善，峰值信噪比提高了1.4%，结构相似性提高了4.5%，平均绝对误差降低了12.6%，基于学习的感知图像块相似性降低了9.1%。结论 FFC-MFMGAN网络能够较好地修复大面积不规则孔洞，同时增强修复图像的全局结构性和清晰度，对实际包装印刷图像的缺陷修复也有一定参考价值。 |
| 英文摘要: |
| The work aims to address the problem that large areas of damage are difficult to repair, the perceptual field and feature space information are under-utilized in the repair process, and the structure, texture and style between the repaired cavity areas and the background are inconsistent. An inpainting network FFC-MFMGAN based on Fourier convolution and multi-feature modulation was proposed. Fourier convolution had a large perceptual field in the shallow layer of the network, especially in wide masks, which could skip the mask zone to capture the effective features especially when the mask was wide. The generative network based on the multi-feature modulation was able to enhance the semantic coherence with undamaged regions and the diversity of restoration at large void rates with information from intact regions and random pattern manipulation, respectively. Experiments were conducted to compare the proposed method with other state-of-the-art image restoration methods on the Place 2 dataset, and the following categories were tested to show significant improvements, including a 1.4% improvement in PSNR, a 4.5% improvement in SSIM, a 12.6% reduction in MAE, and a 9.1% reduction in LPIPS. The FFC-MFMGAN network can better repair large irregular holes, while enhancing the global structure and clarity of the repaired images, which is also a reference value for the repair of defects in actual packaging printing images. |
|
查看全文
查看/发表评论 下载PDF阅读器 |
| 关闭 |
|
|
|
渝公网安备 50010702501716号