How shift equivariance impacts metric learning for instance segmentation

Tools

Item Type:	Conference or Workshop Item
Title:	How shift equivariance impacts metric learning for instance segmentation
Creators Name:	Rumberger, J.L., Yu, X., Hirsch, P., Dohmen, M., Guarino, V.E., Mokarian, A., Mais, L., Funke, J. and Kainmueller, D.
Abstract:	Metric learning has received conflicting assessments concerning its suitability for solving instance segmentation tasks. It has been dismissed as theoretically flawed due to the shift equivariance of the employed CNNs and their respective inability to distinguish same-looking objects. Yet it has been shown to yield state of the art results for a variety of tasks, and practical issues have mainly been reported in the context of tile-and-stitch approaches, where discontinuities at tile boundaries have been observed. To date, neither of the reported issues have undergone thorough formal analysis. In our work, we contribute a comprehensive formal analysis of the shift equivariance properties of encoder-decoder-style CNNs, which yields a clear picture of what can and cannot be achieved with metric learning in the face of same-looking objects. In particular, we prove that a standard encoder-decoder network that takes d-dimensional images as input, with l pooling layers and pooling factor f, has the capacity to distinguish at most f(dl) same-looking objects, and we show that this upper limit can be reached. Furthermore, we show that to avoid discontinuities in a tile-and-stitch approach, assuming standard batch size 1, it is necessary to employ valid convolutions in combination with a training output window size strictly greater than f(l), while at test-time it is necessary to crop tiles to size n • f(l) before stitching, with n ≥ 1. We complement these theoretical findings by discussing a number of insightful special cases for which we show empirical results on synthetic and real data. Code:https://github.com/Kainmueller-Lab/shift_equivariance_unet
Keywords:	Segmentation, Grouping and Shape, Medical, Biological, and Cell Microscopy, Representation Learning
Source:	2021 IEEE/CVF International Conference on Computer Vision (ICCV)
Series Name:	IEEE/CVF International Conference on Computer Vision (ICCV)
Title of Book:	2021 IEEE/CVF International Conference on Computer Vision (ICCV)
ISSN:	2380-7504
ISBN:	978-1-6654-2812-5
Publisher:	IEEE
Page Range:	7108-7116
Date:	28 February 2022
Additional Information:	Copyright © 2021 IEEE
Official Publication:	https://doi.org/10.1109/iccv48922.2021.00704
External Fulltext:	View full text on external repository or document server

Repository Staff Only: item control page