Deep Image Clustering with Tensor Kernels and Unsupervised Companion Objectives
Permanent lenke
https://hdl.handle.net/10037/15661Dato
2019-05-31Type
Master thesisMastergradsoppgave
Forfatter
Trosten, Daniel JohansenSammendrag
Deep image clustering is a rapidly growing branch of machine learning and computer vision, in which deep neural networks are trained to discover groups within a set of images, in an unsupervised manner. Deep neural networks have proven to be immensely successful in several machine learning tasks, but the majority of these advances have been in supervised settings. The process of labeling data for supervised applications can be extremely time-consuming, or even completely infeasible in many domains. This has led researchers to shift their focus towards the deep clustering field. However, this field is still in its infancy, meaning that it includes several open research questions, regarding e.g. the design and optimization of the algorithms, the discovery of meaningful clusters, and the initialization of model parameters.
In an attempt to address some of these open questions, a new algorithm for deep image clustering is developed in this thesis. The proposed Deep Tensor Kernel Clustering (DTKC) consists of a convolutional neural network (CNN), which is trained to reflect a common cluster structure at the output of all its intermediate layers. Encouraging a consistent cluster structure throughout the network has the potential to guide it towards meaningful clusters, even though these clusters might appear to be nonlinear in the input space. The cluster structure is enforced through the idea of companion objectives, where separate loss functions are attached to each of the layers in the network. These companion objectives are constructed based on a proposed generalization of the Cauchy-Schwarz (CS) divergence, from vectors to tensors of arbitrary rank. Generalizing the CS divergence to tensor-valued data is a crucial step, due to the tensorial nature of the intermediate representations in the CNN. Furthermore, an alternate initialization strategy based on self-supervised learning, is also employed. To the author's best knowledge, this is the first attempt at using this particular self-supervised learning approach to initialize a deep clustering algorithm.
Several experiments are conducted to thoroughly assess the performance of the proposed DTKC model, with and without self-supervised pre-training. The results show that the models outperform, or perform comparable to, a wide range of benchmark algorithms from the literature.
Forlag
UiT Norges arktiske universitetUiT The Arctic University of Norway
Metadata
Vis full innførselSamlinger
Copyright 2019 The Author(s)
Følgende lisensfil er knyttet til denne innførselen:
Med mindre det står noe annet, er denne innførselens lisens beskrevet som Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0)
Relaterte innførsler
Viser innførsler relatert til tittel, forfatter og emneord.
-
Engineering methods for enhancing railway geometry and winter road assessment: A safety and maintenance perspective
Brustad, Tanita Fossli (Doctoral thesis; Doktorgradsavhandling, 2020-06-22)In many areas around the world there are limited transportation possibilities when travelling between key cities. If these areas also experience demanding weather conditions or geography, getting from A to B, during difficult conditions, is usually not optimal in regards to accessibility, safety, and comfort. Under challenging conditions, two essential elements in strengthening accessibility, safety, ... -
Geometric Modeling- and Sensor Technology Applications for Engineering Problems
Pedersen, Aleksander (Doctoral thesis; Doktorgradsavhandling, 2020-10-20)In applications for technical problems, Geometric modeling and sensor technology are key in both scientific and industrial development. Simulations and visualization techniques are the next step after defining geometry models and data types. This thesis attempts to combine different aspects of geometric modeling and sensor technology as well as to facilitate simulation and visualization. It includes ... -
Iceberg Drift-Trajectory Modelling and Probability Distributions of the Predictions
Baadshaug, Ole (Master thesis; Mastergradsoppgave, 2018-06-29)Moving icebergs represent a major problem for shipping, as well as for oil and gas installations in ice infested waters. To be able to take actions against hazardous icebergs, it is necessary to develop models for prediction of iceberg drift trajectories. Many models have been developed in order to do so, using different approaches. These approaches can be divided into two main categories, dynamic ...