The Experts below are selected from a list of 81483 Experts worldwide ranked by ideXlab platform
Andrew Y Ng - One of the best experts on this subject based on the ideXlab platform.
-
Unsupervised Learning of hierarchical representations with convolutional deep belief networks
Commun. ACM, 2011Co-Authors: Honglak Lee, Roger Grosse, Rajesh Ranganath, Andrew Y NgAbstract:There has been much interest in Unsupervised Learning of hierarchical generative models such as deep belief networks (DBNs); however, scaling such models to full-sized, high-dimensional images remains a difficult problem. To address this problem, we present the convolutional deep belief network, a hierarchical generative model that scales to realistic image sizes. This model is translation-invariant and supports efficient bottom-up and top-down probabilistic inference. Key to our approach is probabilistic max-pooling, a novel technique that shrinks the representations of higher layers in a probabilistically sound way. Our experiments show that the algorithm learns useful high-level visual features, such as object parts, from unlabeled images of objects and natural scenes. We demonstrate excellent performance on several visual recognition tasks and show that our model can perform hierarchical (bottom-up and top-down) inference over full-sized images.
-
Large-scale deep Unsupervised Learning using graphics processors.
Icml, 2009Co-Authors: Rajat Raina, Anand Madhavan, Andrew Y NgAbstract:The promise of Unsupervised Learning methods lies in their potential to use vast amounts of unlabeled data to learn complex, highly nonlinear models with millions of free parameters. We consider two well-known Unsupervised Learning models, deep belief networks (DBNs) and sparse coding, that have recently been applied to a flurry of machine Learning applications (Hinton & Salakhutdinov, 2006; Raina et al., 2007). Unfortunately, current Learning algorithms for both models are too slow for large-scale applications, forcing researchers to focus on smaller-scale models, or to use fewer training examples. In this paper, we suggest massively parallel methods to help resolve these problems. We argue that modern graphics processors far surpass the computational capabilities of multicore CPUs, and have the potential to revolutionize the applicability of deep Unsupervised Learning methods. We develop general principles for massively parallelizing Unsupervised Learning tasks using graphics processors. We show that these principles can be applied to successfully scaling up Learning algorithms for both DBNs and sparse coding. Our implementation of DBN Learning is up to 70 times faster than a dual-core CPU implementation for large models. For example, we are able to reduce the time required to learn a four-layer DBN with 100 million free parameters from several weeks to around a single day. For sparse coding, we develop a simple, inherently parallel algorithm, that leads to a 5 to 15-fold speedup over previous methods.
Alexander Rives - One of the best experts on this subject based on the ideXlab platform.
-
biological structure and function emerge from scaling Unsupervised Learning to 250 million protein sequences
Proceedings of the National Academy of Sciences of the United States of America, 2021Co-Authors: Alexander Rives, Siddharth Goyal, Joshua Meier, Demi Guo, Myle Ott, Tom Sercu, Zeming Lin, Jason Liu, Lawrence C ZitnickAbstract:In the field of artificial intelligence, a combination of scale in data and model capacity enabled by Unsupervised Learning has led to major advances in representation Learning and statistical generation. In the life sciences, the anticipated growth of sequencing promises unprecedented data on natural sequence diversity. Protein language modeling at the scale of evolution is a logical step toward predictive and generative artificial intelligence for biology. To this end, we use Unsupervised Learning to train a deep contextual language model on 86 billion amino acids across 250 million protein sequences spanning evolutionary diversity. The resulting model contains information about biological properties in its representations. The representations are learned from sequence data alone. The learned representation space has a multiscale organization reflecting structure from the level of biochemical properties of amino acids to remote homology of proteins. Information about secondary and tertiary structure is encoded in the representations and can be identified by linear projections. Representation Learning produces features that generalize across a range of applications, enabling state-of-the-art supervised prediction of mutational effect and secondary structure and improving state-of-the-art features for long-range contact prediction.
-
biological structure and function emerge from scaling Unsupervised Learning to 250 million protein sequences
bioRxiv, 2019Co-Authors: Alexander Rives, Siddharth Goyal, Joshua Meier, Demi Guo, Myle Ott, Lawrence C Zitnick, Rob FergusAbstract:Abstract In the field of artificial intelligence, a combination of scale in data and model capacity enabled by Unsupervised Learning has led to major advances in representation Learning and statistical generation. In biology, the anticipated growth of sequencing promises unprecedented data on natural sequence diversity. Learning the natural distribution of evolutionary protein sequence variation is a logical step toward predictive and generative modeling for biology. To this end we use Unsupervised Learning to train a deep contextual language model on 86 billion amino acids across 250 million sequences spanning evolutionary diversity. The resulting model maps raw sequences to representations of biological properties without labels or prior domain knowledge. The learned representation space organizes sequences at multiple levels of biological granularity from the biochemical to proteomic levels. Learning recovers information about protein structure: secondary structure and residue-residue contacts can be extracted by linear projections from learned representations. With small amounts of labeled data, the ability to identify tertiary contacts is further improved. Learning on full sequence diversity rather than individual protein families increases recoverable information about secondary structure. We show the networks generalize by adapting them to variant activity prediction from sequences only, with results that are comparable to a state-of-the-art variant predictor that uses evolutionary and structurally derived features.
Lawrence C Zitnick - One of the best experts on this subject based on the ideXlab platform.
-
biological structure and function emerge from scaling Unsupervised Learning to 250 million protein sequences
Proceedings of the National Academy of Sciences of the United States of America, 2021Co-Authors: Alexander Rives, Siddharth Goyal, Joshua Meier, Demi Guo, Myle Ott, Tom Sercu, Zeming Lin, Jason Liu, Lawrence C ZitnickAbstract:In the field of artificial intelligence, a combination of scale in data and model capacity enabled by Unsupervised Learning has led to major advances in representation Learning and statistical generation. In the life sciences, the anticipated growth of sequencing promises unprecedented data on natural sequence diversity. Protein language modeling at the scale of evolution is a logical step toward predictive and generative artificial intelligence for biology. To this end, we use Unsupervised Learning to train a deep contextual language model on 86 billion amino acids across 250 million protein sequences spanning evolutionary diversity. The resulting model contains information about biological properties in its representations. The representations are learned from sequence data alone. The learned representation space has a multiscale organization reflecting structure from the level of biochemical properties of amino acids to remote homology of proteins. Information about secondary and tertiary structure is encoded in the representations and can be identified by linear projections. Representation Learning produces features that generalize across a range of applications, enabling state-of-the-art supervised prediction of mutational effect and secondary structure and improving state-of-the-art features for long-range contact prediction.
-
biological structure and function emerge from scaling Unsupervised Learning to 250 million protein sequences
bioRxiv, 2019Co-Authors: Alexander Rives, Siddharth Goyal, Joshua Meier, Demi Guo, Myle Ott, Lawrence C Zitnick, Rob FergusAbstract:Abstract In the field of artificial intelligence, a combination of scale in data and model capacity enabled by Unsupervised Learning has led to major advances in representation Learning and statistical generation. In biology, the anticipated growth of sequencing promises unprecedented data on natural sequence diversity. Learning the natural distribution of evolutionary protein sequence variation is a logical step toward predictive and generative modeling for biology. To this end we use Unsupervised Learning to train a deep contextual language model on 86 billion amino acids across 250 million sequences spanning evolutionary diversity. The resulting model maps raw sequences to representations of biological properties without labels or prior domain knowledge. The learned representation space organizes sequences at multiple levels of biological granularity from the biochemical to proteomic levels. Learning recovers information about protein structure: secondary structure and residue-residue contacts can be extracted by linear projections from learned representations. With small amounts of labeled data, the ability to identify tertiary contacts is further improved. Learning on full sequence diversity rather than individual protein families increases recoverable information about secondary structure. We show the networks generalize by adapting them to variant activity prediction from sequences only, with results that are comparable to a state-of-the-art variant predictor that uses evolutionary and structurally derived features.
Uday K Chakraborty - One of the best experts on this subject based on the ideXlab platform.
-
reasoning and Unsupervised Learning in a fuzzy cognitive map
Information Sciences, 2005Co-Authors: Amit Konar, Uday K ChakrabortyAbstract:This paper presents a new model for Unsupervised Learning and reasoning on a special type of cognitive maps realized with Petri nets. The Unsupervised Learning process in the present context adapts the weights of the directed arcs from transitions to places in the Petri net. A Hebbian-type Learning algorithm with a natural decay in weights is employed to study the dynamic behavior of the algorithm. The algorithm is conditionally stable for a suitable range of the mortality rate. After convergence of the Learning algorithm, the network may be used for computing the beliefs of the desired propositions from the supplied beliefs of the axioms (places with no input arcs). Because of the conditional stability of the algorithm, it may be used in complex decision-making and Learning such as automated car driving in an accident-prone environment. The paper also presents a new model for knowledge refinement by adaptation of weights in a fuzzy Petri net using a different form of Hebbian Learning. This second model converges to stable points in both encoding and recall phases.
Xi Zheng - One of the best experts on this subject based on the ideXlab platform.
-
Unsupervised-Learning-Based Continuous Depth and Motion Estimation With Monocular Endoscopy for Virtual Reality Minimally Invasive Surgery
IEEE Transactions on Industrial Informatics, 2021Co-Authors: Shanlin Yang, Shuai Ding, Alireza Jolfaei, Xi ZhengAbstract:Three-dimensional display and virtual reality technology have been applied in minimally invasive surgery to provide doctors with a more immersive surgical experience. One of the most popular systems based on this technology is the Da Vinci surgical robot system. The key to build the in vivo 3-D virtual reality model with a monocular endoscope is an accurate estimation of depth and motion. In this article, a fully Unsupervised Learning method for depth and motion estimation using the continuous monocular endoscopic video is proposed. After the detection of highlighted regions, EndoMotionNet and EndoDepthNet are designed to estimate ego-motion and depth, respectively. The timing information between consecutive frames is considered with a long short-term memory layer by EndoMotionNet to enhance the accuracy of ego-motion estimation. The estimated depth value of the previous frame is used to estimate the depth of the next frame by EndoDepthNet with a multimode fusion mechanism. The custom loss function is defined to improve the robustness and accuracy of the proposed Unsupervised-Learning-based method. Experiments with the public datasets verify that the proposed Unsupervised-Learning-based continuous depth and motion estimation method can effectively improve the accuracy of depth and motion estimation, especially after processing the frame.
