The Experts below are selected from a list of 8061 Experts worldwide ranked by ideXlab platform
Jascha Sohldickstein - One of the best experts on this subject based on the ideXlab platform.
-
svcca Singular Vector canonical correlation analysis for deep learning dynamics and interpretability
arXiv: Machine Learning, 2017Co-Authors: Maithra Raghu, Justin Gilmer, Jason Yosinski, Jascha SohldicksteinAbstract:We propose a new technique, Singular Vector Canonical Correlation Analysis (SVCCA), a tool for quickly comparing two representations in a way that is both invariant to affine transform (allowing comparison between different layers and networks) and fast to compute (allowing more comparisons to be calculated than with previous methods). We deploy this tool to measure the intrinsic dimensionality of layers, showing in some cases needless over-parameterization; to probe learning dynamics throughout training, finding that networks converge to final representations from the bottom up; to show where class-specific information in networks is formed; and to suggest new training regimes that simultaneously save computation and overfit less. Code: this https URL
-
svcca Singular Vector canonical correlation analysis for deep learning dynamics and interpretability
Neural Information Processing Systems, 2017Co-Authors: Maithra Raghu, Justin Gilmer, Jason Yosinski, Jascha SohldicksteinAbstract:We propose a new technique, Singular Vector Canonical Correlation Analysis (SVCCA), a tool for quickly comparing two representations in a way that is both invariant to affine transform (allowing comparison between different layers and networks) and fast to compute (allowing more comparisons to be calculated than with previous methods). We deploy this tool to measure the intrinsic dimensionality of layers, showing in some cases needless over-parameterization; to probe learning dynamics throughout training, finding that networks converge to final representations from the bottom up; to show where class-specific information in networks is formed; and to suggest new training regimes that simultaneously save computation and overfit less.
T N Palmer - One of the best experts on this subject based on the ideXlab platform.
-
predictability of a coupled model of enso using Singular Vector analysis part ii optimal growth and forecast skill
Monthly Weather Review, 1997Co-Authors: Mark A Cane, Stephen E Zebiak, T N PalmerAbstract:Abstract The fastest perturbation growth (optimal growth) in forecasts of El Nino–Southern Oscillation (ENSO) with the Zebiak and Cane model is analyzed by Singular value decomposition of forward tangent models along forecast trajectories in a reduced EOF space. The authors study optimal growth in forecast runs using two different initialization procedures and discuss the relationship between optimal growth and forecast skill. Consistent with Part I of this work, one dominant growing Singular Vector is found. Most of the variation of optimal growth, measured by the largest Singular value, for warm events and mean condition is seasonal, attributable to the seasonal variations in the background states. For cold events the seasonal optimal growth is substantially suppressed. The first Singular Vector is approximately white in EOF space, while its final pattern after a 6-month evolution is dominated by the first EOF. The energy norm amplifies between 5- and 24-fold in 6 months. This indicates that small-scale...
-
a study of the predictability of tropical pacific sst in a coupled atmosphere ocean model using Singular Vector analysis the role of the annual cycle and the enso cycle
Monthly Weather Review, 1997Co-Authors: Yingquei Chen, T N Palmer, David S Battisti, Joseph J Barsugli, E S SarachikAbstract:Abstract The authors examine the sensitivity of the Battisti coupled atmosphere–ocean model—considered as a forecast model for the El Nino–Southern Oscillation (ENSO)—to perturbations in the sea surface temperature (SST) field applied at the beginning of a model integration. The spatial structures of the fastest growing SST perturbations are determined by Singular Vector analysis of an approximation to the propagator for the linearized system. Perturbation growth about the following four reference trajectories is considered: (i) the annual cycle, (ii) a freely evolving model ENSO cycle with an annual cycle in the basic state, (iii) the annual mean basic state, and (iv) a freely evolving model ENSO cycle with an annual mean basic state. Singular Vectors with optimal growth over periods of 3, 6, and 9 months are computed. The magnitude of maximum perturbation growth is highly dependent on both the phase of the seasonal cycle and the phase of the ENSO cycle at which the perturbation is applied and on the dur...
-
the Singular Vector structure of the atmospheric global circulation
Journal of the Atmospheric Sciences, 1995Co-Authors: Roberto Buizza, T N PalmerAbstract:Abstract The local phase-space instability Of the atmospheric global circulation is Characterized by its (nonmodal) Singular Vectors. The formalism of Singular Vector analysis is described. The relations between Singular Vectors, normal modes, adjoint modes, Lyapunov Vectors, perturbations produced by the so-called breeding method, and wave pseudomomentum are outlined. Techniques to estimate the dominant part of the Singular spectrum using large-dimensional primitive equation models are discussed. These include the use of forward and adjoint tangent propagators with a Lanczos iterative algorithm. Results are described, based first on statistics of routine calculations made between December 1992 and August 1993, and second on three specific case studies. Results define three dominant geographical areas of instability in the Northern Hemisphere: the two regions of storm track cyclogenesis, and the North African subtropical jet Singular Vectors can amplify as much as tenfold over 36 hours, and in winter ther...
Peng Zhao - One of the best experts on this subject based on the ideXlab platform.
-
the role of nonlinear forcing Singular Vector tendency error in causing the spring predictability barrier for enso
Journal of meteorological research, 2016Co-Authors: Wansuo Duan, Peng Zhao, Junya Hu, Hui XuAbstract:With the Zebiak–Cane model, the present study investigates the role of model errors represented by the nonlinear forcing Singular Vector (NFSV) in the “spring predictability barrier” (SPB) phenomenon in ENSO prediction. The NFSV-related model errors are found to have the largest negative effect on the uncertainties of El Ni˜no prediction and they can be classified into two types: the first is featured with a zonal dipolar pattern of SST anomalies (SSTA), with the western poles centered in the equatorial central–western Pacific exhibiting positive anomalies and the eastern poles in the equatorial eastern Pacific exhibiting negative anomalies; and the second is characterized by a pattern almost opposite to the first type. The first type of error tends to have the worst effects on El Ni˜no growth-phase predictions, whereas the latter often yields the largest negative effects on decaying-phase predictions. The evolution of prediction errors caused by NFSVrelated errors exhibits prominent seasonality, with the fastest error growth in spring and/or summer; hence, these errors result in a significant SPB related to El Ni˜no events. The linear counterpart of NFSVs, the (linear) forcing Singular Vector (FSV), induces a less significant SPB because it contains smaller prediction errors. Random errors cannot generate an SPB for El Ni˜no events. These results show that the occurrence of an SPB is related to the spatial patterns of tendency errors. The NFSV tendency errors cause the most significant SPB for El Ni˜no events. In addition, NFSVs often concentrate these large value errors in a few areas within the equatorial eastern and central–western Pacific, which likely represent those areas sensitive to El Ni˜no predictions associated with model errors. Meanwhile, these areas are also exactly consistent with the sensitive areas related to initial errors determined by previous studies. This implies that additional observations in the sensitive areas would not only improve the accuracy of the initial field but also promote the reduction of model errors to greatly improve ENSO forecasts.
-
revealing the most disturbing tendency error of zebiak cane model associated with el nino predictions by nonlinear forcing Singular Vector approach
Climate Dynamics, 2015Co-Authors: Wansuo Duan, Peng ZhaoAbstract:The nonlinear forcing Singular Vector (NFSV) approach is used to identify the most disturbing tendency error of the Zebiak–Cane model associated with El Nino predictions, which is most potential for yielding aggressively large prediction errors of El Nino events. The results show that only one NFSV exists for each of the predictions for the predetermined model El Nino events. These NFSVs cause the largest prediction error for the corresponding El Nino event in perfect initial condition scenario. It is found that the NFSVs often present large-scale zonal dipolar structures and are insensitive to the intensities of El Nino events, but are dependent on the prediction periods. In particular, the NFSVs associated with the predictions crossing through the growth phase of El Nino tend to exhibit a zonal dipolar pattern with positive anomalies in the equatorial central-western Pacific and negative anomalies in the equatorial eastern Pacific (denoted as “NFSV1”). Meanwhile, those associated with the predictions through the decaying phase of El Nino are inclined to present another zonal dipolar pattern (denoted as “NFSV2”), which is almost opposite to the NFSV1. Similarly, the linear forcing Singular Vectors (FSVs), which are computed based on the tangent linear model, can also be classified into two types “FSV1” and “FSV2”. We find that both FSV1 and NFSV1 often cause negative prediction errors for Nino-3 SSTA of the El Nino events, while the FSV2 and NFSV2 usually yield positive prediction errors. However, due to the effect of nonlinearities, the NFSVs usually have the western pole of the zonal dipolar pattern much farther west, and covering much broader region. The nonlinearities have a suppression effect on the growth of the prediction errors caused by the FSVs and the particular structure of the NFSVs tends to reduce such suppression effect of nonlinearities, finally making the NFSV-type tendency error yield much large prediction error for Nino-3 SSTA of El Nino events. The NFSVs, compared to the FSVs, are more applicable in describing the most disturbing tendency error of the Zebiak–Cane model since they consider the effect of nonlinearities. The NFSV-type tendency errors may provide information concerning the sensitive areas where the model errors are much more likely to yield large prediction errors for El Nino events. If the simulation skills of the states in the sensitive areas can be improved, the ENSO forecast skill may in turn be greatly increased.
Roberto Buizza - One of the best experts on this subject based on the ideXlab platform.
-
impact of Singular Vector based satellite data thinning on nwp
Quarterly Journal of the Royal Meteorological Society, 2011Co-Authors: Peter Bauer, Roberto Buizza, Carla Cardinali, J N ThepautAbstract:Singular-Vector(SV)-based selective satellite data thinning is applied to the Southern Hemisphere (SH) extratropics to reduce analysis uncertainty and forecast error. For two seasons, the European Centre for Medium-Range Weather Forecasts (ECMWF) four-dimensional variational data assimilation system has been run in five different configurations with different satellite data coverage: two reference experiments used low-density and high-density coverage over the globe; in the SH two SV-based selective thinning experiments used low-density data everywhere apart from targeted regions; and one random-based thinning experiment used low-density data everywhere apart from randomly defined regions. The SV-based target regions have been defined either by daily operational SVs computed for the ECMWF Ensemble Prediction System, or by the previous year's mean seasonal distribution. Results indicate that the impact of the additional data largely depends on the season. Overall, forecast errors grow faster in the SH cold season than in the warm season. In the SH cold season, the general impact of adding data is smaller and the relative difference between the impact of the individual targeting experiments is smaller as well. In the cold season, the data assimilation system failed to extract the meteorological signal carried by the extra satellite data in sensitive regions. In the SH warm season, all experiments with more data produce a statistically more significant and longer-lasting positive impact on forecast skill. In this season, the SV-based targeting experiment performs best and as well as the reference experiment in which the data density is increased globally. Copyright © 2011 Royal Meteorological Society
-
a comparison of ensemble transform kalman filter targeting guidance with ecmwf and nrl total energy Singular Vector guidance
Quarterly Journal of the Royal Meteorological Society, 2002Co-Authors: Sharanya J Majumdar, Roberto Buizza, Craig H Bishop, Ronald GelaroAbstract:Ongoing adaptive observing programmes strive to improve short- and medium-range forecasts of winter weather over populated areas. Aircraft equipped with Global Positioning System (GPS) dropwindsondes are directed over ordinarily data-sparse regions (oceans) to collect observations to enhance the subsequent operational analysis–forecast cycle. Two objective techniques that have been used to identify optimal ‘target regions’ are the ensemble-transform Kalman filter (ET KF) and the Singular-Vector technique. The similarities and differences between targeting guidance based on the ET KF and total-energy Singular Vectors (TESVs) are assessed for ten cases during the North Pacific Experiment (NORPEX). TESVs are computed at the European Centre for Medium-Range Weather Forecasts (ECMWF) and the Naval Research Laboratory (NRL) using their respective global models, and the ET KF uses 25 ECMWF ensemble perturbations. Using measures based on (i) rankings of aircraft flight tracks and (ii) spatial similarities between targeting guidance maps, the main finding is that (a) the ET KF guidance is reasonably correlated with TESV guidance for at least seven of the ten NORPEX cases. Other findings include: (b) the ECMWF and NRL TESV targets sometimes differ significantly, (c) the ET KF and TESV guidance maps often display different optimal locations for targeting on smaller scales, but larger-scale aspects are usually similar, (d) the ET KF generally identifies larger regions over which useful observations can be taken compared with the SV technique, and (e) regions that are deemed effective for targeting by both techniques often correspond to baroclinic zones. The ET KF and SV techniques may identify similar regions for targeting if locations of large ensemble-based analysis-error variance coincide with areas where rapid perturbation growth occurs. On the other hand, they may identify different targeting locations for the following two reasons. First, the ET KF implicitly accounts for error correlation length-scales in its predictions of forecast-error variance reduction produced by any set of targeted observations. Hence, it can identify locations for targeted observations that are distant from the regions of high analysis sensitivity that are selected for targeting by the SV technique. Second, ET KF estimates of analysis-error variance are constrained to a subspace of evolved ensemble perturbations and are, therefore, rank deficient. Copyright © 2002 Royal Meteorological Society
-
the nature of Singular Vector growth and structure
Quarterly Journal of the Royal Meteorological Society, 2000Co-Authors: Brian J Hoskins, Roberto Buizza, J BadgerAbstract:The aim of this paper is to produce a basic understanding of the nature of Singular Vector growth and structure. The approach is through a sequence of Singular Vector calculations based on the European Centre for Medium-Range Weather Forecasts Ensemble Prediction System. Comparison is made for a range of cases based on trajectories that are evolving or constant atmosphere states, all cases having the same optimization time interval and resolution in the Singular Vector calculation. From these it is deduced that the constant or evolutionary nature of the trajectory is not as relevant as the smoothness of the trajectory. Unless the trajectory is very smooth, Singular Vectors tend to propagate from the upstream end of the storm-track baroclinic regions into their downstream end. The upscale transfer of energy in Singular Vector development is seen mostly as a kinematic effect of the strengthening flow in the downstream direction. Short-optimization-time Singular Vectors are found to grow by unshielding potential-vorticity (PV) structures associated with their compactness in the vertical and by untilting their westward tilted troughs as they propagate upwards. Longer-optimization-time Singular Vectors are lower in the troposphere and their interaction with the near-surface region enables longer-term normal-mode-like growth through PV coupling in the vertical. The division between short and longer optimization times is consistent with the time taken to propagate to the tropopause.
-
the Singular Vector structure of the atmospheric global circulation
Journal of the Atmospheric Sciences, 1995Co-Authors: Roberto Buizza, T N PalmerAbstract:Abstract The local phase-space instability Of the atmospheric global circulation is Characterized by its (nonmodal) Singular Vectors. The formalism of Singular Vector analysis is described. The relations between Singular Vectors, normal modes, adjoint modes, Lyapunov Vectors, perturbations produced by the so-called breeding method, and wave pseudomomentum are outlined. Techniques to estimate the dominant part of the Singular spectrum using large-dimensional primitive equation models are discussed. These include the use of forward and adjoint tangent propagators with a Lanczos iterative algorithm. Results are described, based first on statistics of routine calculations made between December 1992 and August 1993, and second on three specific case studies. Results define three dominant geographical areas of instability in the Northern Hemisphere: the two regions of storm track cyclogenesis, and the North African subtropical jet Singular Vectors can amplify as much as tenfold over 36 hours, and in winter ther...
-
Singular Vectors and the Predictability of Weather and Climate
Philosophical Transactions of the Royal Society A, 1994Co-Authors: Tim Palmer, Roberto Buizza, Franco Molteni, Y. Q. Chen, Susanna CortiAbstract:The local instability properties of a chaotic system are determined by the Singular Vectors and Singular values of the dynamical evolution operator, linearized about a finite trajectory portion of the integral curves of the nonlinear equations. Knowledge of these quantities allows an assessment of the reliability of a finite-time forecast from a chaotic system. After a brief study of the Lorenz model, Singular Vector analysis is applied to study three predictability issues in atmosphere-ocean dynamics. The first concerns the predictability of weather forecasts of a few days, and Singular Vector calculations are made from a large-dimensional numerical weather prediction model using an interative Lanczos algorithm. The second concerns the predictability of El Nino on seasonal to interannual timescales. Here Singular Vector calculations are made using a coupled ocean-atmosphere model of the tropical Pacific region. Finally we show results from a multi-decadal integration of a medium-resolution quasi-geostrophic model, and discuss the possible relevance of Singular Vector analysis for the problem of climate change.
Maithra Raghu - One of the best experts on this subject based on the ideXlab platform.
-
svcca Singular Vector canonical correlation analysis for deep learning dynamics and interpretability
arXiv: Machine Learning, 2017Co-Authors: Maithra Raghu, Justin Gilmer, Jason Yosinski, Jascha SohldicksteinAbstract:We propose a new technique, Singular Vector Canonical Correlation Analysis (SVCCA), a tool for quickly comparing two representations in a way that is both invariant to affine transform (allowing comparison between different layers and networks) and fast to compute (allowing more comparisons to be calculated than with previous methods). We deploy this tool to measure the intrinsic dimensionality of layers, showing in some cases needless over-parameterization; to probe learning dynamics throughout training, finding that networks converge to final representations from the bottom up; to show where class-specific information in networks is formed; and to suggest new training regimes that simultaneously save computation and overfit less. Code: this https URL
-
svcca Singular Vector canonical correlation analysis for deep learning dynamics and interpretability
Neural Information Processing Systems, 2017Co-Authors: Maithra Raghu, Justin Gilmer, Jason Yosinski, Jascha SohldicksteinAbstract:We propose a new technique, Singular Vector Canonical Correlation Analysis (SVCCA), a tool for quickly comparing two representations in a way that is both invariant to affine transform (allowing comparison between different layers and networks) and fast to compute (allowing more comparisons to be calculated than with previous methods). We deploy this tool to measure the intrinsic dimensionality of layers, showing in some cases needless over-parameterization; to probe learning dynamics throughout training, finding that networks converge to final representations from the bottom up; to show where class-specific information in networks is formed; and to suggest new training regimes that simultaneously save computation and overfit less.
