您好，欢迎访问中国农业科学院 机构知识库！

全部

  • 全部
  • 标题
  • 学者
  • 机构
  • 关键词

Topological Analysis of Deep Network Training

文献类型： 会议论文

第一作者： Andrea De Gaetano

作者： Andrea De Gaetano 1 ;

作者机构： 1.BioMatLab Laboratorio di BioMatematica CNR-IASI Roma, CNR-IRIB, Palermo, Italy

关键词： Training;Manifolds;Estimation;Artificial neural networks;Manifold learning;Classification algorithms;Task analysis

会议名称： [ "International Conference on Intelligent Engineering Systems" , "IEEE International Conference on Intelligent Engineering Systems"]

主办单位：

页码： 11-12

摘要： When training a Deep Neural Network, consisting of hundreds or thousands of nodes with correspondingly many more weights and biases, the actual convergence value of these coefficients is typically of secondary or negligible interest compared with the performance of the network on the classification or "estimation" task. Such weights are typically initialized by some random scheme, then adapted through some training algorithm until convergence occurs to the best performance achievable in the given situation. There is no requirement that such "best" coefficient values be unique, and in fact, from a statistical estimation point of view, the network is typically macroscopically overparametrized with respect to the task at hand. In other words, if n is the number of coefficients, then an n-dimensional "best" value obtained through one training exercise, starting from some random value, could well be equivalent in performance to (many) other different optimal values starting from different random initial values. Supposing to have conducted the training exercise m times, it is thus natural to ask how the m optimal values are positioned in Euclidean n-space: whether homogeneously dispersed or, as we suspect, compacted into a (generally nonlinear) d-dimensional manifold embedded in n-space, where d should be much smaller than n if the network is severely overparametrized with respect to the task. The scope of the present work is to define the problem and characterize, by means of some Manifold Learning technique, the dimension of the solutions manifold in a standard classification problem.

分类号： tp313

  • 相关文献

[1]XNet: Wavelet-Based Low and High Frequency Fusion Networks for Fully- and Semi-Supervised Semantic Segmentation of Biomedical Images. Yanfeng Zhou,Jiaxing Huang,Chenlong Wang,Le Song,Ge Yang. 2023

[2]Automated Fruit Classification Based on Deep Learning Utilizing Yolov8. Christine Dewi,Oksam M Kamlasi,Gunjan Chhabra,Guowei Dai,Keshav Kaushik,Inam Ullah Khan. 2023

[3]Non-contact Apnea-Hypopnea Index Estimation using Near Infrared Video *. Kaiyin Zhu,Azadeh Yadollahi,Babak Taati. 2019

[4]A new force profile signal for a convex solution of muscle force estimation from electromyographic signals*. Mehdi Shirzadi,Arezoo Mirshamsi,Alp E?refo?lu,Monica Rojas-Martínez,Hamid Reza Marateb,Miquel Angel Ma?anas,Taian Vieira. 2023

[5]Estimation of carbon and nitrogen contents in citrus canopy by low-altitude remote sensing. Liu Xuefeng,Lyu Qiang,He Shaolan,Yi Shilai,Wang Zhitao,Xie Rangjin,Zheng Yongqiang,Deng Lie,Liu Xuefeng,Lyu Qiang,Deng Lie,Hu Deyu. 2016

[6]Estimating the Contribution of New Seed Cultivars to Increases in Crop Yields: A Case Study for Corn. Qian, Jiarong,Zhao, Zhijun. 2017

[7]Systematical estimation of GPM-based global satellite mapping of precipitation products over China. Zhao, Haigen,Yang, Bogang,Huang, Yingchun,Yang, Shengtian,Bai, Juan,Wang, Zhiwei,Dong, Guotao. 2018

[8]Parameter estimation and the space of random variables. Andrea De Gaetano,Giulia Aliffi. 2023

[9]Extraction of Settlements in Plains and Mountainous Areas Based on High-Resolution Remote Sensing Data and SAM Segmentation Algorithm. Miao Liu,Shijun Wang,Zhenhai Li,Jing Chen. 2024

[10]Automatic Multi-Atlas Liver Segmentation and Couinaud Classification from CT Volumes. Sofia Pla-Alemany,Juan Antonio Romero,Jose Manuel Santabarbara,Roberto Aliaga,Alicia M. Maceira,David Moratal. 2021

[11]Nonparametric Early Stopping Detection for c-VEP-based Brain-Computer Interfaces: A Pilot Study. Víctor Martínez-Cagigal,Eduardo Santamaría-Vázquez,Sergio Pérez-Velasco,Diego Marcos-Martínez,Selene Moreno-Calderón,Roberto Hornero. 2023

[12]Smartphone PPG Validation for a Depression Assessment Protocol. Diego Cajal,Mar Posadas-de Miguel,Concepción de la Cámara,Spyridon Kontaxis,Jesús Lázaro,Raquel Bailón. 2022

[13]Increasing The Convergence Speed of Robust Fixed Point Transformaton-based Adaptive Control by Steffensen's Method in SISO Case. Krisztián Kósi,Atinga Awudu. 2022

[14]Gridded Glyphmaps for Supporting Spatial COVID-19 Modelling. Aidan Slingsby,Richard Reeve,Claire Harris. 2023

[15]The Associations Between HPV-infections Associated Risk Factors and Cervical Cancer Associated Risk Factors Using Chi-square Method. Ogbolu Melvin Omone,Miklós Kozlovszky. 2022

[16]Automatically estimated meals in Model Predictive Control-Moving Horizon Estimation control strategy. Máté Siket,Kamilla Novák,Levente Kovács,Gy?rgy Eigner. 2022

[17]Estimation of Sleep Status in Sleep Apnea Patients Using a Novel Head Actigraphy Technique. Richard Hummel,T. Douglas Bradley,Geoff R. Fernie,S.J. Isaac Chang,Hisham Alshaer. 2015

[18]A High-Resolution Grassland Aboveground Biomass Estimation Method Based on SfM and Grassland Height Factor. Mengdi Cao,Shanshan Cao,Wei Sun,Fantao Kong. 2023

[19]A Revised Two-Leaf Light Use Efficiency Model for Improving Gross Primary Production Estimation at a Tropical Evergreen Broadleaved Forest Site. Lingxiao Huang,Meng Liu,Yazhen Jiang,Ronglin Tang. 2023

[20]Estimation of daily evapotranspiration using MODIS data to calculate instantaneous decoupling coefficient and resistances. Yazhen Jiang,Xiaoguang Jiang,Ronglin Tang,Zhao-Liang Li,Chen Ru,Yuze Zhang,Cheng Huang. 2017

作者其他论文 更多>>
置顶
购物车
反馈

© 京ICP备09089781号-29 主办单位：中国农业科学院

学术资源： 中国农业科学院 农业专业知识服务系统 农科联盟资源共建共享平台 中国农业科技文献与信息服务平台 中国农业期刊集成服务平台