hh.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
A simple trick for estimating the weight decay parameter
Halmstad University, School of Information Technology, Halmstad Embedded and Intelligent Systems Research (EIS).
1998 (English)In: Neural Networks: Tricks of the Trade / [ed] Genevieve B. Orr and Klaus-Robert Müller, Berlin: Springer Berlin/Heidelberg, 1998, 71-92 p.Conference paper, Published paper (Refereed)
Abstract [en]

We present a simple trick to get an approximate estimate of the weight decay parameter lambda. The method combines early stopping and weight decay, into the estimate lambda=parallel to del E(W(es))parallel to/parallel to 2W(es)parallel to, where W(es) is the set of weights at the early stopping point, and E(W) is the training data fit error. The estimate is demonstrated and compared to the standard cross-validation procedure for lambda selection on one synthetic and four real life data sets. The result is that lambda is as good an estimator for the optimal weight decay parameter value as the standard search estimate, but orders of magnitude quicker to compute. The results also show that weight decay can produce solutions that are significantly superior to committees of networks trained with early stop ping.

Place, publisher, year, edition, pages
Berlin: Springer Berlin/Heidelberg, 1998. 71-92 p.
Series
Lecture Notes in Computer Science, ISSN 0302-9743 ; 1524
National Category
Engineering and Technology
Identifiers
URN: urn:nbn:se:hh:diva-18803DOI: 10.1007/3-540-49430-8_4ISI: 000083105900004ISBN: 3-540-65311-2 (print)OAI: oai:DiVA.org:hh-18803DiVA: diva2:541008
Available from: 2012-07-13 Created: 2012-06-25 Last updated: 2017-06-02Bibliographically approved

Open Access in DiVA

No full text

Other links

Publisher's full text

Search in DiVA

By author/editor
Rögnvaldsson, Thorsteinn
By organisation
Halmstad Embedded and Intelligent Systems Research (EIS)
Engineering and Technology

Search outside of DiVA

GoogleGoogle Scholar

Altmetric score

Total: 124 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf