Bohebbian Rhapsody

Douglas Eck (University of Montreal) Bram Bakker (IDSIA, Lugano Switzerland) Freddie Mercury (Queen's University) Is this the real life Is this just fantasy Caught in a state space No escape from dimensionality Open your gates Look up to the weights and see I'm just a poor boy, i need no sympathy Because I'm temporal error flow My system is dynamical Anyway the error flows doesn't really matter to me to me. Yoshua just killed a net [Also Sepp] He showed decaying gradient [Also Sepp] Over time the error's spent Yoshua, life had just begun And now I'm in a local minimum Juergen Schmidh--ooooo--ber, Didn't mean to make you cry If I don't converge again this time tomorrow Try to guess all my weights, as if nothing really matters Too late, my time has come, Divergence won't abate Activations saturate Goodbye everybody, I've got to go Gonna change my name to something with ``kernel'' in it Mama ooo I don't want to die, I sometimes wish I'd had the name ``manifold'' (Loopy nets? Recurrent Control? . . . Oh it doesn't really matter) I see a little silhouetto of a man, Paul Werbos Paul Werbos will you do a tutorial? Thunderbolt and lightning very very frightening me Zimmermann, Pearlmutter, Hochreiter, John Moody, P. Tino Figaro Magnifico But I'm just a poor boy and nobody loves me He's just a network very untrendy We do not care about sequentiality! Easy come easy go, will you let me flow? BAYESmillah! no, we will not let you flow (let him flow) BAYESmillah! no, we will not let you flow (let him flow) Will not let you flow (let me flow) Will not let you flow (let me flow) No,no,no,no,no,no,no- Mama mia, mama mia, mama mia let me flow Marvin Minsky has a devil put aside for me,for me,for me- So you think you'll learn sequences without memory? So its only reduction of dimensionality? Well maybe. But don't forget speech, robots and finance And then theres control, and cars made by Ford, and there's music It's memory that matters The world is POMDP Gradients may vanish But sequence learning is the place to be Anyway, the error flows....