Unrolling [Deep] Neural Networks

When I wrote my last post, I was not aware of research avenues that link Neural Networks (specially, the "deep" ones) to "iterative" algorithms used in signal and image processing.

I just found by chance the following paper by Prof. Vishal Monga, Yuelong Li and Prof. Yonina Eldar:

1) "Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processing"
Vishal Monga, Yuelong Li, Yonina C. Eldar
https://arxiv.org/abs/1912.10557

Good news when I read the word "interpretable" close to the words "Neural Networks"...

Having nice conversations with Generative Adversarial Networks (GANs)

I recently came across a blog by Bram Cohen and read a very interesting post (which can be found HERE) about one of the most intriguing flaws current neural networks have (so to speak): why they fail so tremendously (in some cases, under some circumstances) when a small amount of graciously created noise is introduced into the input data.

He particularly recommended reading the paper:

Adi Shamir, Itay Safran, Eyal Ronen, Orr Dunkelman
"A Simple Explanation for the Existence of Adversarial Examples with Small Hamming Distance"
https://arxiv.org/abs/1901.10861

Look at (the text surrounding and) Figure 1 (What the h...?)

That is the reason why I find particularly interesting the research approaches that Prof. Dr. Antonio Torralba and his PhD student David Bau are carrying out in a particular type of neural networks called Generative Adversarial Networks.

Not the same, but, (well) in some sense...

A summarising video of what I mean by this would be the one below:

Causal Discovery For Climate

I am very happy to see how Machine Learning is contributing to climate prediction. In particular, the initiative Causality 4 Climate tries to establish the maturity some models may have to infer causes leading to extreme climate processes happening nowadys.

The following paper is particularly in order:

Jakob Runge, Sebastian Bathiany, Erik Bollt, Gustau Camps-Valls, Dim Coumou, Ethan Deyle, Clark Glymour, Marlene Kretschmer, Miguel D. Mahecha, Jordi Muñoz-Marí, Egbert H. van Nes, Jonas Peters, Rick Quax, Markus Reichstein, Marten Scheffer, Bernhard Schölkopf, Peter Spirtes, George Sugihara, Jie Sun, Kun Zhang & Jakob Zscheischler, 
"Inferring causation from time series in Earth system sciences"
Nature Communications volume 10, Article number: 2553 (2019)

Link to www of the paper, HERE.

I am also proud to know some of the authors :-)

Try not to learn my [confounding] biases!

Dr. David Lopez-Paz's research never deceives. His (I think) last published work goes to the main core of the classifier training process, i. e., we should try to somehow avoid classifiers to learn spurious (not consistent) relationships that establish on the training data and which make generalization ability difficult to carry out. Their aim is to develop "invariant" and "causal predictors", to enable a "good generalisation behaviour".

Important (well, at least I consider so) piece of research:

Martin Arjovsky, Léon Bottou, Ishaan Gulrajani and David Lopez-Paz, "Invariant Risk Minimization", arXiv:1907.02893, 2019.

https://arxiv.org/abs/1907.02893

Link to pdf file HERE.

No reason not to see you

Walls and dark illumination conditions are becoming less of a problem to identify and infer the presence of people and even actions taking place "out of our sight".

Nice work by Prof. Dina Katabi (MIT Computer Science & Artificial Intelligence Lab, MIT-CSAIL) on "Making the invisible visible":

1) Tianhong Li, Lijie Fan, Mingmin Zhao, Yingcheng Liu, Dina Katabi, "Making the Invisible Visible: Action Recognition Through Walls and Occlusions"
https://arxiv.org/abs/1909.09300

PDF file of the paper available HERE.

Dignified AI

Happy to learn about this work made at the University of Minnesota

Too much -- too little

Reading Dr. Paulina Jaramillo I found the following opinion report.

I completely agree with Shoshanna Saxe. I think there will be a time when we will not be able to efficiently use all the data we are generating and our life may become quite difficult to manage.

Life seems to evolve in cycles and in an action-reaction dynamics...Difficult to "see the horizon"...

Will we choose?

I do not know what my future will be, but something is a cornerstone in my life: try to do your part in science. It will be bigger or smaller, better or worse, but, if honest, it will contribute somehow.

I recall President John F. Kennedy's speech "We choose to go to the moon...":

So, when I see pieces of work like the one made by Dr. Laure Zanna I, again, think to myself "Good for you!":

Laure Zanna, Samar Khatiwala, Jonathan M. Gregory, Jonathan Ison, and Patrick Heimbach, "Global reconstruction of historical ocean heat storage and transport", Proceedings of the National Academy of Sciences of the USA (PNAS) January 22, 2019 116 (4) 1126-1131; https://doi.org/10.1073/pnas.1808838115.

Equilibrium

Old (and not so) school rules...

Rita Pierson

Shred the tendency

Open access to textbooks simply means that more students are able to have access to knowledge, allowing them to eliminate a part of their financial problems in their university path. However, this means what it means, no more and no less. 

We have the tendency to look at a problem, and its *derivatives*, *consequences*, *similarities* to other problems, etc.

A recent study finds no significant effect on the student learning success when using open access textbooks, as compared to other resources:

Phillip J. Grimaldi, Debshila Basu Mallick, Andrew E. Waters, Richard G. Baraniuk, "Do open educational resources improve student learning? Implications of the access hypothesis", PLoS ONE 14(3): e0212508. https://doi.org/10.1371/journal.pone.0212508

Link: HERE

I did not expect another outcome. However, this study is not *definitive*...