Preliminary Objectives and Ideas

Let me put it clearly: although I am knowledgeable in deep learning and algorithm theory, I don’t know much about MANNs yet (by the way I’m not a big fan of acronyms, but memory-augmented neural networks is very long to write, so MANN it will be). In fact this blog will accompany me along my journey of discovering MANNs, as it will be the main subject of my PhD research.

I am one year into my PhD; in the first twelve months I worked mostly on classical deep learning, focusing on binary activated neural networks and PAC-Bayes statistical guarantees, mostly because of my advisor’s research interests. It led to a paper which hopefully will be accepted at AIStats. But I insisted on working on my own stuff after the submission of the paper, because I don’t really believe PAC-Bayes will revolutionize the world, nor am I interested in spending my life doing statistics stuff, as I am a computer scientist, not an actuary.

On the other hand, my master’s degree was about algorithmic complexity issues in post-quantum cryptography. I enjoyed the complexity part a lot, but did not feel at my place in the cryptography world, because I don’t like the rest of computer security and do not care too much about online privacy in general. Except my bank account. Don’t go there.

I have about three years to go, maybe a little more, to complete my PhD, and I wanted to do my thing. I like algorithms, I like complexity, I like artificial intelligence, I like neural networks… is there something for me out there?

Well, actually if there were already something, I’d be sad. Remember, I want to do my own thing. While MANNs are indeed at the crossroads of artificial intelligence, neural networks and algorithms, it does not take complexity into account. I did come across one paper claiming they could learn to sort with \Theta(nlogn) operations, which is great as sorting provably cannot get any better, but this is still very far from the complexity landscape we have with classical algorithms.

Not a very accurate diagram, but I think it illustrates my point.

Me? I want to define the landscape of the complexity of algorithms learned with MANNs. I’ll give more details on MANNs in a subsequent post, but for now I can quickly state that MANNs are neural networks that are optimized not just to classifier, but to become whole algorithms, with loops, memory accesses, etc. The obtained algorithms should have a complexity, just as classical ones do. Can we understand what complexity is possible to achieve for a given problem? Can we force the optimization process to find efficient solutions, by trying to force it to find a solution within a given complexity class?

Why did I choose to write a blog? Firstly, my wife suggested I do so, and I do everything she says because I love her a lot. But mostly, after writing papers and grant submissions in the cold, length-limited, so-called scientific style, I want to feel the freedom of writing however I feel like. I want to be able to write standard deviation instead of std because I have the space to do so. I want to be able to write a paragraph-long analogy to make my readers really understand what I am talking about. I want to be able to repeat « I want to be able » at every sentence, not combine it into one long, painful, sentence.

What I have read so far in the MANN community was disastrously unclear, even for blog posts, an exception being the foundation paper on the subject, Neural Turing Machines, and maybe this blog post which seems promisingly clear. Furthermore, it seems most papers in the field of MANNs have only been published on Arxiv, not in any peer-reviewed conference. This makes me think the field really lacks theoretical foundations and I’d be pleased to contribute to that.

4 Commentaires

Passer au formulaire de commentaire

  1. May I simply say what a relief to uncover somebody that really understands what theyre discussing over the internet. You certainly know how to bring a problem to light and make it important. More and more people really need to check this out and understand this side of the story. I was surprised that youre not more popular because you certainly have the gift.

  2. Quality WordPress links in the comments from 5000 uniques. domains order here.

  3. Use artificial intelligence to cut turnaround time, extend your budget, and create more high-quality content that Google and readers will love. How It Works? WordAi is extremely fast and intuitive. Just enter your content, click rewrite, and in a matter of seconds, WordAi will rewrite an entire piece of content for you. WordAi comes up with different ways to express the same ideas by rewriting every sentence from scratch. This not only removes duplicate content, it also makes the rewritten content read completely naturally. Scale Your Business Make your entire content production process 10x more efficient. Use AI to create more high-quality, unique content that impresses clients and engages readers, all without having to put in more hours Register here and get a bonus.

  4. Hey there are using WordPress for your site platform? I’m new to the blog world but I’m trying to get started and set up my own. Do you need any html coding expertise to make your own blog? Any help would be really appreciated!

Les commentaires sont désactivés.