Baidu researchers propose ERNIE-ViLG, a 10-billion parameter scale pretraining framework for bidirectional text-image generation. Pretrained on 145 million (Chinese) image-text pairs, ERNIE-ViLG achieves state-of-the-art performance on both text-to-image and image-to-text generation tasks.

Here is a quick read: Baidu’s 10-Billion Scale ERNIE-ViLG Unified Generative Pretraining Framework Achieves SOTA Performance on Bidirectional Vision-Language Generation Tasks.

The paper ERNIE-ViLG: Unified Generative Pre-training for Bidirectional Vision-Language Generation is on arXiv.

Content provided by Junjie Shen, the first-author of the paper Drift with Devil: Security of Multi-Sensor Fusion based Localization in High-Level Autonomous Driving under GPS Spoofing.

In this work, we perform the first study on the security of MSF-based localization in AV settings. We find that the state-of-the-art MSF-based AD localization algorithm can indeed generally enhance the security, but have a take-over vulnerability that can fundamentally defeat the design principle of MSF, but only appear dynamically and non-deterministically. Leveraging this insight, we design FusionRipper, a novel and general attack that opportunistically captures and exploits take-over vulnerabilities. We perform both trace-based and simulation-based evaluations, and find that FusionRipper can achieve >= 97% and 91.3% success rates in all traces for off-road and wrong way attacks respectively, with high robustness to practical factors such as spoofing inaccuracies.

Hey. I'm thrilled to announce that my new book, Machine Learning Engineering, was just released and is now available on Amazon and Leanpub, as both a paperback edition and an e-book!

I've been working on the book for the last eleven months and I'm happy (and relieved!) that the work is now over. Just like my previous The Hundred-Page Machine Learning Book, this new book is distributed on the “read-first, buy-later” principle. That means that you can freely download the book, read it, and share it with your friends and colleagues, before buying.

The new book can be bought on Leanpub as a PDF file and on Amazon as a paperback and Kindle. The hardcover edition will be released later this week.

Here's the book's wiki with the drafts of all chapters. You can read them before buying the book: http://www.mlebook.com/wiki/doku.php

I will be here to answer your questions. Or just read the awesome Foreword by Cassie Kozyrkov!

​

https://preview.redd.it/9swslee6i8m51.jpg?width=1600&format=pjpg&auto=webp&s=bf83d13301b1f381cd5e1e1544bec4e8a1fff8c7

To alleviate the quadratic dependency of transformers, a team of researchers from Google Research recently proposed a new sparse attention mechanism dubbed BigBird. In their paper Big Bird: Transformers for Longer Sequences, the team demonstrates that despite being a sparse attention mechanism, BigBird preserves all known theoretical properties of quadratic full attention models. In experiments, BigBird is shown to dramatically improve performance across long-context NLP tasks, producing SOTA results in question answering and summarization.

Here is a quick read: Google ‘BigBird’ Achieves SOTA Performance on Long-Context NLP Tasks

The paper Big Bird: Transformers for Longer Sequences is on arXiv.

Link to the website and the paper on the methodology.

video

demo

github

Very impressive results:

The research team says their proposed method’s 88.4 percent accuracy on ImageNet is 2.0 percent better than the SOTA model that requires 3.5B weakly labelled Instagram images. And that’s not all: “On robustness test sets, it improves ImageNet-A top-1 accuracy from 61.0% to 83.7%, reduces ImageNet-C mean corruption error from 45.7 to 28.3, and reduces ImageNet-P mean flip rate from 27.8 to 12.2.”

A quick read: Google Brain & CMU Semi-Supervised ‘Noisy Student’ Achieves 88.4% Top-1 Accuracy on ImageNet

The paper: Self-training with Noisy Student improves ImageNet classification

The trio of researchers have made deep neural networks a critical component of computing, and in individual talks and a panel discussion they discussed their views on current challenges facing deep learning and where it should be heading.

Read more

In a new paper, researchers from the University of Toronto, Vector Institute, and University of Wisconsin-Madison propose SISA training, a new framework that helps models “unlearn” information by reducing the number of updates that need to be computed when data points are removed.

Read more.

Google’s Meena was released in a preprint recently stating that it could create its own joke, but the threat of racism in the system and its logical inconsistencies aren’t ready to be deployed in a corporate environment. Change my mind

In this tutorial author and teacher Ahmed Fawzy Gad covers a thorough introduction to autoencoders and how to use them for image compression in Keras.

Article link: https://blog.paperspace.com/autoencoder-image-compression-keras/

How do you analyze the distribution of scores produced from a binary classification model to make sure it makes sense?

I am using a decision tree to predict how likely an individual is to vote or not. One idea is to analyze the splits of the tree to see why an individual was given that score. For example, people that got a score below 25% had these characteristics, people that got a score between 25-50% had these characteristics, etc. Is there a better way to do it?