whoa there, pardner!

reddit's awesome and all, but you may have a bit of a problem.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please contact us at .

when contacting us, please include your ip address which is: 195.201.137.88 and reddit account

whoa there, pardner!

reddit's awesome and all, but you may have a bit of a problem.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please contact us at .

when contacting us, please include your ip address which is: 195.201.137.88 and reddit account

whoa there, pardner!

reddit's awesome and all, but you may have a bit of a problem.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please contact us at .

when contacting us, please include your ip address which is: 195.201.137.88 and reddit account

whoa there, pardner!

reddit's awesome and all, but you may have a bit of a problem.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please contact us at .

when contacting us, please include your ip address which is: 195.201.137.88 and reddit account

whoa there, pardner!

reddit's awesome and all, but you may have a bit of a problem.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please contact us at .

when contacting us, please include your ip address which is: 195.201.137.88 and reddit account

whoa there, pardner!

reddit's awesome and all, but you may have a bit of a problem.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please contact us at .

when contacting us, please include your ip address which is: 195.201.137.88 and reddit account

whoa there, pardner!

reddit's awesome and all, but you may have a bit of a problem.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please contact us at .

when contacting us, please include your ip address which is: 195.201.137.88 and reddit account

whoa there, pardner!

reddit's awesome and all, but you may have a bit of a problem.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please contact us at .

when contacting us, please include your ip address which is: 195.201.137.88 and reddit account

whoa there, pardner!

reddit's awesome and all, but you may have a bit of a problem.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please contact us at .

when contacting us, please include your ip address which is: 195.201.137.88 and reddit account

whoa there, pardner!

reddit's awesome and all, but you may have a bit of a problem.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please contact us at .

when contacting us, please include your ip address which is: 195.201.137.88 and reddit account

whoa there, pardner!

reddit's awesome and all, but you may have a bit of a problem.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please contact us at .

when contacting us, please include your ip address which is: 195.201.137.88 and reddit account

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

Understanding the building blocks and design choices of graph neural networks.

What components are needed for building learning algorithms that leverage the structure and properties of graphs?

The core idea behind wav2vec 2.0 is to teach the model to do two things in parallel:Quantize continuous speech data into discrete units automatically.

Wav2vec uses 2 groups with 320 possible words in each group, hence a theoretical maximum of 320 x 320 = 102,400 speech units.

The final context vectors then go through the last projection layer to match the dimension of the quantized speech units Qt.

Fine-tuning and downstream tasksThis concludes our tour of wav2vec 2.0 and its pre-training process.

HuBERT re-uses embeddings from the BERT encoder to improve targets, while wav2vec 2.0 only uses the output of the convolutional network for quantization.

Graph Neural Networks (GNNs) are by far the most common among graph ML methods and the most popular neural network architectures overall [2].

CitationFor attribution in academic contexts or books, please cite this work asMichael Bronstein, "Beyond Message Passing, a Physics-Inspired Paradigm for Graph Neural Networks", The Gradient, 2022.

BibTeX citation:@article{dlneuro2022,author = {Bronstein, Michael},title = {Beyond Message Passing, a Physics-Inspired Paradigm for Graph Neural Networks},journal = {The Gradient},year = {2022},howpublished = {\url{https://thegradient.pub/graph-neural-networks-beyond-message-passing-and-weisfeiler-lehman}},}[1] See e.g.

A general form of message passing an…

On their own, AI ethics principles are insufficient to improve AI systems.

Instead, I suggest that each organization should articulate its own AI ethics principles, and I sketch ways to do so responsibly.

The search for universal AI ethics principlesIn recent years, several research groups have sought unifying themes in current AI ethics principles.

A key question is how to formulate AI ethics principles responsibly and how to tell that an organization has developed its principles responsibly.

But the first question to ask is “which principles?”, and my answer is: Don’t settle for “universal” AI ethics principles.

Specifically, this overview will first explain some common neuroimaging modalities more in-depth and then discuss applications of deep learning in conjunction with some of the unique characteristics of neuroimaging data.

These unique characteristics tie into a broader movement in deep learning, namely that data understanding should be a goal in itself to maximize the impact of applied deep learning.

Unique and leverageable aspects of neuroimaging dataAmong others [20], one critical challenge with deep learning in the field of neuroimaging is the limited number of samples; many neuroimaging datasets range from roughly 300 to 1300 subjects.

Data understanding and interpretation is an essentia…

Tech and AI startups have continued to gain prominence as the AI wave swept the country in 2018.

Hanoi and Ho Chi Minh City have developed a robust ecosystem for tech startups, with dominating sectors including AI, e-commerce, fin-tech, and enterprise solutions.

The ecosystem of 149 AI startups has attracted funding from both domestic and regional venture capital firms.

However, according to computer science professor Than Khoat, these structural initiatives have only started rolling out since 2020, and have not yet translated to a significant boost in tech talent supply to meet the pressing demand of tech talent of the fast growing tech ecosystem.

CitationFor attribution in academic contex…

Recently, NLP technology facilitated access and synthesis of COVID-19 research with the release of a public, annotated research dataset and the creation of public response resources.

Wikipedia serves as a source for BERT, GPT and many other language models.

Responding to this, MIT researchers have released StereoSet, a dataset for measuring bias in language models across several dimensions.

If NLP technology is going to be something revolutionary, it will need to be better and different.

CitationFor attribution in academic contexts or books, please cite this work asBenjamin Batorsky, "New technology, old problems: The missing voices in Natural Language Processing", The Gradient, 2022.

Somehow I think, in terms of chess, engines are always capable of bringing out the key points of a position but never the actual key point itself.

Discussing this pre-neural era of chess engines, Sadler suggests that the elements of the hand-crafted function are uninteresting despite being inherently interpretable.

Before chess engines were unambiguously better than humans (and even for a while after they were), Seconds faced the challenge of doubting the validity of an engine prediction.

In this gap, end-users resort to reading the tea leaves of their model’s predictions.

Sadler additionally has extensive, high quality content on his personal and book-related YouTube channels: Personal, Ga…

Active learning: Find samples for human-in-the-loopIn active learning, we select the most “interesting” unlabeled samples for labeling via human-in-the-loop (HITL).

Facebook added a twist to active learning by including similarity search as a filter (Similarity Search for Efficient Active Learning and Search aka SEALS).

Snorkel DryBell then applies these labeling functions on unlabeled examples before loading the output and labeling functions into a generative model.

Even Tesla, which probably has one of the most sophisticated data labeling systems to support Autopilot, hasn’t worked out all the kinks.

How to bootstrap labels at various stages of maturityGetting labels for machine learning …

Another problem arises if you try to find a high quality VAD with a permissible license.

Voice Activity Detection is the problem of looking for voice activity – or in other words, someone speaking – in a continuous audio stream.

The input is just a small audio chunk, and the output is a probability that this chunk contains speech given its history.

CitationFor attribution in academic contexts or books, please cite this work asAlexander Veysov and Dimitrii Voronin, "One Voice Detector to Rule Them All", The Gradient, 2022.

BibTeX citation:@article{veysov2020towardimagenetstt,author = {Veysov, Alexander and Voronin, Dimitrii},title = {One Voice Detector to Rule Them All},journal = {The Gradie…

Yet many of the key ingredients described by Aristotle are conspicuously absent from modern AI, especially in deep learning.

In this essay I will outline some key points of Aristotelian logic and epistemology to show how their absence in traditional deep learning is responsible for deep learning’s well-known limitations.

However, inductive reasoning is a process distinct from deductive reasoning, while traditional deep learning does not respect this separation.

Because traditional deep learning does not learn generalizations, it is simply a sophisticated pattern-matching technology, not a true learning method.

CitationFor attribution in academic contexts or books, please cite this work asPa…

To give the reader some perspective, it has been estimated that the pharmacologically active chemical space (i.e.

Despite being in its infancy, the use of AI to explore chemical space has already shown some great promise.

It has given us a new paradigm for exploring the chemical space; a new way to test theories and hypotheses.

The overall chemical space is infinite, so the most we can hope for is a large enough sample that will help us generalize.

CitationFor attribution in academic contexts or books, please cite this work asVictor Cano Gil, "How AI is Changing Chemical Discovery", The Gradient, 2022.

Some universities like Nanyang Technological University (NTU), Singapore, have dedicated AV testing environments on campus to simulate real-life road conditions.

New York is reported to be using archaic AV policies that do not benefit AV firms.

I foresee this body regulating the issue of licenses for those interested in purchasing and using self-driving vehicles.

Up-and-coming smart cities must anticipate what it takes to accommodate self-driving vehicles alongside regular vehicles.

CitationFor attribution in academic contexts or books, please cite this work asRishabh Anand, "Engaging with Disengagement", The Gradient, 2021.

My original copy of Gödel, Escher, Bach, as well as a punch card for programmingThough I considered pursuing a career in computer science or cognitive science, I wound up taking a different path.

This type of creativity, I believe, is one of the greatest things AI has to offer.

Human thought has expanded to include machine thought.

When I interviewed Gary Marcus, author of Rebooting AI: Building Artificial Intelligence We Can Trust, he said, “I think that at some point AI will fundamentally transform science and technology and medicine.

As a freelance science journalist, she regularly contributes to Muse magazine, Front Vision, and Science News for Students.

Quotations colored in blue are directly extracted from the MIT Work of the Future task force reports.

These ideas come from the task force research brief on “Artificial Intelligence and the Future of Work”.

[7] This description of the purpose of the MIT Future of Work Task Force is stated on their website homepage at https://workofthefuture.mit.edu/.

[9] All of the MIT Future of Work Task Force field study reports can be found on either their Research Brief webpage https://workofthefuture.mit.edu/research-type/briefs/ or their Working Paper webpage https://workofthefuture.mit.edu/research-type/working-papers/ .

[18] Thomas W. Malone, Daniela Rus, Robert Laubacher, “Artificial Intelligence a…

Hopsworks Feature Store: A Transparent Dual Storage SystemThe Hopsworks Feature Store is a dual storage system, consisting of the high-bandwidth (low-cost) offline storage and the low-latency online store.

Feature freshness is defined as the end-to-end latency between an event arriving that triggers a feature recomputation and the recomputed feature being published in the online feature store.

The Hopsworks Online Feature Store is built around four pillars in order to satisfy the requirements while scaling to manage large amounts of data:HSFS API: The Hopsworks Feature Store library is the main entry point to the feature store for developers, and is available for both Python and Scala/Java.…

💥 Deep Dive: Inside Predibase, the enterprise declarative machine learning platformLow-code ML platforms have received a lot of attention in the past few years, but haven’t yet achieved widespread adoption.

Predibase looks to deliver a high-performance, low-code approach to machine learning (ML) for individuals and organizations who have tried operationalizing ML but found themselves re-inventing the wheel each step of the way.

Declarative ML Systems: LEGO for Machine LearningThe basic idea behind declarative ML systems is to let users specify entire model pipelines as configurations and be intentional about the parts they care about while automating the rest.

Declarative ML systems were pi…

In this article, Run:AI’s team introduces rntop , a new super useful open-source tool that measures GPU cluster utilization.

Why measure GPU cluster utilization?

It’s almost impossible to get an accurate measurement of GPU cluster utilization without a tool, even in the most advanced teams running AI in production.

rntop (pronounced “run top”) available here on GitHub, enables GPU utilization monitoring, anywhere, anytime.

GPU utilization is calculated by averaging the utilization of all the GPUs in the cluster.

In this issue:we discuss the fundamental enabler of distributed training: message passing interface (MPI) ;we overview Google’s paper about General and Scalable Parallelization for ML Computation Graphs ;we share the most relevant technology stacks to enable distributed training in TensorFlow applications.

💡 ML Concept of the Day: MPI: The Enabler of Distributed TrainingDuring this series about distributed training, we have covered some of the main methods that enable the scaling of training across large clusters of nodes.

However, one question that is on everyone’s mind when learning about distributed training is about the technologies that make this possible.

To conclude this series, we w…

We are happy to support the 3rd Annual MLOps World 2022!

The MLOps World Committee would like to invite you this June 9-10th for a truly must-attend event, and an unforgettable experience in Toronto, Canada!

Help find TalentFor anyone building towards or looking to sustain models in production, this is a must-attend event!

You’ll get to experience:Live coding tutorials, virtual and in-person sessions, case studies, demo and brain dates5 tracks (intermediate to advanced)Fun social parties downtown Toronto, Canada 🎉Limited SPECIAL OFFER Tickets are only $135-250!

Download the PDF full calendar of events here.

The 2022 edition of Google I/O took place last week and machine learning (ML) was front and center.

Just like Microsoft’s Ignore AWS re:Invent, I/O provides a first row seat to the ML innovation happening at Google and the new additions to its ML stack.

Google also made available new versions of the LaMDA (Language Model for Dialog Applications) and Pathways Language Model (PaLM) which power systems such as the Google Assistant.

Finally, there was an unexpected announcement of a new form of augmented reality glasses that leverage sophisticated computer vision and language models.

I/O 2022 provided a glimpse of Google’s investments in ML research and technology.

Join us this Wednesday and Thursday at apply() to hear the best ML practitioners and thought leaders discuss data engineering for operational ML!

Speakers include practitioners from the Wikimedia Foundation, Facebook, Gojek, Snapchat, Instacart, Walmart, Stripe, Snowflake, Databricks, Hugging Face, Tecton, and more.

We’d love for you to join us!

REGISTER FOR FREE🔦 A Few Highlights of the Agenda 🔦Clem Delangue, CEO at Hugging Face: Is Open-Source Machine Learning Becoming the Most Impactful Technology of the Decade?

Check out the full agenda and register here.

💥 Deep Dive: Continuous Model Observability With SuperwiseThe MLOps lifecycle is often fraught with information overload, which can lead to sub-optimal decision making.

This is where continuous model observability comes in.

Let’s dive into the Superwise platform and see Continuous Model Observability, in action.

How to Implement Continuous Model ObservabilityContinuous model observability can be tough to implement on your own.

You can start with Superwise’s generous community edition that includes all of the features necessary for continuous model observability.

In this article, Masterful AI’s team suggests that instead of throwing more training data at a deep learning model, one should consider semi-supervised learning (SSL) to unlock the information in unlabeled data.

Image source: Masterful AIIntroPreviously, we showed that throwing more training data at a deep learning model has rapidly diminishing returns.

Try SSLSemi-supervised learning (SSL) means learning from both labeled and unlabeled data.

The key insight: labeling is not your only source of information... unlabeled data also has information!

Semi-supervised learning is the key to unlocking the information in unlabeled data.

In this issue:we discuss pipeline parallelism ;we explore PipeDream, an important Microsoft Research initiative to scale deep learning architectures;we overview BigDL, Intel’s open-source library for distributed deep learning on Spark.

💡 ML Concept of the Day: What is Pipeline Parallelism?

Recently, we have seen a third type of architecture known as pipeline parallelism (PP) gaining significant traction in the deep learning space.

The typical bottleneck in data parallelism methods is the high communication costs between nodes.

Similarly, model parallelism is typically prompt to inefficiencies given the mismatch between a model architecture and the underlying hardware topologies.

📝 EditorialLarge language models are the norm of the day in deep learning.

Large language models’ high computational and energy requirements represent a high barrier to entry for most organizations.

Regardless of the challenges, we have seen notable steps toward responsible open-sourcing large language models.

Last week, Meta AI open-sourced the first version of OPT-175B, an astonishing 175 billion parameter language model that is able to master multiple language tasks.

The release of OPT-175B is an important step toward making large language models more accessible to the broader deep learning community.

In Active Learning 101: A Complete Guide to Higher Quality Data (Part 1), they outlined the basics of Active Learning, how it’s different from other types of computer vision, and its popular subtypes.

Rather than weighing each piece of data equally, active learning works to find the most valuable data that will lead to higher accuracy and efficiency.

Entropy: What it is and How it Fuels Active Learning 🚀‍To build an effective machine learning model, data must be diverse and represent highly possible, yet different, outcomes.

Highlighting differences and encouraging diverse examples in an active learning model leads to better results and faster adoption of the desired behavior.

The Active Le…

On Thursdays, we dive deep into one of the freshest research papers or technology frameworks that is worth your attention.

💥 What’s New in AI: Inside Merlin, the Platform Powering Machine Learning at ShopifySome of the best inspiration for modern machine learning (ML) architectures can be found in the work of technology incumbents like Facebook, Uber, Airbnb, LinkedIn, and several others.

In the past, architectures such as Uber’s Michelangelo or Airbnb’s Bighead have become reference points for modern ML solutions.

Last week, Shopify joined this group by disclosing details about the platform powering its ML workflows.

Code name Merlin, the platform is optimized for three key goals:

So to this end, here are some useful guidelines and examples on how you can test your feature logic and your feature pipelines with pytest.

The phase in the MLOPs lifecycle covered in this article – offline feature testing, feature pipeline testing, and transformation function testing.

In pytest, unit tests may be written either as functions or as methods in classes.

Now, we have looked at pytest for unit tests, let us look at pytest to run integration or end-to-end tests for feature pipelines.

Pytest for Jupyter notebooksYou can also use pytest to test your feature engineering code in your Jupyter notebook provided that you (1) refactor your feature engineering code into functions, and (2)…

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

В следующей статье побеседуем о ключевых трендах в развитии методов причинно-следственного анализа в машинном обучении в 2020-2021 гг.

Что такое причинно-следственный анализCorrelation doesn't imply causationГлавный тезис эконометрики, который в последние 5 лет прочно пришел и в ML: «Корреляция не подразумевает причинно-следственную связь».

В целом, о кейсах бизнес-применения causal inference 2021 г. я рассказывала в одном из постов tg-канала @Reliable_ML еще в начале года.

Causal Inference как ключ к балансу классического ML и эконометрикиCausal Inference в MLВ 2020 году в отчете State of AI впервые в явном виде была обозначена необходимость интеграции классического ML c методами Causal In…

Другими словами, сегодня мы поговорим про то, как автоматически восстановить пунктуацию и капитализацию (сделать нужные буквы заглавными).

Если вы делаете модель для малоресурсного языка, то можно воспользоваться проектом Lingtrain, который я делаю для создания параллельных книг (проект открытый и идеи приветствуются).

Если же вы делаете модель для какого-то популярного языка, то можно воспользоваться готовыми датасетами типа Tatoeba.

Для удобства я оформил скрипты для подготовки и обучения в репозиторий multipunct, поэтому дальше я буду обращаться к нему.

Если понадобится экспортировать модель для инференса, например, в TorchScript, то для этого есть метод export().

Мы использовали их при развитии нашего open-source AutoML фреймворка FEDOT , у которого безусловно есть свои особенности как в архитектуре, так и в парадигме разработки.

Если хотите подробнее почитать про предобработку табличных данных и какой она бывает, то можете начать с Предварительная обработка данных и продолжить c Data Preprocessing: Concepts .

Оговоримся сразу, что рассматривать будем наиболее критические изменения в данных, преобразования в духе “трансформация одномерного target массива в вектор-столбец” подразумевается, что производится при необходимости в любой сколько-нибудь крупной AutoML библиотеке.

Также имеется возможность опциональной предобработки для некоторых моделей - н…

Почему ML-модели часто не справляются с такой задачей и что с этим делать – мы рассмотрим далее.

Конечно, этим проблемы машинного обучения не ограничиваются, существуют также сложности с интерпретацией моделей, проблемы предвзятости и этики, ресурсоемкости обучения и прочие.

Начнем с проблемы утечки данных, и затем рассмотрим более общую проблему ограниченного разнообразия обучающих данных и ситуации, когда модель лишь "делает вид", что обучается.

Проблема shortcut learning присутствует и в тех моделях, которые специально дообучались для лидербордов GLUE (Wang et al., 2018) и SuperGLUE (Wang et al., 2019).

Сеть использует признаки, которые позволяют эффективно предсказывать ответ на обучающ…

TL;DR: Этой весной сообщество Open Data Science и компания Huawei делают новый запуск курса по обработке естесственного языка.

Мы делаем новый запуск курса Natural Language Processing.

Лекции и семинары будут онлайн.

Я буду читать лекции, в области NLP я работаю последние 10 лет, успел поработать в Яндексе и ВКонтакте, защитить кандидатскую диссертацию.

Ссылка будет в канале курса в слаке ODS.

Для каждого листа, признака и порога определяем значения и на листьях и считаем функцию потерь получившегося дерева.

Запишем суммарную функцию потерь после разделения, и минимизируем ее по и :Первое слагаемое зависит только от , второе только от , поэтому их можно минимизировать независимо по и , если известны примеры, попавшие в один и в другой лист.

Суммируем ошибку по всем классам и будем искать минимум по :Из данной формулы можно исключить , и тогда задача нахождения оптимального распределения сводится к минимизации перекрестной энтропии между распределениями и .

Это эквивалентно минимизации расхождения Кульбака-Лейблера между и и достигается при для всех .

Другие особенности решающих д…

Также поговорим о проблемах метода SHAP и его дальнейшем развитии в виде метода Shapley Flow, объединяющего интерпретацию модели и многообразия данных.

Утечка данных является частным случаем сдвига данных, поскольку зависимость ID диагноз была в и , но ее не будет в .

Проблемы и ограничения SHAP valuesНа практике рассчет SHAP values позволяет интерпретировать модель, выявлять скрытые проблемы в модели и данных и даже выполнять кластеризацию, что мы увидим далее в разделе "SHAP на практике".

SHAP values можно посчитать по формуле , но количество слагаемых экспоненциально зависит от количества признаков, и алгоритм рассчета SHAP values в общем случае является NP-полным.

Можно также заметить, …

Список фичей в платной версиии вполне достойный, хоть и не самый впечатляющий (ничего такого, что я не нашел бы в labelstudio).

Документация на троечку и не отвечает на многие вопросы.

И это не так легко, особенно когда речь идет об аудиозаписях.

Это самая большая проблема подобных платформ - не имея собственных разметчиков вы не можете контролировать качество разметки и обучать людей тому, как размечать правильно (да, это не так просто, как кажется!).

Через UI загрузка и выгрузка работает отлично - ты кидаешь .txt или .csv/.tsv файл - и на каждую строчку создается отдельная задача внутри проекта.

В этом посте я расскажу историю появления Open Source библиотеки Albumentations как я ее запомнил.

Я, как и остальные участники команды, включая Александра Буслаева, начали пилить свои аугментационные велосипеды, которые планировалось сделать быстрее и удобнее.

Миша залез в соревнование Severstal: Steel Defect Detection и в кернелах увидел, что участники используют Albumentations.

Но потом, в разговоре всплыло, что для его компании самое большое откровение от соревнования было - что все в топе используют Albumentations.

Она активно использовала Albumentations по работе и, скорее всего, выбралась со мной только потому что ей было интересно пообщаться с её разработчиком :)Что мы по метрикам?

Кластеризация и маскирование таблицы MSAСложность вычислений и объем требуемой памяти в AlphaFold квадратично зависит от количества последовательностей в MSA, и лишь линейно от длины последовательности, поэтому количество последовательностей в MSA желательно уменьшить.

Этот блок принимает на вход массивы pair representation и MSA representation и возвращает два массива таких же размеров.

Таким образом, в MSA representation собирается вся доступная информация о позициях, а в pair representation собирается вся доступная информация о парах позиций.

Дополнительно выполняются следующие операции:Pair bias и outer product mean – операции, позволяющие передавать информацию от pair representation в …

TL;DR: Этой осенью сообщество Open Data Science и компания Huawei делают новый запуск курса.

Мы делаем новый запуск курса Natural Language Processing.

Полный syllabus курса можно посмотреть здесь.

После каждой лекции будет квиз.

Пару слов обо мне, как авторе курса, - в области NLP я работаю последние 9 лет, успел поработать в Яндексе и ВКонтакте, защитить кандидатскую диссертацию.

Делимся нашим исследованием вакансий и зарплат в сфере data science и data engineering.

Посмотрим еще на распределение грейдов для каждой специальностиСпрос на джунов в дата инжиниринге ниже, чем в аналитике и data science.

Зачастую зарплата указывается в тексте вакансии в неструктурированном виде, иногда в одной вакансии идет несколько позиций и несколько вилок.

Анализ зарплат по грейдам и направлениямТеперь мы почти готовы к ответу на главный вопрос: сколько зарабатывают в data science?

В целом, знания deep learning становятся более востребованными: указаны в более 30% вакансий в 2021 году.

Также неправильно было бы говорить о том, что в отличии от классических компьютеров, где есть лишь и в квантовых есть все состояния сразу.

Но это для классического компьютера.

И сегодня мы вынуждены использовать лишь очень приближенные решения и концепции, точности которых часто не хватает.

На сегодня почти все известные технологии создания квантовых компьютеров требуют чего-то из:сверхнизкие температурысверхвысокий вакуумсверхточная юстировка лазеров на оптическом столеИли даже всего сразу.

Да и в этом нет особого смысла, ведь мало кому дома нужно взламывать биткоин, решать логистическую проблему или разрабатывать высокотемпературный сверхпроводник.

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

The experimentation working group is a group of individuals whose goal is to implement the experimentation program i.e.

Data Science Participation in the Working Group Which data science team members should participate in the working group?

This person is responsible for organizing the working group, leading the working group meetings (we’ll discuss this next), and influencing the working group participants.

The Working Group Meeting The working group should meet regularly to achieve it’s goal of implementing the ExPr.

Tactical Tips for Running an Experimentation Working Group Lets discuss a few tactical tips for improving the the working group’s probability of success.

An experimentation program is the mechanism by which a company uses randomized controlled experiments to generate positive business results.

An experimentation program can succeed only when the right people are involved.

Exactly how this ideation, planning, implementation, and analysis is completed is the process component of an experimentation program.

Data Science Data science plays two important roles in an effective experimentation program.

This second role should be played by data science managers or product managers who sit on a data science team.

These are some of the words used to describe the products and services offered by the world’s largest and most successful businesses.

But it’s very likely that this experience wasn’t crafted in some sort of top-down, divine-intervention-like manner either.

Many business leaders today are familiar with examples of companies that have evolved their products and services, and correspondingly optimized their profit-and-loss statements, through experimentation.

Data science teams don’t need to be convinced of the benefits of running experiments.

But often they lack the business knowledge, cross-team relationships, and structured processes for engaging with business teams and helping them optimiz…

My goal is to help data scientists, ML engineers, and AI product managers, build and operate machine learning systems in production.

Learn more about why I started MLinProduction.

Lots of big communities have little interest in r/place, and lots of little communities have outsized presence in r/place.

The Dustforce Discord talked about doing something for r/place, but hadn’t done anything, so I made a pixel art template in hopes it would get the ball rolling.

After scanning existing r/place pixel art, I realized our target image was somewhat big for our community size, so I prepared a smaller version instead.

Our art template and their art template overlapped by 1 pixel, and we both really wanted that pixel.

We even had time to adjust our template and fill in more space with Dustforce pixel art, adding the S+ icon we had last time r/place happened.

SayCan is a robot learning system that we’ve been developing for about the past year.

The language generation is the easy part, while the value function + policy are the hard parts.

Meanwhile, Google Brain announced their PaLM language model, trained with 540B parameters on 780 billion tokens.

Let’s just say it’s not a good look for anyone claiming deep learning models are plateauing.

Similar to language generation, progress here might overstate the state of the field, because it’s improving things we naturally find interesting.

This has spoilers for MIT Mystery Hunt 2022.

Since the winner of MIT Mystery Hunt has to write the next year’s hunt, things will be…interesting for the next year.

Previous Mystery Hunt puzzles have used gimmicked dice before.

The gimmicked dice from the previous Mystery Hunt puzzle were more clearly gimmicked than our dice.

Maybe I’m a little jaded, but winning Mystery Hunt didn’t light any big fires of motivation for me.

In that moment, the brony fandom they never predicted clarified into something real.

Elements of this appear in many fandoms, but brony fandom was unusually prolific.

I think the explosion of MLP fan content is most easily explained by the MLP universe’s untapped potential.

Sturgeon’s Law still applies, but the MLP fandom was prolific enough to produce lots of content in the 10% that wasn’t crap.

Pony fandom was a sizable part of my life for 10 years.

How to Invent Everything is a book by Ryan North, of Dinosaur Comics fame.

How to Invent Everything doesn’t literally cover everything, but you can see it as a book about the highlights of civilization.

Hot air balloons took a really long time to invent, given that baskets, controlled fire, and fabric all existed for centuries.

One question How To Invent Everything toys with is what point in history would let you most influence the trajectory of humanity.

Both these technologies are so incredibly useful to society that it’s hard to see how progress was made before their invention, and they look a long time to invent.

That means I’ve felt greater feelings of responsibility and urgency for puzzlehunt writing compared to blogging.

markdown 6597 2021 - 01 - 29 - mh - 2021. markdown 4249 2021 - 02 - 18 - flash - games .

View CountsThese are the view counts from August 18, 2020 to today, for the posts I’ve written this year.

markdown 912 2021 - 01 - 29 - mh - 2021. markdown 167 2021 - 02 - 18 - flash - games .

Post about Dominion Online:Odds of writing this year: 25%Odds of writing eventually: 50%Post about puzzlehunts:Odds of writing this year: 70%Odds of writing eventually: 90%

Active learning is one paradigm to deal with not enough labeled data, when there are resources for labeling more data samples but under a limited budget.

(2019) proposed a GAN-like setup, named VAAL (Variational Adversarial Active Learning), where a discriminator is trained to distinguish unlabeled data from labeled data.

Bayesian Active Learning for Classification and Preference Learning.” arXiv preprint arXiv:1112.5745 (2020).

“Learning Loss for Active Learning.” CVPR 2019.

“When Deep Learners Change Their Mind: Learning Dynamics for Active Learning.” CAIP 2021.

Semi-supervised learning is one candidate, utilizing a large amount of labeled data conjunction with a small amount of labeled data.

They further set a minimum number of labeled samples in every mini batch by oversampling the labeled samples.

Given a batch of labeled data \(\mathcal{X}\) and unlabeled data \(\mathcal{U}\), we create augmented versions of them via \(\text{MixMatch}(.

A semi-supervised learning framework leverages unlabeled data corpus by (Left) task-agnostic unsupervised pretraining and (Right) task-specific self-training and distillation.

2020:Bigger models are more label-efficient;Bigger/deeper project heads in SimCLR improve representation learning;Distillation using unla…

Additionally training a large model often pairs with a large training corpus and thus a single process may just take forever.

2019)Pipeline ParallelismPipeline parallelism (PP) combines model parallelism with data parallelism to reduce inefficient time “bubbles’’.

The switch transformer paper summarized different data and model parallelism strategies for training large models with a nice illustration:Fig.

2019) optimizes the memory used for training large models based on the observation about two major memory consumption of large model training:The majority is occupied by model states, including optimizer states (e.g.

Cited as:@article{weng2021large, title = "How to Train Really Large Model…

March 3, 2022Implicit Bayesian Inference in Large Language Models?

Exchangeable sequences as Implicit Learning MachinesBefore talking about the paper, let me first refresh those old ideas about exchangeable sequences and implicit learning.

The de Finetti theorem connects such sequence models to Bayesian inference, saying that any such distribution can be decomposed as a mixture over i.i.d.

From Exchangeable sequences to Mixtures of HMMsBut GPT-3 is a language models, and clearly language tokens are not exchangeble.

In context learningThe core idea of this paper is that perhaps in-context learning exploits this implicit Bayesian inference, inherent to statistical models of language, to solve…

One phrase I often use to describe what it's like to read reference letters for Eastern European applicants to PhD and Master's programs in Cambridge.

I decided to write this guide for students so they can share it with their professors when asking for reference letters.

Reference letters are often used as ammunition to justify decisions internally, and to determine who gets prioritised for various scholarship and funding competitions.

Reference letters are often used as ammunition to justify decisions internally, and to determine who gets prioritised for various scholarship and funding competitions.

If you have a candidate you enthusiastically support, don't be afraid to ask for help writi…

Since Content Quality cannot be defined by formula, raters were given carefully chosen guidelines designed to evoke their individual judgment.

They knew this effort would be worthwhile because Content Quality was likely to be a factor in many important decisions.

Their chain of reasoning can be written out as follows:We wish to measure Content Quality.

Since Content Quality cannot be defined by formula, raters were given carefully chosen guidelines designed to evoke their individual judgment.

They knew this effort would be worthwhile because Content Quality was likely to be a factor in many important decisions.

Does Gradient Flow Over Neural Networks Really Represent Gradient Descent?

Researchers often conduct such studies by considering gradient flow (GF), equivalent to GD with infinitesimally small step size.

This would imply $\boldsymbol\theta ( t + \eta ) = \boldsymbol\theta ( t ) - \etaabla f ( \boldsymbol\theta ( t ) )$, which we can write as $\frac{1}{\eta} ( \boldsymbol\theta ( t + \eta ) - \boldsymbol\theta ( t ) ) = -abla f ( \boldsymbol\theta ( t ) )$.

Figure 2: Experiment with NN comparing every iteration of GD with step size $\eta_0 := 0.001$, to every $r$'th iteration of GD with step size $\eta_0 / r$, where $r = 2 , 5 , 10 , 20$.

Left plot shows training loss values; right one shows…

The company trains massive language models (both GPT-like and BERT-like) and offers them as an API (which also supports finetuning).

Semantic search has to be one of the most exciting applications of sentence embedding models.

Finetuning tends to lead to the best results language models can achieve.

This article explains the intuitions around finetuning representation/sentence embedding models.

This is a walkthrough of one of the most common use cases of embedding models -- text classification.

The last few years saw the rise of Large Language Models (LLMs) – machine learning models that rapidly improve how machines process and generate language.

By including a retrieval method in the language model, the model can be much smaller.

Aiding language models with retrieval methods allows us to reduce the amount of information a language model needs to encode in its parameters to perform well at text generation.

Aiding language models with retrieval methods allows us to reduce the amount of information a language model needs to encode in its parameters to perform well at text generation.

Input prompt reaches RETRO Decoder block to start information retrieval Input prompt reaches RETRO D…

And that is exactly what is going on with Tesla FSD.

The excitement and media coverage of this event gathered attention of Silicon Valley and gave rise to a self driving car project at Google in January of 2009.

The appearance of Google "self driving" prototypes in the streets of San Francisco in the early part of 2010's ignited a hysteria in the Bay Area, where the "self driving car" became the next big thing.

It is not clear if a control system for a complex environment can exist in a form that does not continuously learn online.

What we have instead of a safe self driving car, is a farcical comedy of silly mistakes, even in the best of conditions.

We learned for example that everything we can write an equation for can be in principle calculated on a computer.

We take for granted that we can write equations for molecules, yet this isn't really the case.

We can write equations for "approximations" of molecules, ignoring some of the details.

This entire mental exercise that tries to fit the entire Universe into a giant Turing machine is fundamentally flawed!!!

And the straightforward consequence of this flawed philosophy is that brain is just yet another computer, and since we build faster computers every day, it's just a matter of time when we build one as sophisticated as the brain.

It’s all very recognizable and type checks as a modern deep learning paper, except it is from 33 years ago.

misses: 102While my training script repro.py in its current form prints at the end of the 23rd pass:eval: split train.

This change dramatically improved the training error, completely overfitting the training set:eval: split train.

Leaving the baseline training running for 100 passes already shows some improvement from the added data alone:eval: split train.

2055 neural nets are basically the same as 2022 neural nets on the macro level, except bigger.

Reinforcement learning agents typically perform poorly if provided with inputs that were not clearly defined in training.

For instance, most reinforcement learning (RL) agents require their inputs to be in a pre-specified format, or else they will fail.

Each sensory neuron integrates over time information from only their particular sensory input channel.

This action is also fed back into each sensory neuron in the next time-step, closing the communication loop.

Each sensory neuron is an identical neural network that is not confined to only process information from one particular sensory input.

“A drawing of a cat”CLIP + ES + TrianglesRedirecting to es-clip.github.io, where the article resides.

Unsupervised domain adaptive (UDA) person re-identification (ReID) has gained increasing attention for its effectiveness on the target domain without manual annotations.

Most fine-tuning based UDA person ReID methods focus on encoding global features for pseudo labels generation, neglecting the local feature that can provide for the fine-grained information.

To handle this issue, we propose a Learning Feature Fusion (LF2) framework for adaptively learning to fuse global and local features to obtain a more comprehensive fusion feature representation.

The average weighting teacher network is designed to encode global features, while the student network updating at each iteration is responsibl…

An established model for sound energy decay functions (EDFs) is the superposition of multiple exponentials and a noise term.

This work proposes a neural-network-based approach for estimating the model parameters from EDFs.

The network is trained on synthetic EDFs and evaluated on two large datasets of over 20000 EDF measurements conducted in various acoustic environments.

The evaluation shows that the proposed neural network architecture robustly estimates the model parameters from large datasets of measured EDFs, while being lightweight and computationally efficient.

An implementation of the proposed neural network is publicly available.

Despite the significant progress made in both research and practice of recommender systems, to date, there is a lack of a widely-recognized benchmarking system in this field.

However, such conventions not only increase the difficulty in reproducing existing studies, but also lead to inconsistent experimental results among them.

This largely limits the credibility and practical value of research results in this field.

To tackle these issues, we present an initiative project aimed for open benchamrking for recommender systems.

We believe that our benchmark could not only reduce the redundant efforts of researchers to re-implement existing baselines, but also drive more solid and reproducible …

Advances in Natural Language Processing (NLP) have revolutionized the way researchers and practitioners address crucial societal problems.

Large language models are now the standard to develop state-of-the-art solutions for text detection and classification tasks.

However, the development of advanced computational techniques and resources is disproportionately focused on the English language, sidelining a majority of the languages spoken globally.

While existing research has developed better multilingual and monolingual language models to bridge this language disparity between English and non-English languages, we explore the promise of incorporating the information contained in images via …

We present the Berkeley Crossword Solver, a state-of-the-art approach for automatically solving crossword puzzles.

Our system works by generating answer candidates for each crossword clue using neural question answering models and then combines loopy belief propagation with local search to find full puzzle solutions.

Our system also won first place at the top human crossword tournament, which marks the first time that a computer program has surpassed human performance at this event.

To facilitate research on question answering and crossword solving, we analyze our system's remaining errors and release a dataset of over six million question-answer pairs.

PDFAbstractACL 2022 PDFACL 2022 Abstr…

Federated learning is a distributed machine learning approach which enables a shared server model to learn by aggregating the locally-computed parameter updates with the training data from spatially-distributed client silos.

Though successfully possessing advantages in both scale and privacy, federated learning is hurt by domain shift problems, where the learning models are unable to generalize to unseen domains whose data distribution is non-i.i.d.

with respect to the training domains.

In this study, we propose the Federated Invariant Learning Consistency (FedILC) approach, which leverages the gradient covariance and the geometric mean of Hessians to capture both inter-silo and intra-silo …

Pre-trained language models are increasingly important components across multiple information retrieval (IR) paradigms.

Late interaction, introduced with the ColBERT model and recently refined in ColBERTv2, is a popular paradigm that holds state-of-the-art status across many benchmarks.

To dramatically speed up the search latency of late interaction, we introduce the Performance-optimized Late Interaction Driver (PLAID).

Without impacting quality, PLAID swiftly eliminates low-scoring passages using a novel centroid interaction mechanism that treats every passage as a lightweight bag of centroids.

PLAID uses centroid interaction as well as centroid pruning, a mechanism for sparsifying the ba…

In this work, we provide an exact likelihood alternative to the variational training of generative autoencoders.

We show that VAE-style autoencoders can be constructed using invertible layers, which offer a tractable exact likelihood without the need for any regularization terms.

This is achieved while leaving complete freedom in the choice of encoder, decoder and prior architectures, making our approach a drop-in replacement for the training of existing VAEs and VAE-style models.

We refer to the resulting models as Autoencoders within Flows (AEF), since the encoder, decoder and prior are defined as individual layers of an overall invertible architecture.

In a broad sense, the main ambition…

Nowadays, Machine Learning and Deep Learning methods have become the state-of-the-art approach to solve data classification tasks.

This challenge can be tackled by means of semi-supervised learning methods that take advantage of both labelled and unlabelled data.

In this work, we present new semi-supervised learning methods based on techniques from Topological Data Analysis (TDA), a field that is gaining importance for analysing large amounts of data with high variety and dimensionality.

In particular, we have created two semi-supervised learning methods following two different topological approaches.

The results show that the semi-supervised methods developed in this work outperform both t…

However, this quickly becomes impractical as medical data is time-consuming to acquire and expensive to annotate [2].

Thus, the problem of "data-efficient generalization" presents an ongoing difficulty for Medical AI development.

To meet these challenges, we present REMEDIS, a unified representation learning strategy to improve robustness and data-efficiency of medical imaging AI.

We study a diverse range of medical imaging tasks and simulate three realistic application scenarios using retrospective data.

These results suggest that REMEDIS can significantly accelerate the life-cycle of medical imaging AI development thereby presenting an important step forward for medical imaging AI to deli…

Developments in reinforcement learning (RL) have allowed algorithms to achieve impressive performance in highly complex, but largely static problems.

In contrast, biological learning seems to value efficiency of adaptation to a constantly-changing world.

Here we build on a recently-proposed neuronal learning rule that assumes each neuron can optimize its energy balance by predicting its own future activity.

That assumption leads to a neuronal learning rule that uses presynaptic input to modulate prediction error.

We argue that an analogous RL rule would use action probability to modulate reward prediction error.

Differential Privacy (DP) is a formal definition of privacy that provides rigorous guarantees against risks of privacy breaches during data processing.

It makes no assumptions about the knowledge or computational power of adversaries, and provides an interpretable, quantifiable and composable formalism.

DP has been actively researched during the last 15 years, but it is still hard to master for many Machine Learning (ML)) practitioners.

This paper aims to provide an overview of the most important ideas, concepts and uses of DP in ML, with special focus on its intersection with Federated Learning (FL).

PDFAbstract

Depression is a prominent health challenge to the world, and early risk detection (ERD) of depression from online posts can be a promising technique for combating the threat.

Early depression detection faces the challenge of efficiently tackling streaming data, balancing the tradeoff between timeliness, accuracy and explainability.

To tackle these challenges, we propose a psychiatric scale guided risky post screening method that can capture risky posts related to the dimensions defined in clinical depression scales, and providing interpretable diagnostic basis.

For ERD, we propose an online algorithm based on an evolving queue of risky posts that can significantly reduce the number of model…

Large pretrained multilingual models, trained on dozens of languages, have delivered promising results due to cross-lingual learning capabilities on variety of language tasks.

Further adapting these models to specific languages, especially ones unseen during pre-training, is an important goal towards expanding the coverage of language technologies.

In this study, we show how we can use language phylogenetic information to improve cross-lingual transfer leveraging closely related languages in a structured, linguistically-informed manner.

We perform adapter-based training on languages from diverse language families (Germanic, Uralic, Tupian, Uto-Aztecan) and evaluate on both syntactic and sem…

We present SDS-200, a corpus of Swiss German dialectal speech with Standard German text translations, annotated with dialect, age, and gender information of the speakers.

The dataset allows for training speech translation, dialect recognition, and speech synthesis systems, among others.

Each participant was given a text in Standard German and asked to translate it to their Swiss German dialect before recording it.

The data consists of 200 hours of speech by around 4000 different speakers and covers a large part of the Swiss-German dialect landscape.

We release SDS-200 alongside a baseline speech translation model, which achieves a word error rate (WER) of 30.3 and a BLEU score of 53.1 on th…

However, most of these repositories are not labeled or inadequately so, making it harder for users to find relevant projects.

This work proposes GitRanking, a framework for creating a classification ranked into discrete levels based on how general or specific their meaning is.

We collected 121K topics from GitHub and considered $60\%$ of the most frequent ones for the ranking.

This ranking would be an essential asset for developers to build upon, allowing them to complement their annotations with more precise topics.

This paper is the first collective attempt to build a ground-up taxonomy of software domains.

The vertex model is widely used to simulate the mechanical properties of confluent epithelia and other multicellular tissues.

This inherently discrete framework allows a Cauchy stress to be attributed to each cell, and its symmetric component has been widely reported, at least for planar monolayers.

We develop discrete potential theory for localised monolayers having disordered internal structure and use this to derive the analogues of Airy and Mindlin stress functions.

These scalar potentials typically have broad-banded spectra, highlighting the contributions of small-scale defects and boundary-layers to global stress patterns.

An affine approximation attributes couple stresses to pressure…

With the explosive growth of multi-source data, multi-view clustering has attracted great attention in recent years.

Most existing multi-view methods operate in raw feature space and heavily depend on the quality of original feature representation.

Moreover, they are often designed for feature data and ignore the rich topology structure information.

Accordingly, in this paper, we propose a generic framework to cluster both attribute and graph data with heterogeneous features.

Specifically, we first adopt graph filtering technique to eliminate high-frequency noise to achieve a clustering-friendly smooth representation.

The multi-armed bandit (MAB) problem models a decision-maker that optimizes its actions based on current and acquired new knowledge to maximize its reward.

This type of online decision is prominent in many procedures of Brain-Computer Interfaces (BCIs) and MAB has previously been used to investigate, e.g., what mental commands to use to optimize BCI performance.

However, MAB optimization in the context of BCI is still relatively unexplored, even though it has the potential to improve BCI performance during both calibration and real-time implementation.

The review includes a background on MAB problems and standard solution methods, and interpretations related to BCI systems.

Moreover, it inc…

Supervised learning has been widely used for attack detection, which requires large amounts of high-quality data and labels.

Moreover, these supervised models are subject to real-world deployment issues, such as defending against unseen artificial attacks.

We propose a semi-supervised fine-grained attack categorization framework consisting of an encoder and a two-branch structure to integrate information from labeled and unlabeled data to tackle these practical challenges.

The Recurrent Prototype Module (RPM) is proposed to train the encoder effectively in a semi-supervised manner.

Experimental results show that our model outperforms the state-of-the-art semi-supervised attack detection met…

In this paper, we propose a robust estimator for the location function from multi-dimensional functional data.

The proposed estimators are based on the deep neural networks with ReLU activation function.

For any multi-dimensional functional data, we provide the uniform convergence rates for the proposed robust deep neural networks estimators.

Simulation studies illustrate the competitive performance of the robust deep neural network estimators on regular data and their superior performance on data that contain anomalies.

The proposed method is also applied to analyze 2D and 3D images of patients with Alzheimer's disease obtained from the Alzheimer Disease Neuroimaging Initiative database.

Density estimation is a fundamental problem in statistics, and any attempt to do so in high dimensions typically requires strong assumptions or complex deep learning architectures.

An important application for density estimators is synthetic data generation, an area currently dominated by neural networks that often demand enormous training datasets and extensive tuning.

We propose a new method based on unsupervised random forests for estimating smooth densities in arbitrary dimensions without parametric constraints, as well as generating realistic synthetic data.

We prove the consistency of our approach and demonstrate its advantages over existing tree-based density estimators, which genera…

Relation classification models are conventionally evaluated using only a single measure, e.g., micro-F1, macro-F1 or AUC.

In this work, we analyze weighting schemes, such as micro and macro, for imbalanced datasets.

We introduce a framework for weighting schemes, where existing schemes are extremes, and two new intermediate schemes.

We show that reporting results of different weighting schemes better highlights strengths and weaknesses of a model.

PDFAbstract

For such noisy data, most available algorithms perform poorly and often produce negative edge weights in representative trees.

First, we propose a new approach to tree-metric denoising (HyperAid) in hyperbolic spaces which transforms the original data into data that is ``more'' tree-like, when evaluated in terms of Gromov's $\delta$ hyperbolicity.

Second, we perform an ablation study involving two choices for the approximation objective, $\ell_p$ norms and the Dasgupta loss.

As a result, the HyperAid platform outperforms all other existing methods in the literature, including Neighbor Joining (NJ), TreeRep and T-REX, both on synthetic and real-world data.

Synthetic data is represented by ed…

In this paper, we present BEVerse, a unified framework for 3D perception and prediction based on multi-camera systems.

Unlike existing studies focusing on the improvement of single-task approaches, BEVerse features in producing spatio-temporal Birds-Eye-View (BEV) representations from multi-camera videos and jointly reasoning about multiple tasks for vision-centric autonomous driving.

Specifically, BEVerse first performs shared feature extraction and lifting to generate 4D BEV representations from multi-timestamp and multi-view images.

We show that the temporal information improves 3D object detection and semantic map construction, while the multi-task learning can implicitly benefit motion…

We present PYSKL: an open-source toolbox for skeleton-based action recognition based on PyTorch.

The toolbox supports a wide variety of skeleton action recognition algorithms, including approaches based on GCN and CNN.

In contrast to existing open-source skeleton action recognition projects that include only one or two algorithms, PYSKL implements six different algorithms under a unified framework with both the latest and original good practices to ease the comparison of efficacy and efficiency.

We also provide an original GCN-based skeleton action recognition model named ST-GCN++, which achieves competitive recognition performance without any complicated attention schemes, serving as a str…

In this work, we tackle the challenging problem of arbitrary image style transfer using a novel style feature representation learning method.

A suitable style representation, as a key component in image stylization tasks, is essential to achieve satisfactory results.

To address these issues, we propose to learn style representation directly from image features instead of their second-order statistics, by analyzing the similarities and differences between multiple styles and considering the style distribution.

Specifically, we present Contrastive Arbitrary Style Transfer (CAST), which is a new style representation learning and style transfer method via contrastive learning.

Our framework con…

Predicting fetal weight at birth is an important aspect of perinatal care, particularly in the context of antenatal management, which includes the planned timing and the mode of delivery.

In this paper we propose the Residual Transformer Module which extends a 3D ResNet-based network for analysis of 2D+t spatio-temporal ultrasound video scans.

Our end-to-end method, called BabyNet, automatically predicts fetal birth weight based on fetal ultrasound video scans.

We evaluate BabyNet using a dedicated clinical set comprising 225 2D fetal ultrasound videos of pregnancies from 75 patients performed one day prior to delivery.

Experimental results show that BabyNet outperforms several state-of-the…

Given an input sequence (or prefix), modern language models often assign high probabilities to output sequences that are repetitive, incoherent, or irrelevant to the prefix; as such, model-generated text also contains such artifacts.

To address these issues, we present RankGen, an encoder model (1.2B parameters) that scores model generations given a prefix.

RankGen can be flexibly incorporated as a scoring function in beam search and used to decode from any pretrained language model.

Analysis reveals that RankGen outputs are more relevant to the prefix and improve continuity and coherence compared to baselines.

We open source our model checkpoints, code, and human preferences with detailed …

Subtitle segmentation can be evaluated with sequence segmentation metrics against a human reference.

However, standard segmentation metrics cannot be applied when systems generate outputs different than the reference, e.g.

We first conduct a systematic analysis of existing metrics for evaluating subtitle segmentation.

We then introduce $Sigma$, a new Subtitle Segmentation Score derived from an approximate upper-bound of BLEU on segmentation boundaries, which allows us to disentangle the effect of good segmentation from text quality.

Our thorough analyses suggest $Sigma$ is a promising segmentation candidate but its reliability over other segmentation metrics remains to be validated through …

But for a typical visual model to learn a new task, it must be trained on tens of thousands of examples specifically labelled for that task.

Today, in the preprint of our paper, we introduce Flamingo, a single visual language model (VLM) that sets a new state of the art in few-shot learning on a wide range of open-ended multimodal tasks.

This should allow non-expert people to quickly and easily use accurate visual language models on new tasks at hand.

Following this method, we start from Chinchilla, our recently introduced compute-optimal 70B parameter language model, to train our final Flamingo model, an 80B parameter VLM.

Flamingo’s abilities pave the way towards rich interactions with …

At DeepMind…I build bespoke software tools for our developers.

We’re working a hybrid 3:2 model - Monday through Wednesday in the office, Thursday and Friday from anywhere.

Playing music helped tremendously when we were working remotely during the pandemic.

We’re trying to develop tools that can be used by everybody, so we have to design in a very collaborative way.

One of the things I'm advocating for is to take a step back and decide what our principles are for evolving this part of the programming language we’re working on.

Working toward greater generalisability in artificial intelligenceToday, conference season is kicking off with The Tenth International Conference on Learning Representations (ICLR 2022), running virtually from 25-29 April, 2022.

Here’s a brief glimpse into our upcoming oral, spotlight, and poster presentations:Optimising learningA number of key papers focus on the critical ways we’re making the learning process of our AI systems more efficient.

Similarly, exploration mechanisms allow AI agents to go beyond preexisting knowledge and discover the unknown or try something new.

Expanding the research into these mechanisms, we present an experimental framework that enables a fine-grained ana…

Provided with these inputs, Ithaca restores the text, and identifies the time and place in which the text was written.

However, many of the inscriptions historians are interested in analysing with Ithaca are damaged and often missing chunks of text.

Chronological attribution : When dating a text, Ithaca produces a distribution of predicted dates across all decades from 800 BCE to 800 CE.

: When dating a text, Ithaca produces a distribution of predicted dates across all decades from 800 BCE to 800 CE.

Saliency maps: To convey the results to historians, Ithaca uses a technique commonly used in computer vision that identifies which input sequences contribute most to a prediction.

The birth of human writing marked the dawn of History and is crucial to our understanding of past civilisations and the world we live in today.

Many of the surviving inscriptions have been damaged over the centuries or moved from their original location.

In a paper published today in Nature, we jointly introduce Ithaca, the first deep neural network that can restore the missing text of damaged inscriptions, identify their original location, and help establish the date they were created.

Ithaca is named after the Greek island in Homer’s Odyssey and builds upon and extends Pythia, our previous system that focused on textual restoration.

Our evaluations show that Ithaca achieves 62% accuracy i…

Successfully controlling the nuclear fusion plasma in a tokamak with deep reinforcement learningTo solve the global energy crisis, researchers have long sought a source of clean, limitless energy.

However, the plasmas in these machines are inherently unstable, making sustaining the process required for nuclear fusion a complex challenge.

In a paper published today in Nature, we describe how we can successfully control nuclear fusion plasma by building and running controllers on the Variable Configuration Tokamak (TCV) in Lausanne, Switzerland.

(credit: DeepMind & SPC/EPFL)In the video above, we see the plasma at the top of TCV at the instant our system takes control.

We then created a range…

Learning when data is hard to acquireResearch into nuclear fusion is currently limited by researchers’ ability to run experiments.

For example, TCV can only sustain the plasma in a single experiment for up to three seconds, after which it needs 15 minutes to cool down and reset before the next attempt.

For example, plasma simulators are slow and require many hours of computer time to simulate one second of real time.

Each controller uses algorithms to estimate the properties of the plasma in real time and adjust the voltage of the magnets accordingly.

As a demonstration, we first showed that we could manipulate many aspects of the plasma with a single controller.

Collaborating with YouTube to optimise video compression in the open source VP9 codec.

In 2016, we introduced AlphaGo, the first artificial intelligence program to defeat humans at the ancient game of Go.

Now, in pursuit of DeepMind’s mission to solve intelligence, MuZero has taken a first step towards mastering a real-world task by optimising video on YouTube.

In a preprint published today, we detail our collaboration with YouTube to explore the potential for MuZero to improve video compression.

Analysts predicted that streaming video will have accounted for the vast majority of internet traffic in 2021.

Collaborating with YouTube to optimise video compression in the open source VP9 codec.

In a preprint published on arXiv, we detail our collaboration with YouTube to explore the potential for MuZero to improve video compression.

Given a target bitrate, QPs for video frames are decided sequentially to maximize overall video quality.

This works especially well in large, combinatorial action spaces, making it an ideal candidate solution for the problem of rate control in video compression.

Beyond video compression, this first step in applying MuZero beyond research environments serves as an example of how our RL agents can solve real-world problems.

Solving novel problems and setting a new milestone in competitive programming.

AlphaCode placed at about the level of the median competitor, marking the first time an AI code generation system has reached a competitive level of performance in programming competitions.

We pre-train our model on selected public GitHub code and fine-tune it on our relatively small competitive programming dataset.

Solving competitive programming problems is a really hard thing to do, requiring both good coding skills and problem solving creativity in humans.

AlphaCode ranked within the top 54% in real-world programming competitions, an advancement that demonstrates the potential of deep learning models for task…

Solving novel problems and setting a new milestone in competitive programming.

As part of DeepMind’s mission to solve intelligence, we created a system called AlphaCode that writes computer programs at a competitive level.

AlphaCode achieved an estimated rank within the top 54% of participants in programming competitions by solving new problems that require a combination of critical thinking, logic, algorithms, coding, and natural language understanding.

AlphaCode placed at about the level of the median competitor, marking the first time an AI code generation system has reached a competitive level of performance in programming competitions.

To help others build on our results, we’re releasi…

We believe artificial intelligence (AI) is one of the most significant technologies of our age and we want to help people understand its potential and how it’s being created.

In 2019, we released DeepMind: The Podcast to explore these ideas, answer common questions and give an inside look at how AI research happens at a lab like DeepMind.

Today, we’re proud to launch a new season, with stories of the latest breakthroughs, innovations, and challenges.

Listeners can find the new episodes on Apple Podcasts, Google Podcasts, Spotify, or their favourite podcast app by searching for “DeepMind: The Podcast”.

In 2019, we released DeepMind: The Podcast to explore these ideas, answer common questions and give an inside look at how AI research happens at a lab like DeepMind.

Today, we’re proud to launch a new season, with stories of the latest breakthroughs, innovations, and challenges.

Listeners can find the new episodes on Apple Podcasts, Google Podcasts, Spotify, or their favourite podcast app by searching for “DeepMind: The Podcast”.

And we especially want to thank our listeners, who took the time to listen, share, and offer feedback on Season 1.

We hope you enjoy listening to DeepMind: The Podcast, Season 2 as much as we loved making it for you.

Despite decades of effort and several significant advances, accurately modelling the quantum mechanical behaviour of electrons remains an open challenge.

Instead, knowing the probability for any electron to be at each position (i.e., the electron density) is sufficient to exactly compute all interactions.

Kohn received a Nobel Prize in Chemistry after proving this, thus founding Density Functional Theory (DFT).

Over the years, researchers have proposed many approximations to the exact functional with varying levels of accuracy.

Despite their popularity, all of these approximations suffer from systematic errors because they fail to capture certain crucial mathematical properties of the exact…

With billions of searches each day, Google is where people turn when they’re looking to buy something, learn about a product, or complete an important task. With Google’s Business Messages, customers can go straight from Google Search to a one-on-one conversation with a brand and quickly get the information they need. Today, we’re thrilled to announce that we’re expanding our Business Messages partner ecosystem to include Twilio and Genesys, and in the coming months, Avaya – all widely recognized global leaders in customer care and communications. To support businesses of all types and sizes, we’ve worked hard to build a large community of partners that provide everything from hands-on deve…

In “Vector-Quantized Image Modeling with Improved VQGAN”, we propose a two-stage model that reconceives traditional image quantization techniques to yield improved performance on image generation and image understanding tasks.

This approach, which we call Vector-quantized Image Modeling (VIM), can be used for both image generation and unsupervised image representation learning.

We describe multiple improvements to the image quantizer and show that training a stronger image quantizer is a key component for improving both image generation and image understanding.

To test the image understanding capabilities of VIM, we also fine-tune a linear projection layer to perform ImageNet classification…

Conversation on a smart display device, where Assistant understands multiple contextual follow-up queries, allowing the user to have a more natural conversation.

We demonstrate how Assistant is now able to rephrase follow-up queries, adding contextual information before providing an answer.

High level architecture of Google Assistant contextual rephraser.

Candidate ScoringWe extract a number of signals for each rephrasing candidate and use an ML model to select the most promising candidate.

Example conversation on a phone where Assistant understands a sequence of contextual queries.

Multi-objective Optimization via Local SearchCombinatorial optimization remains a difficult task, so we created a domain-specific local search algorithm to optimize network quality.

The local search algorithmic paradigm is widely applied to address computationally-hard optimization problems.

To evaluate the quality of a candidate network, we combine the different objective functions into a single one, as described in the following section.

Three candidate local search moves.

Using combinatorial optimization in concert with geospatial and radio propagation modeling, we built a scalable auto-planner for wireless telecommunication networks.

Artificial intelligence (AI) and particularly machine learning (ML) continue to advance at breakneck pace. We see it throughout projects and commentaries across the broader technology industry. We see it in the amazing things our customers are doing, from creating friendly robots to aid childhood development, to leveraging data for better manufacturing and distribution, to fostering internal innovation through hackathons. And we see it in our own research and product development at Google, from improved machine learning models for our Speech API, to integrations that streamline data management and ML modeling, to making AlphaFold (DeepMind’s breakthrough protein structure prediction system)…

At Google, the state-of-the-art capabilities you see in our products such as Search and YouTube are made possible by Tensor Processing Units (TPUs), our custom machine learning (ML) accelerators. We offer these accelerators to Google Cloud customers as Cloud TPUs. Customer demand for ML capacity, performance, and scale continues to increase at an unprecedented rate. To support the next generation of fundamental advances in artificial intelligence (AI), today we announced Google Cloud's machine learning cluster with Cloud TPU v4 Pods in Preview — one of the fastest, most efficient, and most sustainable ML infrastructure hubs in the world.Powered by Cloud TPU v4 Pods, Google Cloud’s ML cluste…

In “Chain of Thought Prompting Elicits Reasoning in Large Language Models,” we explore a prompting method for improving the reasoning abilities of language models.

With chain of thought prompting, language models of sufficient scale (~100B parameters) can solve complex reasoning problems that are not solvable with standard prompting methods.

We evaluate both the LaMDA collection of language models ranging from 422M to 137B parameters, as well as the PaLM collection of language models ranging from 8B to 540B parameters.

ConclusionsChain of thought prompting is a simple and broadly applicable method for improving the ability of language models to perform various reasoning tasks.

Broadening th…

We’ve been using machine learning in Google Workspace for some years to help make people’s work day more productive and impactful. Today, we’re announcing new features in Google Workspace that tap into our industry-leading AI to help people thrive and get more done in a hybrid work world.We hear from customers—and observe in many of our own teams—that staying on top of the vast amount of information flowing across desks and devices can be a challenge. Information overload isn’t a new phenomenon, but many of our customers say that hybrid work has increased the sheer volume of emails, chats, and meetings for their organizations. Our latest AI innovations are designed to help employees bring f…

There are two key bottlenecks towards building functioning translation models for the long tail of languages.

Both of these challenges need to be addressed for translation models to reach sufficient quality.

The amount of monolingual data per language versus the amount of parallel (translated) data per language.

A small number of languages have large amounts of parallel data, but there is a long tail of languages with only monolingual data.

Our additional innovation is to use the same special tokens for both the monolingual MASS task and the translation task.

Earlier last year, Cloud TPU VMs on Google Cloud were introduced to make it easier to use the TPU hardware by providing direct access to TPU host machines. Today, we are excited to announce the general availability (GA) of TPU VMs.With Cloud TPU VMs you can work interactively on the same hosts where the physical TPU hardware is attached. Our rapidly growing TPU user community has enthusiastically adopted this access mechanism, because it not only makes it possible to have a better debugging experience, but it also enables certain training setups such as Distributed Reinforcement Learning which were not feasible with TPU Node (networks accessed) architecture.What’s new for the GA release?Clo…

Numerous factors underpin the challenges, including access to and storage of data, inconsistent tools, new and evolving data sources and formats, compliance concerns, and security considerations.

To help you identify and solve these challenges, we’ve created a new whitepaper, “The future of data will be unified, flexible, and accessible,” which explores many of the most common reasons our customers tell us they’re choosing Google Cloud to get the most out of their data.

Does this mean moving all your data to the cloud?

The modern tech stack should be a streaming stack that scales with your data, provides real-time analytics, incorporates and understands different types of data, and lets you…

The safe learning framework switches between the safe recovery policy and the learner policy to enable robots to safely acquire novel and agile motor skills.

(2) If the learner policy cannot ensure safety in the near future after switching to the safe recovery policy, we keep using the safe recovery policy.

For the two-leg balance task, the percentage drops from near 82.5% to 67.5%, suggesting that the two-leg balance is substantially harder than the previous two tasks.

The reward learning curve (blue) and the percentage of safe recovery policy activations (red) using our safe RL algorithm in the real world.

Our results suggest that learning legged locomotion skills autonomously and safely …

With over ⅓ of the food in the USA ending up as waste according to the USDA, it is a compelling challenge to address this travesty. What will happen to hunger, food prices, trash reduction, water consumption, and overall sustainability when we stop squandering this abundance?Beginning with the departure from the farm to the back of the store, the freshness clock continues to run. Grocers work very hard to purchase high quality produce items for their customers and the journey to the shelf can take a toll in both quality and remaining shelf life. Suppliers focus on delivering their items through the arduous supply chain journey to the store with speed and gentle handling. The baton is then p…

It translates machine data into a digestible dataset and sends it to the Manufacturing Data Engine for processing, contextualization and storage.

Built on the Manufacturing Data Engine are a growing set of data analytics and AI use cases, enabled by Google Cloud and our partners:Manufacturing analytics & insights : An out-of-the-box integration with Looker templates that delivers a dashboarding and analytics experience.

Manufacturers can continuously improve their models and refine them in collaboration with Google Cloud engineers.

We are excited to partner with Google Cloud as we implement new manufacturing solutions to optimize production operations and consistently increase quality.”“As …

The recently-introduced Open Graph Benchmark (OGB) is an open-source package for benchmarking GNNs on a handful of massive-scale graph datasets across a variety of tasks, facilitating consistent GNN experimental design.

However, the OGB datasets are sourced from many of the same domains as existing datasets, such as citation and molecular networks.

The animation below visualizes GNN performance data from the GraphWorld node classification pipeline (using the SBM as the dataset generator).

First, GraphWorld generates regions of graph datasets that extend well-beyond the regions covered by the standard datasets.

With GraphWorld, researchers can also investigate novel random/generative graph m…

Brad Lightcap has been pivotal in OpenAI's growth, scaling our structure, team, and capital base through his oversight of our Finance, Legal, People, and Operations organizations.

Mira is taking on the role of Chief Technology Officer, reflecting her leadership across these critical areas within OpenAI.

He will lead the operations of OpenAI’s nonprofit parent and key strategic projects including our relationships with mission-aligned partners.

These executives are supported by world-class teams who are the lifeblood of OpenAI, constantly advancing the state of the art in artificial intelligence research and deployment.

It’s a pleasure to work alongside such incredible talent and leadership …

But it’s important to keep track of how well the true objective is being optimized.

We’ll focus on a setting that is particularly clean to analyze, in which we have access to the true objective.

In addition, best-of-$n$ sampling has reliable performance and is straightforward to analyze mathematically, making it well-suited to empirical studies of Goodhart’s law and related phenomena.

Together, these estimators allow us to easily analyze how the true objective varies with the amount of optimization applied to the proxy objective.

In the settings we’ve studied so far, such as summarization, we’ve typically been able to reach a KL of around 10 nats using reinforcement learning before the true…

DALL·E 2 is a new AI system that can create realistic images and art from a description in natural language.

def ___fib(10) def fib(n): if n <= 1: return 1 return fib(n-1) + fib(n-2)fib(10) def fib(n): if n <= 1: return 1 return fib(n-1) + fib(n-2)fib(10) Improve def fib(n): if n <= 1: return 1 return fib(n-1) + fib(n-2)fib(10) Improve the def fib(n): if n <= 1: return 1 return fib(n-1) + fib(n-2)fib(10) Improve the runtime def fib(n): if n <= 1: return 1 return fib(n-1) + fib(n-2)fib(10) Improve the runtime complexity def fib(n): if n <= 1: return 1 return fib(n-1) + fib(n-2)fib(10) Improve the runtime complexity of the def fib(n): if n <= 1: return 1 return fib(n-1) + fib(n-2)fib(10) Improve the runtime complexity of the function def fib(n): if n <= 1: return 1 return fib(n-1) + fib(n-2)fib(10) …

Notably:API-based language model misuse often comes in different forms than we feared most.

We have identified limitations in existing language model evaluations that we are addressing with novel benchmarks and classifiers.

Overview of Our Model Deployment ApproachLarge language models are now capable of performing a very wide range of tasks, often out of the box.

To support the study of language model misuse and mitigation thereof, we are actively exploring opportunities to share statistics on safety incidents this year, in order to concretize discussions about language model misuse.

We are actively discussing many of these issues with other companies deploying language models.

This describes our preliminary priorities for research on the economic impacts of code generation models broadly.

Today, we are excited to complement this research agenda with concrete action to facilitate improved measurement of the economic impacts of our models.

Importance of Studying Economic ImpactsAs an AI research and deployment company, OpenAI recognizes that our decisions around AI system design and deployment can influence economic impacts and the distribution of economic benefits from advances in AI.

Additionally, consider emailing us your questions at [email protected] to learn more about our goals for economic impacts research and how you can be involved.

If you’re interested in …

In the proof below, the model starts by using contraposition leading to the existential statement ( ∃ (x : ℝ), f x ≠ a * x + b ).

Problem 4 Adapted from IMO 1964 Problem 2 Suppose $a$, $b$, $c$ are the sides of a triangle.

Prove that $a^2(b + c − a) + b^2(c + a − b) + c^2(a + b − c) \leq 3abc$.

Prove that $a^2(b + c − a) + b^2(c + a − b) + c^2(a + b − c) \leq 3abc$.

code Formal Informal theorem imo_longlist_1990_p77 (a b c : ℝ) : (a * b + b * c + c * a)^3 ≤ (a^2 + a * b + b^2) * (b^2 + b * c + c^2) * (c^2 + c * a + a^2) := begin let u : euclidean_space ℝ (fin 2) := !

These InstructGPT models, which are trained with humans in the loop, are now deployed as the default language models on our API.

The OpenAI API is powered by GPT-3 language models which can be coaxed to perform natural language tasks using carefully engineered text prompts.

These InstructGPT models, which have been in beta on the API for more than a year, are now the default language models accessible on our API.

Our work is also related to recent research that fine-tunes language models to follow instructions using academic NLP datasets, notably FLAN and T0.

We find that InstructGPT models are significantly preferred on prompts submitted to both the InstructGPT and GPT-3 models on the API.

text-search-{ada, babbage, curie, davinci}-{query, doc}-001 Search, context relevance, information retrieval Code search : Find relevant code with a query in natural language.

code-search-{ada, babbage}-{code, text}-001 Code search and relevanceText Similarity ModelsText similarity models provide embeddings that capture the semantic similarity of pieces of text.

To compare the similarity of two pieces of text, you simply use the dot product on the text embeddings.

2021 50.2% text-search-davinci-{doc, query}-001 52.8% Show more text-search-curie-{doc, query}-001 50.9% text-search-babbage-{doc, query}-001 50.4% text-search-ada-{doc, query}-001 49.0%Code Search ModelsCode search models provide…

It is trained to cite its sources, which makes it easier to give feedback to improve factual accuracy.

Evaluating factual accuracyIn order to provide feedback to improve factual accuracy, humans must be able to evaluate the factual accuracy of claims produced by models.

This allows humans to evaluate factual accuracy by checking whether a claim is supported by a reliable source.

What trade-off should be made between evaluations of factual accuracy and other criteria such as coherence?

Eventually, having models cite their sources will not be enough to evaluate factual accuracy.

Customizing GPT-3 can yield even better results because you can provide many more examples than what’s possible with prompt design.

Customizing GPT-3 improves the reliability of output, offering more consistent results that you can count on for production use-cases.

One customer found that customizing GPT-3 reduced the frequency of unreliable outputs from 17% to 5%.

Whether text generation, summarization, classification, or any other natural language task GPT-3 is capable of performing, customizing GPT-3 will improve performance.

By customizing GPT-3 with their data, Sana’s question and content generation went from grammatically correct but general responses to highly accurate outputs.

As part of our effort to support and develop AI talent, we're excited to announce the OpenAI Residency.

This new program offers a pathway to a full-time role at OpenAI for researchers and engineers who don't currently focus on artificial intelligence.

The Residency shifts the focus away from curriculum-based learning, instead giving Residents an opportunity to work collaboratively alongside OpenAI teams on active projects.

“The Residency aims to address that, by teaching participants the most important practical AI skills in a hands-on way as quickly as possible.

Join us for a discussion with a panel of former OpenAI Scholars and Fellows on December 8 to learn more about the program.

Since the launch of our API, we’ve made deploying applications faster and more streamlined while adding new safety features.

Starting today, developers in supported countries can sign up and start experimenting with our API right away.

Tens of thousands of developers are already taking advantage of powerful AI models through our platform.

As another step in this direction, we are also updating our Content Guidelines to clarify what kind of content our API can be used to generate.

As our safeguards continue to improve, we will expand how the API can be used while further improving the experience for our users.

We’ve trained a system that solves grade school math problems with nearly twice the accuracy of a fine-tuned GPT-3 model.

However, they struggle to perform tasks that require accurate multistep reasoning, like solving grade school math word problems.

GSM8K DatasetGSM8K consists of 8.5K high quality grade school math word problems.

Training Verifiers: Models that Learn from their MistakesOne significant challenge in mathematical reasoning is the high sensitivity to individual mistakes.

Grade school math is an ideal testbed for these capabilities.

Microsoft and SAP are celebrating the one-year anniversary of RISE with SAP on the Microsoft Cloud, which helps organizations of all sizes accelerate their move of SAP solutions to the cloud. As you prepare your agenda for the event, be sure to take in some of the following sessions featuring Microsoft so you can better understand how Microsoft Cloud can help accelerate your SAP transition.

Azure Neural Text to Speech (TTS), a powerful speech synthesis capability of Azure Cognitive Services, enables developers to convert text to lifelike speech using AI. Enterprises and agencies utilize Azure Neural TTS for video game characters, chatbots, content readers, and more. The Azure TTS product team is continuously working on bringing new voice styles and emotions to the US market and beyond.

By encoding the state of a single logical qubit into many physical qubits, quantum error correction (QEC) has the ability to detect and correct most errors that occur on the physical qubits.

Topological qubits promise lower error rates than conventional qubits, and as such can perform scalable quantum computation at lower overhead.

On top of that, in these papers we introduce a new class of quantum error correction codes, called Floquet codes, which are particularly suited to topological qubits.

Unlike most other qubits, our topological qubits employ a measurement-based scheme, where direct measurements between adjacent qubits are the native set of operations.

Along with our recent demonstr…

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

The post MoLeR: Creating a path to more efficient drug design appeared first on Microsoft Research.

In this post, we introduce Path Predictive Elimination (PPE), the first RL algorithm that can solve the problem of exogenous noise with a mathematical guarantee.

Real-world RL and exogenous noiseTo understand how PPE works, it’s important to first discuss how a real-world RL agent (the decision-maker) operates.

That is, taking a fixed action in an endogenous state always leads to the same next endogenous state in most cases.

The circles denote an endogenous state and the arrows denote possible ways to navigate from one endogenous state to another.

The road aheadWhile PPE is the first RL algorithm that offers a mathematical guarantee in the presence of exogenous noise, there is still work to…

Edge computing has come of age, with deployments enabling many applications that process data from IoT sensors and cameras.

In 2017, we identified the symbiotic relationship between edge computing and video analytics in an article, noting that live video analytics is the “killer app” for edge computing.

Using techniques for model specialization and compression, the community has obtained edge models whose compute and memory footprints are substantially lower (by 96x for object detector models).

We achieved the phenomenally low compute footprints for edge models only because we specialized the models to be specific to the camera streams.

For more details, take a look at our paper: Ekya: Cont…

COSTANZA-CHOCK: Design justice is a term—you know, I didn’t create this term; it comes out of a community of practice called the Design Justice Network.

I think for me, um, so design and technology design in particular, I think, for me, practice came first.

And design justice really puts front and center this critical approach.

You know, there’s a whole rich history of writing and thinking and practice, you know, in codesign.

So, there’s codesign, participatory design, human-centered design, design justice.

With the advance of AI and machine learning, companies start to use complex machine learning pipelines in various applications, such as recommendation systems, fraud detection, and more. These complex systems usually require hundreds to thousands of features to support time-sensitive business applications, and the feature pipelines are maintained by different team members across various business groups.

This website is using a security service to protect itself from online attacks.

The action you just performed triggered the security solution.

There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

You can email the site owner to let them know you were blocked.

Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

Any developer can be a space developer with Azure. Microsoft has a long history of empowering the software development community. We have the world’s most comprehensive developer tools and platforms from Github to Visual Studio, and we support a wide range of industries and use cases from healthcare, financial services, critical industries, and now space.

In our research paper, Jigsaw: Large Language Models meet Program Synthesis, which has been accepted at the International Conference on Software Engineering (ICSE 2022), we introduce a new tool that can improve the performance of these large language models.

The promise, and perils, of machine-written softwareLarge language models like OpenAI’s Codex are redefining the landscape of programming.

In our ICSE 2022 paper, Jigsaw: Large Language Models meet Program Synthesis, we evaluate this approach on Python Pandas.

With Jigsaw, the user provides a description of the intended transformation in English, an input dataframe, and the corresponding output dataframe, and then lets Jigsaw synthesize…

Examples of bias in natural language processing are boundless — but MIT scientists have investigated another important, largely underexplored modality: medical images.

Using both private and public datasets, the team found that AI can accurately predict self-reported race of patients from medical images alone.

Using imaging data of chest X-rays, limb X-rays, chest CT scans, and mammograms, the team trained a deep learning model to identify race as white, Black, or Asian — even though the images themselves contained no explicit mention of the patient’s race.

“Even when you filter medical images past where the images are recognizable as medical images at all, deep models maintain a very high …

“Engineering systems are not divorced from the social systems on which they intervene,” Cen says.

There, she developed an interest in AI algorithms, curious about when and why they misbehave.

In one such project, Cen studies options for regulating social media.

How should auditors check that a social media platform complies with this regulation?

Designing an auditing procedure is difficult in large part because there are so many stakeholders when it comes to social media.

Even as rapid improvements in artificial intelligence have led to speculation over significant changes in the health care landscape, the adoption of AI in health care has been minimal.

A 2020 survey by Brookings, for example, found that less than 1 percent of job postings in health care required AI-related skills.

MGB’s chief academic officer and AI Cures co-chair Ravi Thadhani thinks that five times that amount would be necessary in order to do more transformative work.

The presented research spanned different areas of focus from clinical AI and AI for biology to AI-powered systems and others.

“I was really impressed with the breadth of work going on in this space,” Collin Stultz, a profes…

Typical approaches for tackling intersection control problems use mathematical models to solve one simple, ideal intersection.

Reinforcement learning is a trial-and-error method where the control algorithm learns to make a sequence of decisions.

That means we don’t have to wait until we get to 100 percent autonomous vehicles to get benefits from this approach,” she says.

In addition, they intend to study how their control system could impact safety when autonomous vehicles and human drivers share the road.

For instance, even though autonomous vehicles may drive differently than human drivers, slower roadways and roadways with more consistent speeds could improve safety, Wu says.

This technique allows a client to query a database without revealing what it is searching for, Servan-Schreiber explains.

Overcoming security challengesBut while private information retrieval is secure on the client side, it doesn’t provide database privacy on its own.

This efficiently preserves database privacy, so the client won’t learn anything about the feature vectors in the database.

In the future, the researchers plan to adjust the protocol so it can preserve privacy using only one server.

“Nearest neighbor search undergirds many critical machine-learning driven applications, from providing users with content recommendations to classifying medical conditions.

When Chris Rackauckas has a spare moment, he often uses it to answer questions about numerical differential equations that people have posed online.

His research, unsurprisingly, revolves around differential equations and on computational methods — using AI and other techniques — to solve them quickly and efficiently.

Simulations, which are experiments that we carry out on computers, can involve solving thousands upon thousands of differential equations.

Moderna also publicly used Pumas and our clinical analysis methods in its clinical analysis of the Covid-19 vaccine and other drugs.

In between these projects, I might take a moment to answer the odd question about differential equations.

For example, they may highlight words in a movie review that influenced the model’s decision that the review was positive.

So, MIT researchers created a mathematical framework to formally quantify and evaluate the understandability of explanations for machine-learning models.

Instead, researchers resort to using local explanations that focus on individual inputs.

They are then likely to assume that all positive words make positive contributions to a model’s predictions, but that might not always be the case, Zhou says.

“Going from local explanations to global understanding was a big gap in the literature.

Liu and his collaborators developed an artificial intelligence technique that learns to represent data in a way that captures concepts which are shared between visual and audio modalities.

For example, in the video-audio dataset, the model chose 1,000 words to represent the actions in the videos.

Then, when the researchers fed it audio queries, the model tried to find the clip that best matched those spoken words.

For one, their research focused on data from two modalities at a time, but in the real world humans encounter many data modalities simultaneously, Liu says.

“And we know 1,000 words works on this kind of dataset, but we don’t know if it can be generalized to a real-world problem,”…

From “Star Wars” to “Happy Feet,” many beloved films contain scenes that were made possible by motion capture technology, which records movement of objects or people through video.

But what you really care about is the dynamic motion: the joint angles of the leopard — not if they look light or dark,” Du says.

Here, system parameters and actions are entered into a differentiable simulation.

Finally, the researchers deployed their RISP system to infer the motion of a real-world quadrotor, which has complex dynamics, from video.

In nearly all of the experiments, the RISP procedure outperformed similar or the state-of-the-art methods available, imitating or reproducing the desired parameters or…

But until now, the equations have not been able to capture the complexity of breaking waves.

Their results, published today in the journal Nature Communications, will help scientists understand how a breaking wave affects the water around it.

They aimed to improve the model by “training” the model on data of breaking waves from actual experiments.

“We had a simple model that doesn’t capture wave breaking, and then we had the truth, meaning experiments that involve wave breaking,” Eeltink explains.

“If you don’t model wave breaking right, it would have tremendous implications for how structures behave.

This computing energy projection draws from the Semiconductor Research Corporations’s decadal report.

Panel topics ranged from “Custom hardware for efficient computing” to “Hardware for new architectures” to “Algorithms for efficient computing,” among others.

Visual representation of the conversation during the workshop session entitled "Energy Efficient Systems."

Image: Haley McDevitt Previous item Next itemThe goal, said Yildiz, is to improve energy efficiency associated with computing by more than a million-fold.

“5G is the most energy efficient standard ever,” said Scharp.

“We were very pleased that many groups across MIT were eager to contribute their expertise and creativity to that goal.

The team will pilot their technology in a small study at Boston Medical Center in collaboration with neurosurgeon James Holsapple.

Numerous recent studies have highlighted a potential role for immune inflammation in Alzheimer’s disease.

Working in mice, Choi’s lab will test whether such activity is prone to increase in Alzheimer’s and whether it contributes to disease.

By studying biochemical signaling at the junction the lab hopes to discover new targets that could be therapeutically modified.

Instead, the researchers used an optimization methodology developed for machine learning to dramatically reduce the CPU time required while maintaining the accuracy of the solution.

Fusion energyFusion offers the promise of unlimited, carbon-free energy through the same physical process that powers the sun and the stars.

It is no accident that fusion researchers have been pioneers in computational physics for the last 50 years.

Predicting the performance of a self-heated fusion plasma therefore requires a calculation of the power balance between the fusion power input and the losses due to turbulence.

New approach increases confidence in predictionsThis work, described in a recent publicati…

But there’s a major hurdle that holds these systems back: The models often suggest new molecular structures that are difficult or impossible to produce in a laboratory.

A new approach from MIT researchers constrains a machine-learning model so it only suggests molecular structures that can be synthesized.

When compared to other methods, their model proposed molecular structures that scored as high and sometimes better using popular evaluations, but were guaranteed to be synthesizable.

“This process reformulates how we ask these models to generate new molecular structures.

Many of these models think about building new molecular structures atom by atom or bond by bond.

But how much information is any piece of data likely to contain?

One of Shannon’s breakthrough results is the idea of entropy, which lets us quantify the amount of information inherent in any random object, including random variables that model observed data.

It requires precisely calculating the probability of the data, which in turn requires calculating every possible way the data could have arisen under a probabilistic model.

MIT researchers have developed a new method to estimate good approximations to many information quantities such as Shannon entropy by using probabilistic inference.

Estimating entropy and information in a probabilistic model is fundamentally hard because it often re…

Rethinking Human-in-the-Loop for Artificial Augmented IntelligenceFigure 1: In real-world applications, we think there exist a human-machine loop where humans and machines are mutually augmenting each other.

For demonstration, we designed a recognition framework that was a combination of active learning, semi-supervised learning, and human-in-the-loop (Figure 3).

Low-confidence predictions are sent for human annotation, and high-confidence predictions are trusted for downstream tasks or pseudo-labels for model updates.

Thus, the goal of AI development changes from replacing human intelligence to mutually augmenting both human and machine intelligence.

However, this goal of replacing human e…

Designing Societally Beneficial Reinforcement Learning SystemsDeep reinforcement learning (DRL) is transitioning from a research field focused on game playing to a technology with real-world applications.

At the same time as the emergence of powerful RL systems in the real world, the public and researchers are expressing an increased appetite for fair, aligned, and safe machine learning systems.

A Taxonomy of FeedbackReinforcement learning systems are often spotlighted for their ability to act in an environment, rather than passively make predictions.

Other supervised machine learning systems, such as computer vision, consume data and return a prediction that can be used by some decision ma…

Should I Use Offline RL or Imitation Learning?

Are there fundamental limitations to methods that rely on some form of imitation (BC, conditional BC, filtered BC) that offline RL addresses?

While it might be clear that offline RL should enjoy a large advantage over imitation learning when learning from diverse datasets that contain a lot of suboptimal behavior, we will also discuss how even cases that might seem BC-friendly can still allow offline RL to attain significantly better results.

Empirical Results Comparing Offline RL and BCIn our discussion so far, we have already studied settings such as the antmazes, where offline RL methods can significantly outperform imitation-style methods d…

Offline RL Made Easier: No TD Learning, Advantage Reweighting, or TransformersA demonstration of the RvS policy we learn with just supervised learning and a depth-two MLP.

Offline reinforcement learning (RL) is conventionally approached using value-based methods based on temporal difference (TD) learning.

These algorithms learn conditional policies by conditioning on goal states (Lynch et al., 2019; Ghosh et al., 2021), reward-to-go (Kumar et al., 2019; Chen et al., 2021), or language descriptions of the task (Lynch and Sermanet, 2021).

The video above shows the complex behavior we learn using just supervised learning with a depth-two MLP – no TD learning, data reweighting, or Transformer…

EO imagery is commonplace—anyone who has used Google Maps or similar mapping software has interacted with EO satellite imagery.

In general, existing computer vision methods on other, non-aerial RGB imagery transfer very well to satellite imagery.

Synthetic Aperture Radar ImagerySynthetic aperture radar (SAR) imagery is an active form of remote sensing in which a satellite transmits pulses of microwave radar waves down to the surface of the Earth.

Computer Vision on SAR Imagery for UkraineImagery analysts are currently relying on both EO and SAR imagery where available over Ukraine.

Our top performing method, MAERS, for representation learning on RGB, SAR, and co-registered RGB + SAR build…

Unsupervised Skill Discovery with Contrastive Intrinsic ControlUnsupervised Reinforcement Learning (RL), where RL agents pre-train with self-supervised rewards, is an emerging paradigm for developing RL agents that are capable of generalization.

This tension between the need to support large skill spaces and the limitation of current discriminators leads us to propose Contrastive Intrinsic Control (CIC).

Contrastive Intrinsic Control (CIC) introduces a new contrastive density estimator to approximate the conditional entropy (the discriminator).

For a practical algorithm, we use the CIC contrastive skill learning as an auxiliary loss during pre-training.

Our hope is that our approach encoura…

imodels: leveraging the unreasonable effectiveness of rulesimodels: A python package with cutting-edge techniques for concise, transparent, and accurate predictive modeling.

Moreover, interpretable models help with all kinds of things, such as identifying errors, leveraging domain knowledge, and speeding up inference.

Fig 1 shows four possible forms an interpretable model in the imodels package could take.

model = BoostedRulesClassifier () # initialize a model model .

This post is based on the imodels package (github, paper), published in the Journal of Open Source Software, 2021.

While large-scale RL agents can achieve stunning results, even the best RL agents today are narrow.

Unsupervised RL as a path forwardTo date, the most promising path toward generalist AI systems in language and vision has been through unsupervised pre-training.

The unsupervised RL frameworkUnsupervised RL is very similar to supervised RL.

The Unsupervised Reinforcement Learning Benchmark (URLB)While a variety of unsupervised RL algorithms have been proposed over the last few years, it has been impossible to compare them fairly due to differences in evaluation, environments, and optimization.

Paper: URLB: Unsupervised Reinforcement Learning Benchmark Michael Laskin*, Denis Yarats*, Hao Liu, …

Which Mutual Information Representation Learning Objectives are Sufficient for Control?

To simplify the analysis, we analyze representation learning in isolation from the other aspects of RL by assuming the existence of an offline dataset on which to perform representation learning.

An objective may have more than one maximizing representation, so we call a representation learning objective sufficient if all the representations that maximize that objective are sufficient.

To separate representation learning from RL, we first optimize each representation learning objective on a dataset of offline data, (similar to the protocol in Stooke et al.

This post is based on the paper Which Mutual Inf…

Sequence Modeling Solutionsfor Reinforcement Learning ProblemsSequence Modeling Solutions for Reinforcement Learning ProblemsLong-horizon predictions of (top) the Trajectory Transformer compared to those of (bottom) a single-step dynamics model.

While it has been possible to apply reinforcement learning algorithms to large-scale problems, generally there has been much more friction in doing so.

In this post, we explore whether we can alleviate these difficulties by tackling the reinforcement learning problem with the toolbox of sequence modeling.

Though there is value in studying the most streamlined approaches that can tackle reinforcement learning problems, it is possible that the most ef…

Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain DatasetsFig.

For comparison, we include the performance of the policy when trained only on the target domain data, without bridge data (Target Domain Only), a baseline that uses only the bridge data without any target domain data (Direct Transfer), as well as a baseline that trains a single-task policy on data in the target domain only (Single Task).

However, we do include the “direct transfer” baseline, which utilizes a policy trained only on the bridge data.

Figure 10: Example rollouts of policies jointly trained on target domain data and bridge data in each of the three transfer scenarios.

This means that bridge…

Epistemic POMDPs and Implicit Partial ObservabilityActively steering towards regions of low uncertainty or taking information-gathering actions are two of a multitude of avenues an RL agent has to handle its epistemic uncertainty.

What makes the epistemic POMDP particularly exciting is the following equivalence:An RL agent is Bayes-optimal for generalization if and only if it maximizes expected return in the corresponding epistemic POMDP.

LEEP, an algorithm based on the epistemic POMDP objective, generalizes better than PPO in four Procgen tasks.

This is surprisingly not true; limited training data in RL introduces implicit partial observability into an otherwise fully-observable problem.

T…

RECON: Learning to Explore the Real World with a Ground RobotAn example of our method deployed on a Clearpath Jackal ground robot (left) exploring a suburban environment to find a visual target (inset).

To build a robot capable of exploring and navigating like this, we need to learn from diverse prior datasets in the real world.

Our method learns both components using datasets or real-world robot interactions gathered in prior work.

In a new environment, RECON encourages the robot to explore at the frontier of the map – while the robot is not at the frontier, RECON directs it to navigate to a previously seen subgoal at the frontier of the map.

This dataset presents an exciting benchmark f…

Epistemic POMDPs and Implicit Partial ObservabilityActively steering towards regions of low uncertainty or taking information-gathering actions are two of a multitude of avenues an RL agent has to handle its epistemic uncertainty.

What makes the epistemic POMDP particularly exciting is the following equivalence:An RL agent is Bayes-optimal for generalization if and only if it maximizes expected return in the corresponding epistemic POMDP.

LEEP, an algorithm based on the epistemic POMDP objective, generalizes better than PPO in four Procgen tasks.

This is surprisingly not true; limited training data in RL introduces implicit partial observability into an otherwise fully-observable problem.

T…

In this post, we demonstrate how to use Amazon Neptune ML to detect fake news based on the content and social context of the news on social media.

Neptune ML is a new capability of Amazon Neptune that uses graph neural networks (GNNs), a machine learning (ML) technique purpose-built for graphs, to make easy, fast, and accurate predictions using graph data.

The DGL makes it easy to apply deep learning to graph data, and Neptune ML automates the heavy lifting of selecting and training the best ML model for graph data.

Neptune ML uses the DGL to automatically choose and train the best ML model for your workload.

A model training strategy is a configuration set that specifies what type of model…

With CFD, F1 aerodynamicists test different geometry concepts, assess their aerodynamic impact, and iteratively optimize their designs.

Problem statementWhen exploring new aerodynamic concepts, F1 aerodynamicists sometimes employ a process called Design of Experiments (DoE).

Generating the next design candidate to test in CFDSelecting which candidate to test next requires careful consideration.

By doing so, we’re sampling in the region of the design space where the regressor is least confident about its prediction.

In the following figure, we present the permutation importance for a Gaussian Process Regressor (GP) predicting aerodynamic downforce (Cz).

Data engineers can use Amazon SageMaker Data Wrangler to quickly aggregate and prepare data for model building without writing code.

Amazon Simple Storage Service (Amazon S3) acts as our data repository for raw data, engineered data, and model artifacts.

Therefore, we give this responsibility to data engineers so they can transform data without writing code with Data Wrangler.

Import the dataCreate a new Data Wrangler data flow from the Amazon SageMaker Studio UI.

A data engineer can easily prepare data using Data Wrangler without writing any code and pass the prepared dataset to a business analyst.

Solution overviewFor this example, we’ll use an Amazon Lex bot that provides self-service capabilities as part of an Amazon Connect contact flow.

Solution architectureWe’ll use an Amazon Lex bot integrated with Amazon Connect in this solution.

The following AWS Regions support Amazon Lex, Amazon Connect, and Amazon Location Service: US East (N. Virginia), US West (Oregon), Europe (Frankfurt), Asia Pacific (Singapore), Asia Pacific (Sydney) Region, and Asia Pacific (Tokyo).

Enter the ARN (Amazon Resource Name) for the Amazon Connect instance that you’ll use for testing the solution.

However, you can easily integrate Amazon Lex with the Amazon Location Service to look up the correct address, …

However, in some situations you may want to customize the way Amazon Polly pronounces a word.

For such scenarios, Amazon Polly supports phonetic pronunciation, which you can use to achieve a pronunciation that is close to the correct pronunciation in the foreign language.

Now, let’s look at how in such scenarios we can use phonetic pronunciation using SSML tag to customize the speech produced by Amazon Polly.

A good practice is that after you test the custom pronunciation on the Amazon Polly console using the tag, you create a library of customized pronunciations using lexicons.

Upload and apply the lexicon fileUpload your lexicon file to Amazon Polly using the following instructions:On t…

Translators who enhance their workflow with machine translation capabilities such as Amazon Translate often expect fuzzy matching data to be used as part of the automated translation solution.

In this post, you learn how to customize output from Amazon Translate according to translation memory fuzzy match quality scores.

The first segment with 99% match quality isn’t machine translated, whereas the second segment is, because its match quality is below the defined threshold.

Amazon Translate supports customization of machine translation using translation memory thanks to the parallel data feature.

ConclusionIn this post, you learned how to customize your Amazon Translate translation jobs bas…

Amazon Lex now supports a hints capability to enhance the recognition of relevant phrases in a conversation.

Solution overviewLet’s review the overall architecture for the solution (see the following diagram):We use an Amazon Lex bot integrated with an Amazon Connect contact flow to deliver the conversational experience.

This creates an Amazon Lex bot called BankingBot , and one slot type ( accountNumber ).

In the Amazon Lex section, select your Amazon Lex bot and make it available for use in the Amazon Connect contact flow.

The capability is available in all AWS Regions where Amazon Lex operates in the English (Australia), English (UK), and English (US) locales.

In this post, we demonstrate how to run automatic model tuning with JumpStart.

With SageMaker automatic model tuning, ML engineers and data scientists can offload the time-consuming task of optimizing their model and let SageMaker run the experimentation.

In this post, we showed the value of running automatic model tuning on a JumpStart pre-trained model using SageMaker APIs.

For more details on how to optimize a JumpStart model with automatic model tuning, refer to our example notebook.

Dr. Ashish Khetan is a Senior Applied Scientist with Amazon SageMaker JumpStart and Amazon SageMaker built-in algorithms and helps develop machine learning algorithms.

Rekognition Custom Labels abstracts away the complexity involved in building a custom model.

In the search bar, enter Rekognition Custom Labels and choose the Rekognition Custom Labels for Vision notebook.

We encourage you learn more about Rekognition Custom Labels and try it out with your business-specific datasets.

To get started, you can navigate to the Rekognition Custom Labels example notebook in SageMaker JumpStart.

About the AuthorsPashmeen Mistry is the Senior Product Manager for Amazon Rekognition Custom Labels.

You can now use the Amazon Kendra Jira cloud connector to index issues, comments, and attachments in your Jira projects, and search this content using Amazon Kendra intelligent search, powered by machine learning (ML).

This post shows how to use the Amazon Kendra Jira cloud connector to configure a Jira cloud instance as a data source for an Amazon Kendra index, and intelligently search the contents of the projects in it.

In our solution, we configure a Jira cloud instance as a data source to an Amazon Kendra search index using the Amazon Kendra Jira connector.

ConclusionWith the Amazon Kendra Jira connector, your organization can make invaluable knowledge in your Jira projects available to…

The creation of a user group requires a project ID, pipeline parameter set ID, user group name, and user group description.

The creation of a video requires a job ID, video path, video results path, video progress percentage, and video status.

This API requires a user ID, project ID, pipeline ID, pipeline parameter set ID, job parameters, and job status.

POST create_ job Inserts a new job record with user ID, project ID, pipeline ID, pipeline parameter set ID, job results path, job parameters, and job status.

The user ID, project ID, pipeline ID, pipeline parameter set ID, job results path, job parameters, and job status are required for job creation.

We show how you can use Amazon Rekognition and Amazon Textract to optimize and reduce human efforts in processing documents.

Amazon Rekognition identifies moderation labels in your document and classify them using Amazon Rekognition Custom Labels.

Classify documents into different categories such as W-2s, invoices, bank statements, and pay stubs using Rekognition Custom Labels.

Training pipelineBefore we deploy this architecture, we train a custom model to classify documents into different categories using Rekognition Custom Labels.

For more information, see the Amazon Rekognition Custom Labels guide, Amazon Rekognition developer guide and Amazon Textract developer guide.

Once again, the personalized model beat the baselines (see the following table), reinforcing the conclusion that the personalized model is best.

Model MSE(lower is better) Non-personalized baseline 60.885 Personalized baseline 69.902 Non-personalized model 24.823 Personalized model 18.059ConclusionIn this post, we demonstrated how to apply machine learning to achieve personalized in-vehicle thermal comfort.

Yifu Hu is an Applied Scientist in the Amazon Machine Learning Solutions lab, where he helps design creative ML solutions to address customers’ business problems in various industries.

Jennifer Zhu is an Applied Scientist from the Amazon AI Machine Learning Solutions Lab.

Ivan Sosnovik i…

This post walks you through setting up a no-code workflow for creating video subtitles using Amazon Transcribe within your Amazon Web Services account.

Solution overviewThis post walks through a no-code workflow for generating subtitles using Amazon Simple Storage Service (Amazon S3) and Amazon Transcribe.

If you prefer a video walkthrough, refer to the Amazon Transcribe video snacks episode Creating video subtitles without writing any code.

Before you get started, review the Amazon Transcribe and Amazon S3 pricing pages for service pricing.

Create a transcription jobWith the input file ready in Amazon S3, we now create a transcription job in Amazon Transcribe.

You can significantly reduce complexity by using AWS AI capabilities to automate tasks, update prediction models, and integrate human review stages.

The following diagram illustrates the architecture of AWS AI services in a content moderation solution.

You can use the following AWS AI services for moderation, contextual insights, and human-in-the-loop moderation:Amazon Augmented AI (Amazon A2I) makes it easy to build the workflows required for human review, whether moderation runs on AWS or not.

Check out Content Moderation Design Patterns to learn more about how to combine AWS AI services into a multi-modal solution.

For additional information about how to contact our sales and specialist …

Advances in extended reality have already changed the way we work, live and play, and it’s just getting started.

Extended reality, or XR, is an umbrella category that covers a spectrum of newer, immersive technologies, including virtual reality, augmented reality and mixed reality.

What Is Extended Reality?

Virtual reality puts users inside a virtual environment.

Augmented reality is when a rendered image is overlaid onto the real world.

Explainable AI (XAI) is a rapidly advancing field looking to provide insights into the complex decision-making processes of AI algorithms.

Fortunately, model validation can be performed using high-quality AI-generated synthetic data that serves as a highly accurate, anonymized, drop-in replacement for sensitive data.

For example, MOSTLY AI’s synthetic data platform enables organizations to generate synthetic datasets in a fully self-service, automated manner.

A snapshot of real and synthetic data samplesGiven the generated synthetic data, you can then use GPU-accelerated XAI libraries to compute statistics of interest to assess model behavior.

Synthetic data enables a collaborative, broad s…

Building next-generation intelligent vehicles requires an AI infrastructure that pushes the cutting edge.

Electric vehicle maker NIO is using NVIDIA HGX to build a comprehensive data center infrastructure for developing AI-powered, software-defined vehicles.

“By using NVIDIA high-performance compute solutions, NIO can accelerate the path to autonomous driving.”NIO has already launched intelligent vehicles developed on this infrastructure, such as its fully electric, intelligent flagship sedan, the ET7.

NIO’s scalable AI infrastructure is powered by NVIDIA HGX with eight A100 Tensor Core GPUs and NVIDIA ConnectX-6 InfiniBand adapters.

NVIDIA HGX A100 is a high-performance server platform des…

Fortnite on GeForce NOW with touch controls on mobile is now available to all members, streaming through the Safari web browser on iOS and the GeForce NOW Android app.

Streaming ‘Fortnite’ on GeForce NOWGeForce NOW provides millions of gamers the opportunity to stream their favorite PC games — like Fortnite — to nearly any device.

With the addition of touch controls, Fortnite mobile players get GeForce performance they can touch.

When no-build Battle Royale arrived in Fortnite with the launch of Fortnite Zero Build, GeForce NOW members were able to play right away.

Priority members get higher performance, access to premium servers and extended six-hour session lengths, for an upgraded gamin…

Machine learning (ML) employs algorithms and statistical models that enable computer systems to find patterns in massive amounts of data.

Accelerating application development with AI software and infrastructureIf you’re already building data science applications, you’re 90% of the way to using RAPIDS.

Before you dive into data processing, you can look at the details about the GPU using the NVIDIA SMI command.

cuDF is the RAPIDS GPU DataFrame library, which provides everything you need to efficiently transform, load, and aggregate data on the GPU.

In this section, you use SHapley Additive exPlanation (SHAP) values to gain insight into the ML model.

Real-time rendering is helping one studio take virtual production to impossible heights.

In their latest project, the creators at Los Angeles-based company Impossible Objects were tasked with depicting an epic battle between characters from the upcoming video game, Diablo Immortal.

The team at Impossible Objects brought this vision to life using accelerated virtual production workflows to blend visual effects with live action.

The A6000 GPU delivers 48 gigabytes of VRAM, a crucial spec when offline rendering in Unreal Engine.

Impossible Objects used Autodesk Maya to up-res the characters and scale them to perform better in a cinematic setting.

Eyal Ben-Ari just took his first shot on a goal of bringing professional-class analytics to amateur soccer players.

The CEO of startup Track160, in Tel Aviv, has seen his company’s AI-powered sports analytics software tested and used in the big leagues.

Now he’s turning his attention to underserved amateurs in the clubs and community teams he says make up “the bigger opportunity” among the world’s 250 million soccer players.

“Almost everyone in professional sports uses data analytics today.

Track160 frequently signed on as a member of NVIDIA Metropolis, a program for companies in intelligent video analytics.

Editor’s note: This post is part of our weekly In the NVIDIA Studio series, which celebrates featured artists, offers creative tips and tricks, and demonstrates how NVIDIA Studio technology accelerates creative workflows.

Concept artist Pablo Muñoz Gómez dives In the NVIDIA Studio this week, showcasing artwork that depicts a fantastical myth.

Gómez then turns to Adobe Substance 3D Painter to apply various colors and materials directly to his 3D models.

Follow NVIDIA Studio on Facebook, Twitter and Instagram.

Access tutorials on the Studio YouTube channel and get updates directly in your inbox by subscribing to the NVIDIA Studio newsletter.

Phishing todayMost phishing cybersecurity defenses combine rules-based email filters and human training to detect a fraudulent email.

Phishing detection with NVIDIA MorpheusNVIDIA Morpheus, now generally available for download from NVIDIA NGC and the NVIDIA/Morpheus GitHub repo, is an open AI framework for implementing cybersecurity-specific inference pipelines.

With NVIDIA Morpheus, our cybersecurity team applied natural language processing (NLP), a popular AI technique, to create a phishing detection application that correctly classified phishing emails at a 99%+ accuracy rate.

Using the Morpheus pipeline for phishing detection, you can use your own models to improve the accuracy further.…

Date: Wednesday, May 25, 2022Time: 10:00 - 11:00 PTDuration: 1 hourOrganizations across industries are leveraging machine learning (ML) to gather insights, improve the customer experience, and drive operational efficiencies.

However, building an ML application requires large amounts of training data, software and hardware infrastructure to train the ML models, and tools to run real-time inference that scales with demand.

Join this webinar to learn how performance-optimized AI software from the NVIDIA NGC™ catalog helps companies build ML-powered applications faster.

Vertex AI Workbench minimizes context switching through native integration of BigQuery and Cloud Storage, supports distributed…

When it comes to safety, efficiency and sustainability, autonomous vehicles are delivering a clean sweep.

Autonomous vehicle company and NVIDIA Inception member WeRide this month began a public road pilot of its Robo Street Sweepers.

Sweep SmartsWhile street sweepers typically operate at lower speeds and in more constrained environments than robotaxis, trucks or other autonomous vehicles, they still require robust AI compute to safely operate.

A Model LineupThe WeRide Robo Street Sweepers are the latest in the company’s stable of autonomous vehicles and its second purpose-built and mass-produced self-driving vehicle model.

The company is currently building its next-generation self-driving s…

An AI-powered initiative is spotlighting lesser-known endangered creatures on Times Square billboards this month, nightly in the few minutes before midnight across nearly 100 screens.

It’s the first deep learning art display in the Times Square Arts program’s decade-long history.

So the deep learning model, a generative adversarial network, does the best it can, guessing the features of a given endangered species based on related species.

The project began as an exhibition on Instagram, with the goal of adding representation of critically endangered species to social media conversations.

Times Square images courtesy of Times Square Arts, photographed by Michael Hull.

Get ready for some horrifyingly good fun with Evil Dead: The Game streaming on GeForce NOW tomorrow at release.

It’s the 1,300th game to join GeForce NOW, joining on Friday the 13th.

And it’s part of eight total games joining the GeForce NOW library this week.

Hail to the King, BabyStep into the shoes of Ash Williams and friends from the iconic Evil Dead franchise in Evil Dead: The Game (Epic Games Store), streaming on GeForce NOW at release tomorrow.

Work together in a game loaded with over-the-top co-op and PvP action across nearly all your devices.

$ canary version Container Canary Version: VERSION ...Validating containers with a Kubeflow exampleWith Container Canary installed, you can begin validating containers.

Container Canary can help you ensure that your container manifests will run in your own deployments and in third-party platforms.

name: ci on: push: pull_request: jobs: canary: runs-on: ubuntu-latest steps: - name: Checkout uses: actions/[email protected] - name: Install Container Canary run: | curl -L https://github.com/NVIDIA/container-canary/releases/download/v0.2.0/canary_linux_amd64 > /usr/local/bin/canary chmod +x /usr/local/bin/canary - name: Build Container run: docker build -t foo/canary-ci-example:latest .

- name: Valida…

Editor’s note: This post is part of our weekly In the NVIDIA Studio series, which celebrates featured artists, offers creative tips and tricks, and demonstrates how NVIDIA Studio technology accelerates creative workflows.

This week In the NVIDIA Studio, discover how AI-assisted painting is bringing a new level of inspiration to the next generation of artists.

San Francisco-based creator Karen X. Cheng is on the forefront of using AI to design amazing visuals.

With AI, Anything Is PossibleEmpowering scribble-turn-van Gogh painting abilities is just one of the ways that NVIDIA Studio is transforming creative technology through AI.

Follow NVIDIA Studio on Instagram, Twitter and Facebook; acces…

What the research is:Recent research has demonstrated that binary optimization is important for achieving peak performance for various applications.

VESPA expands on ESP in several ways to make it useful in the context of binary optimizers.

VESPA increases the scope where binary optimizers can be used, thus enhancing the range of applications that can leverage these tools to improve their performance.

Once the static profile data produced by VESPA is injected into a binary optimizer, this tool can proceed with its optimization steps as usual, completely oblivious to how the profile data was computed.

VESPA, therefore, can very easily be integrated into existing binary optimizers, which we d…

By training machine learning (ML) models on top of the road graph prediction using historical data in combination with real-time signals, we can refine ETAs that better predict real-world outcomes.

To meet these challenges, Uber AI partnered with Uber’s Maps team on a project called DeepETA to develop a low-latency deep neural network architecture for global ETA prediction.

We take a similar approach to ETA prediction at Uber.

Conclusions and Future WorkWe have launched this DeepETA model into production for global 4-wheel ETA prediction.

The DeepETA model launch makes it both possible and efficient to train and serve large-scale Deep Learning models that predict ETAs better than XGBoost ap…

Industry-wide, payment fraud losses are measured in terms of the fraction of gross amounts processed.

RADAR is an AI fraud detection and mitigation system with humans in the loop.

RADAR fraud protection rules are generally short-lived and targeted reactions to the unexpected attacks.

We will define some terminology to discuss time series data below:OT – “Order time” when the specific order has been fulfilled.

ConclusionIn this blog, we presented the RADAR system and how it brings together many components of Uber’s technical ecosystem to solve a complex business problem.

We recently built a new system, Capacity Recommendation Engine (CRE), with a new algorithm that relies on throughput and utilization based scaling with machine learning modeling.

Take throughput estimation for weekly scaling as an example: the target throughput RPS Target should be the estimation of the next week’s peak traffic.

Define Target UtilizationTarget utilization (Utilization Target ) is the one of the signals required to deduce the capacity number in CRE.

Linear Regression: Normalized Throughput and UtilizationFor utilization-bound service, utilization, throughput, capacity, service, and hardware performance are common factors, which influence each other.

ConclusionIn this article…

Some of the key technologies that we will be using are:TensorFlow,Neptune.ai,GitHub Actions,Docker,Kubernetes,and Google Cloud Build.

MLOps pipeline for machine translation: model developmentFor the sake of this article, we will be using the Notebook provided on the Tensorflow website.

./ RUN ls -la $APP_HOME/ RUN pip install -r requirements.txt CMD [ "python3" , "app.py" ]Google Cloud developmentAfter configuring the Dockerfile we will then use Google Cloud Development or GCD to automate the CI/CD pipeline.

Setting up a plan with Google Cloud Build | Source: AuthorSetting up a plan with Google Cloud Build | Source: AuthorClick on configure.

Monitoring in Google Cloud | Source: AuthorYou wi…

You will learn key aspects of model deployment including the tools, best practices, and things to consider when deploying computer vision models.

Computer vision API providersThere are many service providers out there who provide computer vision APIs that can be directly integrated with your application.

MLflow models | SourceAn example of productionizing computer vision models using MLflow & Redis AI can be seen on this link.

It’s important to note that close to 90% of ML models never make it to production; the same goes for Computer vision models.

Cross-language and tools support issuesMany of the older computer vision models are built with open CV (primarily built with C++).

Neptune.ai has been named to the 2022 CB Insights AI 100 List of Most Promising AI Startups.

The CB Insights team picked 100 private market vendors from a pool of over 7,000 companies.

There are a ton of great startups in the top 100 – congrats to all of them!

Being noticed by CB Insights is motivating but also important for Neptune.ai as a company.

We’re going to work even harder to continue making experiment tracking and model registry “just work” for ML teams around the world.

Error analysis: preliminary knowledgeBefore exploring why each error analysis is distinct and must be done, we first have to properly understand the core concepts of ML models to realize its inherent constraints.

This leads to the first and most important error analysis that everyone must do before production: determine whether the size of the training and validation dataset is sufficient.

When the validation set is too small: a different problem may occur when the validation set is too small.

Error analysis 2: Balance of a dataset and accuracy per classThe second error analysis digs into the content of the dataset to find the balance of labels.

Error analysis 4: Investigate the level of ov…

Do you struggle with monitoring and optimizing the training of Deep Neural Networks on multiple GPUs?

In this article, we will discuss multi GPU training with Pytorch Lightning and find out the best practices that should be adopted to optimize the training process.

Model parallelismModel parallelism partitions a model among multiple GPUs, where each GPU is responsible for the weight updates of the assigned layers of a model.

In this section, we will use Neptune for monitoring the GPU and GPU memory while training over multiple GPUs.

Monitor training on multiple GPUs | SourceLet’s see what kind of meaningful insights we can infer from GPU utilization graphs.

To understand how to track experiments in Kubeflow Pipelines, we need to understand what Kubeflow is.

Example pipelines can be found in the `samples` directory within the Kubeflow Pipelines repository.

Graph illustrating finished pipeline run in Kubeflow Pipelines | Source: image by the authorExperiment tracking in Kubeflow PipelinesSurprisingly, Kubeflow supports experiment tracking natively.

MLFlow Tracking UI | Source: DatabricksThe UI of MLFlow Tracking is rather raw and simple, similar to what we have seen in Kubeflow Pipelines.

Comparison of experiment tracking tools for Kubeflow PipelinesTo sum up, let’s now see a comparison table for the tools I already described.

Read also 👉 Best Practices for Improving Your Machine Learning and Deep Learning ModelsWhat is pipeline debt?

Great Expectations accommodate this via the “mostly” keyword which allows for describing how often should the expectation be matched.

Key features of Great ExpectationsGreat Expectations offer three very useful features:1 Automated data profiling, to create the expectations suite from the data at hand.

This brings us to the second great feature of Great Expectations.

While by no means do they provide an ultimate answer, Great Expectations can at least help us in detecting dangerous biases.

Machine Learning pipelines are complex and there are several ways they can fail or be misused.

Stakeholders involved in ML projects need to understand how Machine Learning pipelines can fail, possible pitfalls, and how to avoid such pitfalls.

There are several pitfalls you should be aware of when building machine learning pipelines.

Common pitfalls in the ML pipeline stepsRunning ML pipelines from ingesting data to modeling, operationalizing the models can be very tedious.

In this section, you will learn the common pitfalls you may encounter in each step of building ML pipelines.

Most importantly, the ML community realized that building a POC model in a notebook is not the end goal.

Speaking of progress, I am really happy to share that we’ve just raised an $8M Series A to continue building Neptune.ai.

I believe that by focusing on providing a great developer experience for experiment tracking and model registry, we can become one of the pillars on which teams build their MLOps tool stacks.

And to make this happen, we will invest a big chunk of that $8M in developer experience.

But first and foremost, we’ll continue making experiment tracking and model registry “just work” for ML teams around the world.

Since you are here, you probably know that time series data is a bit different than static ML data.

This article is sort of a database of time series tools and packages.

Time series forecasting packagesProbably the most important part of the time series project is forecasting.

More about Prophet library is presented below:Time series forecasting with PycaretPyCaret is an open-source machine learning library in Python that automates machine learning workflows.

It can be used to find the best time series forecasting model both for univariate and multivariate time series.

Using Kedro and Optuna together to run hyperparameter sweepsKedro and Optuna complement each other in automating ML workflows.

Kedro handles the high-level pipelines, feature transformations, and pre-processing, while Optuna focuses on the core model optimization.

To create the data processing template, follow the command below:(kedro-environment) [email protected] tutorial % kedro pipeline create data_processingNow, kedro pipelines are very similar to airflow directed acyclic graphs (DAGs).

from typing import Dict from kedro.pipeline import Pipeline, pipeline from tutorial.pipelines import data_processing as dp from tutorial.pipelines import data_science as ds def register…

As the name suggests, a data science pipeline involves the seamless linkage of various components to facilitate the smooth movement of data as intended.

Importance of data science pipelinesIt is vital to appreciate the importance and benefits of data science pipelines in the first place because they require effort to build.

Here are the business benefits that data science pipelines can bring:Speeds up data-driven decision-making to respond to evolving business needs and customer preferences.

Data science pipeline use casesData science pipelines are industry-agnostic, so we can expect them to deliver huge benefits across different fields potentially.

Step 6: Build a data science pipelineThe …

We will also compare the main techniques of building machine learning models for recommender systems and take a look at metrics and business evaluation techniques.

The essential part of content-based systems is to pick similarity metrics.

In terms of content-based filtering, we should choose from similarity metrics, while for collaborative methods – predictive and classification metrics depend on whether we predict score or binary output.

Jaccard similarity | SourceThe difference from other similarity metrics in this article is that Jaccard similarity takes sets or binary vectors as an input.

For content-based filtering, similarity metrics should be considered to evaluate model performance …

These issues are referred to as the Vanishing and Exploding Gradients, respectively.

This is known as the problem of vanishing gradients, and it’s one example of unstable behaviors of neural nets.

Why vanishing or exploding gradients problem happens?

Solutions for when gradients explodeFor the exploding gradients issue, let’s take a look at this regression model.

In this article, we have discussed two major issues associated with neural network training – the Vanishing and Exploding gradients problems.

Model deployment strategies | SourceTo begin with, let’s have a quick overview of what model lifecycle and model deployment refers to.

Model deployment strategiesStrategies allow us to evaluate the ML model performances, capabilities and discover issues concerning the model.

Shadow deployment strategyIn shadow deployment or shadow mode, the new model is deployed with new features alongside the live model.

Canary deployment strategy | SourceMethodologySimilar to other deployment strategies in canary deployment, the new model is tested alongside the current live model but here the new model is tested on a few users to check its reliability, errors, performance et cetera.

Model release (deploy…

#flamingo #mlnews #tech Your updates directly from the state of the art in Machine Learning! OUTLINE:

0:00 - Intro

0:30 - DeepMind's Flamingo: Unified Vision-Language Model

8:25 - LiT: Locked Image Tuning

10:20 - Jurassic X & MRKL Systems

15:05 - Helpful Things

22:40 - This AI does not exist References:

DeepMind's Flamingo: Unified Vision-Language Model

https://www.deepmind.com/blog/tackling-multiple-tasks-with-a-single-visual-language-model

https://storage.googleapis.com/deepmind-media/DeepMind.com/Blog/tackling-multiple-tasks-with-a-single-visual-language-model/flamingo.pdf LiT: Locked Image Tuning

https://ai.googleblog.com/2022/04/locked-image-tuning-adding-language.html

https://google-r…

#mlnews #dalle #gpt3 An inside look of what's happening in the ML world! Sponsor: Weights & Biases

https://wandb.me/yannic OUTLINE:

0:00 - Intro

0:20 - Sponsor: Weights & Biases

1:40 - Meta AI releases OPT-175B

4:55 - CoCa: New CLIP-Competitor

8:15 - DALL-E Mega is training

10:05 - TorToiSe TTS is amazing!

11:50 - Investigating Vision Transformers

12:50 - Hugging Face Deep RL class launched

13:40 - Helpful Things

17:00 - John Deere's driverless tractors References:

Meta AI releases OPT-175B

https://ai.facebook.com/blog/democratizing-access-to-large-scale-language-models-with-opt-175b/

https://arxiv.org/abs/2205.01068

https://arxiv.org/pdf/2205.01068.pdf

https://github.com/facebookresearch/m…

#nft #gan #ai Today we build our own AI that can create as many bored apes as we want! Fungibility for everyone! Try the model here: https://huggingface.co/spaces/ykilcher/apes

or here: https://ykilcher.com/apes

Files & Models here: https://huggingface.co/ykilcher/apes/tree/main

Code here: https://github.com/yk/apes-public This video is sponsored by BrightData, use this link for 25$ free credits (and they match your first deposit up to 250$):

https://brightdata.grsm.io/yannickilcher OUTLINE:

0:00 - Introduction

2:05 - Generative Adversarial Networks

3:40 - Scraping Opensea with BrightData

7:55 - Training the GAN

11:35 - Here are the results!

15:20 - Diving deeper into BrightData References:…

#saycan #robots #ai This is an interview with the authors Brian Ichter, Karol Hausman, and Fei Xia.

Original Paper Review Video: https://youtu.be/Ru23eWAQ6_E

Large Language Models are excellent at generating plausible plans in response to real-world problems, but without interacting with the environment, they have no abilities to estimate which of these plans are feasible or appropriate. SayCan combines the semantic capabilities of language models with a bank of low-level skills, which are available to the agent as individual policies to execute. SayCan automatically finds the best policy to execute by considering a trade-off between the policy's ability to progress towards the goal, given …

#saycan #robots #ai Large Language Models are excellent at generating plausible plans in response to real-world problems, but without interacting with the environment, they have no abilities to estimate which of these plans are feasible or appropriate. SayCan combines the semantic capabilities of language models with a bank of low-level skills, which are available to the agent as individual policies to execute. SayCan automatically finds the best policy to execute by considering a trade-off between the policy's ability to progress towards the goal, given by the language model, and the policy's probability of executing successfully, given by the respective value function. The result is a sys…

#ai #accel #evolution This is an interview with the authors Jack Parker-Holder and Minqi Jiang.

Automatic curriculum generation is one of the most promising avenues for Reinforcement Learning today. Multiple approaches have been proposed, each with their own set of advantages and drawbacks. This paper presents ACCEL, which takes the next step into the direction of constructing curricula for multi-capable agents. ACCEL combines the adversarial adaptiveness of regret-based sampling methods with the capabilities of level-editing, usually found in Evolutionary Methods. OUTLINE:

0:00 - Intro

1:00 - Start of interview

4:45 - How did you get into this field?

8:10 - What is minimax regret?

11:45 - …

#ai #accel #evolution Automatic curriculum generation is one of the most promising avenues for Reinforcement Learning today. Multiple approaches have been proposed, each with their own set of advantages and drawbacks. This paper presents ACCEL, which takes the next step into the direction of constructing curricula for multi-capable agents. ACCEL combines the adversarial adaptiveness of regret-based sampling methods with the capabilities of level-editing, usually found in Evolutionary Methods. OUTLINE:

0:00 - Intro & Demonstration

3:50 - Paper overview

5:20 - The ACCEL algorithm

15:25 - Looking at the pseudocode

23:10 - Approximating regret

33:45 - Experimental results

40:00 - Discussion & C…

#laion #clip #dalle LAION-5B is an open, free dataset consisting of over 5 billion image-text-pairs. Today's video is an interview with three of its creators. We dive into the mechanics and challenges of operating at such large scale, how to keep cost low, what new possibilities are enabled with open datasets like this, and how to best handle safety and legal concerns. OUTLINE:

0:00 - Intro

1:30 - Start of Interview

2:30 - What is LAION?

11:10 - What are the effects of CLIP filtering?

16:40 - How big is this dataset?

19:05 - Does the text always come from the alt-property?

22:45 - What does it take to work at scale?

25:50 -When will we replicate DALL-E?

31:30 - The surprisingly efficient pi…

#nlp #sparsity #transformers This video is an interview with Barret Zoph and William Fedus of Google Brain about Sparse Expert Models.

Sparse Expert models have been hugely successful at distributing parts of models, mostly Transformers, across large array of machines and use a routing function to effectively route signals between them. This means that even though these models have a huge number of parameters, the computational load for a given signal does not increase because the model is only sparsely activated. Sparse expert models, such as Switch Transformers and GLAM can scale up to trillions of parameters and bring a number of desirable properties. We discuss everything from the funda…

#neuralsearch #interview #google This is an interview with the authors Yi Tay and Don Metzler.

Paper Review Video: https://youtu.be/qlB0TPBQ7YY Search engines work by building an index and then looking up things in it. Usually, that index is a separate data structure. In keyword search, we build and store reverse indices. In neural search, we build nearest-neighbor indices. This paper does something different: It directly trains a Transformer to return the ID of the most relevant document. No similarity search over embeddings or anything like this is performed, and no external data structure is needed, as the entire index is essentially captured by the model's weights. The paper experiments…

#dsi #search #google Search engines work by building an index and then looking up things in it. Usually, that index is a separate data structure. In keyword search, we build and store reverse indices. In neural search, we build nearest-neighbor indices. This paper does something different: It directly trains a Transformer to return the ID of the most relevant document. No similarity search over embeddings or anything like this is performed, and no external data structure is needed, as the entire index is essentially captured by the model's weights. The paper experiments with various ways of representing documents and training the system, which works surprisingly well! Sponsor: Diffgram

http…

#mlnews #palm #dalle2 Google releases PaLM and OpenAI releases DALL-E 2 (and more news). Sponsor: Weights & BIases

Start here: https://wandb.me/yannic Thumbnail credit: DALL-E 2 via Sam Altman OUTLINE

0:00 - Street interview w/ random stranger

2:25 - Intro

2:50 - PaLM - Google's 540B Pathways Language Model

7:50 - Sponsor: Weights & Biases

9:10 - OpenAI releases DALL-E 2

12:05 - Open Source Datasets and Models

13:20 - Salesforce releases CodeGen My Live Reaction to DALL-E 2: https://youtu.be/gGPv_SYVDC8

My Video on GLIDE: https://youtu.be/gwI6g1pBD84

My Video on the Pathways System: https://youtu.be/vGFaiLeoLWw References:

PaLM - Google's 540B Pathways Language Model

https://ai.googleblog.c…

Chatting & Coding

#aiart #deeplearning #clip Since the release of CLIP, the world of AI art has seen an unprecedented level of acceleration in what's possible to do. Whereas image generation had previously been mostly in the domain of scientists, now a community of professional artists, researchers, and amateurs are sending around colab notebooks and sharing their creations via social media. How did this happen? What is going on? And where do we go from here? Jack Morris and I attempt to answer some of these questions, following his blog post "The Weird and Wonderful World of AI Art" (linked below). OUTLINE:

0:00 - Intro

2:30 - How does one get into AI art?

5:00 - Deep Dream & Style Transfer: the early days …

#reinforcementlearning #ai #explained This is an interview with Jesse Mu, first author of the paper.

Original Paper Review: https://youtu.be/NeGJAUSQEJI Exploration is one of the oldest challenges for Reinforcement Learning algorithms, with no clear solution to date. Especially in environments with sparse rewards, agents face significant challenges in deciding which parts of the environment to explore further. Providing intrinsic motivation in form of a pseudo-reward is sometimes used to overcome this challenge, but often relies on hand-crafted heuristics, and can lead to deceptive dead-ends. This paper proposes to use language descriptions of encountered states as a method of assessing nov…

Please check out Eric Jang's article "Ranking YC Companies with a Neural Net": https://evjang.com/2022/04/02/yc-rank.html Please subscribe to SeMI Technologies on YouTube! https://www.youtube.com/c/SeMI-and-Weaviate Timecodes

0:00 Introduction

0:58 Weaviate Demo

3:40 Article Overview

10:45 NLP for Venture Capital and Data-Centric AI

Please leave a star! https://github.com/mosaicml/composer Thank you so much for watching! This video presents some details of MosaicML's Composer launch and how to use it in Python. I am really excited about this company and their mission to deliver faster and cheaper Deep Learning training! I hope you find this video useful, happy to answer any questions you might have about this or these ideas in Efficient Deep Learning generally! The full Weaviate podcast with Jonathan Frankle will be uploaded very soon on SeMI Technologies YouTube, please subscribe!

https://www.youtube.com/c/SeMI-and-Weaviate Chapters

0:00 Introduction

1:45 Documentation Intro

4:20 Composer Notebooks

5:35 Functional API…

I hope you found this useful, please let me know if you have any questions or ideas! Docarray Documentation: https://docarray.jina.ai/ Full-Length Podcast: https://www.youtube.com/watch?v=HIGAQAE_xaI Code Tutorial (Weaviate + Jina AI for Image Search): https://www.youtube.com/watch?v=rBKvoIGihnY Please check out Jina AI on YouTube: https://www.youtube.com/c/JinaAI Please check out SeMI Technologies on YouTube: https://www.youtube.com/c/SeMI-and-Weaviate/videos

This video explains one of the biggest lessons for me in interviewing Han Xiao from Jina AI. I hope this was a good explanation of the preprocessing / granularity of embeddings and how that can enable different kinds of search applications. Full-Length Podcast: https://www.youtube.com/watch?v=HIGAQAE_xaI Code Tutorial (Weaviate + Jina AI for Image Search): https://www.youtube.com/watch?v=rBKvoIGihnY Please check out Jina AI on YouTube: https://www.youtube.com/c/JinaAI Please check out SeMI Technologies on YouTube: https://www.youtube.com/c/SeMI-and-Weaviate/videos Chapters

0:00 Introduction

This video explains one of the biggest lessons for me in interviewing Han Xiao from Jina AI. I hope this was a good explanation of the preprocessing / granularity of embeddings and how that can enable different kinds of search applications. Full-Length Podcast: https://www.youtube.com/watch?v=HIGAQAE_xaI Code Tutorial (Weaviate + Jina AI for Image Search): https://www.youtube.com/watch?v=rBKvoIGihnY Please check out Jina AI on YouTube: https://www.youtube.com/c/JinaAI Please check out SeMI Technologies on YouTube: https://www.youtube.com/c/SeMI-and-Weaviate/videos Chapters

0:00 Please check out SeMI YouTube!

0:15 My takeaways on Full Stack Neural Search

11:04 Podcast Clip - Han Xiao

I hope this video helps you get started with Image Search using Weaviate and Jina AI - happy to answer any questions / help solve problems! Check out the full tutorial explanation from Laura Ham: https://www.youtube.com/watch?v=rBKvoIGihnY New podcast with Jina AI CEO Han Xiao! https://www.youtube.com/watch?v=HIGAQAE_xaI Full notebook code: https://github.com/laura-ham/HM-Fashion-image-neural-search/blob/main/hm-fashion-image-neural-search.ipynb Get started with the Weaviate Cloud Service: console.semi.technology

Hey everyone! Hopefully this video helps supplement the new Weaviate podcast with Brady Neal, I hope you find this interesting / useful! Check out Brady Neal on YouTube! https://www.youtube.com/c/BradyNealCausalInference/featured Weaviate Podcast: https://www.youtube.com/watch?v=t7g9s1GWcB8 0:00 New Weaviate Podcast!

0:42 Brady Neal Causal Inference

1:34 Oogway.ai

2:45 Whiteboard Ideas

5:35 Discussion Topics

Hey everyone! I recently interviewed Arvind Neelakantan from OpenAI about the new OpenAI Embeddings API on the Weaviate Podcast! This video provides some additional detail for the different topics that were discussed. If you find this video to be informative, please check out SeMI technologies on youtube where we are working hard on developing content explaining concepts in Deep Learning for Search. Full Podcast: https://www.youtube.com/watch?v=uFxfZ0vLsoU SeMI Technologies on YouTube: https://www.youtube.com/channel/UCJKT6kJ3IFYybWnL7jbXxhQ

Thanks for watching! Please subscribe for more Deep Learning and AI videos, the list of papers is below under "Content Links" Content Links:

Fully Online Meta-Learning without Task Boundaries: https://arxiv.org/abs/2202.00263

Datamodels: Predicting Predictions from Training Data: https://arxiv.org/abs/2202.00622

Adaptive Discrete Communication Bottlenecks with Dynamic Vector Quantization: https://arxiv.org/abs/2202.01334

Competition-Level Code Generation with AlphaCode: https://storage.googleapis.com/deepmind-media/AlphaCode/competition_level_code_generation_with_alphacode.pdf

GPT-NeoX-20B: https://blog.eleuther.ai/announcing-20b/

PromptSource: https://arxiv.org/abs/2202.01279

Chain of Thou…

This video gives an overview of the latest Weaviate podcast, please subscribe to see future episodes!

https://www.youtube.com/c/SeMI-and-Weaviate/videos Thanks for watching! Chapters

0:00 Overview

7:53 Ideas for Podcast Search

10:44 Weaviate Podcast so far

Thank you so much for watching, please subscribe for more Deep Learning and Ai videos! Please check out SeMI Technologies on YouTube as well, where I am hosting a podcast on Deep Learning for Search! Paper Links:

Text and Code Embeddings by Contrastive Pre-Training: https://cdn.openai.com/papers/Text_and_Code_Embeddings_by_Contrastive_Pre_Training.pdf

Introducing Text and Code Embeddings in the OpenAI API (Blog Post): https://openai.com/blog/introducing-text-and-code-embeddings/

Nils Reimers - OpenAI GPT-3 Text Embeddings - Really a new state-of-the-art in dense text embeddings? https://medium.com/@nils_reimers/openai-gpt-3-text-embeddings-really-a-new-state-of-the-art-in-dense-text-embeddi…

Thank you so much for watching, please subscribe for more Deep Learning and AI videos! Please check out SeMI Technologies on YouTube as well! Paper Links:

CM3: https://arxiv.org/abs/2201.07520

data2vec: https://scontent.fmia1-1.fna.fbcdn.net/v/t39.8562-6/271974914_483120576492438_4239522333319653600_n.pdf?_nc_cat=107&ccb=1-5&_nc_sid=ae5e01&_nc_ohc=4-cMR5tUq4QAX-Of7fj&_nc_ht=scontent.fmia1-1.fna&oh=00_AT9ymN9dNPt1p8zWQClW6MSZikaCTT8gobc2LqxW4OhzZQ&oe=61F3F7D1

LaMDA: https://arxiv.org/abs/2201.08239

PromptBERT: https://arxiv.org/abs/2201.04337

UnifiedSKG: https://arxiv.org/abs/2201.05966

Collapse by Conditioning: https://arxiv.org/abs/2201.06578

GradTail: https://arxiv.org/abs/2201.05938

CLIP…

This video quickly presents the idea of having some kind of trusted 3rd party library to handle data privacy for Deep Learning applications. The current state of Deep Learning applications are very data heavy, although recent advances such as Data Augmentation and Transfer Learning may circumvent that bottleneck. Federated Learning is a promising solution to data privacy and Deep Learning. In this framework, only the model weights are sent to a local user and the updates are done within the local machine. Only the parameters are traded globally, rather than having a central data store with sensitive user data. Federated Learning seems to be the leading technique for this -- Differentiable P…

One of the biggest lessons I've learned in developing KerasBERT and trying to apply language modeling to Keras information is the value of data. More particularly, the value of automated pipelines to collect this data. In this clip, I asked Michael about the ease of integrating slack conversations into 3rd party applications built on Deep Learning. This reminded me of the conversation with Charles Pierse at Keenious (Weaviate Podcast #2) and how they integrated their search and recommendation system directly into Google Docs and Microsoft Word. I hope you find this video interesting, it is really exciting to see this modularity with established software platforms to allow Deep Learning deve…

Democratic AI is an important goal to enable entrepreneurship and the development of AI technology. Particularly what we mean by this is generally overcoming bottlenecks of needing very expensive computers, massive private datasets, and long training times to get started with Deep Learning. I think that the decomposition of Retrieve-then-Read can be very promising for overcoming this bottleneck, in addition to ideas around efficient training. For those interested, I highly recommend checking out the "methods" outlined on the MosaicML website. They are a leading research lab doing amazing work on efficiency in Deep Learning, which has additional implications such as limiting climate damage f…

A slight correction to the previous video, with some more details about how the best first word was chosen.

Special thanks to these supporters: https://3b1b.co/lessons/wordle#thanks

Help fund future projects: https://www.patreon.com/3blue1brown​

An equally valuable form of support is to simply share the videos. Contents:

0:00 - The Bug

3:31 - How the best first guess is chosen

8:54 - Does this ruin the game? Nice post by Jonathan Olson on optimal wordle algorithms:

https://jonathanolson.net/experiments/optimal-wordle-solutions ------------------ These animations are largely made using a custom python library, manim. See the FAQ comments here:

https://www.3blue1brown.com/faq#manim

https://gi…

An excuse to teach a lesson on information theory and entropy.

Help fund future projects: https://www.patreon.com/3blue1brown​

Special thanks to these supporters: https://3b1b.co/thanks

An equally valuable form of support is to simply share the videos. Contents:

0:00 - What is Wordle?

2:43 - Initial ideas

8:04 - Information theory basics

18:15 - Incorporating word frequencies

27:49 - Final performance Original wordle site:

https://www.powerlanguage.co.uk/wordle/ Music by Vincent Rubinetti.

https://www.vincentrubinetti.com/ Shannon and von Neumann artwork by Kurt Bruns. Code for this video:

https://github.com/3b1b/videos/blob/master/_2022/wordle.py These animations are largely made using a c…

A tale of two problem solvers.

Numberphile video on Bertrand's paradox: https://youtu.be/mZBwsm6B280

Help fund future projects: https://www.patreon.com/3blue1brown​

Special thanks to these supporters: https://3b1b.co/lessons/newtons-fractal#thanks

An equally valuable form of support is to simply share the videos. The general result here was originally proved by Cauchy.

Mémoire sur la rectification des courbes et la quadrature des surfaces courbes par M. Augustin Cauchy

https://ia600208.us.archive.org/27/items/bub_gb_EomNI7m8__UC/bub_gb_EomNI7m8__UC.pdf ------------------- Timestamps

0:00 - The players

5:22 - How to start

9:12 - Alice's initial thoughts

13:37 - Piecing together the cube

22:1…

Take a look at the full playlist (really): https://www.youtube.com/watch?v=fJWnA4j0_ho&list=PLnQX-jgAF5pTkwtUuVpqS5tuWmJ-6ZM-Z

Blog post with more details: https://3b1b.co/some1-results

Thanks, as always, to the supporters of this channel for helping to make this whole project possible: http://3b1b.co/thanks ------------------ Typo at 2:00, it should read "Burkard Polster" Videos and posts mentioned in this video. That weird light at the bottom of a mug — ENVELOPES

https://youtu.be/fJWnA4j0_ho Hiding Images in Plain Sight: The Physics Of Magic Windows

https://mattferraro.dev/posts/caustics-engineering The Beauty of Bézier Curves

https://youtu.be/aVwxzDHniEw What Is The Most Complicated Lock…

An intro to holomorphic dynamics, the study of iterated complex functions.

Video on Newton's fractal: https://youtu.be/-RdOwhmqP5s

Special thanks to these supporters: https://3b1b.co/lessons/holomorphic-dynamics#thanks Extra special thanks to Sergey Shemyakov, of Aix-Marseille University, for helpful conversations and for introducing me to this phenomenon. Introduction to Fatou sets and Julia sets, including a discussion of Montel's theorem and its consequences:

http://www.math.stonybrook.edu/~scott/Papers/India/Fatou-Julia.pdf Numberphile with Ben Sparks on the Mandelbrot set:

https://youtu.be/FFftmWSzgmk Ben explains how he made the Geogebra files on his channel here: https://youtu.be/ICq…

Who knew root-finding could be so complicated?

Next part: https://youtu.be/LqbZpur38nw

Special thanks to the following supporters: https://3b1b.co/lessons/newtons-fractal#thanks

An equally valuable form of support is to simply share the videos. ------------------ Interactive for this video:

https://www.3blue1brown.com/lessons/newtons-fractal On fractal dimension:

https://youtu.be/gB9n2gHsHN4 Mathologer on the cubic formula:

https://youtu.be/N-KXStupwsc Some articles on Newton's Fractal, and its cousins:

https://www.chiark.greenend.org.uk/~sgtatham/newton/

https://blbadger.github.io/polynomial-roots.html Some of the videos from this year's Summer of Math Exposition are fairly relevant to the…

❤️ Check out Weights & Biases and sign up for a free demo here: https://wandb.com/papers ❤️ Their mentioned post is available here (thank you Soumik!): http://wandb.me/3d-inverse-rendering 📝 The paper "Extracting Triangular 3D Models, Materials, and Lighting From Images" is available here:

https://research.nvidia.com/publication/2021-11_Extracting-Triangular-3D

https://nvlabs.github.io/nvdiffrec/ ❤️ Watch these videos in early access on our Patreon page or join us here on YouTube: - https://www.patreon.com/TwoMinutePapers

- https://www.youtube.com/channel/UCbfYPyITQ-7l4upoX8nvctg/join 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:

Aleksandr Mas…

❤️ Check out Lambda here and sign up for their GPU Cloud: https://lambdalabs.com/papers 📝 The paper "Learning Robust Real-Time Cultural Transmission without Human Data" is available here:

https://www.deepmind.com/research/publications/2022/Learning-Robust-Real-Time-Cultural-Transmission-without-Human-Data

https://sites.google.com/view/dm-cgi ❤️ Watch these videos in early access on our Patreon page or join us here on YouTube: - https://www.patreon.com/TwoMinutePapers

- https://www.youtube.com/channel/UCbfYPyITQ-7l4upoX8nvctg/join 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:

Aleksandr Mashrabov, Alex Balfanz, Alex Haro, Andrew Melnychuk, Angel…

❤️ Check out Lambda here and sign up for their GPU Cloud: https://lambdalabs.com/papers 📝 The paper "Creating Multimodal Interactive Agents with Imitation and Self-Supervised Learning" is available here:

https://deepmind.com/research/publications/2021/Creating-Interactive-Agents-with-Imitation-Learning ❤️ Watch these videos in early access on our Patreon page or join us here on YouTube: - https://www.patreon.com/TwoMinutePapers

- https://www.youtube.com/channel/UCbfYPyITQ-7l4upoX8nvctg/join 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:

Aleksandr Mashrabov, Alex Balfanz, Alex Haro, Andrew Melnychuk, Angelos Evripiotis, Benji Rabhan, Bryan Learn…

❤️ Check out Lambda here and sign up for their GPU Cloud: https://lambdalabs.com/papers 📝 The paper "CLIPort: What and Where Pathways for Robotic Manipulation" is available here:

https://cliport.github.io/ ❤️ Watch these videos in early access on our Patreon page or join us here on YouTube: - https://www.patreon.com/TwoMinutePapers

- https://www.youtube.com/channel/UCbfYPyITQ-7l4upoX8nvctg/join 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:

Aleksandr Mashrabov, Alex Balfanz, Alex Haro, Andrew Melnychuk, Angelos Evripiotis, Benji Rabhan, Bryan Learn, B Shang, Christian Ahlin, Eric Martel, Gordon Child, Ivo Galic, Jace O'Brien, Jack Lukic, Javier…

❤️ Check out Cohere and sign up for free today: https://cohere.ai/papers 📝 The paper "MuZero with Self-competition for Rate Control in VP9 Video Compression" is available here:

https://deepmind.com/blog/article/MuZeros-first-step-from-research-into-the-real-world

https://storage.googleapis.com/deepmind-media/MuZero/MuZero%20with%20self-competition.pdf deepmind

https://arxiv.org/abs/2202.06626 ❤️ Watch these videos in early access on our Patreon page or join us here on YouTube: - https://www.patreon.com/TwoMinutePapers

- https://www.youtube.com/channel/UCbfYPyITQ-7l4upoX8nvctg/join 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:

Aleksandr Mashrab…

❤️ Check out Weights & Biases and sign up for a free demo here: https://wandb.com/papers ❤️ Their mentioned post is available here (thank you Soumik Rakshit!): http://wandb.me/barbershop 📝 The paper "Barbershop: GAN-based Image Compositing using Segmentation Masks" is available here:

https://zpdesu.github.io/Barbershop/

https://github.com/ZPdesu/Barbershop ❤️ Watch these videos in early access on our Patreon page or join us here on YouTube: - https://www.patreon.com/TwoMinutePapers

- https://www.youtube.com/channel/UCbfYPyITQ-7l4upoX8nvctg/join 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:

Aleksandr Mashrabov, Alex Balfanz, Alex Haro, Andrew M…

❤️ Check out Weights & Biases and say hi in their community forum here: https://wandb.me/paperforum 📝 The paper "Neural Control Variates" is available here:

https://research.nvidia.com/publication/2021-01_Neural-Control-Variates

https://tom94.net/data/publications/mueller20neural/interactive-viewer/ 🔆 The free light transport course is available here. You'll love it!

https://users.cg.tuwien.ac.at/zsolnai/gfx/rendering-course/ ❤️ Watch these videos in early access on our Patreon page or join us here on YouTube: - https://www.patreon.com/TwoMinutePapers

- https://www.youtube.com/channel/UCbfYPyITQ-7l4upoX8nvctg/join 🙏 We would like to thank our generous Patreon supporters who make Two Minute …

❤️ Check out Lambda here and sign up for their GPU Cloud: https://lambdalabs.com/papers 📝 The post about GPT-3's Edit and Insert capabilities are available here:

https://openai.com/blog/gpt-3-edit-insert/ ❤️ Watch these videos in early access on our Patreon page or join us here on YouTube: - https://www.patreon.com/TwoMinutePapers

- https://www.youtube.com/channel/UCbfYPyITQ-7l4upoX8nvctg/join 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:

Aleksandr Mashrabov, Alex Balfanz, Alex Haro, Andrew Melnychuk, Angelos Evripiotis, Benji Rabhan, Bryan Learn, B Shang, Christian Ahlin, Eric Martel, Gordon Child, Ivo Galic, Jace O'Brien, Jack Lukic, Javier …

❤️ Check out Lambda here and sign up for their GPU Cloud: https://lambdalabs.com/papers 📝 The paper "Hierarchical Text-Conditional Image Generation with CLIP Latents" is available here:

https://openai.com/dall-e-2/

https://www.instagram.com/openaidalle/ ❤️ Watch these videos in early access on our Patreon page or join us here on YouTube: - https://www.patreon.com/TwoMinutePapers

- https://www.youtube.com/channel/UCbfYPyITQ-7l4upoX8nvctg/join 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:

Aleksandr Mashrabov, Alex Balfanz, Alex Haro, Andrew Melnychuk, Angelos Evripiotis, Benji Rabhan, Bryan Learn, B Shang, Christian Ahlin, Eric Martel, Gordon Ch…

❤️ Check out Weights & Biases and sign up for a free demo here: https://wandb.com/papers ❤️ Their mentioned post is available here (thank you Soumik Rakshit!): http://wandb.me/EditGAN 📝 The paper "EditGAN: High-Precision Semantic Image Editing" is available here:

https://nv-tlabs.github.io/editGAN/

https://arxiv.org/abs/2111.03186

https://github.com/nv-tlabs/editGAN_release

https://nv-tlabs.github.io/editGAN/editGAN_supp_compressed.pdf ❤️ Watch these videos in early access on our Patreon page or join us here on YouTube: - https://www.patreon.com/TwoMinutePapers

- https://www.youtube.com/channel/UCbfYPyITQ-7l4upoX8nvctg/join 🙏 We would like to thank our generous Patreon supporters who make T…

❤️ Check out Weights & Biases and sign up for a free demo here: https://wandb.com/papers 📝 The paper "Stitch it in Time: GAN-Based Facial Editing of Real Videos" is available here:

https://stitch-time.github.io/ ❤️ Watch these videos in early access on our Patreon page or join us here on YouTube: - https://www.patreon.com/TwoMinutePapers

- https://www.youtube.com/channel/UCbfYPyITQ-7l4upoX8nvctg/join 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:

Aleksandr Mashrabov, Alex Balfanz, Alex Haro, Andrew Melnychuk, Angelos Evripiotis, Benji Rabhan, Bryan Learn, B Shang, Christian Ahlin, Eric Martel, Gordon Child, Ivo Galic, Jace O'Brien, Javier Busta…

❤️ Check out Lambda here and sign up for their GPU Cloud: https://lambdalabs.com/papers 📝 The #OpenAI paper "Aligning Language Models to Follow Instructions" is available here:

https://openai.com/blog/instruction-following/ ❤️ Watch these videos in early access on our Patreon page or join us here on YouTube: - https://www.patreon.com/TwoMinutePapers

- https://www.youtube.com/channel/UCbfYPyITQ-7l4upoX8nvctg/join 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:

Aleksandr Mashrabov, Alex Balfanz, Alex Haro, Andrew Melnychuk, Angelos Evripiotis, Benji Rabhan, Bryan Learn, Christian Ahlin, Eric Martel, Gordon Child, Ivo Galic, Jace O'Brien, Javier Bu…

❤️ Check out Cohere and sign up for free today: https://cohere.ai/papers 📝 The paper "GANgealing GAN-Supervised Dense Visual Alignment" is available here:

https://www.wpeebles.com/gangealing Note that this work is a collaboration between Adobe Research, UC Berkeley, CMU and MIT CSAIL. Try it!:

- https://colab.research.google.com/drive/1JkUjhTjR8MyLxwarJjqnh836BICfocTu?usp=sharing

- https://github.com/wpeebles/gangealing ❤️ Watch these videos in early access on our Patreon page or join us here on YouTube: - https://www.patreon.com/TwoMinutePapers

- https://www.youtube.com/channel/UCbfYPyITQ-7l4upoX8nvctg/join 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers…

❤️ Check out Weights & Biases and sign up for a free demo here: https://wandb.com/papers ❤️ Their mentioned post is available here (Thank you Soumik Rakshit!): http://wandb.me/2min-block-nerf 📝 The paper "Block-NeRF Scalable Large Scene Neural View Synthesis" from #Waymo is available here:

https://waymo.com/research/block-nerf/ ❤️ Watch these videos in early access on our Patreon page or join us here on YouTube: - https://www.patreon.com/TwoMinutePapers

- https://www.youtube.com/channel/UCbfYPyITQ-7l4upoX8nvctg/join 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:

Aleksandr Mashrabov, Alex Balfanz, Alex Haro, Andrew Melnychuk, Angelos Evripiotis,…

Data Fest Online 2021 https://fest.ai/2021/

ML Art track https://ods.ai/tracks/ml-art-df2021 Speaker introduces himself in Russian, and then presents the material in English.

Несмотря на то, что многие из последних достижений в области машинного обучения связаны с глубоким обучением с подкреплением, Deep RL алгоритмы остаются ненадёжными (по сравнению с классическими моделями глубокого обучения) и трудновоспроизводимыми (с точки зрения результата). Авторы статьи связывают описанные недостатки с проблемой отсутствия понимания того как внутренние механизмы, используемые в RL алгоритмах, влияют на поведение агента по отдельности и вместе взятые. На семинаре мы поговорим о поднятой авторами проблеме на примере алгоритмов Trust Region Policy Optimization (TRPO) и Proximal Policy Optimization (PPO), рассмотрим эксперименты по оценке влияния составных частей этих алгор…

Зачастую в прикладных задачах собрать достаточно большой, подходящим образом размеченный датасет для обучения модели не представляется возможным. Популярным решением в такой ситуации является Self-Supervised Learning. В рамках этого подхода модель сначала предобучают на синтетической, искусственно выдуманной задаче, выборку для которой автоматически формируют из неразмеченных данных. Примерами таких синтетических задач являются восстановление маскированных токенов в NLP (этот же подход используется и в некоторых моделях для работы с кодом), восстановление фрагментов или удаление искусственного шума при работе с картинками, восстановление последовательности кадров при работе с видео и т.д.. …

Многие из самых захватывающих новых прорывов в области искусственного интеллекта произошли благодаря двум недавним инновациям: самоконтролируемое обучение, который позволяет машинам учиться на случайных немаркированных примерах, а также Трансформеры, которые позволяют моделям ИИ выборочно сосредотачиваться на определенных частях своего ввода и, таким образом, рассуждать более эффективно. На семинара будет разобрана новая статья "Emerging Properties in Self-Supervised Vision Transforms", в которой авторы используются ранее упомянутые техники для решения задач компьютерного зрения. Докладчик: Ольга Лавриченко.

Существующие фреймворки для генерации изображений могут обуславливаться на пользовательский ввод в одной модальности — например, на текст, эскиз, маску сегментации или пример изображения со стилем. При этом, такие подходы не используют доступные мультимодальные данные. Авторы данной статьи предлагают Product-of-Experts Generative Adversarial Networks (PoE-GAN) фреймворк, который позволяет синтезировать изображение на основе условий в нескольких модальностях или любом их подмножестве, а также осуществлять безусловную генерацию. Данная модель также превосходит другие подходы в условиях унимодальной условной генерации. Докладчик: Дарья Евсикова.

Трансформеры уже давно господствуют во многих задачах NLP. И если с задачами где длина последовательности относительно мала (не более 512 токенов) проблем не возникает, то с обработкой больших текстов не все так ясно. Проблема в том, что потребление памяти увеличивается квадратично с ростом обрабатываемой последовательности. Существуют различные подходы к решению проблемы, например, можно линеаризовать softmax в модуле внимания, снизив асимптотику до O(N) (linear transformers); или же исследовать разреженность (BigBird). В свою очередь, авторы статьи продолжают идеи sliding-window и Transformer-XL. Поэтому на семинаре поговорим об этих подходах и архитектуре Block-Recurrent Transformer. Док…

Мы расскажем про исследование, посвященное дообучению ML4SE моделей под конкретный проект. В то время как большинство исследователей обучает и тестирует модели на непересекающихся наборах проектов, мы задались вопросом: “А что будет, если показать модели данные из целевого проекта?“. Мы поговорим об особенностях оценки качества проектно-дообученных моделей и презентуем полученные результаты для трех моделей в задаче предсказания имен методов.

Докладчик – Егор Богомолов

На семинаре мы поговорим о нашей работе в области предварительной тренировки векторных представлений графовых нейронных сетей (GNN) для исходного кода. Качество векторов мы оцениваем с помощью задачи предсказания типов для языка с динамической типизацией Python. Для предварительной тренировки используется задача предсказания имён. По результатам наших экспериментов векторные представления GNN позволяют достичь точности классификации типов, сравнимой с CodeBERT. Вдобавок, объединение CodeBERT и GNN векторов в гибридную модель позволяет улучшить точность классификации типов. При этом, улучшения достигаются даже после тренировки GNN модели в течение всего одной эпохи, что намного меньше чем тр…

В крупных компаниях, где весь код лежит в едином репозитории, очень важно уметь оперативно локализовать баг. Задача усложняется, когда отельные файлы состоят из сотен строк, а проблема выявляется на этапе End-to-End тестирования или в продакшене. В такой ситуации необходимо автоматическое решение, которое способно быстро найти ломающий коммит, несмотря на то, что сообщения об ошибке зачастую трудночитаемые и содержат большой объём информации. На этом семинаре мы разберём статью от Facebook (https://arxiv.org/pdf/2010.09977.pdf), в которой авторы предлагают эффективный unsupervised алгоритм локализации бага к коммиту, использующий методы информационного поиска. Описанный алгоритм приспособле…

Разработка программного обеспечения сопряжена с поиском и исправлением ошибок. В программной инженерии уже давно изучаются и описываются запахи кода – формальные признаки, индицирующие о возможном наличии проблем. Примерами запахов кода могут быть завистливая функция (метод обращается к данным чужого класса чаще, чем к данным собственного) или параллельная иерархия (ситуация, когда при создании нового класса в одной иерархии классов вам почти всегда приходится создавать парный к нему класс в другой иерархии). Для каждого запаха кода описаны потенциальные пути исправления, часто сводящиеся к какому-то рефакторингу.

Однако, проекты, связанные с машинным обучением, обладают особой спецификой и…

Трансформер - очень хорошая модель для понимания текста, однако она не эффективна из-за квадратичной асимптотической сложности по длине входящей последовательности. Хотя существует множество методов ускорения трансформера, они все еще недостаточно эффективны на длинных последовательностях. Авторы статьи предлагают Fastformer, эффективную модель трансформера, основанную на аддитивном внимании (additive attention). На семинаре мы вспомним, как работают трансформеры, познакомимся с additive attention и Fastformer и посмотрим, как он справляется с различными задачами. Докладчик: Тимур Хабибуллин

Задачи обработки естественного языка, такие как машинный перевод, ответы на вопросы и обобщения текстов, как правило решаются с помощью обучения с учителем на специально подобранных под конкретное задание датасетах. Авторы статьи показывают, что можно обучить модель, которая будет способна решать различные задачи с минимальным количеством обучения с учителем, используя для этого датасет Webtext, состоящий из миллионов различных веб-страниц. На семинаре мы обсудим, как модель справляется с заданиями различной специфики и сравним результаты авторов с результатами state-of-the art моделей. Докладчики: Маргарита Чудова

Я расскажу про процесс создания командой AI Team системы автодополнения для языка R. Будет рассказано о том, с какими трудностями можно столкнуться при разработке и внедрении системы автодополнения, основанной на нейросетях. Также мы рассмотрим некоторые нерешённые исследовательские проблемы в области нейросетевого автодополнения и обсудим возможные способы их решения. Большая часть рассказа будет основана на статье Time-Efficient Code Completion Model for the R Programming Language (https://aclanthology.org/2021.nlp4prog-1.4/), опубликованной на воркшопе NLP4prog 2021 (https://nlp4prog.github.io/2021/) конференции ACL. Докладчики: Артем Попов.

Графовые структуры, описывающие зависимости между сущностями, широко используются для повышения эффективности моделей машинного обучения, обучаемых на потоковых данных. Для того, чтобы использовать классические методы машинного обучения в таких задачах, необходимо иметь возможность строить векторные представления компонентов графа (вершин и/или ребер) с учетом их атрибутов. Хотя для статических графов существует большое количество методов построения таких представлений, задача оказывается гораздо сложнее, когда у графа меняется структура с течением времени. В рамках семинара мы рассмотрим задачи, которые возникают при работе с динамическими графами, рассмотрим классификацию методов построен…

Знание полного целевого пространства лекарственных средств дает важную информацию о потенциальном терапевтическом применении агентов для модулирования или предотвращения их различных целевых и побочных эффектов в области разработки лекарств и точной медицины. На семинаре мы рассмотрим комплексное использование обширных данных о биоактивности соединения-мишени из Drug Target Commons (DTC) и баз данных связанных лекарств с помощью Drug Target Profiler (DTP), программного обеспечения с открытым исходным кодом и веб-инструмента для интерактивного исследования сетей взаимодействия лекарств и мишеней. Докладчик: Юлия Волкова

Дерево отрезков. Задача RSQ (range sum query). Задачи LCA (least common ancestor) и RMQ (range minimum query). Решение RMQ с помощью sparse table. Сведение LCA к RMQ (алгоритм Фарах-Колтона-Бендера). Сведение RMQ к LCA. Задача LA (level ancestors). Подробнее о поступлении в Школу анализа данных от Академии Яндекса: https://clck.ru/geqRt

Кратчайшие пути в графах. Оценки расстояний и их релаксация. Алгоритмы Беллмана-Форда, Флойда и Дийкстры. Потенциалы. Критерий консервативности длин в терминах наличия допустимых потенциалов. Нахождение допустимых потенциалов с помощью алгоритма Беллмана-Форда. Алгоритм Джонсона. Подробнее о поступлении в Школу анализа данных от Академии Яндекса: https://clck.ru/geqRt

Остовы минимального веса. Лемма о минимальном ребре в разрезе. Алгоритмы Краскала и Прима. Структура DSU (disjoint set union) Реализация с использованием леса. Ранги вершин, эвристика ранга. Логарифмическая оценка ранга через количество элементов. Эвристика сжатия путей. Оценка учетной стоимости операций (без доказательства). Подробнее о поступлении в Школу анализа данных от Академии Яндекса: https://clck.ru/geqRt

Splay-деревья. Операция splay: zig, zig-zig и zig-zag шаги. Реализация операций вставки, удаления, слияния и разделения для splay-деревьев. Обход в глубину. Топологическая сортировка Подробнее о поступлении в Школу анализа данных от Академии Яндекса: https://clck.ru/geqRt

Хеш-функции. Коллизии. Разрешение коллизий методом цепочек. Гипотеза простого равномерного хеширования, оценка средней длины цепочки хеш-функции. Подробнее о поступлении в Школу анализа данных от Академии Яндекса: https://clck.ru/geqRt

Модель разрешающих деревьев. Нижняя оценка на число сравнений при сортировке и поиске в модели разрешающих деревьев.

Сортировка слиянием (Merge-Sort). Top-down и bottom-up подходы. Сортировка слиянием во внешней памяти. Inplace Merge-Sort. Подробнее о поступлении в Школу анализа данных от Академии Яндекса: https://clck.ru/geqRt

Задача о длиннейшой возрастающей подпоследовательности. Алгоритмы сортировки Heap-Sort и Intro-Sort. Частичная сортировка с помощью кучи и поиска порядковой статистики. Подробнее о поступлении в Школу анализа данных от Академии Яндекса: https://clck.ru/geqRt

Построение совершенной хеш-функции методом двухуровненого хеширования. Построение совершенной хеш-функции методом ациклических графов. Фильтр Блюма (Bloom filter). Оценка вероятности ложноположительного срабатывания. Count-min sketch. Подробнее о поступлении в Школу анализа данных от Академии Яндекса: https://clck.ru/geqRt

Сильно связанные компоненты. Точки сочленения: определение и нахождение с помощью обхода в глубину. Мосты. Подробнее о поступлении в Школу анализа данных от Академии Яндекса: https://clck.ru/geqRt

Время и память как основные ресурсы. RAM машина. Сложность на заданном входе, сложность в худшем случае, сложность в среднем случае, рандомизированная сложность.

Учетная стоимость операций, метод потенциалов, банковский метод анализа сложности.

Массивы переменного размера. Реаллокация. Анализ учетной сложности операции push-back. Подробнее о поступлении в Школу анализа данных от Академии Яндекса: https://clck.ru/geqRt

Время и память как основные ресурсы. RAM машина. Сложность на заданном входе, сложность в худшем случае, сложность в среднем случае, рандомизированная сложность.

Учетная стоимость операций, метод потенциалов, банковский метод анализа сложности.

Массивы переменного размера. Реаллокация. Анализ учетной сложности операции push-back. Подробнее о поступлении в Школу анализа данных от Академии Яндекса: https://clck.ru/geqRt

Реализация очереди на паре стеков с константной учетной сложностью. Динамические минимумы-максимумы в стеках и очередях. Персистентные структуры данных. Виды персистентности. Модель вычислений Pointer Machine. Персистентные стеки и очереди. Подробнее о поступлении в Школу анализа данных от Академии Яндекса: https://clck.ru/geqRt

Алгоритм Misra-Gries.

Деревья поиска. Вставка и удаление элементов. Inorder-обход дерева. Красно черные деревья: определение и основные свойства. Реализация операций вставки для красно-черного дерева. Дучи (treaps). Единственность дучи для заданного набора различных ключей и приоритетов. Логарифмическая оценка матожидания высоты дучи. Операции слияния и разделения для дуч. Операции вставки и удаления элементов для дуч. Подробнее о поступлении в Школу анализа данных: https://academy.yandex.ru/dataschool

Быстрая сортировка (Quick-Sort). Способы выбора разделяющего элемента. Элиминация хвостовой рекурсии. Порядковые статистики. Рандомизированный алгоритм Quick-Select. Детермининированный алгоритм поиска (метод "медианы медиан").

Бинарный поиск. Galloping.

Линейное по времени слияние упорядоченных последовательностей. Оптимальное по числу сравнений слияние упорядоченных последовательностей.

Задача о длиннейшей возврастающей подпоследовательности. Динамическое программирование. O(n log n)-алгоритм. Подробнее о поступлении в Школу анализа данных от Академии Яндекса: https://clck.ru/geqRt

Дерево отрезков. Задача RSQ (range sum query). Задачи LCA (least common ancestor) и RMQ (range minimum query). Решение RMQ с помощью sparse table. Сведение LCA к RMQ (алгоритм Фарах-Колтона-Бендера). Сведение RMQ к LCA. Задача LA (level ancestors). Подробнее о поступлении в Школу анализа данных от Академии Яндекса: https://clck.ru/geqRt

Open ML Course: Линейные модели https://ods.ai/tracks/linear-models-spring22

Лекция 3. Выбор модели. Создание новых признаков | Иван Комаров Канал курса:

Telegram chat: https://t.me/+88vIIR6Ch3c0N2Ji Соцсети Open Data Science:

https://t.me/datafest

https://t.me/ods_ru

https://vk.com/datafest Регистрация на мероприятия сообщества: https://ods.ai/events

Хабы сообщества: https://ods.ai/hubs

Open ML Course: Линейные модели https://ods.ai/tracks/linear-models-spring22

Лекция 2. Логистическая регрессия. Метрики качества | Иван Комаров Канал курса:

Telegram chat: https://t.me/+88vIIR6Ch3c0N2Ji Соцсети Open Data Science:

https://t.me/datafest

https://t.me/ods_ru

https://vk.com/datafest Регистрация на мероприятия сообщества: https://ods.ai/events

Хабы сообщества: https://ods.ai/hubs

Open ML Course: Линейные модели https://ods.ai/tracks/linear-models-spring22

Лекция 1. Линейная регрессия | Иван Комаров Канал курса:

Telegram chat: https://t.me/+88vIIR6Ch3c0N2Ji Соцсети Open Data Science:

https://t.me/datafest

https://t.me/ods_ru

https://vk.com/datafest Регистрация на мероприятия сообщества: https://ods.ai/events

Хабы сообщества: https://ods.ai/hubs

Сезон открытых бесплатных курсов ODS продолжается в весеннем семестре! 🍿 Стрим с анонсами и обсуждением курсов

🔥 Митапы, встречи с участниками курсов и сообщества, знакомства и ламповое общение в Spatial.chat ODS, включая чекпойнт зимнего хакатона ODS #ods_pet_projects Winter Hack Регистрация и полное расписание мероприятия доступны на ODS.AI: https://ods.ai/events/course_fest_spring_22 Весенние курсы ODS: https://ods.ai/tracks/groups/courses

Вступить в сообщество: https://ods.ai/ Соцсети Data Fest & Course Fest: https://t.me/datafest

https://vk.com/datafest

https://twitter.com/NewsOds

- расскажу про задачу ранжирования, решаемую в Lamoda для каталога товаров, и про ее решение, не опирающееся на синхронный запуск модели в ответ на событие пользователя

- приложу подход к нескольким примерам других задач и на их примере расскажу для каких случаев он может быть полезен

- на примере рабочей системы обработки данных покажу как она отличается от классического варианта (с деплоем модельки для синхронного ответа на событие)

Закрываем наш небольшой должок с прошедшей Ёлки 🤗 В программе ещё 100 минут итогов года:

1. Итоги 2021 в NLP - Валентин Малых

2. Итоги 2021 в Quantum Computing - Сергей Ширкин

3. Итоги 2021 в GraphML - Михаил Галкин, Антон Цицулин, Анвар Курмуков

4. Итоги 2021 в A/B тестировании - Аслан Байрамкулов

...

5. ODS Awards 2021 recap + Зимняя школа ODS 2022 Плейлист лучших видео 2021: https://www.youtube.com/playlist?list=PLTlO6nV_TaGA9qov5W093NlMJbN7aeGa1

Зимняя школа ODS: https://ods.ai/competitions/ods-winter-projects-22

Владимир Грановский, Data Scientist at KION

Нейросетевая матричная факторизация и dssm (Практика)

В лекции мы соберем простой пайплайн для обучения DSSM:

- в очередной раз разгребем данные;

- сформируем датасет для обучения;

- напишем функцию потерь;

- соберем и обучим модель;

- посмотрим как ее применять. В конце видео вас ждет пара бонусов, которые позволят вам использовать разработанную модель по максимуму🔥 Соцсети Open Data Science:

https://t.me/datafest

https://t.me/ods_ru

https://vk.com/datafest Регистрация на мероприятия соощества: https://ods.ai/events

Хабы сообщества: https://ods.ai/hubs

Спикер: Глеб Волохов, [email protected] Нейросетевая матричная факторизация и dssm (Теория) Мы поговорим о применении нейросетевых подходов для построения рекомендаций. В частности мы поговорим о:

-автоенкодерах

-item 2 vec подходе

-графовом представлении матричной факторизации и подробно обсудим нейронную матричную факторизацию, лоссы для ранжирования и сиамкие сети. Соцсети Open Data Science:

https://t.me/datafest

https://t.me/ods_ru

https://vk.com/datafest Регистрация на мероприятия соощества: https://ods.ai/events

Хабы сообщества: https://ods.ai/hubs

Ирина Елисова, DS Teamlead [RecSys] В этой лекции вас ждет рассказ про двухуровневую или двухэтапную модель. Мы разберем в теории и на примере с кодом зачем ее строить, какие модели стоит использовать на каждом этапе. Как правильно собрать трейн и тест и настроить схему валидации. Первая часть лекции - теория, вторая - практика на датасете от онлайн-кинотеатра Кион. Соцсети Open Data Science:

https://t.me/datafest

https://t.me/ods_ru

https://vk.com/datafest Регистрация на мероприятия соощества: https://ods.ai/events

Хабы сообщества: https://ods.ai/hubs

ODS Course Fest 2021

https://ods.ai/events/course_fest_1 Спикер: Михаил Хасыков, ML Engineer в [email protected] В видео мы расскажем про визуальный анализ рекомендаций и про сложные метрики (diversity + novelty + serendipity) с примером кода Занятие состоит из двух частей - теории и практики. В первой половине лекции обсудим способы получения качественной оценки качества рекомендаций. В частности, познакомимся с «визуальным анализом» - методом оценки рекомендательной системы «глазами», узнаем кто такие «аватары» и как они применяются. Во второй половине теоретической части поговорим о таких свойствах рекомендаций как разнообразие и новизна: зачем они нужны, как их оценивать и оптимизировать. В п…

ODS Course Fest 2021

https://ods.ai/events/course_fest_1 Спикер: Ильдар Сафило, Head of Recommender Systems, [email protected] Мы расскажем о том, как можно оценивать эффект от влияния рекомендаций на продукт, зачем делать А/Б тест и что с помощью него оценивать. В конце мы поговорим о том, как можно связывать оффлайн и онлайн метрики( бизнесовые метрики) в рекомендательных системах, также обсудим проблему bias и feedback loop

Спикер: Александр Бутенко, MLE @ BigData MTS На лекции поговорим про некоторые способы ускорения инференса рекомендательных систем преимущественно в nearline/online сеттингах.

В частности, обсудим методы приближенного поиска соседей, почему их можно применять и какие могут быть трудности и ограничения. Подробнее разберем один.

Также немного затронем тему кэширования. Соцсети Open Data Science:

https://t.me/datafest

https://t.me/ods_ru

https://vk.com/datafest Регистрация на мероприятия соощества: https://ods.ai/events

Хабы сообщества: https://ods.ai/hubs

Спикер: Эмилий Фельдман, ML developer @ BigData MTS На лекции обсудим, что такое production (прод) для рекомендательных систем и зачем он нужен. Затем рассмотрим различные варианты реализации прода - offline, nearine, online - и поговорим о том, когда какой вариант лучше использовать. После чего детально рассмотрим как грамотно построить офлайн-продд, какие инструменты использовать и на что обратить внимание. В конце осветим несколько общих моментов про то, как сделать прод лучше.

Наше финальное мероприятие 2021 года уже завтра! Завтрашняя трансляция уже ждёт вас на нашем ютубе: https://youtu.be/xkl5oXtsjTc Проголосовать в премии ODS Awards нужно тут:

https://ods.ai/tracks/ods-awards-2021 А полное расписание Ёлки доступно на ODS.AI: https://ods.ai/events/data-elka-2021 Ждём вас и ваших голосов!

По уже сложившейся традиции, Дата Ёлка это наш финальный аккорд года:

🎄 Вместе с главными экспертами своих областей, подводим итоги года по самым бурным направлениям современного DS/ML

🏆 А тем, кто знатно фигачил в 2021, самое время вручить подарки за их вклад в Open Data Science на ODS Awards

🔥 Широченный набор подарков и возможностей их заполучить: -Самим победителям премии ODS Awards 2021

-Топ участникам каждого из наших осенних курсов

...а также участникам голосований и активностей Дата Ёлки Полное расписание конференции доступно на ODS.AI: https://ods.ai/events/data-elka-2021 Проголосовать в премии ODS Awards нужно здесь:

https://ods.ai/tracks/ods-awards-2021 И вступить в сообщество, к…

Play at primerlearning.org

Or on Google Play: https://play.google.com/store/apps/details?id=com.Primer.CatchtheCheaters For discussion and updates

- Discord: https://discord.gg/NbruaNW

- Twitter: @primerlearning

- Reddit: r/primerlearning Plush blobs and other merch: https://store.dftba.com/collections/primer

Support these videos on Patreon: https://www.patreon.com/primerlearning Made with Unity

https://github.com/Helpsypoo/PrimerUnity Made possible by support through Patreon:

Anthony Eufemio

Jon Mundle

Spline

Zachariah Richard Fournier

Vladimir Duchenchuk

Roy & BreAnna Steves

Shayn Osborn

Jeremy

Guguke

Anders Fjeldvær

Luc Cedric R.

Erik Broeders

Kairui Wang

Sean Barker

Eric Helps

Stevie Hr…

More than you ever wanted to know about Hamilton's rule:

https://users.ox.ac.uk/~grafen/cv/oseb.pdf For discussion and updates

- Twitter: @primerlearning

- Discord: https://discord.gg/NbruaNW

- Reddit: r/primerlearning Plush blobs and other merch: https://store.dftba.com/collections/primer

Support these videos on Patreon: https://www.patreon.com/primerlearning This video is presented under a Creative Commons Attribution-NonCommercial-ShareAlike License. More at:

https://creativecommons.org/licenses/by-nc-sa/4.0/ Made with Unity and Manim

https://github.com/Helpsypoo/PrimerUnity

https://www.manim.community Music by Mathieu Keith. For business inquiries: [email protected] Several other …

Bobby Lee is a comedian and co-host of TigerBelly and Bad Friends podcasts.

Please support this podcast by checking out our sponsors:– Brave: https://brave.com/lex– GiveWell: https://www.givewell.org/ and use code LEX– Linode: https://linode.com/lex to get $100 free credit– Onnit: https://lexfridman.com/onnit to get up to 10% off– Indeed: https://indeed.com/lex to get $75 creditEPISODE LINKS:Bobby’s Twitter: https://twitter.com/bobbyleeliveBobby’s Instagram: https://instagram.com/bobbyleeliveTigerBelly Podcast: https://youtube.com/c/TigerBellyTigerBelly Merch: https://thetigerbelly.comBad Friends Podcast: https://youtube.com/c/BadFriendsBad Friends Merch: http://badfriendsmerch.comPODCAST I…

Oliver Stone is a filmmaker with 3 Oscar wins and 11 Oscar nominations.

His films include Platoon, Wall Street, Born on the Fourth of July, Scarface, JFK, Nixon, Alexander, W, Snowden, and documentaries where he has interviewed Fidel Castro, Hugo Chavez, and Vladimir Putin.

Please support this podcast by checking out our sponsors:– Novo: https://novo.co/lex– Blinkist: https://blinkist.com/lex and use code LEX to get 25% off premium– SimpliSafe: https://simplisafe.com/lex and use code LEX– ExpressVPN: https://expressvpn.com/lexpod and use code LexPod to get 3 months free– NetSuite: http://netsuite.com/lex to get free product tourEPISODE LINKS:Oliver’s Twitter: https://twitter.com/TheOliverSt…

Glenn Loury is a professor of economics and social sciences at Brown University, and a prominent podcaster and social critic who speaks and writes about race, inequality, and social policy.

Please support this podcast by checking out our sponsors:– Lambda: https://lambdalabs.com/lex– LMNT: https://drinkLMNT.com/lex to get free sample pack– Coinbase: https://coinbase.com/lex to get $10 in free Bitcoin– ROKA: https://roka.com/ and use code LEX to get 20% off your first order– MasterClass: https://masterclass.com/lex to get 15% offEPISODE LINKS:Glenn’s Twitter: https://twitter.com/GlennLouryGlenn’s Substack: https://glennloury.substack.comGlenn’s YouTube: https://youtube.com/c/GlennLouryShowTh…

Saifedean Ammous is an Austrian economist and author of The Bitcoin Standard and The Fiat Standard.

Please support this podcast by checking out our sponsors:– GiveWell: https://www.givewell.org/ and use code LEX– Scale: https://scale.com/lex– Uncruise: https://uncruise.com/pages/lex– BiOptimizers: http://www.magbreakthrough.com/lex to get 10% off– Athletic Greens: https://athleticgreens.com/lex and use code LEX to get 1 month of fish oilEPISODE LINKS:Saifedean’s Twitter: https://twitter.com/saifedeanSaifedean’s Website: https://saifedean.comThe Bitcoin Standard podcast: https://saifedean.com/podcastBooks & resources mentioned:The Fiat Standard (book): https://amzn.to/3FmNfsyThe Bitcoin Stan…

Chris Mason is a professor of genomics, physiology, and biophysics at Cornell, doing research on the long-term effects of space on the human body.

He is the author of The Next 500 Years: Engineering Life to Reach New Worlds.

Please support this podcast by checking out our sponsors:– BetterHelp: https://betterhelp.com/lex to get 10% off– Grammarly: https://grammarly.com/lex to get 20% off premium– Magic Spoon: https://magicspoon.com/lex– Blinkist: https://blinkist.com/lex– Eight Sleep: https://www.eightsleep.com/lexEPISODE LINKS:Chris’s Twitter: https://twitter.com/mason_labChris’s Website: http://masonlab.net/Chris’s Company: https://onegevity.com/The Next 500 Years (book): https://amzn.to/…

David Buss is an evolutionary psychologist at UT Austin.

He is one of the founders of the field of evolutionary psychology.

His current research is on sex differences in mate selection, mate attraction, infidelity, and the emotions of jealousy, lust, and love.

On some podcast players you should be able to click the timestamp to jump to that time.

(00:00) – Introduction(08:04) – Sex vs Violence(14:48) – Mating strategies(26:44) – Social construct of beauty(30:38) – Evolution of mating evaluation(34:40) – Mating selection desires(39:52) – Difficulties of monogamy(46:16) – Importance of male appearance(48:52) – Importance of wealth(52:08) – Penis and breasts(55:59) – Fashion(58:58) – Body obje…

Grimes is a musician, artist, singer, songwriter, producer, and director.

Please support this podcast by checking out our sponsors:– Brave: https://brave.com/lex– Novo: https://novo.co/lex– Lambda: https://lambdalabs.com/lex– Public Goods: https://publicgoods.com/lex and use code LEX to get $15 off– Blinkist: https://blinkist.com/lex and use code LEX to get 25% off premiumEPISODE LINKS:Grimes’s Twitter: https://twitter.com/GrimezszGrimes’s Instagram: https://www.instagram.com/grimesPODCAST INFO:Podcast website: https://lexfridman.com/podcastApple Podcasts: https://apple.co/2lwqZIrSpotify: https://spoti.fi/2nEwCF8RSS: https://lexfridman.com/feed/podcast/YouTube Full Episodes: https://youtube…

Cristiano Amon is the CEO of Qualcomm, world-leader in 5G wireless communication and computation systems inside premium Android phones and other robots.

Please support this podcast by checking out our sponsors:– Scale: https://scale.com/lex– Truebill: https://truebill.com/lex– BiOptimizers: http://www.magbreakthrough.com/lex to get 10% off– Four Sigmatic: https://foursigmatic.com/lex and use code LexPod to get up to 60% off– Fundrise: https://fundrise.com/lexEPISODE LINKS:Cristiano’s Twitter: https://twitter.com/cristianoamonQualcomm’s Website: https://qualcomm.comPODCAST INFO:Podcast website: https://lexfridman.com/podcastApple Podcasts: https://apple.co/2lwqZIrSpotify: https://spoti.fi/2n…

Sara Walker is an astrobiologist and theoretical physicist.

Lee Cronin is a chemist.

This is a conversation and debate about alien life and alien civilizations.

On some podcast players you should be able to click the timestamp to jump to that time.

(29:35) – Assembly theory(52:03) – Math(1:03:45) – Communication with aliens(1:28:38) – Evolution of the universe(1:37:56) – Creating alien life(1:45:29) – Origin of life(1:52:29) – Before the Big Bang(1:59:22) – God(2:09:39) – Goal-directed behavior(2:27:37) – Time(2:35:54) – Free will and imagination(2:51:06) – UFO sightings(2:56:06) – Alien life forms debate(3:11:14) – Robots(3:20:29) – Love and emotion(3:38:55) – Beauty in science(3:49:06) – …

Skye Fitzgerald is a two-time Oscar-nominated documentary filmmaker, his films include Hunger Ward, Lifeboat, and 50 Feet from Syria.

Please support this podcast by checking out our sponsors:– Notion: https://notion.com/startups to get up to $1000 off team plan– Mizzen+Main: https://mizzenandmain.com and use code LEX to get $35 off– BetterHelp: https://betterhelp.com/lex to get 10% off– Onnit: https://lexfridman.com/onnit to get up to 10% off– Eight Sleep: https://www.eightsleep.com/lex and use code LEX to get special savingsEPISODE LINKS:Skye’s Twitter: https://twitter.com/spin_filmSkye’s Instagram: https://instagram.com/spin_filmHunger Ward (movie): https://hungerward.orgLifeboat (movie):…

Andrew Huberman is a neuroscientist at Stanford and host of the Huberman Lab Podcast.

Please support this podcast by checking out our sponsors:– Brave: https://brave.com/lex– LMNT: https://drinkLMNT.com/lex to get free sample pack– ROKA: https://roka.com/ and use code LEX to get 20% off your first order– Indeed: https://indeed.com/lex to get $75 credit– MasterClass: https://masterclass.com/lex to get 15% offEPISODE LINKS:Andrew’s YouTube: https://youtube.com/AndrewHubermanLabHuberman Lab Podcast: https://hubermanlab.comAndrew’s Twitter: https://twitter.com/hubermanlabAndrew’s Instagram: https://instagram.com/hubermanlabPODCAST INFO:Podcast website: https://lexfridman.com/podcastApple Podcas…

Michael Saylor is the CEO of MicroStrategy and a prominent holder and proponent of Bitcoin.

Please support this podcast by checking out our sponsors:– Scale: https://scale.com/lex– Coinbase: https://coinbase.com/lex to get $10 in free Bitcoin– Audible: https://audible.com/lex to get $9.95 a month for 6 months– NetSuite: http://netsuite.com/lex to get free product tour– SimpliSafe: https://simplisafe.com/lex and use code LEXEPISODE LINKS:Michael’s Twitter: https://twitter.com/saylorMicroStrategy: https://microstrategy.com/Michael’s Book: https://amzn.to/37J2iA0Book mentioned: https://amzn.to/3jwsIaPPODCAST INFO:Podcast website: https://lexfridman.com/podcastApple Podcasts: https://apple.co/2…

Rick Rubin is one of the greatest music producers of all time, working with many of the greats including Beastie Boys, Eminem, Metallica, LL Cool J, Kanye West, Slayer, Tom Petty, Johnny Cash, Dixie Chicks, Aerosmith, Adele, Danzig, Red Hot Chili Peppers, System of a Down, Jay-Z, Black Sabbath.

Please support this podcast by checking out our sponsors:– Lambda: https://lambdalabs.com/lex– Theragun: https://therabody.com/lex to get 30 day trial– ROKA: https://roka.com/ and use code LEX to get 20% off your first order– Onnit: https://lexfridman.com/onnit to get up to 10% off– ExpressVPN: https://expressvpn.com/lexpod and use code LexPod to get 3 months freeEPISODE LINKS:Rick’s Twitter: https:/…

Karl Deisseroth is a professor of bioengineering, psychiatry, and behavioral sciences at Stanford University.

Please support this podcast by checking out our sponsors:– BiOptimizers: http://www.magbreakthrough.com/lex to get 10% off– BetterHelp: https://betterhelp.com/lex to get 10% off– Notion: https://notion.com/startups to get up to $1000 off team plan– Blinkist: https://blinkist.com/lex and use code LEX to get 25% off premium– Magic Spoon: https://magicspoon.com/lex and use code LEX to get $5 offEPISODE LINKS:Karl’s Twitter: https://twitter.com/karldeisserothKarl’s Website: https://web.stanford.edu/group/dlabProjections (book): https://amzn.to/3NKmdiJPODCAST INFO:Podcast website: https:…

Chris Blattman is a professor at the University of Chicago studying the causes and consequences of violence and war.

Please support this podcast by checking out our sponsors:– Truebill: https://truebill.com/lex– Mizzen+Main: https://mizzenandmain.com and use code LEX to get $35 off– Grammarly: https://grammarly.com/lex to get 20% off premium– Indeed: https://indeed.com/lex to get $75 credit– Eight Sleep: https://www.eightsleep.com/lex and use code LEX to get special savingsEPISODE LINKS:Chris’s Twitter: https://twitter.com/cblattsChris’s Website: https://chrisblattman.comWhy We Fight (book): https://amzn.to/3702fjbPODCAST INFO:Podcast website: https://lexfridman.com/podcastApple Podcasts: h…

In “Just Tech: Centering Community-Driven Innovation at the Margins,” Senior Principal Researcher Mary L. Gray explores how technology and community intertwine and the role technology can play in supporting community-driven innovation and community-based organizations.

Dr. Gray and her team are working to bring computer science, engineering, social science, and communities together to boost societal resilience in ongoing work with Project Resolve.

She’ll talk with organizers, academics, technology leaders, and activists to understand how to develop tools and frameworks of support alongside members of these communities.

They also discuss how critical thinkers and makers from social movements…

In “Just Tech: Centering Community-Driven Innovation at the Margins,” Senior Principal Researcher Mary Gray explores how technology and community intertwine and the role technology can play in supporting community-driven innovation and community-based organizations.

Dr. Gray and her team are working to bring computer science, engineering, social science, and community together to boost societal resilience in ongoing work with Project Resolve.

She’ll talk with organizers, academics, technology leaders, and activists to understand how to develop tools and frameworks of support alongside members of these communities.

In this episode of the series, Dr. Gray talks with Dr. Tawanna Dillahunt, Ass…

In “Just Tech: Centering Community-Driven Innovation at the Margins,” Senior Principal Researcher Mary Gray explores how technology and community intertwine and the role technology can play in supporting community-driven innovation and community-based organizations.

Dr. Gray and her team are working to bring computer science, engineering, social science, and community together to boost societal resilience in ongoing work with Project Resolve.

She’ll talk with organizers, academics, technology leaders, and activists to understand how to develop tools and frameworks of support alongside members of these communities.

Together, they explore Patton’s learnings about the challenges of using AI in…

For Microsoft researchers, COVID-19 was a call to action.

Teams from across the Microsoft organizational chart pooled their unique expertise together under The New Future of Work initiative.

The results have informed product features designed to better support remote work and are now being used to help companies, including Microsoft, usher their workforces into a future of hybrid work.

In this episode of The New Future of Work series, Chief Scientist Jaime Teevan and Senior Principal Researcher Siddharth Suri explore the many ways people were impacted by work shifts during the COVID-19 pandemic.

The research that Siddharth Suri describes in this podcast was jointly done with Hana Wolf of Li…

For Microsoft researchers, COVID-19 was a call to action.

The reimagining of work practices had long been an area of study, but existing and new questions that needed immediate answers surfaced as companies and their employees quickly adjusted to significantly different working conditions.

Teams from across the Microsoft organizational chart pooled their unique expertise together under The New Future of Work initiative.

The results have informed product features designed to better support remote work and are now being used to help companies, including Microsoft, usher their workforces into a future of hybrid work.

They also talk about what an “anatomy of hybrid work” might look like and som…

For Microsoft researchers, COVID-19 was a call to action.

The reimagining of work practices had long been an area of study, but existing and new questions that needed immediate answers surfaced as companies and their employees quickly adjusted to significantly different working conditions.

Teams from across the Microsoft organizational chart pooled their unique expertise together under The New Future of Work initiative.

The results have informed product features designed to better support remote work and are now being used to help companies, including Microsoft, usher their workforces into a future of hybrid work.

They also explore why remote work came with a spike in phishing threats, what…

This episode is the third in our current series on PhD applications.

In this episode, we talk about what the PhD application process looks like after applications are submitted.

We start with a general overview of the t…

This episode is the second in our current series on PhD applications.

How do PhD programs in Europe differ from PhD programs in the US, and how should people decide between them?

In this episode, we invite Barbara Plank…

This episode is the first in our current series on PhD applications.

How should people prepare their applications to PhD programs in NLP?

In this episode, we invite Nathan Schneider (Professor of Linguistics and Compute…

In this episode, we discussed the Alexa Prize Socialbot Grand Challenge and this year's winning submission, Alquist 4.0, with Petr Marek, a member of the winning team.

Petr gave us an overview of their submission, the de…

What can NLP researchers learn from Human Computer Interaction (HCI) research?

We chatted with Nanna Inie and Leon Derczynski to find out.

We discussed HCI's research processes including methods of inquiry, the data anno…

In this episode, we talk with Lisa Beinborn, an assistant professor at Vrije Universiteit Amsterdam, about how to use human cognitive signals to improve and analyze NLP models.

We start by discussing different kinds of c…

On the show today, we interview Mouhamed Abdulla, a professor of Electrical Engineering at Sheridan Institute of Technology. Mouhamed joins us to discuss his study on remote teaching and learning in applied engineering. He discusses how he embraced the new approach after the pandemic, the challenges he faced and how he tackled them. Listen to find out more.

It is difficult to estimate the effect on remote working across the board. Darja Šmite, who speaks with us today, is a professor of Software Engineering at the Blekinge Institute of Technology. In her recently published paper, she analyzed data on several companies' activities before and after remote working became prevalent. She discussed the results found, why they were and some subtle drawbacks of remote working. Check it out!

In this episode, we speak with Abdullah Kurkcu, a Lead Traffic Modeler. Abdullah joins us to discuss his recent study on the effect of COVID-19 on bicycle usage in the US. He walks us through the data gathering process, data preprocessing, feature engineering, and model building. Abdullah also disclosed his results and key takeaways from the study. Listen to find out more. Click here for additional show notes on our website. Thanks to our sponsor!Astrato is a modern BI and analytics platform built for the Snowflake Data Cloud. A next-generation live query data visualization and analytics solution, empowering everyone to make live data decisions.

Today, we are joined by Jennifer Jacobs and Nadya Peek, who discuss their experience in teaching remote classes for a course that is largely hands-on. The discussion was focused on digital fabrication, why it is important, the prospect for the future, the challenges with remote lectures, and everything in between. Click here for additional show notes on our website! Thanks to our sponsor! https://neptune.ai/ Log, store, query, display, organize, and compare all your model metadata in a single place

Today, we are joined by Danae Ford, a Senior Researcher at Microsoft Research and an Affiliate Assistant Professor at the University of Washington. Danae discusses her work around remote work and its culminating impact on workers. She narrowed down her research to how COVID-19 has affected the working system of software engineers and the emerging challenges it brings. Click here to access additional show notes on our website! Thanks to our sponsor! Weights & Biases : The developer-first MLOps platform. Build better models faster with experiment tracking, dataset versioning, and model management.

In this episode, we interview Jonas Landman, a Postdoc candidate at the University of Edinburg. Jonas discusses his study around quantum learning where he attempted to recreate the conventional k-means clustering algorithm and spectral clustering algorithm using quantum computing. Click here to access additional show notes on our website!

K-means is widely used in real-life business problems. In this episode, Mujtaba Anwer, a researcher and Data Scientist walks us through some use cases of k-means. He also spoke extensively on how to prepare your data for clustering, find the best number of clusters to use, and turn the ‘abstract’ result into real business value. Listen to learn. Click here to access additional show notes on our website! Thanks to our sponsor! ClearML is an open-source MLOps solution users love to customize, helping you easily Track, Orchestrate, and Automate ML workflows at scale.

Many people know K-means clustering as a powerful clustering technique but not all listeners will be as familiar with spectral clustering. In today’s episode, Sybille Hess from the Data Mining group at TU Eindhoven joins us to discuss her work around spectral clustering and how its result could potentially cause a massive shift from the conventional neural networks. Listen to learn about her findings.

Building a fair machine learning model has become a critical consideration in today’s world. In this episode, we speak with Anshuman Chabra, a Ph.D. candidate in Computer Networks. Chhabra joins us to discuss his research on building fair machine learning models and why it is important. Find out how he modeled the problem and the result found.

In today’s episode, Jason, an Assistant Professor of Statistical Science at Duke University talks about his research on K power means. K power means is a newly-developed algorithm by Jason and his team, that aims to solve the problem of local minima in classical K-means, without demanding heavy computational resources. Listen to find out the outcome of Jason's study.

In this episode, Kyle interviews Lucas Murtinho about the paper "Shallow decision treees for explainable k-means clustering" about the use of decision trees to help explain the clustering partitions. Check out our website for extended show notes and images! Thanks to our Sponsors:ClearML is an open-source MLOps solution users love to customize, helping you easily Track, Orchestrate, and Automate ML workflows at scale.

Have you ever wondered how you can use clustering to extract meaningful insight from a time-series single-feature data? In today’s episode, Ehsan speaks about his recent research on actionable feature extraction using clustering techniques. Want to find out more? Listen to discover the methodologies he used for his research and the commensurate results.

Hollywood has officially fallen for the drama of tech startups!

Tune in to hear Jon Krohn review the small-screen adaptations of WeWork (WeCrashed), Uber (Super Pumped), and Theranos (The Dropout).

Additional materials:…

In this episode, the Director of Architecture at NVIDIA, Dr. Magnus Ekman, joins Jon Krohn to discuss how machine learning, including deep learning, can optimize computer hardware design.

The pair also review his excepti…

In this episode, Jon shares how the right music can power your productivity.

It's no secret that he's a big fan of 'deep work,' but this week, he opens up about the artists, sites, and playlists that propel his productiv…

In this episode, co-founder and CEO of Linea, Dr. Doris Xin, joins Jon Krohn to discuss how automating ML model deployment delivers groundbreaking change to data science productivity, and shares what it's like being the …

In this episode, Jon shares his habit of blocking out two hours in his mornings that are free from email and social media distractions.

Tune in to learn how this habit helps him deeply focus on his most delightful tasks …

Einblick co-founder and associate professor at MIT, Tim Kraska, joins Jon Krohn to discuss no-code collaboration tools for data science and uncovers the clever database and machine learning tricks under the hood of the v…

In this episode, Jon is back with another A.I.

model breakthrough!

He updates listeners on OpenAI's outstanding DALL-E 2 model.

The new natural language processing model churns out staggering visual examples of whatever …

Research Scientist at Meta AI, Dr. Noam Brown, joins Jon Krohn to discuss his award-winning no-limit poker-playing algorithms and the real-world implications of his game-playing A.I.

breakthroughs.

In this episode you w…

In this episode, Jon updates listeners on one of the industry's biggest breakthroughs to date –Google's new natural language processing model, PaLM.

The key innovation with PaLM is scaling up Google's Pathways modeling a…

In this episode, the MIT Press Director and Publisher, Dr. Amy Brand, joins Jon Krohn to discuss open-access publishing in data science and how to address the inequalities that exist for women and minorities in STEM.

In this episode, Jon reflects on the Chinese proverb: "The best time to plant a tree was 20 years ago.

The second best time is now."

He also challenges listeners to reflect on their long-term goals that have gone unfulfi…

In this episode, Jeremie Harris dives into the stirring topic of AI Safety and the existential risks that Artificial General Intelligence poses to humankind.

In this episode you will learn:• Why mentorship is crucial i…

In this episode, Jon speaks with the CEO of Hugging Face, Clem Delangue, about open-source machine learning and transformer architectures, while attending the ScaleUp:AI Conference in New York.

Additional materials: www…

In this episode, superstar data science YouTuber Tina Huang joins us to discuss what it's like to work at one of the world's largest tech companies, her strategies for efficient learning, and how best to prepare for a ca…

In this episode, Jon shares his daily technical exercise, which is part of an extensive habit tracking system that allows him to achieve more, create more structure within his day, and cut out bad habits.

By completing m…

May 16, 2022 podcastIn this episode I speak with Matt Swalley, Chief Business Officer of Omneky, an AI platform that generates, analyzes and optimizes personalized ad creatives at scale.

We speak about the way AI is used for generating customized recommendation and creating experiences with data aggregation and analytics.

respecting the privacy of individuals.

LinksGrow your business with personalized ads https://www.omneky.com/Data Science at Home Podcast (Live) https://www.twitch.tv/datascienceathome

May 16, 2022 podcastLet’s take a break and think about the state of AI in 2022.

In this episode I summarize the long report from the Stanford Institute for Human-Centered Artificial Intelligence (HAI)Enjoy!

If you want a new interactive experience, I am scheduling hands-on session on TwitchFeel free to drop by when there is a live session, and interact with me.

I’ll see you there!

Referenceshttps://spectrum.ieee.org/artificial-intelligence-indexhttps://www.twitch.tv/datascienceathome

May 16, 2022 podcastIn this episode I have a conversation with, Itai Bar-Sinai, CPO & Cofounder of Mona.

Why is AI monitoring so different from monitoring classic software?

How to reduce the gap between data science and business?

What is the role of MLOps in the data monitoring field?

With over 10 years of experience with AI and as the CPO and head of customer success at Mona, the leading AI monitoring intelligence company, Itai has a unique view of the AI industry.

April 21, 2022 podcastI am with Ander Steele, data scientist and mathematician with a passion for privacy and Shannon Bayatpur, product manager with a background in technical writing and computer science, from Tonic.ai.

We speak about data.

But all we say is authentic.

LinksTonic websiteCareer pageNeural networks for synthetic data

April 21, 2022 podcastIn this episode my friend and I speak about AI, batteries and automotive.

Dennis Berner, founder of Digitlabs has been operating in the field of automotive and batteries for a long time.

His point of views are absolutely a must to listen to.

Below a list of the links he mentioned in the show.

April 21, 2022 podcastThis is one episode where passion for math, statistics and computers are merged.

I have a very interesting conversation with Ravin, data scientist at Google where he uses data to inform decisions.

All opinions in this episode are his own and none of the companies he has worked for are represented.

and by Amethix TechnologiesAmethix use advanced Artificial Intelligence and Machine Learning to build data platforms and predictive engines in domain like finance, healthcare, pharmaceuticals, logistics, energy.

Amethix provide solutions to collect and secure data with higher transparency and disintermediation, and build the statistical models that will support your business.

We speak about machine learning applied to spatial data, spatial SQL and GIS (Geographic Information System).

Just tell them you came through Data Science at Home podcast.

and by Amethix TechnologiesAmethix use advanced Artificial Intelligence and Machine Learning to build data platforms and predictive engines in domain like finance, healthcare, pharmaceuticals, logistics, energy.

Amethix provide solutions to collect and secure data with higher transparency and disintermediation, and build the statistical models that will support your business.

ReferencesCarto https://carto.comSpatial Feature Engineering: https://geographicdata.science/book/intro.htmlCARTO Blog: https://carto.com/blog/Spati…

We use cookies on our website to give you the most relevant experience by remembering your preferences and repeat visits.

By clicking “Accept”, you consent to the use of ALL the cookies.

We answer such questions and much more in this wonderful episode with Triveni Gandhi, Senior Data Scientist and Shaun McGirr, AI Evangelist at Dataiku.

We cover topics about the history of data science, ethical AI and…This episode is brought to you by DataikuWith Dataiku, you have everything you need to build and deploy AI projects in one place, including easy-to-use data preparation and pipelines, AutoML, and advanced automation.

Get secure and private access to the internet by surfing nordvpn.com/DATASCIENCE or use coupon code DATASCIENCE and get a massive discount.

and by Amethix TechnologiesAmethix use advanced Artificial Intelligence and Machine Learning to build data platforms and pre…

April 1, 2022 podcastIn this episode I speak about AI and cloud automation with Leon Kuperman, co-founder and CTO at CAST AI.

Chat with meJoin us on Discord community chat to discuss the show, suggest new episodes and chat with other listeners!

Sponsored by Amethix TechnologiesAmethix use advanced Artificial Intelligence and Machine Learning to build data platforms and predictive engines in domain like finance, healthcare, pharmaceuticals, logistics, energy.

Amethix provide solutions to collect and secure data with higher transparency and disintermediation, and build the statistical models that will support your business.

Get secure and private access to the internet by surfing nordvpn.com/…

February 3, 2022 podcastThis is the last episode of the series “Embedded ML” and I made it for the bravest 🙂I speak about machine learning compiler optimization to a much greater detail.

Enjoy the episode!

Chat with meJoin us on Discord community chat to discuss the show, suggest new episodes and chat with other listeners!

Sponsored by Amethix TechnologiesAmethix use advanced Artificial Intelligence and Machine Learning to build data platforms and predictive engines in domain like finance, healthcare, pharmaceuticals, logistics, energy.

Amethix provide solutions to collect and secure data with higher transparency and disintermediation, and build the statistical models that will support your…

January 25, 2022 podcastIn this episode I speak about machine learning compilers, the most important tools to bridge the gap between high level frontends, ML backends and hardware target architectures.

There are several compilers one can choose.

Chat with meJoin us on Discord community chat to discuss the show, suggest new episodes and chat with other listeners!

Sponsored by Amethix TechnologiesAmethix use advanced Artificial Intelligence and Machine Learning to build data platforms and predictive engines in domain like finance, healthcare, pharmaceuticals, logistics, energy.

Amethix provide solutions to collect and secure data with higher transparency and disintermediation, and build the s…

January 20, 2022 podcastIn this episode I speak about neural network quantization, a technique that makes networks feasible for embedded systems and small devices.

There are many quantization techniques depending on several factors that are all important to consider during design and implementation.

Chat with meJoin us on Discord community chat to discuss the show, suggest new episodes and chat with other listeners!

Sponsored by Amethix TechnologiesAmethix use advanced Artificial Intelligence and Machine Learning to build data platforms and predictive engines in domain like finance, healthcare, pharmaceuticals, logistics, energy.

Amethix provide solutions to collect and secure data with hig…

We use cookies on our website to give you the most relevant experience by remembering your preferences and repeat visits.

By clicking “Accept”, you consent to the use of ALL the cookies.

January 10, 2022 podcastThis episode is the first of a series about Embedded Machine Learning.

I explain the requirements of tiny devices and how it is possible to run machine learning models.

Join us on Discord community chat to discuss the show, suggest new episodes and chat with other listeners!

Sponsored by Amethix TechnologiesAmethix use advanced Artificial Intelligence and Machine Learning to build data platforms and predictive engines in domain like finance, healthcare, pharmaceuticals, logistics, energy.

Amethix provide solutions to collect and secure data with higher transparency and disintermediation, and build the statistical models that will support your business.