Your request has been blocked due to a network policy.

If you're running a script or application, please register or sign in with your developer credentials here.

Additionally make sure your User-Agent is not empty and is something unique and descriptive and try again.

if you're supplying an alternate User-Agent string, try changing back to default as that can sometimes result in a block.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please file a ticket here.

Your request has been blocked due to a network policy.

If you're running a script or application, please register or sign in with your developer credentials here.

Additionally make sure your User-Agent is not empty and is something unique and descriptive and try again.

if you're supplying an alternate User-Agent string, try changing back to default as that can sometimes result in a block.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please file a ticket here.

Your request has been blocked due to a network policy.

If you're running a script or application, please register or sign in with your developer credentials here.

Additionally make sure your User-Agent is not empty and is something unique and descriptive and try again.

if you're supplying an alternate User-Agent string, try changing back to default as that can sometimes result in a block.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please file a ticket here.

Your request has been blocked due to a network policy.

If you're running a script or application, please register or sign in with your developer credentials here.

Additionally make sure your User-Agent is not empty and is something unique and descriptive and try again.

if you're supplying an alternate User-Agent string, try changing back to default as that can sometimes result in a block.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please file a ticket here.

Your request has been blocked due to a network policy.

If you're running a script or application, please register or sign in with your developer credentials here.

Additionally make sure your User-Agent is not empty and is something unique and descriptive and try again.

if you're supplying an alternate User-Agent string, try changing back to default as that can sometimes result in a block.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please file a ticket here.

Your request has been blocked due to a network policy.

If you're running a script or application, please register or sign in with your developer credentials here.

Additionally make sure your User-Agent is not empty and is something unique and descriptive and try again.

if you're supplying an alternate User-Agent string, try changing back to default as that can sometimes result in a block.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please file a ticket here.

Your request has been blocked due to a network policy.

If you're running a script or application, please register or sign in with your developer credentials here.

Additionally make sure your User-Agent is not empty and is something unique and descriptive and try again.

if you're supplying an alternate User-Agent string, try changing back to default as that can sometimes result in a block.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please file a ticket here.

Your request has been blocked due to a network policy.

If you're running a script or application, please register or sign in with your developer credentials here.

Additionally make sure your User-Agent is not empty and is something unique and descriptive and try again.

if you're supplying an alternate User-Agent string, try changing back to default as that can sometimes result in a block.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please file a ticket here.

Your request has been blocked due to a network policy.

If you're running a script or application, please register or sign in with your developer credentials here.

Additionally make sure your User-Agent is not empty and is something unique and descriptive and try again.

if you're supplying an alternate User-Agent string, try changing back to default as that can sometimes result in a block.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please file a ticket here.

Your request has been blocked due to a network policy.

If you're running a script or application, please register or sign in with your developer credentials here.

Additionally make sure your User-Agent is not empty and is something unique and descriptive and try again.

if you're supplying an alternate User-Agent string, try changing back to default as that can sometimes result in a block.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please file a ticket here.

Your request has been blocked due to a network policy.

If you're running a script or application, please register or sign in with your developer credentials here.

Additionally make sure your User-Agent is not empty and is something unique and descriptive and try again.

if you're supplying an alternate User-Agent string, try changing back to default as that can sometimes result in a block.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please file a ticket here.

Your request has been blocked due to a network policy.

If you're running a script or application, please register or sign in with your developer credentials here.

Additionally make sure your User-Agent is not empty and is something unique and descriptive and try again.

if you're supplying an alternate User-Agent string, try changing back to default as that can sometimes result in a block.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please file a ticket here.

Learn from Software Engineers and Discover the Joy of ‘Worse is Better’ Thinkingsource: unsplash.comRecently, I have had the fortune of speaking to a number of data engineers and data architects about the problems they face with data in their businesses. The main pain points I heard time and time again were:Not knowing why something brokeGetting burnt with high cloud compute costsTaking too long to build data solutions/complete data projectsNeeding expertise on many tools and technologiesThese problems aren’t new. I’ve experienced them, you’ve probably experienced them. Yet, we can’t seem to find a solution that solves all of these issues in the long run. You might think to yourself, ‘well …

Machine Learning unknowns that researchers struggle to understand — from Batch Norm to what SGD hidesContinue reading on Towards Data Science »

Moving data around can be slow. Here’s how you can squeeze every bit of performance optimization out of Python.Continue reading on Towards Data Science »

How to use Python and OSMnx to create beautiful visuals of global cities’ road networks.Continue reading on Towards Data Science »

Photo by Zdeněk Macháček on UnsplashData modeling is a process of creating a conceptual representation of the data and its relationships within an organization or system. Dimensional modeling is an advanced technique that attempts to present data in a way that is intuitive and understandable for any user. It also allows for high-performance access, flexibility, and scalability to accommodate changes in business needs.In this article, I will provide an in-depth overview of data modeling, with a specific focus on Kimball’s methodology. Additionally, I will introduce other techniques used to present data in a user-friendly and intuitive manner. One particularly interesting technique for modern…

A visual explanation of the math behind decision trees and gradient boostingContinue reading on Towards Data Science »

Navigating the Latest GenAI Announcements — July 2024A guide to new models GPT-4o mini, Llama 3.1, Mistral NeMo 12B and other GenAI trendsImage Created by Author with GPT-4o to represent different modelsIntroductionSince the launch of ChatGPT in November 2022, it feels like almost every week there’s a new model, novel prompting approach, innovative agent framework, or other exciting GenAI breakthrough. July 2024 is no different: this month alone we’ve seen the release of Mistral Codestral Mamba, Mistral NeMo 12B, GPT-4o mini, and Llama 3.1 amongst others. These models bring significant enhancements to areas like inference speed, reasoning ability, coding ability, and tool calling performanc…

Understanding its role in future AI, brain research, and consciousnessContinue reading on Towards Data Science »

Whether you are a Data Engineer, Machine Learning Engineer or Web developer, you ought to get used to this toolHow the antic sun shines upon PydAntic users. Image by Vladimir Timofeev under license to Ilija Lazarevic.There are quite a few use cases where Pydantic fits almost seamlessly. Data processing, among others, benefits from using Pydantic as well. However, it can be used in web development for parsing and structuring data in expected formats.Today’s idea is to define a couple of pain points and show how Pydantic can be used. Let’s start with the most familiar use case, and that is data parsing and processing.Let’s say we have a CSV file with a dozen columns and thousands of rows. The…

How to stop worrying and love the dataGenerated by the author using Midjourney Version 6Definition: eval (short for evaluation). A critical phase in a model’s development lifecycle. The process that helps a team understand if an AI model is actually doing what they want it to. The evaluation process applies to all types of models from basic classifiers to LLMs like ChatGPT. The term eval is also used to refer to the dataset or list of test cases used in the evaluation.Depending on the model, an eval may involve quantitative, qualitative, human-led assessments, or all of the above. Most evals I’ve encountered in my career involved running the model on a curated dataset to calculate key metri…

Modern RAG for modern models.Continue reading on Towards Data Science »

Feeling inspired to write your first TDS post? We’re always open to contributions from new authors.When we encounter a new question, topic, or challenge, taking the first step forward is often the most difficult part. That’s the moment where self-doubt kicks in, our existing knowledge feels hazy and inadequate, and procrastination often appears like the only acceptable choice.Our standout articles this week won’t magically solve every single challenge you’ll ever face as a data scientist or machine learning engineer, but what they do all offer is a pragmatic, action-focused roadmap for overcoming those initial hurdles in the learning process.From expanding your foundational statistics knowl…

An overview of how to leverage streaming using open source tools applied to building a simple agentic chat botThe model of our agentic application. We’ll show how you can build this with streaming so you can create a great user experience. Image by author.In this post we will go over how to build an agentic chatbot that streams responses to the user, leveraging Burr’s (I’m an author) streaming capabilities, FastAPI’s StreamingResponse, and server-sent-events (SSEs) queried by React. All of these are open source tools. This is aimed at those who want to learn more about streaming in Python and how to add interactivity to their agent/application. While the tools we use will be fairly specific…

You will not think about using Regular Expressions anymore.Continue reading on Towards Data Science »

How to extend Bounded Kernel Density Estimation to the 2D case? Let’s explore how to fix boundary bias around the unit disk.Photo by Leo_Visions on Unsplash0. IntroductionMonteCarlo IntegrationNumerical methods become essential when closed-form solutions for integrals are unavailable. While traditional numerical integration techniques like trapezoidal integration are highly effective for low-dimensional and smooth integrals, their efficiency diminishes rapidly, becoming clearly intractable as the dimensionality of the integrand increases.Unlike traditional techniques, the convergence rate of Monte Carlo methods, which leverage randomness to evaluate integrals, does not depend on the dimensi…

Looked at from another angle, there is much more noise than signal in financial data.

Another financial market application of LLMs might be synthetic data creation [4,8].

Then precious real market data could be employed to fine-tune the predictions and determine precisely the optimal speed to trade.

Financial market practitioners are often interested in extreme events, the times when trading strategies are more likely to experience significant gains or losses.

CitationFor attribution in academic contexts or books, please cite this work asRichard Dewey and Ciamac Moallemi, "Financial Market Applications of LLMs," The Gradient, 2024

All of these terms (“AI”, “gender”, and “bias”) can be somewhat overused and ambiguous.

A Short History of Studying Gender Bias in AIHere, I cover a very small sample of papers I’ve found influential studying gender bias in AI.

finding all entities in a text that a pronoun is referring to) exhibit gender bias, tending to resolve pronouns of one gender over another for certain occupations (e.g.

This article mainly focused on gender bias — and particularly, on binary gender.

AcknowledgementsThis post was originally posted on Art Fish IntelligenceCitationFor attribution in academic contexts or books, please cite this work asYennie Jun, "Gender Bias in AI," The Gradient, 2024@article{Jun2024bia…

As a general sequence model backbone, Mamba achieves state-of-the-art performance across several modalities such as language, audio, and genomics.

Here we’ll discuss:The advantages (and disadvantages) of Mamba (🐍) vs Transformers (🤖),Analogies and intuitions for thinking about Mamba, andWhat Mamba means for Interpretability, AI Safety and Applications.

The Mamba BlockLike a Transformer made up of stacked transformer blocks, Mamba is made up of stacked Mamba blocks as above.

The Mamba authors write, “the efficiency vs. effectiveness tradeoff of sequence models is characterised by how well they compress their state”.

Thanks to Gonçalo for reading an early draft, Jaden for the nnsight library …

We've just seen 3 prominent families of LLM usage in self-driving cars: Perception, Planning, and Generation.

The first wave of papers mentioning LLMs in Self-Driving Cars is from mid-2023, so let's give it some time.

Next StepsIf you want to get started on LLMs for self-driving cars, there are several things you can do:⚠️ Before this, the most important : If you want to keep learning about self-driving cars.

Author BioJérémy Cohen is a self-driving car engineer and founder of Think Autonomous, a platform to help engineers learn about cutting-edge technologies such as self-driving cars and advanced Computer Vision.

CitationFor attribution in academic contexts or books, please cite this work…

Beyond the requirements of semantic similarity, there are no constraints on what embedding must be assigned for a given text input.

What if someone hacks into the database and gains access to all your text embedding vectors – would this be bad?

From text to embeddings...back to textThe problem of recovering text from embeddings is exactly the scenario we tackle in our paper Text Embeddings Reveal As Much as Text (EMNLP 2023).

Scaling and future workThe fact that text embeddings can be perfectly inverted raises many follow-up questions.

CitationFor attribution in academic contexts or books, please cite this work asJack Morris, "Do text embeddings perfectly encode text?

If your model latches on to these during training, it will appear to work well, but may not work on new data.

This happens when the model training pipeline has access to information it shouldn’t have access to, particularly information that confers an advantage to the model.

This means that knowledge of the test data is implicitly entering the model training pipeline, even if it is not explicitly used to train the model.

Well, this is a common thing to do, but if you’re developing a model iteratively and using the same test set to evaluate the model after each iteration, then you’re basically using that test set to guide the development of the model.

CitationFor attribution in academic cont…

Evolution of single-cell sequencing over timeHaving explored the panorama of single-cell sequencing, let us now delve into the role of deep learning in the context of single-cell sequencing.

Deep Learning on single-cell sequencingDeep learning is increasingly employed in single-cell analysis due to its capacity to handle the complexity of single-cell sequencing data.

The deep learning approach, however, autonomously captures relevant characteristics from single-cell sequencing data, addressing the heterogeneity between single-cell sequencing experiments, as well as the associated noise and sparsity in such data.

As we explore the reasons behind using deep learning in single-cell sequencing …

In order to obtain a license or permit from FERC, hydroelectric dam operators must submit detailed plans and studies demonstrating that their facility meets regulations.

Typically, a hydroelectric dam requires lots of space to store water on one side of it, which means they tend to be located away from population centers.

Enter Computer VisionSome organizations are exploring the use of computer vision and machine learning to significantly automate fish counting.

The annotated images are then used to train a machine learning model.

CitationFor attribution of this in academic contexts or books, please cite this work as:Kevin McCraney, "Salmon in the Loop", The Gradient, 2023.

In recent work with computer networking and machine learning collaborators from ETH Zürich, we studied the applicability of neural algorithmic reasoning in computer networking [27].

Our proposal, the neural algorithmic reasoning blueprint [32], aims to bridge this divide by neuralising the target algorithm.

Neural Algorithmic Reasoning with Causal Regularisation.

Neural Algorithmic Reasoning.

CitationFor attribution in academic contexts or books, please cite this work asPetar Veličković, "Neural Algorithmic Reasoning", The Gradient, 2023.

This community has developed an extensive vocabulary around theories of AI safety and alignment, many first introduced as detailed blog posts in forums like LessWrong and AI Alignment Forum.

One such idea that is useful for contextualizing technical alignment work — and is perhaps the more formal version of what Bostrom was referring to — is the concept of intent alignment.

Anthropic’s product marketing pages are plastered with notes and phrases about their alignment work —“HHH” is also Claude's biggest selling point.

The site uses the phrasing “AI Safety” instead of “AI Alignment” in the title, but the article itself proceeds to use “safety” and “alignment” interchangeably without differen…

Apple has an ideal playground for innovating in one of the hottest areas of the next wave of generative AI: on-device multimodal models.

However, most of Apple’s efforts in small on-device models have been somewhat underwhelming.

Recently, Apple released what I consider its most impressive work in small, on-device foundation models with the publication and open source release of 4M-21, a multimodal model that work seamlessly across 21 modalities!

The work definitely signals the path for Apple on-device model strategy and the large number of modalities is quite shocking.

However, this work builds on a previous research work that Apple published months ago with the release of its 4M model.

Created Using IdeogramIn this issue:An overview of procedural memory in autonomos agents.

A review of Microsoft’s JARVIS-1, a memory-agumented LLM agents.

💡 ML Concept of the Day: Procedural Memory in Autonomous AgentsTo finalize this segment about memory in autonomous agents, today we will dive into procedural memory as a form of long-term memory.

As it names indicates, procedural memory is related to procedures and actions an LLM agent will perform in a given environment.

Procedural memory has a strong parallel to human cognition as we learn to perform actions since we are babies.

In this guest post, Stephen Batifol from Zilliz discusses how to build agents capable of tool-calling using LangGraph with Llama 3 and Milvus.

Example 1: Questions requiring factual answers might be routed to a document retrieval node searching a pre-indexed knowledge base (powered by Milvus).

General ideas for AgentsReflection – The self-correction mechanism is a form of reflection where the LangGraph agent reflects on its retrieval and generations.

Tool use – The LangGraph agent’s control flow incorporates specific nodes for various tools.

for output in app.stream(inputs):for key, value in output.items():pprint(f"Finished running: {key}:")pprint(value["generation"])```ConclusionIn this bl…

You can subscribe to The Sequence below:📝 Editorial: What a Week for Foundation ModelsBuilding high-quality, large-scale foundation models is hard.

Small Models: 500M-10B parameter models that can run inference on commodity hardware, IoT, or mobile devices.

As you can see, the releases emphasize the domain specialization and small model trends.

Even by the crazy standards of the generative AI market, last week was a remarkable week in terms of model releases.

Text to Image at PinterestPinterest discusses some details about Canvas, its text-to-image model —> Read more.

Data scientists and engineers face challenges in building and maintaining feature pipelines, ensuring data consistency and freshness, and achieving real-time performance.

In this presentation, Sergio Ferragut, Principal Developer Advocate at Tecton, will discuss how declarative frameworks are transforming production AI.

These frameworks enable seamless collaboration between data scientists and ML engineers, simplifying the creation of production features.

Discover how to improve feature reusability, eliminate training-serving skew, and simplify complex feature development.

Key topics include:Seamless collaboration between data scientists and ML engineers Reuse features and eliminate trainin…

By reasoning, we refer to the ability to decompose a task into smaller subsets and solve those individually.

Chain-of-Thought, Tree-of-Tought, Skeleton-of-Thought, and Reflexion are some of the recent techniques that have tackled reasoning capabilities in LLMs.

In the last couple of years, we have seen models to perform extremely well in specific reasoning techniques, but they failed to generalize across domains.

HUSKY is an open-source language agent designed to handle a variety of complex tasks involving numerical, tabular, and knowledge-based reasoning.

Unlike other agents that focus on specific tasks or use proprietary models, HUSKY operates within a unified framework to manage diverse …

Created Using IdeogramIn this issue:An overview of semantic memory in autonomous agents.

💡 ML Concept of the Day: Semantic Memory in Autonomous AgentsIn the last few issues of this newsletter we have been exploring different memory architectures in autonomous agents.

Semantic memory is, arguably, the most common memory structure in LLMs.

From a conceptual standpoint, semantic memory in autonomous agents draws inspiration from cognitive psychology and is centered around the argument that facts have meaning.

As it name indicates, the core idea of semantic memory structures is to capture knowledge of the agent’s knowledge about the world and itself.

Data scientists and engineers grapple with the complexity of building and maintaining feature pipelines, ensuring consistency, data freshness, and achieving real-time performance.

Sergio will demonstrate how Tecton’s declarative framework streamlines collaboration between data scientists and ML engineers, automates feature materialization, and handles diverse data types.

Discover strategies to promote feature reusability, eliminate training-serving skew, and simplify complex feature creation.

He’ll also discuss how these frameworks drive automation of production-ready pipelines, accelerating AI projects and making AI-powered applications smarter.

Key topics include:Overcoming traditional fe…

Generative Teaching for AgentsMicrosoft Research published a paper unveiling AgentInstruct, an agentic framework for creating syntethic data.

Specifically, AgentInstruct focuses on datasets used for instruction tuning of base models —> Read more.

The core of VBase is based on a property called relaxed monotonicity that enables the unification of the different data types models —> Read more.

MInferenceMicrosoft released some demos of its MInference method for optimizing LLM inference performance —> Read more.

AutoGen ModelsMicrosoft AutoGen added support for non OpenAI models —> Read more.

In the context of AI, a compiler is responsible for translating a neural network architecture into executable code in a specific hardware topology.

Those two areas: model and hardware architectures, have been an explosion in innovation, regularly making AI compilers obsolete.

The challenges in AI compilation are many, from hardware acceleration to computation and memory efficiency.

Microsoft Research has been at the forefront of the AI compiler research, and recently, they unveiled a quartet of cutting-edge AI compilers, each tailored to address specific challenges in the realm of deep neural networks (DNNs).

The list includes the following compilers:· Rammer: For parallelism· Roller: For c…

Created Using Ideogram💡 ML Concept of the Day: Understanding Episodic Memory in Autonomous AgentsIn the previous issue of our series about autonomous agents we introduced the concept of long-term memory and its different forms.

Today, we would like to dive into one of those variations known as episodic memory.

In cognitive science, episodic memory enables humans to recall where they've been, what they've experienced, and the actions they've taken in different situations.

It helps individuals track the world beyond their immediate perception and learn from past experiences.

In the context of LLM agents, episodic memory involves three main processes: capturing and encoding experiences interna…

You can subscribe to The Sequence below:📝 Editorial: Apple Goes Small and Super MultimodalApple has been late to the generative AI game, but lately, it has been pushing the research agenda quite hard.

Apple has an ideal playground for innovating in one of the hottest areas of the next wave of generative AI: on-device multimodal models.

🔎 ML ResearchA Small Any-to-Any ModelApple Research published a paper introducing 4M-21, a small multimodal model optimized for tens of different tasks and modalities.

The post discusses the requirements and challenges to new evaluations and its relevance for improving foundation models —> Read more.

🤖 Cool AI Tech ReleasesGemma2Google released 9B and 27B ver…

Created Using IdeogramImagine the following scenarios in an LLM application:Client A sends requests averaging 4k tokens each.

Client B sends requests averaging 200 tokens each.

Should the requests from both clients follow be served by the same LLM resources.

The answer seems obviously no as the second client requires much less resources than the first client.

Current LLM serving systems rely on a commonly used method for handling incoming requests based on the First-Come-First-Serve (FCFS) approach.

Created Using IdeogramIn this Issue:Different types of long-term memory in autonomous agents.

Microsoft’s LONGMEM research to enable long-term memory in LLMs.

💡 ML Concept of the Day: Long-Term Memory and Autonomous AgentsIn the previous issue of this series we discussed short-term memory in autonomous agents fundamentally powered by LLM context-windows.

Long-term memory represents a natural complement to short-term memory by persisting information in a way that can be used beyond the lifecycle of a conversation.

From a conceptual standpoint, there are three fundamental forms of long-term memory in autonomous agents:

Among them, we distinguish a group of Data Parallelism methods that allow full sharding of weights, gradients, and optimizer states.

During the optimizer step, we update only those weights and optimizer parameters that belong to the particular GPU.

Here are the advantages:FSDP combines multiple layer parameters into a single FlatParameter that gets split during sharding.

— Even if all module input tensors pass gradients, there is no guarantee that backward_hook will run after the .grad of all tensors is computed.

Now we have everything we need for concurrent communications and computations in the backward pass.

Video highlight detection is a crucial task in the field of video content analysis, aiming to automatically identify and extract the most…Continue reading on SyncedReview »

AI research aims to develop autonomous agents that can adaptively operate in complex social environments. Multi-agent reinforcement…Continue reading on SyncedReview »

Continue reading on SyncedReview »

Continue reading on SyncedReview »

Continue reading on SyncedReview »

There is a growing interest in leveraging large language models (LLMs) for various software engineering tasks, including code generation…Continue reading on SyncedReview »

Continue reading on SyncedReview »

Continue reading on SyncedReview »

Continue reading on SyncedReview »

Cultural accumulation has been a driving force behind the open-ended and diverse advancements in human capabilities throughout history. By…Continue reading on SyncedReview »

Sleep is a complex physiological process evaluated through various methods that record electrical brain activity, cardiac activity, and…Continue reading on SyncedReview »

Gboard, Google’s keyboard for mobile devices, utilizes statistical decoding to offer a smooth typing experience.Continue reading on SyncedReview »

AI Pioneers Gather at BAAI 2024: Unveiling Innovations in Large-Scaled AI Models for Language, Multimodal, Embodied, Bio-Computing, and FlagOpen 2.0The 6th annual BAAI Conference hosted by Beijing Academy of Artificial Intelligence (BAAI), commenced today in Beijing, marking a significant gathering for global AI insiders.This premier event, themed “Global Vision, Ideas in Collision, Leading Cutting-Edge Innovations,” brings together top researchers and industry leaders from around the world to share their latest findings and discuss the future of artificial intelligence.This year’s BAAI 2024 conference showcases an esteemed roster of speakers, featuring Turing Award laureate Andrew Chi-Chih…

AI is experiencing a transformative shift with significant advancements driven by the integration of multiple large language models (LLMs)…Continue reading on SyncedReview »

Как это определить и как предсказывать?

- Да... упс.. правда, они же уже определяют целевой признакВовремя не выкрутился, пришел фидбек, что я не понимаю разницу между обычными признаками (а.к.а.

Не, не делал, только DDP?

NVIDIA ищет единорогов, крутых и в рисече, и в инженерии.

Вакансия в Лондоне была, и в итоге они взяли моего бывшего коллегу: он уже в Лондоне и с чуть большим опытом в рекомендашках, явно лучше подходит.

ChatGPT 4 был значительно улучшен по сравнению с ChatGPT 3.5, что делает его более мощным инструментом.

Вам тоже надо учиться — учиться выстраивать взаимоотношение с ChatGPT, учиться общаться с ним.

Помните, вы всегда можете уточнить любую строчку в ответе ChatGPT и, в большинстве случаев, получить исчерпывающий ответ, который, вероятнее всего, обогатит вас новыми знаниями.

Вот несколько идей:Откройте с ChatGPT новый чат и в нём отправьте запрос в другой форме, желательно с новыми деталями.

И поэтому, когда с ChatGPT не удаётся что-то сделать с первого раза за 2–5 минут, возникает возмущение: “Ну, как так?!”.

Или кликбейт — и это в Nature?

Статья ниже — подробный разбор достаточно сложного топика, и в некоторых моментах нужно будет сосредоточиться и вдумчиво читать.

Они же пишут код в помощь разработчикам, да и в целом помогают решать разного рода проблемы.

Однако может так получиться, что сет долгое время не выпадает — и не потому, что игроки проворонили, а потому, что его действительно просто нет на столе.

Надеемся, что после прочтения этой статьи стало ясно, что ошибки нейросетей — это не баг, это фича.

Также присоединяйтесь к моему телеграм каналу AI[ex]Time , где я пишу про машинное обучение, NLP, LLM и в том числе про агентов.

LLaVaВ качестве LLM для генерации текстового ответа используется LLaMA, которая декодирует эмбеддинги (то же, что и векторы) картинок и входного текста в ответ.

На вход модель получает картинку и запрос от пользователя (Normal prompt) и на выходе должна дать ответ (Response).

Модель таким образом от решения задачи напрямую переходит к рассуждению по шагам, что в некотором смысле и является декомпозицией задачи.

И это улучшает качество работы модели — в том числе и ризонинга.

Мы помним, что в начале 2023-го продуктом уже пользовалось больше 100 млн человек в месяц.

У самых любознательных читателей может возникнуть вопрос: а как это вообще работает?

Насколько нам известно, это первый раз, когда модель такого масштаба обучается на синтетических данных, а не на произведенных человеком.

Использование такой модели дешевле в 3 раза на текст из промпта, и в 2 раза на генерируемые токены (их обычно меньше).

Но вот как с ними обращаться, когда использовать и как комбинировать — это уже решает AI по контексту диалога.

In machine learning projects, achieving optimal model performance requires paying attention to various steps in the training process. But before focusing on the technical aspects of model training, it is important to define the problem, understand the context, and analyze the dataset in detail. Once you have a solid grasp of the problem and data, […]

The post Tips for Effectively Training Your Machine Learning Models appeared first on MachineLearningMastery.com.

Reinforcement Learning (RL) is a type of machine learning. It trains an agent to make decisions by interacting with an environment. This article covers the basic concepts of RL. These include states, actions, rewards, policies, and the Markov Decision Process (MDP). By the end, you will understand how RL works. You will also learn how […]

The post Principles of Reinforcement Learning: An Introduction with Python appeared first on MachineLearningMastery.com.

Deep learning is a subset of machine learning that has become a cornerstone in many technological breakthroughs. At the core of deep learning, it’s a model inspired by the human brain, which we call a neural network. Contrary to the traditional machine learning model, deep learning can automatically find feature representations from data. That’s why […]

The post 5 Tips for Getting Started with Deep Learning appeared first on MachineLearningMastery.com.

Feature engineering is an important step in the machine learning pipeline. It is the process of transforming data in its native format into meaningful features to help the machine learning model learn better from the data. If done right, feature engineering can significantly enhance the performance of machine learning algorithms. Beyond the basics of understanding […]

The post Tips for Effective Feature Engineering in Machine Learning appeared first on MachineLearningMastery.com.

Using machine learning to solve real-world problems is exciting. But most eager beginners jump straight to model building—overlooking the fundamentals—resulting in models that aren’t very helpful. From understanding the data to choosing the best machine learning model for the problem, there are some common mistakes that beginners often tend to make. But before we go […]

The post 5 Common Mistakes in Machine Learning and How to Avoid Them appeared first on MachineLearningMastery.com.

Photography has been around for more than a century. There are many old photos around, and probably your family has some, too. Limited by the camera and film of the time, you may have photos of low resolution, blurry, or with folds or scratches. Restoring these old photos and making them like new ones taken […]

The post Stable Diffusion Project: Reviving Old Photos appeared first on MachineLearningMastery.com.

Today’s digital landscape has never been so diverse. Every individual and company selects their preferred tools and operating systems, creating a diverse technological system. However, this diversity often leads to compatibility issues, making it hard to ensure application performance across different environments. This is where Docker plays a key role as an indispensable tool for […]

The post The Ultimate Beginner’s Guide to Docker appeared first on MachineLearningMastery.com.

Stable Diffusion has taken the AI art world by storm, empowering users to generate stunning and imaginative visuals with just a few text prompts. This opens exciting possibilities for creatives, including crafting impactful commercial posters. In this post, we’ll delve into using Stable Diffusion to design a compelling poster for a product. After finishing this […]

The post Stable Diffusion Project: Commercial Poster appeared first on MachineLearningMastery.com.

Introduction Here’s a something that new machine learning practitioners figure out almost immediately: not all datasets are created equal. It may now seem obvious to you, but had you considered this before undertaking machine learning projects on a real world dataset? As an example of a single class vastly outnumbering the rest, take for instance […]

The post 5 Effective Ways to Handle Imbalanced Data in Machine Learning appeared first on MachineLearningMastery.com.

Introduction Choosing the right machine learning model for your data is of major importance in any data science project. The model you select will have a significant impact on the insights you derive from your data, and ultimately determine the usefulness of a project. In this article, we aim to provide practical tips to help […]

The post Tips for Choosing the Right Machine Learning Model for Your Data appeared first on MachineLearningMastery.com.

Many people write in their jobs. Not everyone is a novel writer; some write technical documentation, business plans, news articles, and even blog posts. In those writings, illustrations are not essential but often good to have. They are decorations, interpretations, or visual explanations of the text. However, you probably do not want to spend too […]

The post Stable Diffusion Project: Creating Illustration appeared first on MachineLearningMastery.com.

If you are a machine learning student, researcher, or practitioner, it is crucial for your career growth to have a deep understanding of how each algorithm works and the various techniques to enhance model performance. Nowadays, many individuals tend to focus solely on the code, data, and pre-trained models, often without fully comprehending the machine […]

The post 5 Free Books on Machine Learning Algorithms You Must Read appeared first on MachineLearningMastery.com.

Stable Diffusion is a powerful tool that helps you generate pictures. It is fun to play with the generative AI tool. But it would be useful if the tool could help you in a real job. In this post, you will see how you can leverage the power of Stable Diffusion to work on something […]

The post Stable Diffusion Project: Word Art appeared first on MachineLearningMastery.com.

As a data professional, you should also know how to build predictive models with machine learning to solve business problems. And if you’re interested in machine learning, you’re probably also looking for the best resources to get going. Well, you can always choose a self-paced online course that best aligns with your learning preferences. But […]

The post 5 Free YouTube Channels Dedicated to Machine Learning Education appeared first on MachineLearningMastery.com.

If you’re looking to make a career in data science, you probably know that machine learning is one of the most in-demand skills. Whether you are a beginner looking to break into the field or an experienced professional aiming to level up your expertise, selecting the right machine learning course is super important. So how […]

The post Tips for Choosing the Right Machine Learning Course appeared first on MachineLearningMastery.com.

I would extend it to reality is complicated, relative to code, and in robotics you’re often pushing a messy reality into an abstraction nice enough for code to act on it.

Robotics research relies on building new bridges between reality and software, but that happens outside of robotics too.

Any software that interfaces with reality will have imperfect knowledge of that reality.

However, that means all the messiness of reality is coming for a field that historically does a bad job at considering reality.

I consider the world of bits to be as much a part of reality as the world of atoms.

Most people will have more fun if they solve puzzles than if they don’t, but you don’t have to solve puzzles quickly to have fun.

I’m still going to explain the solving strategies I’ve learned, but puzzle solving is really an activity where you learn by doing.

Puzzle solving often involves relating two parts of the puzzle together.

Search everythingHonestly, a lot of puzzle solving is about taking random parts of the puzzle and throwing them into a search engine.

Bringing This TogetherTo showcase these strategies together, here is a puzzle I remember speedrunning especially quickly: The Three Little Pigs from Hunt 20 2.1 Puzzle Hunt.

This means we can reduce all the crew battle outcomes down to a single \(n \times n\) matrix, which I’ll call the crew battle matrix.

So I Wrote Some Python Code to Compute \(f\) for Arbitrary Crew BattlesLet’s consider the RPS crew battle again.

Here’s the matchup matrix:and here’s what my code outputs for the crew battle matrix, assuming optimal play.

But also, this suggests that crew battle strategy really isn’t that important in the first place!

If your crew loses, you don’t get to blame bad crew battle strategy.

This has spoilers for MIT Mystery Hunt 2024.

I hunted with teammate again this year, because there is nothing quite like writing a Mystery Hunt to forge friends through fire.

I needed to spend some to avoid hitting the vacation cap, and what better time than Mystery Hunt?

If you were forced to pick how a Mystery Hunt runs long, I think most people would pick the “too many puzzles” side of Mystery Hunt 2024 over the “too difficult puzzles” side of Mystery Hunt 2023.

So did Spoilr, the codebase for Mystery Hunt 2022, and tph-site from Mystery Hunt 2023.

In August 2020, I wrote a post about my AI timelines.

Diffusion based image augmentation has been shown to improve robot learning, and Anthropic has based a lot of its branding on constitutional AI and “RL from AI feedback”.

I don’t like AI Twitter for reasons I’ve explained here, but I especially do not AI twitter post-ChatGPT.

In AI, models can never do everything people claim they can, but what the models can do is ever-growing and never slides backward.

The bull is to say that we can figure out how to scale models, and scaled up models will solve all the other hard problems.

You know how much time I have spent studying how to squeeze Neopoints water out of the Neopets stone?

I’d say you can expect to get about 25k NP a day, depending on how many NP rewards you get.

Trudy’s Surprise also gives items for 7 day streaks, but these items are usually junk and not worth anything.

When you win against CPU opponents, you earn a small amount of Neopoints and an item drop.

Or your needs to…see someone do a lot of research into things that don’t matter?

From these excerpts, sociotechnical AI seems to have something to do with being people-centred, having social-purpose, understanding and reducing risks, building trust, widening opportunities, centring rights and values, being accountable for claims, and taking a system-wide view on AI.

To do this we will need more than technical AI research; and that is the role of sociotechnical AI research.

This expresses a hope for significant public benefit, but again, the burden of proof lies with AI designers, and so requires new forms of sociotechnical AI research.

Further developing all these varied forms for social and community involvement is another focus area for sociotechnical AI.

Sociotechnic…

Generative AI is a membership based concept, so it is defined by the set of approaches and applications that get put under that name.

I think this is one part of the intrigue of generative AI for science.

So maybe generative AI applications are part of this drive for a post-theory mode of science.

This generative science, it is hoped, can add vigour into the sciences, especially in cross-disciplinary ways and provide broad-based benefit.

Generative AI for Science presents opportunities for new ways of doing theory and renewed vigour across our sciences.

Keynote at the Stanford Human-centered AI 2023 Fall Conference on New Horizons in Generative AI: Science, Creativity, and Society.

Our conference today on new horizons in generative AI, invites us to think of the frontier of research and innovation.

These stories will expose some features of the sociotechnical foundations of generative AI that is my underlying message and call to action.

What he doesn’t know yet, is that this book will become a cornerstone of the field and industry of weather forecasting.

There is a specific and firm model for responsibility that is built on taking a sociotechnical approach to generative AI and our work.

At Youtube, the relationships between system parameters and metrics often seem simple — straight-line models sometimes fit our data well.

To find optimal values of two parameters experimentally, the obvious strategy would be to experiment with and update them in separate, sequential stages.

We address uncertainty in estimates $\widehat \beta$ of model coefficients with subsampling, our usual strategy for variance estimation in online experiments[12].

If our experiments were smaller then we might need to model our metrics with Poisson, Binomial or other GLMs instead.

We sometimes use $L^1$ regularization when we fit models to our live experiment data to select the most important non-linear c…

Both of these assumptions seem reasonable and general and we can quickly see where Gaussian distribution should start popping out.

Since Gaussian distribution is fully defined by just two numbers - mean and variance, we can, by collecting enough data rather precisely estimate these values.

Below I simulate 400x200=80000 independent pixels, each taking a random value between zero and one [code available here].

These distributions we often refer to as "fat tails" [for putting much more "weight" in the tail of the distribution, as in outside of the "central" part] are really weird and problematic.

Let's get back to why we see Gaussian distribution anywhere at all?

So it should not be surprising that hubris in science, displayed by some scientists strikes me equally as arrogant and irritating as the claims of religious leaders.

With the day of judgment established and imminent we are now ready to spread the gospel of modern day religion and profit.

AGI (artificial general intelligence) isn't even a well defined thing.

We can hardly even imagine such a thing, which is great for religious purposes, because everyone can have their own "AGI" to be scared of.

That we should never question the technocrats, because technology is the only and natural way of making our lives better.

NeurIPS 2023, arguably this year’s biggest AI conference takes place in two weeks from Dec 10–16 in New Orleans.

Here are the main trends I observed:Most NLP work at NeurIPS is related to large language models (LLMs).

Papers studied different aspects of in-context learning such as whether it persists during training and using a Bayesian perspective.

Lexinvariant Language Models (Huang et al.).

Large Language Models Are Latent Variable Models: Explaining and Finding Good Demonstrations for In-Context Learning (Wang et al.).

Sparked by progress in Large Language Models (LLMs), there’s a lot of chatter recently about AGI, its timelines, and what it might look like.

In particular, when I am asked about what I think AGI will look like, I personally like to point to self-driving.

AGISo these are the broad strokes of what I think AGI will look like.

And it looks a lot more like self-driving, the part of our economy that is currently speed-running the development of a major, society-altering automation.

I personally very much look forward to what all the equivalents of that might be with AGI.

Include the markdown at the top of your GitHub README.md file to showcase the performance of the model.

Badges are live and will be dynamically updated with the latest ranking of this paper.

Include the markdown at the top of your GitHub README.md file to showcase the performance of the model.

Badges are live and will be dynamically updated with the latest ranking of this paper.

Technology mapping involves mapping logical circuits to a library of cells.

Traditionally, the full technology library is used, leading to a large search space and potential overhead.

Motivated by randomly sampled technology mapping case studies, we propose MapTune framework that addresses this challenge by utilizing reinforcement learning to make design-specific choices during cell selection.

The effectiveness of MapTune is evaluated on a wide range of benchmarks, different technology libraries and technology mappers.

The improvements are consistently remained for four different technologies (7nm, 45nm, 130nm, and 180 nm) and two different mappers.

Empirically-determined scaling laws have been broadly successful in predicting the evolution of large machine learning models with training data and number of parameters.

In certain applications, storage space is an important constraint, and data format needs to be chosen carefully as a consequence.

Given a dataset of digital images, the number of bits $L$ to store each of them can be further reduced using lossy data compression.

In order to capture this trade-off and optimize storage of training data, we propose a `storage scaling law' that describes the joint evolution of test error with sample size and number of bits per image.

At given storage, models trained on optimally compressed ima…

As a fundamental task in long-form video understanding, temporal action detection (TAD) aims to capture inherent temporal relations in untrimmed videos and identify candidate actions with precise boundaries.

Over the years, various networks, including convolutions, graphs, and transformers, have been explored for effective temporal modeling for TAD.

However, these modules typically treat past and future information equally, overlooking the crucial fact that changes in action boundaries are essentially causal events.

Inspired by this insight, we propose leveraging the temporal causality of actions to enhance TAD representation by restricting the model's access to only past or future context.…

Low-Rank Adaptation, also known as LoRA, has emerged as a prominent method for parameter-efficient fine-tuning foundation models by re-parameterizing the original matrix into the product of two low-rank matrices.

In this paper, we propose LoRA-Pro to bridge this performance gap.

We reveal that while LoRA employs low-rank approximation, it neglects to approximate the optimization process of full fine-tuning.

To address this, we introduce a novel concept called the "equivalent gradient."

Our method constrains the optimization process, shrinking the performance gap between LoRA and full fine-tuning.

Moving object segmentation based on LiDAR is a crucial and challenging task for autonomous driving and mobile robotics.

Most approaches explore spatio-temporal information from LiDAR sequences to predict moving objects in the current frame.

To overcome this issue, we propose a streaming network with a memory mechanism, called StreamMOS, to build the association of features and predictions among multiple inferences.

Specifically, we utilize a short-term memory to convey historical features, which can be regarded as spatial prior of moving objects and adopted to enhance current inference by temporal fusion.

Besides, we present multi-view encoder with cascade projection and asymmetric convolut…

Recent advances in large language models (LLMs) have opened new avenues for applying multi-agent systems in very large-scale simulations.

However, there remain several challenges when conducting multi-agent simulations with existing platforms, such as limited scalability and low efficiency, unsatisfied agent diversity, and effort-intensive management processes.

To address these challenges, we develop several new features and components for AgentScope, a user-friendly multi-agent platform, enhancing its convenience and flexibility for supporting very large-scale multi-agent simulations.

We conduct a comprehensive simulation to demonstrate the effectiveness of the proposed enhancements in Age…

Occluded Person Re-Identification (ReID) is a metric learning task that involves matching occluded individuals based on their appearance.

While many studies have tackled occlusions caused by objects, multi-person occlusions remain less explored.

Inspired by recent work on prompting in vision, we introduce Keypoint Promptable ReID (KPR), a novel formulation of the ReID problem that explicitly complements the input bounding box with a set of semantic keypoints indicating the intended target.

Since promptable re-identification is an unexplored paradigm, existing ReID datasets lack the pixel-level annotations necessary for prompting.

Furthermore, we release custom keypoint labels for four popul…

Recent advances in molecular and genetic research have identified a diverse range of brain tumor sub-types, shedding light on differences in their molecular mechanisms, heterogeneity, and origins.

The present study performs whole-brain connectome analysis using diffusionweighted images.

These streamline mappings form the connectome matrix, on which persistent homology based analysis and graph theoretical analysis are executed to evaluate the discriminatory power between tumor sub-types that include meningioma and glioma.

In classifying tumor sub-types, an accuracy of 80% is attained.

The findings obtained from this study underscore the potential of persistent homology and graph theoretical …

Recently, it was shown that the role of the teacher in knowledge distillation (KD) is to provide the student with an estimate of the true Bayes conditional probability density (BCPD).

Notably, the new findings propose that the student's error rate can be upper-bounded by the mean squared error (MSE) between the teacher's output and BCPD.

Consequently, to enhance KD efficacy, the teacher should be trained such that its output is close to BCPD in MSE sense.

This paper elucidates that training the teacher model with MSE loss equates to minimizing the MSE between its output and BCPD, aligning with its core responsibility of providing the student with a BCPD estimate closely resembling it in MSE…

Here we propose a cost-effective instruction learning framework for conversational pathology named as CLOVER.

CLOVER only trains a lightweight module and uses instruction tuning while freezing the parameters of the large language model.

To augment the use of instructions, we construct a high-quality set of template-based instructions in the context of digital pathology.

Through the instruction tuning, CLOVER exhibits robustness of few-shot learning in the external clinical dataset.

These findings demonstrate that cost-effective modeling of CLOVER could accelerate the adoption of rapid conversational applications in the landscape of digital pathology.

Off-policy evaluation (OPE) is widely applied in sectors such as pharmaceuticals and e-commerce to evaluate the efficacy of novel products or policies from offline datasets.

This paper introduces a causal deepset framework that relaxes several key structural assumptions, primarily the mean-field assumption, prevalent in existing OPE methodologies that handle spatio-temporal interference.

In response, we advocate for the implementation of the permutation invariance (PI) assumption.

Furthermore, we present novel algorithms that incorporate the PI assumption into OPE and thoroughly examine their theoretical foundations.

Our numerical analyses demonstrate that this novel approach yields signifi…

In the field of instruction-following large vision-language models (LVLMs), the efficient deployment of these models faces challenges, notably due to the high memory demands of their key-value (KV) caches.

Conventional cache management strategies for LLMs focus on cache eviction, which often fails to address the specific needs of multimodal instruction-following models.

Recognizing this gap, in this paper, we introduce Elastic Cache, a novel approach that benefits from applying distinct acceleration methods for instruction encoding and output generation stages.

Instead of discarding less important caches, our strategy identifies important key/value vectors as anchor points.

Surrounding less…

Fine-Grained Object Detection (FGOD) is a critical task in high-resolution aerial image analysis.

This letter introduces Orthogonal Mapping (OM), a simple yet effective method aimed at addressing the challenge of semantic confusion inherent in FGOD.

OM introduces orthogonal constraints in the feature space by decoupling features from the last layer of the classification branch with a class-wise orthogonal vector basis.

This effectively mitigates semantic confusion and enhances classification accuracy.

Extensive experiments conducted on three FGOD datasets (FAIR1M, ShipRSImageNet, and MAR20) demonstrate the effectiveness and superiority of the proposed approach.

Kolmogorov-Arnold Networks (KANs), a novel type of neural network, have recently gained popularity and attention due to the ability to substitute multi-layer perceptions (MLPs) in artificial intelligence (AI) with higher accuracy and interoperability.

Furthermore, the corresponding hardware implementation is considered using the Vitis high-level synthesis (HLS) tool.

To the best of our knowledge, this is the first article to implement hardware for KAN.

The results indicate that KANs cannot achieve more accuracy than MLPs in high complex datasets while utilizing substantially higher hardware resources.

Therefore, MLP remains an effective approach for achieving accuracy and efficiency in soft…

The semantic similarity between documents of a text corpus can be visualized using map-like metaphors based on two-dimensional scatterplot layouts.

These layouts result from a dimensionality reduction on the document-term matrix or a representation within a latent embedding, including topic models.

Thereby, the resulting layout depends on the input data and hyperparameters of the dimensionality reduction and is therefore affected by changes in them.

Furthermore, the resulting layout is affected by changes in the input data and hyperparameters of the dimensionality reduction.

First, we derived layouts for the combination of three text corpora and six text embeddings and a grid-search-inspire…

Machine learning techniques have recently been of great interest for solving differential equations.

In this article, we focus on one class of models called nonlinear vector autoregression (NVAR) to solve ordinary differential equations (ODEs).

Motivated by connections to numerical integration and physics-informed neural networks, we explicitly derive the physics-informed NVAR (piNVAR) which enforces the right-hand side of the underlying differential equation regardless of NVAR construction.

Because NVAR and piNVAR completely share their learned parameters, we propose an augmented procedure to jointly train the two models.

Then, using both data-driven and ODE-driven metrics, we evaluate the…

Current methods commonly utilize three-branch structures of inversion, reconstruction, and editing, to tackle consistent image editing task.

However, these methods lack control over the generation position of the edited object and have issues with background preservation.

This approach allows users to simultaneously edit the object's action and control the generation position of the edited object.

In the editing stage, we use the image features in self-attention to query the key and value of the corresponding time step in the inversion to achieve consistent image editing.

Impressive image editing results and quantitative evaluation demonstrate the effectiveness of our method.

To address this, we propose a novel registration-based prompt engineering framework for medical image segmentation using SAM.

This approach utilises established image registration algorithms to align the new image (to-be-segmented) and a small number of reference images, without requiring segmentation labels.

The spatial transformations generated by registration align either the new image or pre-defined point-based prompts, before using them as input to SAM.

This strategy, requiring as few as five reference images with defined point prompts, effectively prompts SAM for inference on new images, without needing any segmentation labels.

This outperforms atlas-based label fusion and is comparab…

Textual entity descriptions have already been shown to be valuable, but models that incorporate numerical literals have shown minor improvements on existing benchmark datasets.

It is unclear whether a model is actually better in using numerical literals, or better capable of utilizing the graph structure.

We propose a methodology to evaluate LP models that incorporate numerical literals.

We propose i) a new synthetic dataset to better understand how well these models use numerical literals and ii) dataset ablations strategies to investigate potential difficulties with the existing datasets.

Our investigation highlights the need for more extensive evaluations when releasing new models and da…

Large language models (LLMs) have demonstrated remarkable capabilities, but their power comes with significant security considerations.

While extensive research has been conducted on the safety of LLMs in chat mode, the security implications of their function calling feature have been largely overlooked.

This paper uncovers a critical vulnerability in the function calling process of LLMs, introducing a novel "jailbreak function" attack method that exploits alignment discrepancies, user coercion, and the absence of rigorous safety filters.

Our empirical study, conducted on six state-of-the-art LLMs including GPT-4o, Claude-3.5-Sonnet, and Gemini-1.5-pro, reveals an alarming average success r…

3D referring segmentation is an emerging and challenging vision-language task that aims to segment the object described by a natural language expression in a point cloud scene.

In this work, we propose RefMask3D to explore the comprehensive multi-modal feature interaction and understanding.

Then, we introduce a Linguistic Primitives Construction to produce semantic primitives representing distinct semantic attributes, which greatly enhance the vision-language understanding at the decoding stage.

The proposed RefMask3D achieves new state-of-the-art performance on 3D referring segmentation, 3D visual grounding, and also 2D referring image segmentation.

Especially, RefMask3D outperforms previo…

While many deep learning methods have been developed for general protein binding site prediction tasks, whether they work for epitope prediction remains an understudied research question.

We introduce a filtered antibody-antigen complex structure dataset, AsEP (Antibody-specific Epitope Prediction).

AsEP is the largest of its kind and provides clustered epitope groups, allowing the community to develop and test novel epitope prediction methods.

AsEP comes with an easy-to-use interface in Python and pre-built graph representations of each antibody-antigen complex while also supporting customizable embedding methods.

Based on this new dataset, we benchmarked various representative general pro…

This paper investigates visual analogical reasoning in large multimodal models (LMMs) compared to human adults and children.

While benchmarks exist for testing visual reasoning in LMMs, they require advanced skills and omit basic visual analogies that even young children can make.

Inspired by developmental psychology, we propose a new benchmark of 1,400 visual transformations of everyday objects to test LMMs on visual analogical reasoning and compare them to children and adults.

In contrast, children and adults exhibit much stronger analogical reasoning at all three stages.

Altogether, these findings highlight the limitations of training models on data that primarily consists of 2D images a…

The ubiquity of missing data has sparked considerable attention and focus on tabular data imputation methods.

Diffusion models, recognized as the cutting-edge technique for data generation, demonstrate significant potential in tabular data imputation tasks.

However, in pursuit of diversity, vanilla diffusion models often exhibit sensitivity to initialized noises, which hinders the models from generating stable and accurate imputation results.

Additionally, the sparsity inherent in tabular data poses challenges for diffusion models in accurately modeling the data manifold, impacting the robustness of these models for data imputation.

To tackle these challenges, this paper introduces an advan…

Due to the complex characteristics of cellular morphology, a large receptive field is considered crucial for generating high-quality segmentation.

To address this issue, we propose LKCell, a high-accuracy and efficient cell segmentation method.

Its core insight lies in unleashing the potential of large convolution kernels to achieve computationally efficient large receptive fields.

Specifically, (1) We transfer pre-trained large convolution kernel models to the medical domain for the first time, demonstrating their effectiveness in cell segmentation.

(2) We analyze the redundancy of previous methods and design a new segmentation decoder based on large convolution kernels.

To our best knowledge, all existing binary image downsampling methods do not have such topology-preserving guarantees.

However, we found the similarity scores to be much worse.

First, generating smaller versions of medical image segmentation masks for easier human inspection.

Second, improving the efficiency of binary image operations, including persistent homology computation and shortest path computation, by substituting the original images with smaller ones.

In particular, the latter is a novel application that is made feasible only by the full topology-preservation guarantee of our method.

Background: Benchmarking medical decision support algorithms often struggles due to limited access to datasets, narrow prediction tasks, and restricted input modalities.

These limitations affect their clinical relevance and performance in high-stakes areas like emergency care, complicating replication, validation, and improvement of benchmarks.

Methods: We introduce a dataset based on MIMIC-IV, benchmarking protocol, and initial results for evaluating multimodal decision support in the emergency department (ED).

Incorporating raw waveform data significantly improves model performance, which represents one of the first robust demonstrations of this effect.

The strong performance, as evidence…

Identifying significant references within the complex interrelations of a citation knowledge graph is challenging, which encompasses connections through citations, authorship, keywords, and other relational attributes.

The Paper Source Tracing (PST) task seeks to automate the identification of pivotal references for given scholarly articles utilizing advanced data mining techniques.

In the KDD CUP 2024, we design a recommendation-based framework tailored for the PST task.

This framework employs the Neural Collaborative Filtering (NCF) model to generate final predictions.

To process the textual attributes of the papers and extract input features for the model, we utilize SciBERT, a pre-train…

Research Google DeepMind at ICML 2024 ShareCopy link ×Exploring AGI, the challenges of scaling and the future of multimodal generative AI Next week the artificial intelligence (AI) community will come together for the 2024 International Conference on Machine Learning (ICML).

Running from July 21-27 in Vienna, Austria, the conference is an international platform for showcasing the latest advances, exchanging ideas and shaping the future of AI research.

Depending on their performance, generality and autonomy, our paper categorizes systems ranging from non-AI calculators to emerging AI models and other novel technologies.

New approaches in generative AI and multimodality Generative AI technolo…

Research Generating audio for video ShareCopy link ×Video-to-audio research uses video pixels and text prompts to generate rich soundtracks Video generation models are advancing at an incredible pace, but many current systems can only generate silent output.

Diagram of our V2A system, taking video pixel and audio prompt input to generate an audio waveform synchronized to the underlying video.

First, V2A encodes the video and audio prompt input and iteratively runs it through the diffusion model.

By training on video, audio and the additional annotations, our technology learns to associate specific audio events with various visual scenes, while responding to the information provided in the a…

How summits in Seoul, France and beyond can galvanize international cooperation on frontier AI safetyLast year, the UK Government hosted the first major global Summit on frontier AI safety at Bletchley Park.

We share below some thoughts on how the summit – and future ones – can drive progress towards a common, global approach to frontier AI safety.

Establishing best practices in evaluations and a coherent governance frameworkEvaluations are a critical component needed to inform AI governance decisions.

Towards a global approach for frontier AI safetyMany of the potential risks that could arise from progress at the frontier of AI are global in nature.

As we head into the AI Seoul Summit, and…

Google DeepMind has consistently pushed the boundaries of AI, developing models that have transformed our understanding of what's possible.

At the same time, we recognize that as we continue to advance the frontier of AI capabilities, these breakthroughs may eventually come with new risks beyond those posed by present-day models.

Today, we are introducing our Frontier Safety Framework - a set of protocols for proactively identifying future AI capabilities that could cause severe harm and putting in place mechanisms to detect and mitigate them.

Our Framework focuses on severe risks resulting from powerful capabilities at the model level, such as exceptional agency or sophisticated cyber capa…

Company Watermarking AI-generated text and video with SynthID ShareCopy link ×Announcing our novel watermarking method for AI-generated text and video, and how we’re bringing SynthID to key Google products Generative AI tools — and the large language model technologies behind them — have captured the public imagination.

Today, we’re expanding SynthID’s capabilities to watermarking AI-generated text in the Gemini app and web experience, and video in Veo, our most capable generative video model.

SynthID text watermarking is less effective on responses to factual prompts because there are fewer opportunities to adjust the token distribution without affecting the factual accuracy.

How video wat…

Over the past year, we’ve made incredible progress in enhancing the quality of our generative media technologies.

We’ve been working closely with the creative community to explore how generative AI can best support the creative process, and to make sure our AI tools are as useful as possible at each stage.

Today, we’re introducing Veo, our latest and most advanced video generation model, and Imagen 3, our highest quality text-to-image model yet.

We’re also sharing some of our recent collaborations with filmmaker Donald Glover and his creative studio, Gilga, and new demo recordings being released by artists Wyclef Jean, Marc Rebillet and songwriter Justin Tranter, made with help from our Mus…

In December, we launched our first natively multimodal model Gemini 1.0 in three sizes: Ultra, Pro and Nano.

Just a few months later we released 1.5 Pro, with enhanced performance and a breakthrough long context window of 1 million tokens.

This inspired us to keep innovating, so today, we’re introducing Gemini 1.5 Flash: a model that’s lighter-weight than 1.5 Pro, and designed to be fast and efficient to serve at scale.

Both 1.5 Pro and 1.5 Flash are available in public preview with a 1 million token context window in Google AI Studio and Vertex AI.

And now, 1.5 Pro is also available with a 2 million token context window via waitlist to developers using the API and to Google Cloud customers.

They’re made up of proteins, DNA and other molecules, but no single piece works on its own.

In a paper published in Nature, we introduce AlphaFold 3, a revolutionary model that can predict the structure and interactions of all life’s molecules with unprecedented accuracy.

To build on AlphaFold 3’s potential for drug design, Isomorphic Labs is already collaborating with pharmaceutical companies to apply it to real-world drug design challenges and, ultimately, develop new life-changing treatments for patients.

Our new model builds on the foundations of AlphaFold 2, which in 2020 made a fundamental breakthrough in protein structure prediction.

This leap could unlock more transformative science…

Research Google DeepMind at ICLR 2024 ShareCopy link ×Developing next-gen AI agents, exploring new modalities, and pioneering foundational learning Next week, AI researchers from around the globe will converge at the 12th International Conference on Learning Representations (ICLR), set to take place May 7-11 in Vienna, Austria.

Teams from across Google DeepMind will present more than 70 papers this year.

For instance, LLM-based AI agents capable of taking effective actions could transform digital assistants into more helpful and intuitive AI tools.

Until recently, large AI models mostly focused on text and images, laying the groundwork for large-scale pattern recognition and data interpreta…

Responsibility & Safety The ethics of advanced AI assistants ShareCopy link ×Exploring the promise and risks of a future with more capable AI Imagine a future where we interact regularly with a range of advanced artificial intelligence (AI) assistants — and where millions of assistants interact with each other on our behalf.

General-purpose foundation models are paving the way for increasingly advanced AI assistants.

Advanced AI assistants could have a profound impact on users and society, and be integrated into most aspects of people’s lives.

Able to fluidly communicate using natural language, the written output and voices of advanced AI assistants may become hard to distinguish from those…

Research TacticAI: an AI assistant for football tactics ShareCopy link ×As part of our multi-year collaboration with Liverpool FC, we develop a full AI system that can advise coaches on corner kicks 'Corner taken quickly… Origi!'

Our first paper, Game Plan, looked at why AI should be used in assisting football tactics, highlighting examples such as analyzing penalty kicks.

Predicting corner kick outcomes with geometric deep learning A corner kick is awarded when the ball passes over the byline, after touching a player of the defending team.

With TacticAI, we have developed a capable AI assistant for football tactics and achieved a milestone in developing useful assistants in sports AI.

We s…

In a new technical report, we introduce SIMA, short for Scalable Instructable Multiworld Agent, a generalist AI agent for 3D virtual settings.

SIMA: a versatile AI agent SIMA is an AI agent that can perceive and understand a variety of environments, then take actions to achieve an instructed goal.

What’s more, an agent trained in all but one game performed nearly as well on that unseen game as an agent trained specifically on it, on average.

We compare this performance with three types of generalist SIMA agent, each trained across multiple environments.

Advancing AI agent research SIMA’s results show the potential to develop a new wave of generalist, language-driven AI agents.

Today, we’re excited to introduce a new generation of open models from Google to assist developers and researchers in building AI responsibly.

Gemma open modelsGemma is a family of lightweight, state-of-the-art open models built from the same research and technology used to create the Gemini models.

This enables Gemma 2B and 7B to achieve best-in-class performance for their sizes compared to other open models.

And Gemma models are capable of running directly on a developer laptop or desktop computer.

Notably, Gemma surpasses significantly larger models on key benchmarks while adhering to our rigorous standards for safe and responsible outputs.

A note from Google and Alphabet CEO Sundar Pichai:Last week, we rolled out our most capable model, Gemini 1.0 Ultra, and took a significant step forward in making Google products more helpful, starting with Gemini Advanced.

Today, developers and Cloud customers can begin building with 1.0 Ultra too — with our Gemini API in AI Studio and in Vertex AI.

Our teams continue pushing the frontiers of our latest models with safety at the core.

In fact, we’re ready to introduce the next generation: Gemini 1.5.

It shows dramatic improvements across a number of dimensions and 1.5 Pro achieves comparable quality to 1.0 Ultra, while using less compute.

Leveraging enterprise data for generative AI and large language models (LLMs) presents significant challenges related to data silos, quality inconsistencies, privacy and security concerns, compliance with data regulations, capturing domain-specific knowledge, and mitigating inherent biases.

Organizations must navigate the complexities of consolidating fragmented data sources, ensuring data integrity, and addressing ethical considerations.

Techniques like retrieval augmented generation (RAG) can help bridge the gap between enterprise generative AI apps and the actual enterprise data.

Although RAG is a great tool, and LLMs have enabled natural language-to-SQL translation, these capabilities f…

Leveraging enterprise data for generative AI and large language models (LLMs) presents significant challenges related to data silos, quality inconsistencies, privacy and security concerns, compliance with data regulations, capturing domain-specific knowledge, and mitigating inherent biases.

Organizations must navigate the complexities of consolidating fragmented data sources, ensuring data integrity, and addressing ethical considerations.

Techniques like retrieval augmented generation (RAG) can help bridge the gap between enterprise generative AI apps and the actual enterprise data.

Although RAG is a great tool, and LLMs have enabled natural language-to-SQL translation, these capabilities f…

Following the exciting preview at Google Cloud Next 2024, we're thrilled to announce the expanded availability of Partner Companion to Google Cloud Services Partners!

It's designed to elevate the professional services they deliver, ensuring their customers receive the highest quality Google Cloud solutions.

If you’re a Google Cloud partner, Partner Companion is ready to amplify your customer engagement journey.

Think of Partner Companion as your dedicated Google Cloud advisor, grounded in the zone of service delivery.

Here’s what we’re hearing:"The Partner Companion perfectly combines the power of LLMs with Google Cloud-specific knowledge.

You can get started with it today in Vertex AI Model Garden.

Our collaboration with Mistral AI is a testament to our open approach, within a unified and an enterprise ready environment.

Vertex AI provides a curated collection of first-party, open-source, and third-party models, many of which — including the new Mistral AI models — can be delivered as a fully-managed Model-as-a-service (MaaS) offering.

As the first hyperscaler to support our new Codestral model, Google Cloud will enable developers worldwide to leverage the power of Mistral AI's proprietary models on Vertex AI.

With this collaboration, we are committed to driving together meaningful innovation in AI and delivering unparallele…

The process of RSS feed ingestion starts with extracting the message URL.

Next, the RSS feed is fetched, and each article is iterated through to verify that it has not already been indexed.

For new articles, the URL in the RSS feed is followed, and the text of the article is retrieved.

Using GKE Autopilot Spot Pods helps to optimize costs, as they provide the lowest cost per pod.

In the screenshots below, you can find a sample BigQuery schema and populated data.

After deciding on Filestore Basic, Shane and his team had to act fast.

They opted for a one-to-one ratio of Filestore instances to JupyterHub deployments, effectively over-provisioning to maximize performance and reliability.

The next challenge was to determine the appropriate size for each Filestore instance.

One thing we're still grappling with is the lack of easy down-scaling in Filestore Basic.

It's not uncommon for us to see a Filestore instance grow by terabytes in a matter of hours.

Today, we’re excited to announce the addition of the Llama 3.1 family of models, including a new 405B model – Meta's most powerful and versatile model to date — to Vertex AI Model Garden.

Vertex AI provides a curated collection of first-party, open-source, and third-party models, many of which — including the new Llama models — can be delivered as fully-managed Model-as-a-service (MaaS) offerings.

Llama 3.1 models also include multilingual support across eight languages, further broadening their reach and applicability.

Using Llama 3.1 in Google CloudGoogle Cloud’s Vertex AI is a comprehensive AI platform for experimenting with, customizing, and deploying, and monitoring foundation models l…

By storing extensive, in-depth company data in Cloud SQL for PostgreSQL, Deckmatch can quickly provide investors with comprehensive insights on startups and their competition.

Putting our money on cloud architectureAt the heart of our cloud architecture is a pgvector-equipped Cloud SQL database that integrates with a powerful suite of Google Cloud services.

In turn, efficient embedding storage and querying powers fast, contextual search capabilities that yield relevant insights quickly.

Rich payoffs with Cloud SQLOur clients have benefited tremendously from the capabilities enabled by Cloud SQL.

From rich documentation resources to expert technical support helping us optimize our cloud arch…

At Lowe's, we are always striving to make the shopping experience more enjoyable and convenient for our customers.

To address this issue and enhance the shopping journey, we introduced Visual Scout — an interactive way to explore the product catalog and quickly find products of interest on lowes.com.

Visual Scout is designed for shoppers who value the visual aspects of products when making certain purchasing decisions.

Visual Scout begins by presenting a panel of 10 items.

Based on this feedback, Visual Scout dynamically updates the panel, with items that reflect customer style and design preferences.

LiveX AI stands at the cutting edge of generative AI technology, building custom, multimodal AI agents that can see, hear, chat, and show to deliver truly human-like customer experiences.

Founded by a team of experienced entrepreneurs and distinguished tech leaders, LiveX AI provides businesses with trusted AI agents that deliver strong customer engagement across a variety of platforms.

LiveX AI generative AI agents provide real-time, immersive, human-like customer experience that offer helpful, real-time solutions to customer questions and concerns in a familiar, conversational manner.

GKE provides a robust foundation for advanced generative AI applicationsFrom the start, Google Cloud and …

Insights are most valuable when they’re easy to understand and communicate to the stakeholders who can use them.

With the preview of Google Slides generation in Gemini in Looker, users can now create presentations with insightful chart summaries from Looker Studio Pro in seconds, to rapidly accelerate data storytelling — complete with charts and summaries in Slides that stay current as your data changes.

Gone are the days of painstakingly moving data and imagery from your BI tools into untitled slides.

Gemini in Looker leverages Google’s AI to eliminate that cold start, transforming your reports to visually appealing slides that tell a full story.

To activate Google Slides generation in Gem…

Enterprises from almost every industry are exploring the possibilities of generative AI, adopting AI agents for purposes ranging from internal productivity to customer-facing support.

A RAG use- case: Cymbal AirLike any data access paradigm, without a careful approach there is risk.

However, giving the AI unrestricted access to the database could lead to accidental leaks of sensitive information about a different user.

How do we ensure data safety while letting the AI assistant retrieve information from the database?

Rather than give the foundation model unbounded access, we can define specific tool functions that the agent uses to access database information securely and predictably.

Editor's note: In this guest post, we will hear from Gramercy Tech about their experience working with Google Cloud as both a customer and vendor.

Organizing engaging events can be quite challenging, but by utilizing Google's Imagen throughout the conference, the Gramercy team was able to demonstrate the possibilities of generative AI for creating real-time experiences.

For example, one of the activations developed at a recent Google Cloud event was the “Imagine Tree” (Interactive Storytelling).

By using Imagen, Gramercy was able to create hyper-realistic visual narratives that were tailored to the needs of each attendee.

Please see below for the prompts, actual generated event images, and …

In today's fast-paced software development landscape, ambitious goals, complex technologies, and shifting priorities can lead to frustrated developer experience.

To maintain product resiliency, operations teams often face immense pressure when issues arise, expected to resolve them with far greater urgency than development teams.

However, navigating the path to AI-driven success and empowering your software development teams requires a strategic approach.

Reach out to your Google Cloud sales for the Gemini Code Assist for Developers Pilot program with guided workshops, program checkpoints and business cases to increase developer productivity and joy.

Our solution experts can share recommend…

“It recognises the value of our award-winning journalism and will give us early insights into how content is surfaced through AI.

“Apart from the benefits to the FT, there are broader implications for the industry.

It’s right, of course, that AI platforms pay publishers for the use of their material.

“We value the opportunity to be inside the development loop as people discover content in new ways.

As with any transformative technology, there is potential for significant advancements and major challenges, but what’s never possible is turning back time.

OpenAI, alongside industry leaders including Amazon, Anthropic, Civitai, Google, Meta, Metaphysic, Microsoft, Mistral AI, and Stability AI, has committed to implementing robust child safety measures in the development, deployment, and maintenance of generative AI technologies as articulated in the Safety by Design principles.

By adopting comprehensive Safety by Design principles, OpenAI and our peers are ensuring that child safety is prioritized at every stage in the development of AI.

Responsibly source our training datasets, detect and remove child sexual abuse material (CSAM) and child sexual exploitation material (CSEM) from training data, and report any confirmed CSAM to the relevant a…

Customers with a sustained level of tokens per minute (TPM) usage on GPT-4 or GPT-4 Turbo can request access to provisioned throughput to get discounts ranging from 10–50% based on the size of the commitment.

Reduced costs on asynchronous workloads: Customers can use our new Batch API to run non-urgent workloads asynchronously.

Batch API requests are priced at 50% off shared prices, offer much higher rate limits, and return results within 24 hours.

We plan to keep adding new features focused on enterprise-grade security, administrative controls, and cost management.

For more information on these launches, visit our API documentation or get in touch with our team to discuss custom solution…

Our new local presence also gets us closer to leading businesses like Daikin, Rakuten, and TOYOTA Connected who are using ChatGPT Enterprise to automate complex business processes, assist in data analysis, and optimize internal reporting.

ChatGPT also helps accelerate the efforts of local governments, such as Yokosuka City, which is leveraging the technology to improve the efficiency of public services in Japan.

Over the past year, the city has gradually provided ChatGPT access to almost all city employees, and 80% have reported increases in productivity.

Now Yokosuka City has formed a network with 21 local governments—including the Tokyo Metropolitan Government and the City of Kobe—to …

Assisted Fine-TuningAt DevDay last November, we announced a Custom Model program designed to train and optimize models for a specific domain, in partnership with a dedicated group of OpenAI researchers.

Since then, we've met with dozens of customers to assess their custom model needs and evolved our program to further maximize performance.

Today, we are formally announcing our assisted fine-tuning offering as part of the Custom Model program.

Fully custom-trained models imbue new knowledge from a specific domain by modifying key steps of the model training process using novel mid-training and post-training techniques.

Our team modified every step of the model training process, from domain-s…

We’ve also introduced additional content safeguards for this experience, such as blocking prompts and generations in a wider range of categories.

There are many benefits to creating an account including the ability to save and review your chat history, share chats, and unlock additional features like voice conversations and custom instructions.

For anyone that has been curious about AI’s potential but didn’t want to go through the steps to set-up an account, start using ChatGPT today.

We recognize that generating speech that resembles people's voices has serious risks, which are especially top of mind in an election year.

We are engaging with U.S. and international partners from across government, media, entertainment, education, civil society and beyond to ensure we are incorporating their feedback as we build.ÂThe partners testing Voice Engine today have agreed to our usage policies, which prohibit the impersonation of another individual or organization without consent or legal right.

In addition, our terms with these partners require explicit and informed consent from the original speaker and we don’t allow developers to build ways for individual users to create the…

Starting his career at DreamWorks Animation, Don Allen III is a multidisciplinary creator, speaker and consultant who collaborates with major tech and entertainment companies on mixed reality, virtual reality and AI applications.

“For a long time I've been making augmented reality hybrid creatures that I think would be fun combinations in my head.

Now I have a much easier way of prototyping the ideas before I fully build out the 3-D characters to place in spatial computers.” Don cites Sora’s “weirdness” as its greatest strength: “It’s not bound by traditional laws of physics or conventions of thought.” He says that working with Sora shifted his focus from “technical hurdle…

Echoing this sentiment, Louis Dreyfus, CEO of Le Monde, stated, "At the moment we are celebrating the 80th anniversary of Le Monde, this partnership with OpenAI allows us to expand our reach and uphold our commitment to providing accurate, verified, balanced news stories at scale.

Collaborating with OpenAI ensures that our authoritative content can be accessed and appreciated by a broader, more diverse audience. ÂEvery shift in the media landscape has presented Le Monde with new opportunities.

From the transition to digital platforms to embracing the era of free media, Le Monde has consistently seized these moments to underscore its commitment to independence, expertise, and journalistic i…

Additionally, Sam Altman, CEO, will rejoin the OpenAI Board of Directors.ÂSue, Nicole and Fidji have experience in leading global organizations and navigating complex regulatory environments, including backgrounds in technology, nonprofit and board governance.

They will work closely with current board members Adam D’Angelo, Larry Summers and Bret Taylor as well as Sam and OpenAI’s senior management.ÂBret Taylor, Chair of the OpenAI board, stated, “I am excited to welcome Sue, Nicole, and Fidji to the OpenAI Board of Directors.

She also served as President of Sony Entertainment, Inc., and simultaneously served as President of Sony Corporation of America.

She also serves as a member of …

The Special Committee of the OpenAI Board today announced the completion of the review by WilmerHale.

The firm conducted dozens of interviews with members of OpenAI’s prior Board, OpenAI executives, advisors to the prior Board, and other pertinent witnesses; reviewed more than 30,000 documents; and evaluated various corporate actions.

“We have unanimously concluded that Sam and Greg are the right leaders for OpenAI,” stated Bret Taylor, Chair of the OpenAI Board.

The Special Committee acknowledged the important work done by WilmerHale in conducting this extensive review and thanked OpenAI current and former Board members, advisors and employees for their cooperation.

The Special Commi…

Date: January 31, 2018 at 11:54:30 PM PSTSubject: Re: Top AI institutions todayWorking at the cutting edge of AI is unfortunately expensive.

For example,In addition to DeepMind, Google also has Google Brain, Research, and Cloud.

If historical trends are any indication, progress in AI is primarily driven by systems - compute, data, infrastructure.

Not only that, but any algorithmic advances published in a paper somewhere can be almost immediately re-implemented and incorporated.

The “second stage” would be a full self driving solution based on large-scale neural network training, which OpenAI expertise could significantly help accelerate.

This technical report focuses on (1) our method for turning visual data of all types into a unified representation that enables large-scale training of generative models, and (2) qualitative evaluation of Sora’s capabilities and limitations.

Model and implementation details are not included in this report.

Much prior work has studied generative modeling of video data using a variety of methods, including recurrent networks,[^1][^2][^3] generative adversarial networks,[^4][^5][^6][^7] autoregressive transformers,[^8][^9] and diffusion models.

[^10][^11][^12] These works often focus on a narrow category of visual data, on shorter videos, or on videos of a fixed size.

Sora is a generalist mo…

Based on collaboration and information sharing with Microsoft, we disrupted five state-affiliated malicious actors: two China-affiliated threat actors known as Charcoal Typhoon and Salmon Typhoon; the Iran-affiliated threat actor known as Crimson Sandstorm; the North Korea-affiliated actor known as Emerald Sleet; and the Russia-affiliated actor known as Forest Blizzard.

The identified OpenAI accounts associated with these actors were terminated.

Salmon Typhoon used our services to translate technical papers, retrieve publicly available information on multiple intelligence agencies and regional threat actors, assist with coding, and research common ways processes could be hidden on a system.…

We’re testing memory with ChatGPT.

Remembering things you discuss across all chats saves you from having to repeat information and makes future conversations more helpful.

You're in control of ChatGPT's memory.

You can explicitly tell it to remember something, ask it what it remembers, and tell it to forget conversationally or through settings.

We are rolling out to a small portion of ChatGPT free and Plus users this week to learn how useful it is.

Today, the AI systems we interact with are more than just neural network models.

With the help of demos, we’ll show you how this powerful tool can be used to build AI agents that learn and adapt from their experiences, eliminating the need for specialized engineering.

Subscribe today Opens in a new tabWarm up: Building a Battleship game AI agent through learningTo start, consider building an AI agent for the classic Battleship board game.

Trace automatically constructs the interaction graph of agents and updates each agent’s behavior factoring in the behavior of other agents.

This innovation could be the key to unlocking the full potential of AI systems, making them more efficient and respo…

In an era increasingly steered by data, machine learning is a pivotal force, transforming vast amounts of information into actionable intelligence with unprecedented speed and accuracy.

For example, recent advances in machine learning have led to breakthroughs in precision health, helping doctors make more informed decisions about patient care.

Similarly, in climate science, machine learning is improving scientists’ ability to predict and mitigate the impact of extreme weather events.

These innovations illustrate that machine learning not only streamlines workflows, it also equips people with the tools to tackle some of today’s most pressing challenges with efficiency and innovation.

As the…

The problem is, however, though, high-quality synthetic data creation requires lots of human effort and expertise.

If you have a way to, actually, generate very high-quality synthetic data, you can fast-track this part of specialization process.

We want to also measure how far we are from those frontier models, so that’s, sort of, our evaluation setup.

Similarly, if I’m focusing on math skill, there are many datasets which test, like, elementary math, high school math, college-level math.

MITRA: The key takeaway would be, like, the AgentInstruct method enables the generation of vast, diverse, and high-quality synthetic data with very minimal human input.

Welcome to Research Focus, a series of blog posts that highlights notable publications, events, code/datasets, new hires and other milestones from across the research community at Microsoft.

In a recent article: MG-TSD: Advancing time series analysis with multi-granularity guided diffusion model, researchers from Microsoft present MG-TSD, a novel approach aimed at tackling this challenge.

The paper introducing this research: MG-TSD: Multi-Granularity Time Series Diffusion Models with Guided Learning Process(opens in new tab) (opens in new tab), was presented at ICLR 2024 (opens in new tab).

Spotlight: Microsoft research newsletter Microsoft Research Newsletter Stay connected to the research…

Together, the joint team aims to achieve carbon neutrality and enhance lithium-ion battery performance prediction by focusing on battery performance degradation.

“Through our collaboration with Microsoft Research Asia, we are innovating battery degradation prediction methods to enhance the effectiveness of battery recycling and promote resource reuse.

Compared with popular state-of-the-art battery prediction methods, this data-driven model improves accuracy by approximately 80% with Nissan’s simulation data and by over 30% with real-world experimental data.

It was surprising that the high sensitivity to certain voltages (3.9V) indicated by the data-driven cathode (NMC) SoH prediction model …

Generative AI has redefined the landscape of AI assistants in software development, with innovations like GitHub Copilot providing real-time, chat-based programming support.

As these tools increase in sophistication and domain specialization, assessing their impact on user interactions becomes more challenging.

Developers frequently question whether modifications to their AI assistants genuinely improve the user experience, as indicated in a recent paper.

To use RUBICON, developers need a small set of labeled conversations from their AI assistant and specifically designed prompts that reflect the criteria for task progression and completion.

As we look ahead, our goal is to broaden the appl…

HUIZINGA: Yeah, yeah.

VASHISTH: Yeah, yeah.

[LAUGHTER] Upstream of that, the design drives environmental health and so on, that’s actually … that’s up to you guys!

VASHISTH: Yeah, yeah.

VASHISTH: Yeah, yeah.

VBase query system: Providing a unified foundation for vector index and scalar index scanningVector databases and scalar databases have different index scan patterns.

Therefore, the lack of a unified foundation is the first problem to be solved in building a unified database.

By analyzing a large number of vector indices, researchers have found that vector index queries do not require strict monotonicity for early termination.

Based on this discovery, researchers have developed the VBase unified database system.

Take the currently popular fine-grained graph-based vector index and coarse-grained cluster-based vector index as examples.

Presentation of generative AI tools and opportunities at Issara Global Forum, Bangkok, November 2023.

For NGO staff members that need to divide their time between frontline assistance and data work, any tool that increases the efficiency and quality of data work can create more time for more effective assistance.. For NGO staff members that need to divide their time between frontline assistance and data work, any tool that increases the efficiency and quality of data work can create more time for more effective assistance.

Generative AI is then used to evaluate and prioritize these matches for human review and potential record linking.

Across multiple stakeholder events, we have helped to r…

Download GraphRAGDownload GraphRAG AcceleratorEarlier this year, we introduced GraphRAG (opens in new tab), a graph-based approach to retrieval-augmented generation (RAG) that enables question-answering over private or previously unseen datasets.

Today, we’re pleased to announce that GraphRAG is now available on GitHub (opens in new tab), offering more structured information retrieval and comprehensive response generation than naive RAG approaches.

The results show that GraphRAG, when using community summaries at any level of the community hierarchy, outperforms naive RAG on comprehensiveness and diversity (~70–80% win rate).

GraphRAG using intermediate- and low-level community summaries al…

Welcome to Research Focus, a series of blog posts that highlights notable publications, events, code/datasets, new hires and other milestones from across the research community at Microsoft.

RENC is purely a research project and there are no current plans to incorporate RENC into a product.

In a recent preprint: MAIRA-2: Grounded Radiology Report Generation, researchers from Microsoft extend report generation to include the localization of individual findings on the image – or grounded report generation.

To enable evaluation of grounded reporting, the researchers propose a novel framework – RadFact – leveraging the reasoning capabilities of LLMs.

Microsoft's secret weapon – research leader …

Software-driven wide area network (SWAN) is a system that enables centralized management and control of network infrastructure to improve reliability and efficiency.

Over the last decade, I’ve had the opportunity to shepherd SWAN from a research idea to a foundational system for Microsoft Azure (opens in new tab).

We decided to explore logically centralized control for both the applications and the network.

Spotlight: Microsoft research newsletter Microsoft Research Newsletter Stay connected to the research community at Microsoft.

Many of these have been deployed in production on the Microsoft WAN.

Figure 1: features of habitual behavior (e.g., eating snack when focusing on work) and goal-directed behavior (planning a meal to lose weight).

The core ideaThe paper proposes the Bayesian behavior framework, which aims to enhance the understanding of behavior in sensorimotor tasks.

The key innovation is the introduction of a pivotal concept: the Bayesian intention variable, designed to bridge habitual behavior and goal-directed behavior.

Transition from goal-directed to habitual behavior: The simulations demonstrated that with repetitive trials, an agent’s behavior naturally transitions from slow, goal-directed behavior to faster, habitual behavior.

The goal-directed intention is inferred …

MicroCode: Mobility-focused visual programmingOur paper, “Meet MicroCode: a Live and Portable Programming Tool for the BBC micro:bit,” presented at IDC 2024, addresses these issues with MicroCode, a portable programming approach that makes it possible to program the micro:bit anywhere—whether in a classroom, outdoors, or on the bus—without needing a separate internet-connected computer.

It provides a color screen and inputs that enable live and portable programming.

The micro:bit V2 (top) is inserted into a Game Bit, a commercially available Arcade shield, which displays a MicroCode program.

Microsoft Research Podcast What’s Your Story: Jacki O’Neill Jacki O’Neill saw an opportunity to expa…

Microsoft is proud to sponsor the 41st annual Conference on Computer Vision and Pattern Recognition (CVPR 2024), held from June 17 to June 21.

It offers a practical solution for creating realistic egocentric training data, with the goal of serving as a useful tool for egocentric computer vision research.

Finally, the authors present Large Language Instructed Segmentation Assistant (LISA), a tool that combines the linguistic capabilities of large language models with the ability to produce segmentation masks.

The process uses confidence-guided optimization to alternately refine human poses and shapes, achieving high-fidelity, consistent 3D models.

This dataset enriches scene comprehension wi…

In a new paper, they show how randomizing a model’s decisions in a structured way can improve fairness in certain situations.

They present a framework one could use to introduce a specific amount of randomization into a model’s decisions by allocating resources through a weighted lottery.

“Even if you could make fair predictions, should you be deciding these social allocations of scarce resources or opportunities strictly off scores or rankings?

In this paper, they explored the question of when randomization can improve fairness.

“When you acknowledge that people have different claims to these scarce resources, fairness is going to require that we respect all claims of individuals.

Interpreting the mechanisms underlying AI models enables us to audit them for safety and biases, with the potential to deepen our understanding of the science behind intelligence itself.

To address this, MIT Computer Science and Artificial Intelligence Laboratory (CSAIL) researchers decided to take an automated approach to interpreting artificial vision models that evaluate different properties of images.

They developed “MAIA” (Multimodal Automated Interpretability Agent), a system that automates a variety of neural network interpretability tasks using a vision-language model backbone equipped with tools for experimenting on other AI systems.

“Our goal is to create an AI researcher that can…

Most proton-conducting inorganic materials available now require undesirably high temperatures to achieve sufficiently high conductivity.

In order to advance the development of proton conductors, MIT engineers have identified certain traits of materials that give rise to fast proton conduction.

Using those traits quantitatively, the team identified a half-dozen new candidates that show promise as fast proton conductors.

As Yildiz explains, proton conduction first involves a proton “hopping from a donor oxygen atom to an acceptor oxygen.

Sure enough, they found six promising materials, with predicted proton conduction speeds faster than the best existing solid acid proton conductors.

One thing that makes large language models (LLMs) so powerful is the diversity of tasks to which they can be applied.

We wanted to illustrate that this force of human generalization is also present in how people form beliefs about language models,” Rambachan says.

Through the survey, they generated a dataset of nearly 19,000 examples of how humans generalize about LLM performance across 79 diverse tasks.

“Human generalization gets applied to language models, but that breaks down because these language models don’t actually show patterns of expertise like people would,” Rambachan says.

In situations where people put more weight on incorrect responses, simpler models outperformed very large m…

Ductal carcinoma in situ (DCIS) is a type of preinvasive tumor that sometimes progresses to a highly deadly form of breast cancer.

Because it is difficult for clinicians to determine the type and stage of DCIS, patients with DCIS are often overtreated.

Their model shows that both the state and arrangement of cells in a tissue sample are important for determining the stage of DCIS.

Researchers can use techniques like multiplexed staining or single-cell RNA sequencing to determine the stage of DCIS in tissue samples.

For this research, they hypothesized that combining this single stain with a carefully designed machine-learning model could provide the same information about cancer stage as co…

Interest in understanding SRO is linked to the excitement around advanced materials called high-entropy alloys, whose complex compositions give them superior properties.

Unlike most traditional alloys, high-entropy alloys have several elements, from three up to 20, in nearly equal proportions.

Fortunately, researchers are leveraging machine learning to overcome the shortcomings of traditional approaches for capturing and quantifying SRO.

A two-pronged machine learning solutionTo study SRO using machine learning, it helps to picture the crystal structure in high-entropy alloys as a connect-the-dots game in an coloring book, Cao says.

Freitas used machine learning to evaluate the different ch…

The traditional way to verify safety and stability is through techniques called Lyapunov functions.

For robots controlled by neural networks, though, prior approaches for verifying Lyapunov conditions didn’t scale well to complex machines.

Then, they developed a novel verification formulation that enables the use of a scalable neural network verifier, α,β-CROWN, to provide rigorous worst-case scenario guarantees beyond the counterexamples.

“Our work bridges the gap between that level of performance from neural network controllers and the safety guarantees needed to deploy more complex neural network controllers in the real world,” notes Yang.

These experiments, though modest, are relatively…

A material’s phonon dispersion relation is the relationship between energy and momentum of phonons in its crystal structure.

For years, researchers have tried to predict phonon dispersion relations using machine learning, but there are so many high-precision calculations involved that models get bogged down.

“If you have 100 CPUs and a few weeks, you could probably calculate the phonon dispersion relation for one material.

The physical location doesn’t matter, and the real nodes don’t even know the virtual nodes are there,” says Chotrattanapituk.

“I find that the level of acceleration in predicting complex phonon properties is amazing, several orders of magnitude faster than a state-of-the-…

Foundation models are massive deep-learning models that have been pretrained on an enormous amount of general-purpose, unlabeled data.

To help prevent such mistakes, researchers from MIT and the MIT-IBM Watson AI Lab developed a technique to estimate the reliability of foundation models before they are deployed to a specific task.

They do this by training a set of foundation models that are slightly different from one another.

Unlike traditional machine-learning models, foundation models don’t give concrete outputs like “cat” or “dog” labels.

However, one limitation comes from the fact that they must train an ensemble of large foundation models, which is computationally expensive.

The gift, made by Stephen A. Schwarzman, the chair, CEO, and co-founder of Blackstone, one of the world’s largest alternative investment firms, was the foundation for establishing the college.

The MIT Stephen A. Schwarzman College of Computing recently marked a significant milestone as it celebrated the completion and inauguration of its new building on Vassar Street with a dedication ceremony.

MIT President Sally Kornbluth told the audience that the “success of the MIT Stephen A. Schwarzman College of Computing is a testament to Steve’s vision.” She pointed out that the new building — with capacity for 50 computing research groups — will foster a remarkable confluence of knowledge and cros…

The mystery surrounding the inner workings of large language models (LLMs) stems from their vast size, complex training methods, hard-to-predict behaviors, and elusive interpretability.

The researchers developed some tests outside the models’ comfort zones by tweaking existing tasks instead of creating entirely new ones.

When users interact with language models, any arithmetic is usually in base-10, the familiar number base to the models.

“We’ve uncovered a fascinating aspect of large language models: they excel in familiar scenarios, almost like a well-worn path, but struggle when the terrain gets unfamiliar.

“As language models scale up, understanding their training data becomes increasin…

If a model says it is 49 percent confident that a medical image shows a pleural effusion, then 49 percent of the time, the model should be right.

Quantifying uncertaintyUncertainty quantification methods often require complex statistical calculations that don’t scale well to machine-learning models with millions of parameters.

These methods also require users to make assumptions about the model and data used to train it.

MDL is used to better quantify and calibrate uncertainty for test points the model has been asked to label.

The technique can also determine whether the model has mislabeled certain data points or reveal which data points are outliers.

This includes increased geostationary Earth orbit (GEO) satellite activity, which brings technologies with global-scale impact, from broadband internet to climate surveillance.

To address this challenge, the MIT Astrodynamics, Space Robotic, and Controls Laboratory (ARCLab) launched the MIT ARCLab Prize for AI Innovation in Space: a first-of-its-kind competition asking contestants to harness AI to characterize satellites’ patterns of life (PoLs) — the long-term behavioral narrative of a satellite in orbit — using purely passively collected information.

With support from the U.S. Department of the Air Force-MIT AI Accelerator, the challenge offers a total of $25,000.

The challenge participan…

“Our friend Victoria noticed that where we live in Marlborough there are lots of trees in our own backyards.

The Day of AI curriculum introduces K-12 students to artificial intelligence.

“They can see themselves shaping the world they live in,” said Cunningham.

Using MIT App Inventor, ​their combined ideas led to a prototype with the potential to make a real-world impact in their community.

The Day of AI curriculum teaches the mechanics of AI, ethical considerations and responsible uses, and interdisciplinary applications for different fields.

GenSQL, a generative AI system for databases, could help users make predictions, detect anomalies, guess missing values, fix errors, or generate synthetic data with just a few keystrokes.

GenSQL automatically integrates a tabular dataset and a generative probabilistic AI model, which can account for uncertainty and adjust their decision-making based on new data.

The researchers noticed that SQL didn’t provide an effective way to incorporate probabilistic AI models, but at the same time, approaches that use probabilistic models to make inferences didn’t support complex database queries.

Then, she can run queries on data that also get input from the probabilistic model running behind the scen…

Launching VHs: The Visual Haystacks Benchmark!

Humans excel at processing vast arrays of visual information, a skill that is crucial for achieving artificial general intelligence (AGI).

Visual Haystacks: the first "visual-centric" Needle-In-A-Haystack (NIAH) benchmark designed to rigorously evaluate Large Multimodal Models (LMMs) in processing long-context visual information.

The first NIAH benchmark for visual reasoning was introduced by Google in the Gemini-v1.5 technical report.

What is the Visual Haystacks (VHs) Benchmark?

TinyAgent: Function Calling at the EdgeThe ability of LLMs to execute commands through plain language (e.g.

The framework is open sourced and available at https://github.com/SqueezeAILab/TinyAgentTeaching LLMs to do Function CallingFigure 1: Overview of the LLMCompiler Function Calling Planner.

Once this function calling plan is generated, we can parse it and call each function based on the dependencies.

With our dataset in place, we can now proceed to fine-tune off-the-shelf SLMs to enhance their function calling capability.

Latency is the end-to-end latency of the function calling planner, including the prompt processing time and generation.

As computer vision researchers, we believe that every pixel can tell a story. However, there seems to be a writer’s block settling into the field when it comes to dealing with large images. Large images are no longer rare—the cameras we carry in our pockets and those orbiting our planet snap pictures so big and detailed that they stretch our current best models and hardware to their breaking points when handling them. Generally, we face a quadratic increase in memory usage as a function of image size.

Today, we make one of two sub-optimal choices when handling large images: down-sampling or cropping. These two methods incur significant losses in the amount of information and context present…

Modeling Extremely Large Images with xTAs computer vision researchers, we believe that every pixel can tell a story.

However, there seems to be a writer’s block settling into the field when it comes to dealing with large images.

Today, we make one of two sub-optimal choices when handling large images: down-sampling or cropping.

Why bother handling large images anyways?

That’s basically what we do with large images with $x$T.

Every year, the Berkeley Artificial Intelligence Research (BAIR) Lab graduates some of the most talented and innovative minds in artificial intelligence and machine learning. Our Ph.D. graduates have each expanded the frontiers of AI research and are now ready to embark on new adventures in academia, industry, and beyond.

These fantastic individuals bring with them a wealth of knowledge, fresh ideas, and a drive to continue contributing to the advancement of AI. Their work at BAIR, ranging from deep learning, robotics, and natural language processing to computer vision, security, and much more, has contributed significantly to their fields and has had transformative impacts on society.

This…

2024 BAIR Graduate DirectoryEvery year, the Berkeley Artificial Intelligence Research (BAIR) Lab graduates some of the most talented and innovative minds in artificial intelligence and machine learning.

Our Ph.D. graduates have each expanded the frontiers of AI research and are now ready to embark on new adventures in academia, industry, and beyond.

These fantastic individuals bring with them a wealth of knowledge, fresh ideas, and a drive to continue contributing to the advancement of AI.

Join us in celebrating the achievements of BAIR’s latest PhD graduates.

Thank you to our friends at the Stanford AI Lab for this idea!

AI caught everyone’s attention in 2023 with Large Language Models (LLMs) that can be instructed to perform general tasks, such as translation or coding, just by prompting. This naturally led to an intense focus on models as the primary ingredient in AI application development, with everyone wondering what capabilities new LLMs will bring.

As more developers begin to build using LLMs, however, we believe that this focus is rapidly changing: state-of-the-art AI results are increasingly obtained by compound systems with multiple components, not just monolithic models.

For example, Google’s AlphaCode 2 set state-of-the-art results in programming through a carefully engineered system that uses L…

In this post, we analyze the trend toward compound AI systems and what it means for AI developers.

We argue that compound AI systems will likely be the best way to maximize AI results in the future, and might be one of the most impactful trends in AI in 2024.

We define a Compound AI System as a system that tackles AI tasks using multiple interacting components, including multiple calls to models, retrievers, or external tools.

Developing Compound AI SystemsWhile compound AI systems can offer clear benefits, the art of designing, optimizing, and operating them is still emerging.

However, new compound AI systems contain non-differentiable components like search engines or code interpreters, a…

Ghostbuster: Detecting Text Ghostwritten by Large Language ModelsThe structure of Ghostbuster, our new state-of-the-art method for detecting AI-generated text.

Large language models like ChatGPT write impressively well—so well, in fact, that they’ve become a problem.

Existing tools to detect AI-generated text sometimes do poorly on data that differs from what they were trained on.

Our recent paper introduces Ghostbuster, a state-of-the-art method for detecting AI-generated text.

Many current AI-generated text detection systems are brittle to classifying different types of text (e.g., different writing styles, or different text generation models or prompts).

The structure of Ghostbuster, our new state-of-the-art method for detecting AI-generated text. Large language models like ChatGPT write impressively well—so well, in fact, that they’ve become a problem. Students have begun using these models to ghostwrite assignments, leading some schools to ban ChatGPT. In addition, these models are also prone to producing text with factual errors, so wary readers may want to know if generative AI tools have been used to ghostwrite news articles or other sources before trusting them.

What can teachers and consumers do? Existing tools to detect AI-generated text sometimes do poorly on data that differs from what they were trained on. In addition, if these m…

Asymmetric Certified Robustness via Feature-Convex Neural NetworksAsymmetric Certified Robustness via Feature-Convex Neural NetworksTLDR: We propose the asymmetric certified robustness problem, which requires certified robustness for only one class and reflects real-world adversarial scenarios.

We argue that these issues can be addressed by refining the certified robustness problem to be more aligned with practical adversarial settings.

The Asymmetric Certified Robustness ProblemCurrent certifiably robust classifiers produce certificates for inputs belonging to any class.

Feature-convex classifiersWe propose feature-convex neural networks to address the asymmetric robustness problem.

Conclu…

Asymmetric Certified Robustness via Feature-Convex Neural Networks TLDR: We propose the asymmetric certified robustness problem, which requires certified robustness for only one class and reflects real-world adversarial scenarios. This focused setting allows us to introduce feature-convex classifiers, which produce closed-form and deterministic certified radii on the order of milliseconds. Figure 1. Illustration of feature-convex classifiers and their certification for sensitive-class inputs. This architecture composes a Lipschitz-continuous feature map $\varphi$ with a learned convex function $g$. Since $g$ is convex, it is globally underapproximated by its tangent plane at $\varphi(x)$, y…

Goal Representations for Instruction Following A longstanding goal of the field of robot learning has been to create generalist agents that can perform tasks for humans. Natural language has the potential to be an easy-to-use interface for humans to specify arbitrary tasks, but it is difficult to train robots to follow language instructions. Approaches like language-conditioned behavioral cloning (LCBC) train policies to directly imitate expert actions conditioned on language, but require humans to annotate all training trajectories and generalize poorly across scenes and behaviors. Meanwhile, recent goal-conditioned approaches perform much better at general manipulation tasks, but do not e…

Goal Representations for Instruction FollowingGoal Representations for Instruction FollowingA longstanding goal of the field of robot learning has been to create generalist agents that can perform tasks for humans.

Goal Representations for Instruction FollowingThe GRIF model consists of a language encoder, a goal encoder, and a policy network.

Our approach, Goal Representations for Instruction Following (GRIF), jointly trains a language- and a goal- conditioned policy with aligned task representations.

In particular, we exploit this structure by requiring that language- and goal- representations be similar for the same semantic task.

We train dual image and text encoders by doing contrastiv…

Rethinking the Role of PPO in RLHF TL;DR: In RLHF, there’s tension between the reward learning phase, which uses human preference in the form of comparisons, and the RL fine-tuning phase, which optimizes a single, non-comparative reward. What if we performed RL in a comparative way? Figure 1: This diagram illustrates the difference between reinforcement learning from absolute feedback and relative feedback. By incorporating a new component - pairwise policy gradient, we can unify the reward modeling stage and RL stage, enabling direct updates based on pairwise responses. Large Language Models (LLMs) have powered increasingly capable virtual assistants, such as GPT-4, Claude-2, Bard and Bing…

Faster auto scaling metricsTo optimize real-time inference workloads, SageMaker employs Application Auto Scaling.

Auto scaling trigger (t1) – If the metric crosses the predefined threshold, the CloudWatch alarm goes into an InAlarm state, invoking an auto scaling action to scale up the resources.

By using these new metrics, auto scaling can now be invoked and scale out significantly faster compared to the older SageMakerVariantInvocationsPerInstance predefined metric type.

You can use the following steps to create a new scaling policy to benefit from faster auto scaling.

After you create your SageMaker endpoint, you define a new auto scaling target for Application Auto Scaling.

SharePoint Sever and SharePoint Online contain pages, files, attachments, links, events, and comments that can be crawled by Amazon Q SharePoint connectors for SharePoint Server and SharePoint Online.

Overview of the SharePoint Online connector for Amazon Q BusinessTo crawl and index contents from SharePoint Online, you can configure the Amazon Q Business SharePoint Online connector as a data source in your Amazon Q business application.

Configure and prepare the Amazon Q connectorBefore you index the content from Microsoft SharePoint online, your need to first establish a secure connection between the Amazon Q Business connector for SharePoint Online with your SharePoint Online instance.

C…

Agent Evaluation, an open source solution using LLMs on Amazon Bedrock, addresses this gap by enabling comprehensive evaluation and validation of conversational AI agents at scale.

Use case overviewTo illustrate how Agent Evaluation can accelerate the development and deployment of conversational AI agents at scale, let’s explore an example scenario: developing an insurance claim processing agent using Agents for Amazon Bedrock.

For the latest pricing details for Amazon Bedrock, refer to Amazon Bedrock pricing.

ConclusionThis post introduced Agent Evaluation, an open source solution that enables developers to seamlessly integrate agent evaluation into their existing CI/CD workflows.

To furth…

In the post, we introduce the AWS Neuron node problem detector and recovery DaemonSet for AWS Trainium and AWS Inferentia on Amazon Elastic Kubernetes Service (Amazon EKS).

The node recovery agent is a separate component that periodically checks the Prometheus metrics exposed by the node problem detector.

Additionally, the node recovery agent will publish Amazon CloudWatch metrics for users to monitor and alert on these events.

With the plugin deployed, the node problem detector will proactively remove the problem node from the cluster.

He currently focuses on enhancing the AI/ML experience through the integration of AWS Neuron with containerized environments and Kubernetes.

Mistral AI’s Mistral Large 2 (24.07) foundation model (FM) is now generally available in Amazon Bedrock.

Mistral Large 2 is the newest version of Mistral Large, and according to Mistral AI offers significant improvements across multilingual capabilities, math, reasoning, coding, and much more.

Overview of Mistral Large 2Mistral Large 2 is an advanced large language model (LLM) with state-of-the-art reasoning, knowledge, and coding capabilities according to Mistral AI.

Get started with Mistral Large 2 on Amazon BedrockIf you’re new to using Mistral AI models, you can request model access on the Amazon Bedrock console.

For more information about Mistral AI on Amazon Bedrock, refer to Mistral …

In this post, we dive into LLM customization using fine-tuning, exploring the key considerations for successful experimentation and how Amazon SageMaker with MLflow can simplify the process using Amazon SageMaker Pipelines.

You can do this using services such as Amazon SageMaker JumpStart and Amazon SageMaker Clarify.

It can also be done at scale, as explained in Operationalize LLM Evaluation at Scale using Amazon SageMaker Clarify and MLOps services.

To simplify this process, you can use Amazon SageMaker with MLflow and SageMaker Pipelines for fine-tuning and evaluation at scale.

Overview of SageMaker Pipelines for experimentation at scaleWe use SageMaker Pipelines to orchestrate LLM fine-…

Finding accurate answers from content in S3 using Amazon Q BusinessAfter you integrate Amazon Q Business with Amazon S3, users can ask questions about the content stored in S3.

You can configure and customize your Amazon Q web experience using either the AWS Management Console for Amazon Q or the Amazon Q API.

Follow the steps for Setting up for Amazon Q Business if you’re using Amazon Q Business for the first time.

TroubleshootingTroubleshooting your Amazon S3 connector provides information about error codes you might see for the Amazon S3 connector and suggested troubleshooting actions.

To delete the Amazon Q application, go to the Amazon Q console and, on the Applications page, select yo…

The Salesforce Einstein AI Platform team is the group supporting development of Einstein applications.

In this post, we share how the Salesforce Einstein AI Platform team boosted latency and throughput of their code generation LLM using Amazon SageMaker.

The Einstein team conducted a comprehensive evaluation of various tools and services, including open source options and paid solutions.

For instance, they needed additional features for NVIDIA’s FasterTransformer to optimize their model performance.

Through a productive collaboration with the SageMaker team, they successfully integrated this support, which initially was not available.

CloudWatch data protection log group policies for data identifiersSensitive data that’s ingested by CloudWatch Logs can be safeguarded by using log group data protection policies.

CloudWatch Logs data protection can detect the categories of sensitive data by using managed data identifiers.

Prevent changes to a CloudWatch Logs log group using an SCPTo prevent changes to a CloudWatch Logs log group using an SCP, create one that denies the specific actions related to modifying or deleting the log group.

It doesn’t restrict other actions, such as creating or deleting log streams within the log group or modifying other log group configurations.

It doesn’t restrict other actions such as creating …

Today, we are excited to announce AWS Trainium and AWS Inferentia support for fine-tuning and inference of the Llama 3.1 models.

Overview of Llama 3.1 modelsThe Llama 3.1 family of multilingual LLMs are a collection of pre-trained and instruction tuned generative models in 8B, 70B, and 405B sizes (text in/text and code out).

Fine-tune Llama 3.1 on TrainiumTo get started with fine-tuning either Llama 3.1 8B or Llama 3.1 70B, you can use the NeuronX Distributed library.

After you create the environment, you can use vLLM to deploy Llama 3.1 8/70B models on AWS Trainium or Inferentia.

In his current role, he works on optimizing training and inference of generative AI models on AWS AI chips.

Generate label data using Llama 3.1 405BBecause Llama 3.1 405B is the most capable of the Llama 3.1 collection of models, and because of its state-of-the-art math and general knowledge capabilities, we run direct inference of the questions in the AQUA-RAT dataset on Llama 3.1 405B using either SageMaker JumpStart or Amazon Bedrock.

In essence, we’re using Llama 3.1 405B as an alternative to human annotation to generate labels for the dataset.

Calculate the total cost: cost + overhead cost = $48 + $14.40 = $62.404.

Step 1: Calculate the cost price of the articleThe cost price of the article is $48.

Dr. Xin Huang is a Senior Applied Scientist for Amazon SageMaker JumpStart and Amazon SageMake…

In this post, we walk through how to discover and deploy Llama 3.1 models using SageMaker JumpStart.

Discover Llama 3.1 models in SageMaker JumpStartSageMaker JumpStart provides FMs through two primary interfaces: Amazon SageMaker Studio and the SageMaker Python SDK.

Alternatively, you can use the SageMaker Python SDK to programmatically access and utilize SageMaker JumpStart models.

Deploy Llama 3.1 models for inference using SageMaker JumpStartOn the SageMaker JumpStart landing page, you can browse for solutions, models, notebooks, and other resources.

You can find the Llama 3.1 models in the Foundation Models: Text Generation carousel.

In this post, we show how to develop an IDP solution using Anthropic Claude 3 Sonnet on Amazon Bedrock.

Solution overviewThe proposed solution uses Amazon Bedrock and the powerful Anthropic Claude 3 Sonnet model to enable IDP capabilities.

This event invokes an AWS Lambda function, responsible for invoking the Anthropic Claude 3 Sonnet model on Amazon Bedrock.

The extracted data from the Anthropic Claude 3 model is sent to an Amazon Simple Queue Service (Amazon SQS) queue.

An AWS account with an AWS Identity and Access Management (IAM) user who has permissions to DynamoDB, Lambda, Amazon Bedrock, Amazon S3, Amazon SQS, Lambda, and IAM.

On March 27, 2024, Amazon Bedrock announced a key new feature called metadata filtering and also changed the default engine.

In this post, we show you how to use the new metadata filtering feature with Knowledge Bases for Amazon Bedrock for such tabular data.

Retrieve data from the knowledge base using metadata filtering.

Retrieve data from the knowledge base using metadata filteringNow let’s retrieve some data from the knowledge base.

For this post, we use Anthropic Claude Sonnet on Amazon Bedrock for our FM, but you can choose from a variety of Amazon Bedrock models.

In this post, we’ll explore how Lili, a financial platform designed specifically for businesses, used Amazon Bedrock to build a secure and intelligent AccountantAI chatbot for small business owners.

Solution overviewThe AccountantAI chatbot provides small business owners with accurate and relevant financial accounting advice in a secure manner.

This process occurs over AWS PrivateLink for Amazon Bedrock, a protected and private connection in your VPC.

Because model providers continually enhance their offerings with innovative updates, Amazon Bedrock simplifies the ability to adopt emerging advancements in generative AI across multiple model providers.

Explore Lili’s AccountantAI feature pow…

You’re finally awake.”It’s the summer of Elder Scrolls — whether a seasoned Dragonborn or a new adventurer, dive into the legendary world of Tamriel this GFN Thursday as The Elder Scrolls V: Skyrim Special Edition joins the cloud.

Epic adventures await, along with nine new games joining the GeForce NOW library this week.

Unleash the DragonbornExperience the legendary adventures, breathtaking landscapes and immersive storytelling of the iconic role-playing game The Elder Scrolls V: Skyrim Special Edition from Bethesda Game Studios — now accessible on any device from the cloud.

Explore a vast landscape, complete quests and improve skills to develop characters in the open world of Skyrim.

Get …

VISTA-2D is a new foundational model from NVIDIA that can quickly and accurately perform cell segmentation, a fundamental task in cell imaging and spatial omics workflows that is critical to the accuracy of all downstream tasks.

Figure 2 shows an example of what the output should look like using the cell image provided in the notebook.

VISTA-2D segmentation resultsAlt: Three images show the result of the VISTA-2D segmentation: the original cell image, segmentation of all the cells from the background, and individual masks for each cell.

This function takes every individual cell segmentation and generates a feature vector.

The idea is that the resulting vector contains all the information th…

Adobe Firefly is Adobe’s family of creative generative AI models that offer new ways to ideate and create while assisting creative workflows using generative AI.

They’re designed to be safe for commercial use and were trained, using NVIDIA GPUs, on licensed content, like Adobe Stock Images, and public domain content where copyright has expired.

With the latest Reference Image feature currently in beta, users can also upload a sample image to get image results closer to their desired output.

NVIDIA will continue working with Adobe to support advanced generative AI models, with a focus on deep integration into the apps the world’s leading creators use.

Generative AI is transforming gaming, vi…

How Georgia Tech’s AI Makerspace Is Preparing the Future Workforce for AIAI is set to transform the workforce — and the Georgia Institute of Technology’s new AI Makerspace is helping tens of thousands of students get ahead of the curve.

Built in collaboration with NVIDIA, the AI Makerspace underscores Georgia Tech’s commitment to preparing students for an AI-driven future, while fostering collaboration with local schools and universities.

14:47: Georgia Tech’s AI-focused minor and coursework19:25: Raychowdhury’s insight on the intersection of AI and higher education23:33: How have industries and jobs already changed as a result of AI?

27:44: What can younger students do to prepare to get a …

The key difference is the product: TSMC produces physical semiconductor chips, while NVIDIA AI Foundry helps create custom models.

“ServiceNow is using NVIDIA AI Foundry to fine-tune and deploy models that can integrate easily within customers’ existing workflows.”The Pillars of NVIDIA AI FoundryNVIDIA AI Foundry is supported by the key pillars of foundation models, enterprise software, accelerated computing, expert support and a broad partner ecosystem.

If an NVIDIA AI Foundry customer needs assistance, NVIDIA AI Enterprise experts are on hand to help.

NVIDIA AI Foundry customers have access to a global ecosystem of partners that can provide a full range of support.

Using the NeMo platform…

To help developers efficiently fetch the best proprietary data to generate knowledgeable responses for their AI applications, NVIDIA today announced four new NVIDIA NeMo Retriever NIM inference microservices.

And with NeMo Retriever NIM microservices, developers can benefit from all of this — superpowered by their data.

When compared with alternate models, NeMo Retriever NIM microservices provided 30% fewer inaccurate answers for enterprise question answering.

Dozens of NVIDIA data platform partners are working with NeMo Retriever NIM microservices to boost their AI models’ accuracy and throughput.

Use With Other NIM MicroservicesNeMo Retriever NIM microservices can be used with NVIDIA Riva…

Team NVIDIA clinches first place across all tracks in the annual challenge with their groundbreaking AI solutions.

Team NVIDIA has triumphed at the Amazon KDD Cup 2024, securing first place Friday across all five competition tracks.

“The new trend in LLM competitions is that they don’t give you training data,” said Deotte, a senior data scientist at NVIDIA.

The team fine-tuned the just-released Qwen2-72B model using eight NVIDIA A100 Tensor Core GPUs for approximately 24 hours.

First, the team generated training datasets based on the provided examples and synthesized additional data using Llama 3 70B hosted on build.nvidia.com.

AI and accelerated computing — twin engines NVIDIA continuously improves — are delivering energy efficiency for many industries.

Why Accelerated Computing Is Sustainable ComputingAccelerated computing uses the parallel processing of NVIDIA GPUs to do more work in less time.

That’s why accelerated computing is sustainable computing.

User Experiences With Accelerated AIUsers worldwide are documenting energy-efficiency gains with AI and accelerated computing.

NVIDIA continues to drive energy efficiency for accelerated AI, helping users in science, government and industry accelerate their journeys toward sustainable computing.

Industry experts gather to share advice on starting a career in AI, highlighting technical training and certifications for career growth.

It includes free access to self-paced introductory courses and webinars on topics such as generative AI, retrieval-augmented generation (RAG) and CUDA.

DLI provides comprehensive training for generative AI, RAG, NVIDIA NIM inference microservices and large language models.

Offerings also include certifications for generative AI LLMs and generative AI multimodal that help learners showcase their expertise and stand out from the crowd.

Get started with AI Learning Essentials, the NVIDIA Deep Learning Institute and on-demand resources.

Research published earlier this month in the science journal Nature used NVIDIA-powered supercomputers to validate a pathway toward the commercialization of quantum computing.

They used these state-of-the-art resources to simulate the behavior of a certain kind of quantum computing system known as a quantum annealer.

Unlike classical computers, which process information in binary — 0s and 1s — quantum computers use quantum bits or qubits that can allow information to be processed in entirely new ways.

Unlike gate-model quantum computers, which operate by applying a sequence of quantum gates, quantum annealers allow a quantum system to evolve freely in time.

Read the full paper and learn mor…

Aside from these shared parameters, IVF-PQ adds a few additional parameters that control the compression.

An interesting fact about the IVF-PQ search is that its core component, the fine search step, doesn’t depend on the original dimensionality of the data.

Using pq_dim * pq_bits >= 128 and having ( pq_dim * pq_bits ) divide evenly by 32 maximizes the GPU memory bandwidth utilization.

n_probes is the most important search parameter, but IVF-PQ provides a few more knobs to adjust the internal workings of the algorithm.

To practice tuning IVF-PQ parameters for your dataset, check out our IVF-PQ notebook on GitHub.

In this blog post, we continue the series on accelerating vector search using cuVS.

Performance: 100-million scale datasetIndex building space and timeBuilding an IVF-PQ index involves the same k-means clustering step as an IVF-Flat index.

This result is somewhat expected, given that the IVF-PQ index is smaller than the IVF-Flat index by a similar ratio.

Getting started with IVF-PQ in RAPIDS cuVScuVS is a library for vector search and clustering on the GPU.

Many of the important building blocks in cuVS can be integrated into vector databases and other vector search libraries.

Mistral AI and NVIDIA today released a new state-of-the-art language model, Mistral NeMo 12B, that developers can easily customize and deploy for enterprise applications supporting chatbots, multilingual tasks, coding and summarization.

With a 128K context length, Mistral NeMo processes extensive and complex information more coherently and accurately, ensuring contextually relevant outputs.

Mistral NeMo comes packaged as an NVIDIA NIM inference microservice, offering performance-optimized inference with NVIDIA TensorRT-LLM engines.

Advanced Model Development and CustomizationThe combined expertise of Mistral AI and NVIDIA engineers has optimized training and inference for Mistral NeMo.

Expe…

And starting today, gamers can directly access supported PC games on GeForce NOW via Xbox.com game pages, enabling them to get into their favorite Xbox PC games even faster.

We Halve a DealTake advantage of a special new discount — one-month and six-month GeForce NOW Priority or Ultimate memberships are now 50% off until Aug. 18.

Priority members enjoy more benefits over free users, including faster access to gaming servers and gaming sessions of up to six hours.

Walk the path of the goddess in the cloud with extended gaming sessions for Ultimate and Priority members.

Ultimate members can also enjoy seeing supernatural and human worlds collide in ultrawide resolutions for an even more immer…

Kaggle string data and benchmarkingString data is complex, and the effectiveness of encoding and compression is data-dependent.

String encodings in ParquetIn Parquet, string data is represented using the byte array physical type.

Of the several encoding methods available for byte array data in Parquet, most writers default to RLE_DICTIONARY encoding for string data.

Total file size for 149 string columns by encoding method and compression method for the RAPIDS libcudf Parquet writerBy default, most Parquet writers use dictionary encoding and SNAPPY compression for string columns.

File size reduction for using delta encoding compared to dictionary encoding with plain fallbackIn Figure 3, eac…

What happens when a team of mechanical engineers get tired of looking at flat images of 3D models over Zoom?

Meet the team behind Caddy, a new CAD app for mixed reality.

They join Pascal Hartig (@passy) on the Meta Tech Podcast to talk about teaching themselves to code, disrupting the CAD software space, and how they integrated Caddy with Llama 3, and so much more!

Download or listen to the podcast episode below:You can also find the episode wherever you get your podcasts, including:The Meta Tech Podcast is a podcast, brought to you by Meta, where we highlight the work Meta’s engineers are doing at every level – from low-level frameworks to end-user features.

And if you’re interested in lea…

The key to developer velocity across AI lies in minimizing time to first batch (TTFB) for machine learning (ML) engineers.

AI Lab prevents TTFB regressions whilst enabling experimentation to develop improvements.

Optimizing TTFB helps ML engineers move fastThe overhead induced from TTFB is on the critical path for most ML development.

Here, we see the true utility of a framework like AI Lab and how it was used to facilitate this sweeping change.

O(Releases): Running a more holistic set of AI Lab tests prior to release and performing a bisect-like attribution process to find the root cause.

Improving tail utilization – the utilization level of the top 5% of the servers when ranked by utilization– within our infrastructure is imperative to operate our fleet efficiently and sustainably.

Challenges of load balancingThere are two approaches to load balancing:Routing load balancing – load balancing across replicas of a single model.

Placement load balancing – balancing load on hosts by moving replicas of a model across hosts.

It also exposed a deeper problem like spiky tail utilization, which was hidden behind the high tail utilization and was fixed once identified .

Optimizing tail utilization within IPnext thereby delivering these benefits to a broader range of expanding machine …

Prediction uncertainty is inherent, which makes it difficult to define, identify, diagnose, reproduce, and debug prediction quality issues.

However, ML prediction stability implies a consistent prediction quality shift, which is harder to distinguish.

Meta’s approach and progress towards prediction robustnessMeta has developed a systematic framework to build prediction robustness.

Additionally, fundamental understanding of training data label consistency has resulted in optimizations in training data generation for better model learning.

Regarding ML development productivity, prediction robustness techniques can facilitate model development, and improve daily operations by reducing the time…

We’re sharing how we streamline system reliability investigations using a new AI-assisted root cause analysis system.

The system uses a combination of heuristic-based retrieval and large language model-based ranking to speed up root cause identification during investigations.

We’ve streamlined our investigations through a combination of heuristic-based retrieval and large language model (LLM)-based ranking to provide AI-assisted root cause analysis.

But identifying the root cause of an issue is necessary to mitigate it properly.

We focused on building a system capable of identifying potential code changes that might be the root cause for a given investigation.

In this process, we’ve built one of the world’s largest AI training infrastructures, and it has been growing exponentially over the last years.

For AI capacity, we have optimized domains that allow for different kinds of AI capacity, very strict SLOs, and a contract with services that allows them to avoid maintenance-train interruptions, if possible.

This is quite normal in traditional capacity but challenging in AI training, since AI jobs are very closely tied to the hardware.

Since interruption costs are high for AI jobs, optimizing this relationship allowed us to significantly reduce the maintenance overhead for AI capacity.

OpsPlanner: Meta disruptive-work orchestratorCritical to AI cap…

To help personalize content, tailor and measure ads and provide a safer experience, we use cookies.

By clicking or navigating the site, you agree to allow our collection of information on and off Facebook through cookies.

Learn more, including about available controls: Cookie PolicyAccept

The next generation of Meta’s large-scale infrastructure is being built with AI in mind, including supporting new generative AI (GenAI) products and services, recommendation systems, and advanced AI research.

It’s an investment we expect will grow in the years ahead as the compute requirements to support AI models increase alongside the models’ sophistication.

Last year, we unveiled the Meta Training and Inference Accelerator (MTIA) v1, our first-generation AI inference accelerator that we designed in-house with Meta’s AI workloads in mind – specifically our deep learning recommendation models that are improving a variety of experiences across our products.

MTIA is a long-term venture to pr…

Bandwidth estimation (BWE) and congestion control play an important role in delivering high-quality real-time communication (RTC) across Meta’s family of apps.

Network characterizationAn ML model-based approach leverages time series data to improve the bandwidth estimation by using offline parameter tuning for characterized network types.

The first component is offline ML model learning using ML to categorize the network type (random packet loss versus bursty loss).

The non-time series data or dense data will pass through a dense layer (i.e., a fully connected layer).

Use case: Random packet loss classificationLet’s consider the use case of categorizing packet loss as either random or conge…

In this post, we present the design behind Logarithm, and show how it powers AI training debugging use cases.

AI training debugging with LogarithmBefore looking at Logarithm’s internals, we present support for training systems and model issue debugging, one of the prominent use cases of Logarithm at Meta.

ML model training workflows tend to have a wide range of failure modes, spanning data inputs, model code and hyperparameters, and systems components (e.g., PyTorch, data readers, checkpointing, framework code, and hardware).

Logarithm ingests both systems logs from the training stack, and model telemetry from training jobs that the stack executes.

Filter–by-callsite enables hiding known lo…

While we’ve had a long history of building AI infrastructure, we first shared details on our AI Research SuperCluster (RSC), featuring 16,000 NVIDIA A100 GPUs, in 2022.

Under the hoodOur newer AI clusters build upon the successes and lessons learned from RSC.

Our out-of-box performance for large clusters was initially poor and inconsistent, compared to optimized small cluster performance.

Commitment to open AI innovationMeta maintains its commitment to open innovation in AI software and hardware.

The future of Meta’s AI infrastructureThese two AI training cluster designs are a part of our larger roadmap for the future of AI.

These improvements helped us find and remove many unnecessary dependencies, making build graph analysis and overall build times much better.

In response to this challenge, we implemented a new solution for the packaging and distribution of Python executables – the Content Addressable Filesystem (CAF).

LazyCAF and enforcing uniform revisions: Areas for further ML iteration improvementsThe improvements we implemented have proven highly effective, drastically reducing the overhead and significantly elevating the efficiency of our ML engineers.

Faster build times and more efficient packaging and distribution of executables have reduced overhead by double-digit percentages.

We plan to enable all…

At Meta, the quest for faster model training has yielded an exciting milestone: the adoption of Lazy Imports and the Python Cinder runtime.

The challenges of adopting Lazy ImportsWhile Lazy Imports’ approach significantly improved ML development, it was not all a bed of roses.

With Lazy Imports, Meta’s ML developers are now equipped to work more efficiently, experiment more rapidly, and achieve results faster.

Here’s a glimpse into our future endeavors:Streamlining developer onboardingThe learning curve associated with Lazy Imports can be a challenge for newcomers.

Building a robust community that helps supporting paradigms and patterns that play well with Lazy Imports is one of our future …

What’s going on with generative AI (GenAI) at Meta?

In this episode of the Meta Tech Podcast, Meta engineer Pascal Hartig (@passy) speaks with Devi Parikh, an AI research director at Meta.

They cover a wide range of topics, including the history and future of GenAI and the most interesting research papers that have come out recently.

And, of course, they discuss some of Meta’s latest GenAI innovations, including:Audiobox, a foundational model for generating sound and soundscapes using natural language prompts.

And if you’re interested in AI career opportunities at Meta visit the Meta Careers page.

In this post, we will provide an overview of the end-to-end debugging workflows supported by HawkEye, components of the system, and the product surface for Meta product and monetization teams to debug AI model and feature issues.

HawkEye includes infrastructure for continuously collecting data on serving and training models, data generation, and analysis components for mining root causes.

However, significant differences indicate problems with either the training data or loss divergence (e.g., loss or gradient explosion) in the bad snapshot.

Such issues can happen for several hard-to-diagnose reasons, ranging from the complex data pipelines behind training data, to data corruptions.

HawkEye…

End-to-end (E2E) MLOps platforms promise to simplify the complicated process of building, deploying, and maintaining ML models in production.

However, while E2E MLOps platforms promise convenience and integration, they may not always align with an organization’s specific needs, existing infrastructure, or long-term goals.

She brings a wealth of experience to the discussion on end-to-end MLOps platforms.

However, she highlighted a core problem:The main challenge of using end-to-end ML platforms for your MLOps stack is that nothing works exactly as you need.

As an avid podcast fan, I’ve also learned much from listening to experienced MLOps engineers share their experiences building platforms …

Defense strategies like adversarial learning, monitoring, defensive distillation, and differential privacy improve robustness against adversarial attacks.

In this article, we’ll review common attack strategies and dive into the latest defense mechanisms for shielding machine learning systems against adversarial attacks.

Regardless of the level of access to the targeted machine learning model, adversarial attacks can be further categorized as:Evasion attacks,Data-poisoning attacks,Byzantine attacks,Model-extraction attacks.

Related Adversarial Attacks on Neural Networks: Exploring the Fast Gradient Sign Method Read moreData-poisoning attacksData-poisoning attacks are another flavor of advers…

Large Language Model (LLM): A machine-learning model that takes in a textual prompt and outputs an answer.

These documents and the user query comprise the prompt for the LLM | Source: AuthorMethods for building LLM applications with vector databasesVector databases for context retrievalThe simplest way to leverage vector databases in LLM systems is to use them to efficiently search for context that can help your LLM provide accurate answers.

Dynamic few-shot prompting: The prompt is constructed by combining the original user query and examples selected through retrieval froma vector database | Source: AuthorRelated Zero-Shot and Few-Shot Learning with LLMs Read moreHow to build LLM applicat…

Your MLflow run logs can easily be exported to the neptune.ai app using a dedicated plugin.

Seven reasons for migrating from MLflow to neptune.aiBefore diving into the migration process, let’s see how MLflow and Neptune differ and why migrating from MLflow to Neptune is worth the effort.

Export MLflow logs to neptune.ai: We will migrate your existing MLflow run logs and models to the Neptune experiment tracker.

Export MLflow logs to NeptuneDepending on whether you store your MLflow data locally or work with a remote tracking server, the export works slightly differently.

Here’s an example of how to create a Neptune tracking URI and pass it to MLflow:import os import mlflow from neptune_mlfl…

We have consolidated the separate runs table, run details, and compare runs views into one single view, freeing up a ton of space at the top of your screen.

Run groups: Level up the analysis of your experimentsTo some extent, grouping experiments has always been possible in Neptune.

But we’re now elevating run groups to be more important citizens of Neptune.

Here’s what’s new:Most importantly, you can now group experiments by string sets, e.g., tags and newly introduced group tags .

A few changes in the runs table include:We’re working on bringing the run name to the front row of the runs table.

There is no one-size-fits-all approach to implementing an ML/AI platform: Building an in-house platform can be fast for specific use cases and help maximize return on investment quickly.

Build vs buy: benefits and drawbacksDespite what marketing will have you believe, you won’t find a ready-to-use ML/AI platform off the shelf.

Many popular ML/AI tools are open-source, and it’s highly unlikely that any team building an ML/AI platform will not use some open-source component.

Related Learnings From Building the ML Platform at Stitch Fix Read moreFactors to consider in the build vs buy decision-making processInvesting in an ML/AI platform is a major decision for any company.

ConclusionThe decis…

Using the following three principles helps you build a mature ML development process: Establish a standard repository structure you can use as a scaffold for your projects.

Where traditional software development follows a widely agreed-upon streamlined process for developing and deploying, ML development is quite different.

What is a mature ML development process?

A mature ML development process is tough to implement since it doesn’t just happen organically—quite the opposite.

A mature ML development process enables teams to ship confidently and without fear of breaking production.

GitHub’s CI/CD solution, GitHub Actions, is popular because it’s directly integrated into the platform and easy to use.

The compute resources offered by GitHub Actions directly are not suitable for larger-scale ML workloads.

We’ll focus on GitHub Actions, the CI/CD platform integrated into GitHub, but the insights also apply to other CI/CD frameworks.

We can also compare two versions of our model training setup to uncover what changed between them.

Running GitHub Actions jobs on your own serversBy default, GitHub Actions executes workflows on servers hosted by GitHub, which are called “runners”.

In this article, we’ll explore how teams achieve this with Encord Active and neptune.ai.

Encord Active is part of the Encord data engine for AI that includes Annotate for data annotation and Index for data curation and management.

Using Encord Active and neptune.ai to build high-performing computer vision modelsIn this walkthrough, you will:Analyze the popular Caltech 101 dataset with Encord Active and curate good-quality data.

To launch the Encord Active web app, run the following command:%cd /ea-caltech/ !encord-active startYour browser should open a new window with Encord Active.

Next stepsIn this guide, we’ve developed a computer vision model using Encord Active and Neptune.

TL;DR Scaling machine learning (ML) experiments is a challenging process that requires efficient resource management, experiment tracking, and infrastructure scalability.

Combining neptune.ai and Kubernetes provides a robust solution for scaling ML experiments, making it easier to manage and scale experiments across multiple environments and team members.

Recommended ML Experiment Tracking: What It Is, Why It Matters, and How to Implement It Read moreUsing neptune.ai and Kubernetes as solutions for scaling ML experimentsNow that we have identified the main challenges of distributed computing, we will explore how combining neptune.ai and Kubernetes can offer a powerful solution to scale dist…

In this episode, Eduardo Bonet shares what he learned from building MLOps capabilities at GitLab as a one-person ML platform team.

I would guess these are organizations that develop regular software but would also like to use GitLab for machine learning.

There was a paragraph about how we are starting new things, like MLOps support or MLOps GitLab offering for the MLOps community.

There are two types of GitLab users: self-managed, where you can deploy your own GitLab.

The regular testing stack that you use for software development doesn’t really apply to machine learning because, by definition, machine learning involves a lot of flaky tests.

Using Optuna and a hands-on example, you will learn about the ideas behind Bayesian hyperparameter optimization, how it works, and how to perform Bayesian optimization for any of your machine-learning models.

Advantages of Bayesian optimization over other hyperparameter optimization methodsWe’ve seen that Bayesian optimization is superior to simpler hyperparameter optimization approaches because it takes into account past information.

Global optimization: Bayesian optimization is well-suited for global optimization tasks where the goal is to find the global optimum rather than just a local one.

Optimizing hyperparameter search using Bayesian optimization and OptunaOptuna is an open-source h…

We can further control the output of LLMs through parameters such as “temperature” or a “frequency penalty,” which influence an LLM’s output on a token-by-token basis.

How the Greedy Decoding, Beam Search, and Sampling post-processing techniques determine the next token to output.

How advanced techniques like frequency penalties, logit bias, and structured output give you even more control over an LLM’s output.

Before we dive into post-processing techniques for customizing LLM outputs, it’s crucial to understand how an LLM generates its output in the first place.

Adjusting the temperature parameter modifies the softmax function, influencing the diversity and predictability of a large langua…

In this article, we’ll survey the most commonly used deep learning optimization algorithms, including Gradient Descent, Stochastic Gradient Descent, and the Adam optimizer.

Understanding different optimization algorithms and their strengths and weaknesses is crucial for any data scientist training deep learning models.

Optimization in deep learning Have a look at other articles on our blog exploring aspects of optimization in deep learning: Deep Learning Model Optimization Methods: Deep learning models exhibit excellent performance but require high computational resources.

:Mini-batch Gradient DescentMini-batch Gradient Descent strikes a balance between the thorough, calculated approach of …

On top of that, neptune.ai integrates with any MLOps stack, and it just works.

Now, with less boilerplate code, you can log and visualize information from your ZenML pipeline steps (e.g., models, parameters, metrics).

You’re looking for a more visually interactive way of navigating the results produced from your ZenML pipeline runs (e.g., models, metrics, datasets).

In this example, we log a simple ZenML pipeline to Neptune using the Experiment Tracker stack component.

The example assumes that you have ZenML installed together with the Neptune integration.

Matrix multiplications (MatMuls) are pervasive throughout modern machine learning architectures. However, they are also very resource intensive and require special accelerators (GPUs). This paper explores architectures that do away with MatMuls and use quantization and recurrence to keep performance up. OUTLINE:

0:00 - Intro

2:30 - MatMul is everywhere

5:55 - Ternary accumulation as a substitute for matrix multiplication

16:35 - Replacing attention layers with recurrent layers

32:40 - Replacing dense layers with ternary channel mixing

38:30 - Language modelling results & scaling laws

45:00 - Other experimental results

48:20 - Conclusion Paper: https://arxiv.org/abs/2406.02528

Code: https://…

#rag #hallucinations #legaltech An in-depth look at a recent Stanford paper examining the degree of hallucinations in various LegalTech tools that incorporate LLMs. OUTLINE:

0:00 - Intro

1:58 - What are legal research tools and how are large language models used by them?

5:30 - Overview and abstract of the paper

9:29 - What is a hallucination and why do they occur?

15:45 - What is retrieval augmented generation (RAG)?

25:00 - Why LLMs are a bad choice when reasoning is involved

29:16 - The products that were tested

32:00 - Some shady practices by the researchers in the back and forth with the legal research companies

37:00 - Legal technology companies’ marketing claims to eliminate or solve…

xLSTM is an architecture that combines the recurrency and constant memory requirement of LSTMs with the large-scale training of transformers and achieves impressive results. Paper: https://arxiv.org/abs/2405.04517 Abstract:

In the 1990s, the constant error carousel and gating were introduced as the central ideas of the Long Short-Term Memory (LSTM). Since then, LSTMs have stood the test of time and contributed to numerous deep learning success stories, in particular they constituted the first Large Language Models (LLMs). However, the advent of the Transformer technology with parallelizable self-attention at its core marked the dawn of a new era, outpacing LSTMs at scale. We now raise a sim…

#gpt4o #sky #scarlettjohansson After the release of their flagship model GPT-4o, OpenAI finds itself in multiple controversies and an exodus of senior personnel - notably Ilya Sutskever References:

https://openai.com/index/gpt-4o-and-more-tools-to-chatgpt-free/

https://openai.com/index/hello-gpt-4o/

https://x.com/LiamFedus/status/1790064963966370209?t=rx2YBT9AdDdKPhI6dUH4zA&s=09

https://x.com/lmsysorg/status/1790097588399779991?t=rx2YBT9AdDdKPhI6dUH4zA&s=09

https://x.com/bindureddy/status/1790127425705120149?t=mMUBqFBRphx-bDuZ1j3mjQ&s=09

https://openai.com/index/improvements-to-data-analysis-in-chatgpt/

https://openai.com/index/openai-and-reddit-partnership/

https://archive.ph/jHlMm

https:/…

Paper: https://arxiv.org/abs/2403.07691 Abstract:

While recent preference alignment algorithms for language models have demonstrated promising results, supervised fine-tuning (SFT) remains imperative for achieving successful convergence. In this paper, we study the crucial role of SFT within the context of preference alignment, emphasizing that a minor penalty for the disfavored generation style is sufficient for preference-aligned SFT. Building on this foundation, we introduce a straightforward and innovative reference model-free monolithic odds ratio preference optimization algorithm, ORPO, eliminating the necessity for an additional preference alignment phase. We demonstrate, both empiri…

OUTLINE:

0:00 - Intro

0:19 - Our next-generation Meta Training and Inference Accelerator

01:39 - ALOHA Unleashed

03:10 - Apple Inks $50M Deal with Shutterstock for AI Training Data

04:28 - OpenAI Researchers, Including Ally of Sutskever, Fired for Alleged Leaking

05:01 - Adobe's Ethical Firefly AI was Trained on Midjourney Images

05:52 - Trudeau announces $2.4billion for AI-related investments

06:48 - RecurrentGemma: Moving Past Transformers for Efficient Open Language Models

07:15 - CodeGemma - an official Google release for code LLMs

07:24 - Mistral AI: Cheaper, Better, Faster, Stronger

08:08 - Vezora/Mistral-22B-v0.1

09:00 - WizardLM-2, next generation state-of-the-art-LLM

09:31 - Idefic…

Paper: https://arxiv.org/abs/2404.09173 Abstract:

While Transformers have revolutionized deep learning, their quadratic attention complexity hinders their ability to process infinitely long inputs. We propose Feedback Attention Memory (FAM), a novel Transformer architecture that leverages a feedback loop to enable the network to attend to its own latent representations. This design fosters the emergence of working memory within the Transformer, allowing it to process indefinitely long sequences. TransformerFAM requires no additional weights, enabling seamless integration with pre-trained models. Our experiments show that TransformerFAM significantly improves Transformer performance on long-…

OUTLINE:

0:00 - Intro

0:21 - Debunking Devin: "First AI Software Engineer" Upwork lie exposed!

07:24 - NeurIPS 2024 will have a track for papers from high schoolers.

13:29 - Opus can operate as a Turing machine.

13:47 - An AI-Powered, Self-Running Propaganda Machine for $105

14:27 - TechScape: How cheap, outsourced labour in Africa is shaping AI English

16:25 - Is ChatGPT Transforming Academics' Writing Style? References:

https://news.ycombinator.com/item?id=40008109&s=09

https://www.youtube.com/watch?v=tNmgmwEtoWE

https://www.youtube.com/watch?v=xE2fxcETP5E

https://twitter.com/itsandrewgao/status/1779369373737668669?t=omW3DvRNmZyce8oo0Ehf1g&s=09

https://twitter.com/0interestrates/status/17…

Google researchers achieve supposedly infinite context attention via compressive memory. Paper: https://arxiv.org/abs/2404.07143 Abstract:

This work introduces an efficient method to scale Transformer-based Large Language Models (LLMs) to infinitely long inputs with bounded memory and computation. A key component in our proposed approach is a new attention technique dubbed Infini-attention. The Infini-attention incorporates a compressive memory into the vanilla attention mechanism and builds in both masked local attention and long-term linear attention mechanisms in a single Transformer block. We demonstrate the effectiveness of our approach on long-context language modeling benchmarks, 1M …

Meta's Llama 3 is out. New model, new license, new opportunities. References:

https://llama.meta.com/llama3/

https://ai.meta.com/blog/meta-llama-3/

https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md

https://llama.meta.com/trust-and-safety/

https://ai.meta.com/research/publications/cyberseceval-2-a-wide-ranging-cybersecurity-evaluation-suite-for-large-language-models/

https://github.com/meta-llama/llama-recipes/tree/main/recipes/responsible_ai

https://llama.meta.com/llama3/license/

https://about.fb.com/news/2024/04/meta-ai-assistant-built-with-llama-3/?utm_source=twitter&utm_medium=organic_social&utm_content=thread&utm_campaign=imagineflash

https://twitter.com/minchoi/status/178277…

Links:

Homepage: https://ykilcher.com

Merch: https://ykilcher.com/merch

YouTube: https://www.youtube.com/c/yannickilcher

Twitter: https://twitter.com/ykilcher

Discord: https://ykilcher.com/discord

LinkedIn: https://www.linkedin.com/in/ykilcher If you want to support me, the best thing to do is to share out the content :) If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):

SubscribeStar: https://www.subscribestar.com/yannickilcher

Patreon: https://www.patreon.com/yannickilcher

Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq

Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2

Litecoin (LTC): LQW2TRyKYetVC8WjFkhpP…

Some updates from industry in the Machine Learning world Links:

Homepage: https://ykilcher.com

Merch: https://ykilcher.com/merch

YouTube: https://www.youtube.com/c/yannickilcher

Twitter: https://twitter.com/ykilcher

Discord: https://ykilcher.com/discord

LinkedIn: https://www.linkedin.com/in/ykilcher If you want to support me, the best thing to do is to share out the content :) If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):

SubscribeStar: https://www.subscribestar.com/yannickilcher

Patreon: https://www.patreon.com/yannickilcher

Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq

Ethereum (ETH): 0x7ad3513E3B8f66799f507…

A flurry of new models continues to appear. Links:

Homepage: https://ykilcher.com

Merch: https://ykilcher.com/merch

YouTube: https://www.youtube.com/c/yannickilcher

Twitter: https://twitter.com/ykilcher

Discord: https://ykilcher.com/discord

LinkedIn: https://www.linkedin.com/in/ykilcher If you want to support me, the best thing to do is to share out the content :) If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):

SubscribeStar: https://www.subscribestar.com/yannickilcher

Patreon: https://www.patreon.com/yannickilcher

Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq

Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC…

Flow matching is a more general method than diffusion and serves as the basis for models like Stable Diffusion 3. Paper: https://arxiv.org/abs/2210.02747 Abstract:

We introduce a new paradigm for generative modeling built on Continuous Normalizing Flows (CNFs), allowing us to train CNFs at unprecedented scale. Specifically, we present the notion of Flow Matching (FM), a simulation-free approach for training CNFs based on regressing vector fields of fixed conditional probability paths. Flow Matching is compatible with a general family of Gaussian probability paths for transforming between noise and data samples -- which subsumes existing diffusion paths as specific instances. Interestingly, …

Paper: https://arxiv.org/abs/2402.14083 Abstract:

While Transformers have enabled tremendous progress in various application settings, such architectures still lag behind traditional symbolic planners for solving complex decision making tasks. In this work, we demonstrate how to train Transformers to solve complex planning tasks and present Searchformer, a Transformer model that optimally solves previously unseen Sokoban puzzles 93.7% of the time, while using up to 26.8% fewer search steps than standard A∗ search. Searchformer is an encoder-decoder Transformer model trained to predict the search dynamics of A∗. This model is then fine-tuned via expert iterations to perform fewer search step…

Hey everyone! Thanks so much for watching this video exploring Gemini Pro 1.5 and Gemini Flash! Long Context LLMs!! This video covers 3 key tests, the classic "Lost in the Middle" exploration, using Long Context LLMs as Re-rankers in Search, and finally, testing Many-Shot In-Context Learning! I am really excited about the potential of Many-Shot In-Context Learning with DSPy's `BootstrapFewShot` and Gemini, curious to know what you think! Notebook: https://github.com/weaviate/recipes/blob/main/integrations/dspy/llms/Gemini-1.5-Pro-and-Flash.ipynb Gemini 1.5 Technical Report: https://storage.googleapis.com/deepmind-media/gemini/gemini_v1_5_report.pdf Chapters

0:00 Gemini 1.5!!

1:25 Setup and …

Hey everyone! Thank you so much for watching this overview of Llama 3 looking at the release notes and seeing a demo of how to integrate it with DSPy through Ollama and how to use DSPy's MIPRO to find the optimal prompt when using this new large language model for RAG! We are hosting an event in San Francisco on May 1st with Arize AI and Cohere, featuring a talk from Omar Khattab, the lead author of DSPy! Hope to see you there! https://lu.ma/dspy Introducing Meta Llama 3: https://ai.meta.com/blog/meta-llama-3/ Ollama Llama 3: https://ollama.com/library/llama3 Weaviate Recipes: https://github.com/weaviate/recipes/blob/main/integrations/dspy/llms/Llama3.ipynb Chapters

0:00 Llama3!!

1:28 Relea…

Hey everyone! Thank you so much for watching this overview of Command R+ showing you how you can use the new model in DSPy and a quick RAG demo, as well as walking through the details of the release post! Congratulations to the Cohere team! Super exciting times to be working with LLM systems! Introducing Command R+: A Scalable LLM Built for Business - https://txt.cohere.com/command-r-plus-microsoft-azure/ Link to demo notebook - https://github.com/weaviate/recipes/blob/main/integrations/dspy/llms/Command-R-Plus.ipynb Chapters

0:00 Welcome! Command R+!

1:12 Demo with Cohere, DSPy, and Weaviate

6:06 Command R+ Announcement Post

9:24 LLM Evals

Unfortunately, Large Language Models will not consistently follow the instructions that you give them. This is a massive problem when you are building AI systems that require a particular type of output from the previous step to feed into the next one! For example, imagine you are building a blog post writing system that first takes a question and retrieved context to output a list of topics. These topics have to be formatted in a particular way, such as a comma-separated list or a JSON of Topic objects, such that the system can continue writing the blog post! I am SUPER excited to share the 4th video in my DSPy series, diving into 3 solutions to structuring outputs in DSPy programs: (1) **…

Hey everyone! Thank you so much for watching the 3rd edition of the DSPy series, Adding Depth to DSPy Programs!! You can find the examples and links to community resources / news on https://github.com/weaviate/recipes! Chapters

0:00 Intro

0:50 Chapters Overview

5:06 Weaviate Recipes

5:24 DSPy News and Community Notes

13:51 Adding Depth to RAG Programs

18:40 Multi-Model DSPy Programs

20:18 DSPy Optimizers

25:30 Deep Dive Optimizers

27:55 Into the Optimizer Code!

37:48 Demo #1: Adding Depth to RAG

1:05:25 Demo #2: Questions to Blogs

1:07:48 Thank you so much for watching!

Hey everyone! Thank you so much for watching this tutorial on getting started with RAG programming in DSPy! This video will take you through 4 major aspects of building DSPy programs (1) Installation, settings, and Datasets with dspy.Example, (2) LLM Metrics, (3) The DSPy programming model, and (4) Optimization!! The notebook used in the video can be found here: https://github.com/weaviate/recipes/blob/main/integrations/dspy/1.Getting-Started-with-RAG-in-DSPy.ipynb All future videos, as well as additional utils like data import scripts, will be in this folder: https://github.com/weaviate/recipes/tree/main/integrations/dspy Please leave a star, it helps a lot! DSPy on GitHub: https://github.…

Hey everyone! Thank you so much for watching this explanation of DSPy! DSPy is a super exciting new framework for developing LLM programs! Pioneered by frameworks such as LangChain and LlamaIndex, we can build much more powerful systems by chaining together LLM calls! This means that the output of one call to an LLM is the input to the next, and so on. We can think of chains as programs, with each LLM call analogous to a function that takes text as input and produces text as output. DSPy offers a new programming model, inspired by PyTorch, that gives you a massive amount of control over these LLM programs. Further the Signature abstraction wraps prompts and structured input / outputs to cle…

A link to the full video is on the screen, or here for reference: https://youtu.be/W3I3kAg2J7w

I had the pleasure of being invited to give Harvey Mudd's commencement speech this year.

Reposted here with permission from the University Timestamps:

0:00 - End of Harriet Nembhard's introduction

0:45 - The cliché

2:28 - The shifting goal

5:57 - Action precedes motivation

7:02 - Timing

10:47 - Know your influence

12:05 - Anticipate change ------------------ 3blue1brown is a channel about animating math, in all senses of the word animate. If you're reading the bottom of a video description, I'm guessing you're more interested than the average viewer in lessons here. It would mean a lot to me if you chose to stay up to date on new ones, either by subscribing here on YouTube or otherwise foll…

This comes from a full video breaking down how LLMs work. The link is on the bottom of the screen (in the shorts feed at least), or here for reference: https://youtu.be/wjZofJX0v4M

This comes from a full video dissecting how LLMs work. In the shorts player, you can click the link at the bottom of the screen, or for reference: https://youtu.be/wjZofJX0v4M

Demystifying attention, the key mechanism inside transformers and LLMs.

Instead of sponsored ad reads, these lessons are funded directly by viewers: https://3b1b.co/support

Special thanks to these supporters: https://www.3blue1brown.com/lessons/attention#thanks

An equally valuable form of support is to simply share the videos. Demystifying self-attention, multiple heads, and cross-attention.

Instead of sponsored ad reads, these lessons are funded directly by viewers: https://3b1b.co/support The first pass for the translated subtitles here is machine-generated, and therefore notably imperfect. To contribute edits or fixes, visit https://translate.3blue1brown.com/ ------------------ Here are …

An introduction to transformers and their prerequisites

Early view of the next chapter for patrons: https://3b1b.co/early-attention Other recommended resources on the topic. Richard Turner's introduction is one of the best starting places:

https://arxiv.org/pdf/2304.10557.pdf Coding a GPT with Andrej Karpathy

https://youtu.be/kCc8FmEb1nY Introduction to self-attention by John Hewitt

https://web.stanford.edu/class/cs224n/readings/cs224n-self-attention-transformers-2023_draft.pdf History of language models by Brit Cruise:

https://youtu.be/OFS90-FX6pg ------------------ Timestamps 0:00 - Predict, sample, repeat

3:03 - Inside a transformer

6:36 - Chapter layout

7:20 - The premise of Deep Learni…

A link to the full video is at the bottom of the screen.

Or, for reference: https://youtu.be/aXRTczANuIs

A link to the full video answering this is at the bottom of the screen. Or, for reference: https://youtu.be/LqbZpur38nw

A link to the full video answering this is at the bottom of the screen. Or, for reference: https://youtu.be/bOXCLR3Wric

A link to the full video is at the bottom of the screen.

Or, for reference: https://youtu.be/pQa_tWZmlGs The full video this comes from proves why slicing a cone gives the same shape as the two-thumbtacks-and-string construction, which is beautiful. Editing from long-form to short by Dawid Kołodziej

A link to the full video is at the bottom of the screen.

Or, for reference: https://youtu.be/HZGCoVF3YvM Editing from long-form to short by Dawid Kołodziej

A link to the full video about Bayesian thinking is at the bottom of the screen.

Or, for reference: https://youtu.be/lG4VkPoG3ko Long-to-short editing by Dawid Kołodziej

A link to the full video is at the bottom of the screen.

Or, for reference: https://youtu.be/OkmNXy7er84 Editing from the original video into this short by Dawid Kołodziej

A link to the full video is at the bottom of the screen.

Or, for reference: https://youtu.be/Cz4Q4QOuoo8 That video answers various viewer questions about the index of refraction. Editing from long-form to short by Dawid Kołodziej

A link to the full video is at the bottom of the screen. Or, for reference: https://youtu.be/ltLUadnCyi0

❤️ Check out Weights & Biases and sign up for a free demo here: https://wandb.me/papersllm 📝 Blender 4.2 and demo files are available here:

https://www.blender.org/download/releases/4-2/

https://www.blender.org/download/demo-files/ 📝 Separable Subsurface Scattering paper:

https://users.cg.tuwien.ac.at/zsolnai/gfx/separable-subsurface-scattering-with-activision-blizzard/ 📝 My paper on simulations that look almost like reality is available for free here:

https://rdcu.be/cWPfD Or this is the orig. Nature Physics link with clickable citations:

https://www.nature.com/articles/s41567-022-01788-5 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:

Alex Bal…

❤️ Check out Lambda here and sign up for their GPU Cloud: https://lambdalabs.com/paper 📝 The paper "Diffusion Texture Painting" is available here:

https://research.nvidia.com/labs/toronto-ai/DiffusionTexturePainting/ 📝 My paper on simulations that look almost like reality is available for free here:

https://rdcu.be/cWPfD Or this is the orig. Nature Physics link with clickable citations:

https://www.nature.com/articles/s41567-022-01788-5 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:

Alex Balfanz, Alex Haro, B Shang, Benji Rabhan, Gaston Ingaramo, Gordon Child, John Le, Kyle Davis, Loyal Alchemist, Lukas Biewald, Martin, Michael Albrecht, Michae…

❤️ Check out Lambda here and sign up for their GPU Cloud: https://lambdalabs.com/paper 📝 The paper "Categorical Codebook Matching for Embodied Character Controllers" is available here:

https://github.com/sebastianstarke/AI4Animation?tab=readme-ov-file#siggraph-2024categorical-codebook-matching-for-embodied-character-controllerssebastian-starkepaul-starkenicky-hetaku-komurayuting-yeacm-trans-graph-43-4-article-142 📝 My paper on simulations that look almost like reality is available for free here:

https://rdcu.be/cWPfD Or this is the orig. Nature Physics link with clickable citations:

https://www.nature.com/articles/s41567-022-01788-5 🙏 We would like to thank our generous Patreon supporters w…

❤️ Check out Lambda here and sign up for their GPU Cloud: https://lambdalabs.com/paper 📝 The paper "NeRF-XL: Scaling NeRFs with Multiple GPUs" is available here:

https://research.nvidia.com/labs/toronto-ai/nerfxl/ 📝 My paper on simulations that look almost like reality is available for free here:

https://rdcu.be/cWPfD Or this is the orig. Nature Physics link with clickable citations:

https://www.nature.com/articles/s41567-022-01788-5 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:

Alex Balfanz, Alex Haro, B Shang, Benji Rabhan, Gaston Ingaramo, Gordon Child, John Le, Kyle Davis, Loyal Alchemist, Lukas Biewald, Martin, Michael Albrecht, Michael T…

❤️ Check out Weights & Biases and sign up for a free demo here: https://wandb.me/papers If you are interested in a sponsorship:

https://form.jotform.com/241831324457354 DeepMind's Veo:

https://deepmind.google/discover/blog/generating-audio-for-video/

https://deepmind.google/technologies/veo/ Gen-3 text to video (note: the new gen-3 requires a subscription):

https://runwayml.com/blog/introducing-gen-3-alpha/

https://runwayml.com/ai-tools/gen-3-alpha/ Video sources:

https://x.com/umpherj/status/1803048403925958862

https://x.com/c_valenzuelab/status/1807101690887475621?s=46

https://x.com/nobanksnearby/status/1807141474888298618?s=46

https://x.com/jon_barron/status/1807180191313396120?s=46

http…

❤️ Check out Lambda here and sign up for their GPU Cloud: https://lambdalabs.com/paper Our Patreon with early access: https://www.patreon.com/TwoMinutePapers 📝 The paper "LLM Critics Help Catch LLM Bugs" is available here:

https://openai.com/index/finding-gpt4s-mistakes-with-gpt-4/ 📝 My paper on simulations that look almost like reality is available for free here:

https://rdcu.be/cWPfD Or this is the orig. Nature Physics link with clickable citations:

https://www.nature.com/articles/s41567-022-01788-5 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:

Alex Balfanz, Alex Haro, B Shang, Benji Rabhan, Gaston Ingaramo, Gordon Child, John Le, Kyle Davis…

❤️ Check out Microsoft Azure AI and try it out for free:

https://azure.microsoft.com/en-us/solutions/ai 📝 The paper "Factory: Fast Contact for Robotic Assembly" is available here:

https://sites.google.com/nvidia.com/factory/ 📝 My paper on simulations that look almost like reality is available for free here:

https://rdcu.be/cWPfD Or this is the orig. Nature Physics link with clickable citations:

https://www.nature.com/articles/s41567-022-01788-5 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:

Alex Balfanz, Alex Haro, B Shang, Benji Rabhan, Gaston Ingaramo, Gordon Child, John Le, Kyle Davis, Loyal Alchemist, Lukas Biewald, Martin, Michael Albrecht…

❤️ Check out Lambda here and sign up for their GPU Cloud: https://lambdalabs.com/paper Try it out: https://lumalabs.ai/dream-machine Full "This is fine" comic strip (you have been warned!): https://www.theverge.com/2016/5/5/11592622/this-is-fine-meme-comic Ben Nash extending the video: https://x.com/bennash/status/1801694774791180658 📝 My paper on simulations that look almost like reality is available for free here:

https://rdcu.be/cWPfD Or this is the orig. Nature Physics link with clickable citations:

https://www.nature.com/articles/s41567-022-01788-5 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:

Alex Balfanz, Alex Haro, B Shang, Benji Rabha…

❤️ Check out Microsoft Azure AI and try it out for free:

https://azure.microsoft.com/en-us/solutions/ai Apple Intelligence: https://www.apple.com/apple-intelligence/ 📝 My earlier paper on brush stroke synthesis: https://users.cg.tuwien.ac.at/zsolnai/gfx/procedural-brush-synthesis-paper/ 📝 My paper on simulations that look almost like reality is available for free here:

https://rdcu.be/cWPfD Or this is the orig. Nature Physics link with clickable citations:

https://www.nature.com/articles/s41567-022-01788-5 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:

Alex Balfanz, Alex Haro, B Shang, Benji Rabhan, Gaston Ingaramo, Gordon Child, John Le, Kyle …

❤️ Check out Microsoft Azure AI and try it out for free:

https://azure.microsoft.com/en-us/solutions/ai 📝 The "Amortizing Samples in Physics-Based Inverse Rendering using ReSTIR" is available here:

https://shuangz.com/projects/psdr-restir-sa23/ Andrew Price's Blender tutorials:

https://www.youtube.com/watch?v=B0J27sf9N1Y&list=PLjEaoINr3zgEPv5y--4MKpciLaoQYZB1Z 📝 My paper on simulations that look almost like reality is available for free here:

https://rdcu.be/cWPfD Or this is the orig. Nature Physics link with clickable citations:

https://www.nature.com/articles/s41567-022-01788-5 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:

Alex Balfanz, Alex…

❤️ Check out Microsoft Azure AI and try it out for free:

https://azure.microsoft.com/en-us/solutions/ai 📝 The paper "Training language models to follow instructions with human feedback" is available here:

https://arxiv.org/abs/2203.02155 📝 My paper on simulations that look almost like reality is available for free here:

https://rdcu.be/cWPfD Or this is the orig. Nature Physics link with clickable citations:

https://www.nature.com/articles/s41567-022-01788-5 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:

Alex Balfanz, Alex Haro, B Shang, Benji Rabhan, Gaston Ingaramo, Gordon Child, John Le, Kyle Davis, Loyal Alchemist, Lukas Biewald, Martin, Mic…

❤️ Check out Lambda here and sign up for their GPU Cloud: https://lambdalabs.com/papers Flood forecasting:

https://sites.research.google/floodforecasting/

Paper: https://www.nature.com/articles/s41586-024-07145-1 Weather forecasting:

https://research.google/blog/generative-ai-to-quantify-uncertainty-in-weather-forecasting/

https://www.science.org/doi/10.1126/sciadv.adk4489 Sustainable flights:

https://sites.research.google/contrails/ 📝 My paper on simulations that look almost like reality is available for free here:

https://rdcu.be/cWPfD Or this is the orig. Nature Physics link with clickable citations:

https://www.nature.com/articles/s41567-022-01788-5 🙏 We would like to thank our generous…

❤️ Check out Lambda here and sign up for their GPU Cloud: https://lambdalabs.com/papers 📝 Papers:

https://arxiv.org/abs/2405.02957

https://arxiv.org/html/2405.11804v1 📝 My paper on simulations that look almost like reality is available for free here:

https://rdcu.be/cWPfD MedQA dataset: https://aclanthology.org/2024.lrec-main.975.pdf Or this is the orig. Nature Physics link with clickable citations:

https://www.nature.com/articles/s41567-022-01788-5 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:

Alex Balfanz, Alex Haro, B Shang, Benji Rabhan, Gaston Ingaramo, Gordon Child, John Le, Kyle Davis, Lukas Biewald, Martin, Michael Albrecht, Michael Te…

❤️ Check out Lambda here and sign up for their GPU Cloud: https://lambdalabs.com/papers 📝 The paper "LATTE3D: Large-scale Amortized Text-To-Enhanced3D Synthesis " is available here:

https://research.nvidia.com/labs/toronto-ai/LATTE3D/ 📝 My paper on simulations that look almost like reality is available for free here:

https://rdcu.be/cWPfD Or this is the orig. Nature Physics link with clickable citations:

https://www.nature.com/articles/s41567-022-01788-5 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:

Alex Balfanz, Alex Haro, B Shang, Benji Rabhan, Gaston Ingaramo, Gordon Child, John Le, Kyle Davis, Lukas Biewald, Martin, Michael Albrecht, Micha…

Подписывайтесь - https://dzen.ru/spbai !

Встерячаемся на втором и заключительном офлайне Data Fest 2024 в Питере! Программа состоит из 3 секций с небольшими вкраплениями в каждой из них :)

12:00 — 14:00, секция Advanced LLM (3 доклада) и 1 доклад ML in Marketing

14:30 — 16:10, секция Random DS (4 доклада)

16:30 — 18:10, секция ML in Manufacturing (3 доклада) и еще 1 доклад ML in Marketing :) Полное расписание:

https://ods.ai/events/fest2024-vk-spb/schedule Информация про Data Fest 2024 доступна на ODS.AI: https://ods.ai/events/fest2024-vk-spb

https://ods.ai/events/datafest2024

https://ods.ai/events/datafest2024/faq Вступить в сообщество: https://ods.ai/ Соцсети Data Fest & Course Fest: https://t.me/datafest

https://vk.com/datafest

Завершаем программу Data Fest 2024 с целым днём секций в Московском офисе Яндекса! Трансляция идёт только из главного зала Экстрополис. Сегодня в заключении программы Феста:

1. 12:00 — 14:10, секция Advanced LLM

...небольшой перерыв...

2. 14:30 — 16:30, начало секции Practical ML от Яндекса

...небольшой перерыв...

3. 17:00 — 18:30, финал секции Practical ML от Яндекса, финальные аккорды Феста в эфире! Полное расписание:

https://ods.ai/events/fest2024-yandex-msc/schedule Информация про Data Fest 2024 доступна на ODS.AI: https://ods.ai/events/fest2024-yandex-msc

https://ods.ai/events/datafest2024

https://ods.ai/events/datafest2024/faq Вступить в сообщество: https://ods.ai/ Соцсети Data Fest &…

Завершаем online программу Data Fest 2024 🎉 Надеемся что все дожили до конца — осталось совсем чуть-чуть 🤗 Сегодня в программе всего 2 секции-трека в 2 комнатах, целиком друг за другом:

1. 10:00 — 12:30, трек Reliable ML

2. 12:30 — 14:00, трек ML in Edtech Полное расписание дня:

https://ods.ai/events/df2024-2-june-online Как подключиться к spatial.chat:

https://ods.ai/events/df2024-2-june-online/networking Информация про Data Fest 2024 доступна на ODS.AI: https://ods.ai/events/datafest2024

https://ods.ai/events/datafest2024/faq Вступить в сообщество: https://ods.ai/ Соцсети Data Fest & Course Fest: https://t.me/datafest

https://vk.com/datafest

Открываем онлайн программу первого дня лета Data Fest 2024 вместе с ODS Siberia! В программе трансляции только доклады в большом зале (3307). Полное расписание:

https://ods.ai/events/fest2024-ngu-nsk/schedule Информация про Data Fest 2024 доступна на ODS.AI: https://ods.ai/events/fest2024-ngu-nsk

https://ods.ai/events/datafest2024

https://ods.ai/events/datafest2024/faq Вступить в сообщество: https://ods.ai/ Соцсети Data Fest & Course Fest: https://t.me/datafest

https://vk.com/datafest

Встречаем Московское лето во вторую и последнюю субботу Data Fest 2024! Встречаемся в гостях у Avito.Tech с супер-плотной и мощной программой:

1. 11:00 — 12:25, секция ML in Marketplace

...небольшой перерыв...

2. 12:50 — 15:00, продолжение секции ML in Marketplace

...большой перерыв...

3. 16:00 — 17:40, секция Time Series + 1 доклад MLOps

...небольшой перерыв...

4. 18:00 — 19:40, 2 доклада секции CV + 2 доклада секции Random Полное расписание:

https://ods.ai/events/fest2024-avito-msc/schedule Информация про Data Fest 2024 доступна на ODS.AI: https://ods.ai/events/fest2024-avito-msc

https://ods.ai/events/datafest2024

https://ods.ai/events/datafest2024/faq Вступить в сообщество: https://ods.a…

Второй день Data Fest 2024 в Алматы! Встречаемся уже днем в 12:00 (по местному) / 10:00 (по мск) в гостях у Citix. Сегодня в программе 4 доклада про CV и NLP, а также дискуссия после докладом. Детальное расписание:

https://ods.ai/events/fest2024-almaty/schedule Информация про Data Fest 2024 доступна на ODS.AI: https://ods.ai/events/fest2024-almaty

https://ods.ai/events/datafest2024

https://ods.ai/events/datafest2024/faq Вступить в сообщество: https://ods.ai/ Соцсети Data Fest & Course Fest: https://t.me/datafest

https://vk.com/datafest

Встречаем лето с гипер-насыщенным днём докладов секций Data Fest 2024 🎉 Сегодня у нас самый насыщенный секциями день в spatial.chat:

1. 11:00 — 16:00, трек Robotics

2. 12:00 — 13:30, трек Speech

3. 12:00 — 14:45, трек ML in Manufacturing

4. 13:00 — 14:30, трек ML in Marketing

5. 15:00 — 16:00, online-mix докладов разных треков

6. 16:00 — 17:00, трек Scoring

7. 17:00 — 20:00, трек Advanced LLM Трансляция будет поочередно заглядывать в каждую из комнат на 1-2 доклада. Посмотреть программу на свой собственный вкус можно в самом spatial.chat ODS. Полное расписание дня:

https://ods.ai/events/df2024-1-june-online Как подключиться к spatial.chat:

https://ods.ai/events/df2024-1-june-online/networki…

Провожаем весну на первой и последней пятнице Data Fest 2024! Сегодня в программе::

1. 12:00 — 17:30, трек Ужасы Медицинских Данных

2. 17:00 — 19:00, Advanced LLM трек Полное расписание дня:

https://ods.ai/events/df2024-31-may-online Как подключиться к spatial.chat:

https://ods.ai/events/df2024-31-may-online/networking Информация про Data Fest 2024 доступна на ODS.AI: https://ods.ai/events/datafest2024

https://ods.ai/events/datafest2024/faq Вступить в сообщество: https://ods.ai/ Соцсети Data Fest & Course Fest: https://t.me/datafest

https://vk.com/datafest

Впервые Data Fest приходит в Алматы! Встречаемся вечером в 19:00 (по местному) / 17:00 (по мск) в гостях у Altel Digital (пространство Tele2 Space). В программе первого дня Феста в Алматы 3 доклада: про геоналатику, составление музыкальных плейлистов и экосистему open source LLM. Детальное расписание:

https://ods.ai/events/fest2024-almaty-31may/schedule Информация про Data Fest 2024 доступна на ODS.AI: https://ods.ai/events/fest2024-almaty-31may

https://ods.ai/events/datafest2024

https://ods.ai/events/datafest2024/faq Вступить в сообщество: https://ods.ai/ Соцсети Data Fest & Course Fest: https://t.me/datafest

https://vk.com/datafest

Провожаем весну на третьем офлайн дне Data Fest 2024 в Москве! Встречаемся в гостях у Цифровой кафедры Альфа-Банка в Финансовом университете. В программе вас ждут:

1. 12:00 — 13:40, первые 3 доклада секции GeoML

...большой перерыв...

2. 14:30 — 16:10, 2 доклада GeoML, 1 про ML в производстве и 1 про Дистилляцию нейросетей

3. 16:30 — 18:10, 4 доклада секции Speech Полное расписание:

https://ods.ai/events/fest2024-alfa-mscschedule Информация про Data Fest 2024 доступна на ODS.AI: https://ods.ai/events/fest2024-alfa-msc

https://ods.ai/events/datafest2024

https://ods.ai/events/datafest2024/faq Вступить в сообщество: https://ods.ai/ Соцсети Data Fest & Course Fest: https://t.me/datafest

https://…

Привет! Ждем тебя в онлайне 31.05.2024 г. в гостях у секции Ужасы медицинских данных

Полное расписание онлайн дня доступно по ссылке https://ods.ai/events/df2024-31-may-online

Инструкция как подключиться: https://ods.ai/events/df2024-31-may-online/networking Расписание секции Ужасы медицинских данных: 12:00, Евгений Никитин, Екатерина Кондратьева Что изменилось в медицинском ML за пять лет? Взгляд стартапера и ML-инженера.

12:40, Михаил Суслов Инциденты ML-сервисов и способы их мониторинга.

13:00, Мария Гарец Supervised labeling: когда рулбук не работает.

13:30, Дарья Цыба

Не диагностикой единой: ML для дизайна терапевтических белков.

14:00, Владимир Шапошников Применение LLМ моделей в диаг…

Первый из двух офлайн дней Data Fest 2024 в Питере! В программе вас ждут:

1. 12:00 — секция про научный Open Source

2. 14:30 — секция про применения ML в Физике

3. 16:20 — доклады из секций GeoML, Time Series, и CV Полное расписание:

https://ods.ai/events/fest2024-alfa-spb/schedule Информация про Data Fest 2024 доступна на ODS.AI: https://ods.ai/events/fest2024-alfa-spb

https://ods.ai/events/datafest2024

https://ods.ai/events/datafest2024/faq Вступить в сообщество: https://ods.ai/ Соцсети Data Fest & Course Fest: https://t.me/datafest

https://vk.com/datafest

Продолжаем серию офлайнов со вторым днём Data Fest 2024 в Москве в гостях у ВТБ! В программе:

13.30-14.50 — доклады сессии NLP

15.10-16.10 — доклады сессии MLOps

16.20-18.20 — доклады сессии Scoring

18.30-19.30 — доклады сессии Data Fusion

После чего, формальная часть трансляции завершится, а нетворкиг активности живьем на площадке продолжатся 🤗 Полное расписание:

https://ods.ai/events/fest2024-vtb-msc/schedule Информация про Data Fest 2024 доступна на ODS.AI: https://ods.ai/events/fest2024-vtb-msc

https://ods.ai/events/datafest2024

https://ods.ai/events/datafest2024/faq Вступить в сообщество: https://ods.ai/ Соцсети Data Fest & Course Fest: https://t.me/datafest

https://vk.com/datafest

Открываем вторую и заключительную неделю Data Fest 2024! Сегодня в программе::

1. 14:00 — 18:30, Open Source трек

2. 15:00 — 17:30, GeoML трек

3. 17:50 — 20:20, Advanced LLM трек Полное расписание дня:

https://ods.ai/events/df2024-27-may-online Как подключиться к spatial.chat:

https://ods.ai/events/df2024-27-may-online/networking Информация про Data Fest 2024 доступна на ODS.AI: https://ods.ai/events/datafest2024

https://ods.ai/events/datafest2024/faq Вступить в сообщество: https://ods.ai/ Соцсети Data Fest & Course Fest: https://t.me/datafest

https://vk.com/datafest

Продолжаем программу Data Fest 2024 с целым днём online активностей! В программе на это воскресенье:

1. 11:00 — 14:00, Data Governance трек

2. 11:10 — 14:10, NLP трек

3. 12:00 — 13:40, CV трек

4. 12:00 — 18:00, OptimalDL трек

5. 12:00 — 14:30, Собеседования в никуда

6. 13:00 — 18:00, ML in Marketplace трек Avito.tech

7. 15:30 — 18:00, Time Series трек Полное расписание дня:

https://ods.ai/events/df2024-26-may-online Как подключиться к spatial.chat:

https://ods.ai/events/df2024-26-may-online/networking Информация про Data Fest 2024 доступна на ODS.AI: https://ods.ai/events/fest2024-vk

https://ods.ai/events/datafest2024

https://ods.ai/events/datafest2024/faq Вступить в сообщество: https://ods…

Twitch: https://www.twitch.tv/justin_helps

Discord: https://discord.gg/NbruaNW

Store: https://store.dftba.com/collections/primer

Patreon: https://www.patreon.com/primerlearning Source and further reading on the common side-blotched lizard:

Sinervo, B.; C.M. Lively (1996). "The rock–paper–scissors game and the evolution of alternative male strategies". Nature. 380 (6571): 240–243.

https://en.wikipedia.org/wiki/Common_side-blotched_lizard Made with Godot

Github: https://github.com/Primer-Learning/PrimerTools Made possible by support from these wonderful Patrons:

abledbody

Alba Caparros-Roissard

Andrew Lang

Anthony Eufemio

Brian Cloutier

Captain Chinchilla

Christoph Grabo (@asaaki)

Christy Ser…

Twitch: https://www.twitch.tv/primerjustin

Discord: https://discord.gg/NbruaNW

Store: https://store.dftba.com/collections/primer

Patreon: https://www.patreon.com/primerlearning

Twitter: @primerlearning 0:00 - Introduction

0:19 - Simulation rules

3:23 - First simulations

5:21 - Game theory analysis

8:45 - Alternate reward matrices

15:58 - Requirements for an evolutionarily stable strategy

16:69 - Discussion questions Made possible by support from these wonderful Patrons:

abledbody

Alba Caparros-Roissard

Andrew Lang

Anthony Eufemio

Brian Cloutier

Captain Chinchilla

Christy Serbus

Daniel Rolandsgard Kjellevold

Erik Broeders

Flavio Kindler

Gabriele Siino

Garrett

Guguke

James Manning

Jeff

Jeremy…

Ivanka Trump is a businesswoman, real estate developer, and former senior advisor to the President of the United States.

Please support this podcast by checking out our sponsors:– Cloaked: https://cloaked.com/lex and use code LexPod to get 25% off– Shopify: https://shopify.com/lex to get $1 per month trial– NetSuite: http://netsuite.com/lex to get free product tour– Eight Sleep: https://eightsleep.com/lex to get $350 off– ExpressVPN: https://expressvpn.com/lexpod to get 3 months freeTranscript: https://lexfridman.com/ivanka-trump-transcriptEPISODE LINKS:Ivanka’s Instagram: https://instagram.com/ivankatrumpIvanka’s X: https://x.com/IvankaTrumpIvanka’s Facebook: https://facebook.com/IvankaTru…

Andrew Huberman is a neuroscientist at Stanford and host of the Huberman Lab Podcast.

Please support this podcast by checking out our sponsors:– Eight Sleep: https://eightsleep.com/lex to get $350 off– LMNT: https://drinkLMNT.com/lex to get free sample pack– AG1: https://drinkag1.com/lex to get 1 month supply of fish oil– Shopify: https://shopify.com/lex to get $1 per month trial– NetSuite: http://netsuite.com/lex to get free product tour– BetterHelp: https://betterhelp.com/lex to get 10% offTranscript: https://lexfridman.com/andrew-huberman-5-transcriptEPISODE LINKS:Andrew’s YouTube: https://youtube.com/AndrewHubermanLabAndrew’s Instagram: https://instagram.com/hubermanlabAndrew’s Website:…

Arvind Srinivas is CEO of Perplexity, a company that aims to revolutionize how we humans find answers to questions on the Internet.

Please support this podcast by checking out our sponsors:– Cloaked: https://cloaked.com/lex and use code LexPod to get 25% off– ShipStation: https://shipstation.com/lex and use code LEX to get 60-day free trial– NetSuite: http://netsuite.com/lex to get free product tour– LMNT: https://drinkLMNT.com/lex to get free sample pack– Shopify: https://shopify.com/lex to get $1 per month trial– BetterHelp: https://betterhelp.com/lex to get 10% offEPISODE LINKS:Aravind’s X: https://x.com/AravSrinivasPerplexity: https://perplexity.ai/Perplexity’s X: https://x.com/perplexi…

Sara Walker is an astrobiologist and theoretical physicist.

She is the author of a new book titled “Life as No One Knows It: The Physics of Life’s Emergence”.

Please support this podcast by checking out our sponsors:– Notion: https://notion.com/lex– Motific: https://motific.ai– Shopify: https://shopify.com/lex to get $1 per month trial– BetterHelp: https://betterhelp.com/lex to get 10% off– AG1: https://drinkag1.com/lex to get 1 month supply of fish oilTranscript: https://lexfridman.com/sara-walker-3-transcriptEPISODE LINKS:Sara’s Book – Life as No One Knows It: https://amzn.to/3wVmOe1Sara’s X: https://x.com/Sara_ImariSara’s Instagram: https://instagram.com/alien_matterPODCAST INFO:Podcast …

Kevin Spacey is a two-time Oscar-winning actor, who starred in Se7en, the Usual Suspects, American Beauty, and House of Cards, creating haunting performances of characters who often embody the dark side of human nature.

Please support this podcast by checking out our sponsors:– ExpressVPN: https://expressvpn.com/lexpod to get 3 months free– Eight Sleep: https://eightsleep.com/lex to get $350 off– BetterHelp: https://betterhelp.com/lex to get 10% off– Shopify: https://shopify.com/lex to get $1 per month trial– AG1: https://drinkag1.com/lex to get 1 month supply of fish oilEPISODE LINKS:Kevin’s X: https://x.com/KevinSpaceyKevin’s Instagram: https://www.instagram.com/kevinspaceyKevin’s YouTube…

Roman Yampolskiy is an AI safety researcher and author of a new book titled AI: Unexplainable, Unpredictable, Uncontrollable.

Please support this podcast by checking out our sponsors:– Yahoo Finance: https://yahoofinance.com– MasterClass: https://masterclass.com/lexpod to get 15% off– NetSuite: http://netsuite.com/lex to get free product tour– LMNT: https://drinkLMNT.com/lex to get free sample pack– Eight Sleep: https://eightsleep.com/lex to get $350 offEPISODE LINKS:Roman’s X: https://twitter.com/romanyamRoman’s Website: http://cecs.louisville.edu/ryRoman’s AI book: https://amzn.to/4aFZuPbPODCAST INFO:Podcast website: https://lexfridman.com/podcastApple Podcasts: https://apple.co/2lwqZIrSp…

Charan Ranganath is a psychologist and neuroscientist at UC Davis, specializing in human memory.

He is the author of a new book titled Why We Remember.

Please support this podcast by checking out our sponsors:– Riverside: https://creators.riverside.fm/LEX and use code LEX to get 30% off– ZipRecruiter: https://ziprecruiter.com/lex– Notion: https://notion.com/lex– MasterClass: https://masterclass.com/lexpod to get 15% off– Shopify: https://shopify.com/lex to get $1 per month trial– LMNT: https://drinkLMNT.com/lex to get free sample packEPISODE LINKS:Charan’s X: https://x.com/CharanRanganathCharan’s Instagram: https://instagram.com/thememorydocCharan’s Website: https://charanranganath.comWhy W…

Paul Rosolie is a naturalist, explorer, author, and founder of Junglekeepers, dedicating his life to protecting the Amazon rainforest.

Support his efforts at https://junglekeepers.orgPlease support this podcast by checking out our sponsors:– ShipStation: https://shipstation.com/lex and use code LEX to get 60-day free trial– Yahoo Finance: https://yahoofinance.com– BetterHelp: https://betterhelp.com/lex to get 10% off– NetSuite: http://netsuite.com/lex to get free product tour– Eight Sleep: https://eightsleep.com/lex to get $350 off– Shopify: https://shopify.com/lex to get $1 per month trialTranscript: https://lexfridman.com/paul-rosolie-2-transcriptEPISODE LINKS:Paul’s Instagram: https://in…

Sean Carroll is a theoretical physicist, author, and host of Mindscape podcast.

Please support this podcast by checking out our sponsors:– HiddenLayer: https://hiddenlayer.com/lex– Cloaked: https://cloaked.com/lex and use code LexPod to get 25% off– Notion: https://notion.com/lex– Shopify: https://shopify.com/lex to get $1 per month trial– NetSuite: http://netsuite.com/lex to get free product tourTranscript: https://lexfridman.com/sean-carroll-3-transcriptEPISODE LINKS:Sean’s Website: https://preposterousuniverse.comMindscape Podcast: https://www.preposterousuniverse.com/podcast/Sean’s YouTube: https://youtube.com/@seancarrollSean’s Patreon: https://www.patreon.com/seanmcarrollSean’s Twitte…

Neil Adams is a judo world champion, 2-time Olympic silver medalist, 5-time European champion, and often referred to as the Voice of Judo.

Please support this podcast by checking out our sponsors:– ZipRecruiter: https://ziprecruiter.com/lex– Eight Sleep: https://eightsleep.com/lex to get special savings– MasterClass: https://masterclass.com/lexpod to get 15% off– LMNT: https://drinkLMNT.com/lex to get free sample pack– NetSuite: http://netsuite.com/lex to get free product tourEPISODE LINKS:Neil’s Instagram: https://instagram.com/naefightingNeil’s YouTube: https://youtube.com/NAEffectiveFightingNeil’s TikTok: https://tiktok.com/@neiladamsmbeNeil’s Facebook: https://facebook.com/NeilAdamsJudo…

Edward Gibson is a psycholinguistics professor at MIT and heads the MIT Language Lab.

Please support this podcast by checking out our sponsors:– Yahoo Finance: https://yahoofinance.com– Listening: https://listening.com/lex and use code LEX to get one month free– Policygenius: https://policygenius.com/lex– Shopify: https://shopify.com/lex to get $1 per month trial– Eight Sleep: https://eightsleep.com/lex to get special savingsTranscript: https://lexfridman.com/edward-gibson-transcriptEPISODE LINKS:Edward’s X: https://x.com/LanguageMITTedLab: https://tedlab.mit.edu/Edward’s Google Scholar: https://scholar.google.com/citations?user=4FsWE64AAAAJTedLab’s YouTube: https://youtube.com/@Tedlab-MITP…

Andrew Callaghan is the host of Channel 5 on YouTube, where he does street interviews with fascinating humans at the edges of society, the so-called vagrants, vagabonds, runaways, outlaws, from QAnon adherents to Phish heads to O Block residents and much more.

Please support this podcast by checking out our sponsors:– ShipStation: https://shipstation.com/lex and use code LEX to get 60-day free trial– BetterHelp: https://betterhelp.com/lex to get 10% off– LMNT: https://drinkLMNT.com/lex to get free sample pack– MasterClass: https://masterclass.com/lexpod to get 15% off– AG1: https://drinkag1.com/lex to get 1 month supply of fish oilTranscript: https://lexfridman.com/andrew-callaghan-transcri…

Bassem Youssef is an Egyptian-American comedian & satirist, referred to as the Jon Stewart of the Arab World.

Please support this podcast by checking out our sponsors:– AG1: https://drinkag1.com/lex to get 1 month supply of fish oil– Shopify: https://shopify.com/lex to get $1 per month trial– Eight Sleep: https://eightsleep.com/lex to get special savings– LMNT: https://drinkLMNT.com/lex to get free sample packEPISODE LINKS:Bassem’s X: https://x.com/ByoussefBassem’s Instagram: https://instagram.com/bassemBassem’s Facebook: https://facebook.com/bassemyousseftvBassem’s Website: https://bassemyoussef.xyzPODCAST INFO:Podcast website: https://lexfridman.com/podcastApple Podcasts: https://apple.co…

Tulsi Gabbard is a politician, veteran, and author of For Love of Country.

Please support this podcast by checking out our sponsors:– Riverside: https://creators.riverside.fm/LEX and use code LEX to get 30% off– ExpressVPN: https://expressvpn.com/lexpod to get 3 months free– NetSuite: http://netsuite.com/lex to get free product tour– Notion: https://notion.com/lexEPISODE LINKS:For Love of Country (book): https://amzn.to/3VLlofMTulsi’s X: https://x.com/tulsigabbardTulsi’s YouTube: https://youtube.com/@TulsiGabbardTulsi’s Podcast: https://youtube.com/@TheTulsiGabbardShowTulsi’s Instagram: https://instagram.com/tulsigabbardTulsi’s Facebook: https://facebook.com/TulsiGabbardTulsi’s Website: htt…

Senior Researcher Arindam Mitra introduces AgentInstruct.

Using raw data sources, the automated multi-agent framework can create diverse, high-quality synthetic data at scale for the post-training of small and large language models.

Microsoft Research PodcastBy Researchers across the Microsoft research communityRedmond, WAAn ongoing series of conversations bringing you right up to the cutting edge of Microsoft Research.

Printed circuit boards are abundant—in the stuff we use and in landfills.

Researcher Jake Smith and professor Aniruddh Vashisth discuss the development of vitrimer-based PCBs that perform comparably to traditional PCBs but have less environmental impact.

Learn more:

Behind every emerging technology is a great idea propelling it forward.

In the new Microsoft Research Podcast series, Ideas, members of the research community at Microsoft discuss the beliefs that animate their research, the experiences and thinkers that inform it, and the positive human impact it targets.

In this episode, host Gretchen Huizinga talks with Principal Researcher Behnaz Arzani.

But the criteria she uses to determine whether a challenge deserves her time has evolved.

These days, a problem must appeal across several dimensions: Does it answer a hard technical question?

WEISHUNG LIU: I’ve always felt like I want the things that I work on to create joy in people.

GEHRKE: Right …LIU: … And that was my interpretation of the prompt at the time.

We did a test, and I was really, really proud of it.

GEHRKE: Right …LIU: You know, we’re not really selling to the kids; we’re, kind of, selling to the parents.

And you’re like, wow, like, that’s, that’s really cool.

In the new Microsoft Research Podcast series, Ideas, members of the research community at Microsoft discuss the beliefs that animate their research, the experiences and thinkers that inform it, and the positive human impact it targets.

In this episode, host Gretchen Huizinga talks with Distinguished Scientist and Lab Director Abigail Sellen.

Today, Sellen and the teams she oversees are studying how AI could—and should—be designed for people, focusing on helping to ensure new developments support people in growing the skills and qualities they value.

Sellen explores those efforts through the AI, Cognition, and the Economy initiative—or AICE, for short—a collective of interdisciplinary scient…

Members of the research community at Microsoft work continuously to advance their respective fields.

Abstracts brings its audience to the cutting edge with them through short, compelling conversations about new and noteworthy achievements.

In this episode, Principal Research Manager Andrey Kolobov joins host Gretchen Huizinga to discuss “WindSeer: Real-time volumetric wind prediction over complex terrain aboard a small uncrewed aerial vehicle,” which was recently published in Nature Communications.

Using just synthetic data, Kolobov and his coauthors train a model to predict wind fields in real time with limited compute and measurement data, an advancement that can help small drones stay in…

O’NEILL: Yes, yes.

O’NEILL: Yeah, yeah.

Wasn’t there …O’NEILL: Yes, yes, yes.

O’NEILL: Yeah, yeah, yes.

O’NEILL: Yeah, yeah.

We are sorry, the page you requested cannot be found.

The page you are looking for could not be found or is no longer available.

Well, I’ve heard other people on your teams use words like surprise, sometimes even shock …HOSN: Yeah, yeah, there are a lot of “wow” factors.

HUIZINGA: Yeah, yeah.

AI research is moving at such speeds that I feel like we need to get accustomed to a timing of now.

HOSN: That’s right.

Well, as we close, Rafah, I want to ask a question anchored on the big idea behind AI Frontiers.

GRETCHEN HUIZINGA: Welcome to Abstracts, a Microsoft Research Podcast that puts the spotlight on world-class research in brief.

CHAKRABORTY: So satellite connectivity is nothing new and has been there for long.

So we are talking about the satellites that are at least 10 to 20 times cheaper and smaller than state-of-the-art satellites.

So the device sends some packet to the satellite; satellite sends some packet to the device—it’s all about packet exchange.

So our vision is clear: to bring 24-7 connectivity for devices anywhere on Earth with just a click of power button.

Behind every emerging technology is a great idea propelling it forward. In the new Microsoft Research Podcast series, Ideas, members of the research community at Microsoft discuss the beliefs that animate their research, the experiences and thinkers that inform it, and the positive human impact it targets. In this episode, host Gretchen Huizinga talks with Principal Researcher Kalika Bali. Inspired by an early vision of “talking computers” and a subsequent career in linguistics, Bali has spent the last two decades bringing the two together. Aided by recent advances in large language models and motivated by her belief that everyone should have access to AI in their own language, Bali and her…

And so I just want to start here: for you, Ida, what is general intelligence?

Different people at different times provide different criteria for what would be the artificial general intelligence notion.

One is artificial general intelligence and the other is humanlike intelligence or human-level intelligence.

Artificial general intelligence and humanlike, human-level intelligence—how do these two concepts relate to you?

LLORENS: So it sounds like a very extensive set of experiments across many different tasks and with many different leading AI models, and you’ve uncovered a lack of robustness across some of these different tasks.

GRETCHEN HUIZINGA: Welcome to Abstracts, a Microsoft Research Podcast that puts the spotlight on world-class research in brief.

These two examples are also the differences from other deep learning OFDFT works.

This is the generalization challenge and is one of the major challenges of deep learning method for molecular science applications.

This somehow shows the benefits of leveraging the OFDFT framework for using a deep learning method to solve molecular tasks.

You can also read it on arXiv, or you can check out the March 2024 issue of Nature Computational Science.

And so we realized that working with generative AI really parallels these different aspects of what a manager does, right.

So this requires having self-awareness of the applicability of generative AI to your workflows and maintaining an appropriate level of confidence in completing tasks manually or relying on generative AI.

For example, whether it’s worth it for you to actually learn how to work with generative AI more effectively.

But I think, given how generative AI has rolled out in the world today, I mean, a lot of the focus has been on productivity and workflows.

If you want to read the full paper on metacognition and generative AI, you can find a link at aka.ms/abstracts, or you can …

NICOLE FORSGREN: Yeah, it’s, you know, it’s, kind of, this ridiculous story.

I was there for, you know, two, three years, and I’m doing really, really well.

GEHRKE: This is just “I had a feeling.”FORSGREN: In my gut, I’m like, I’m doing really well.

After that, things were going really well, but we were also growing and scaling really, really rapidly.

Because I realized there were pieces about research that I really, really loved.

Curious about the safety of LLMs?

🤔 Join us for an insightful new episode featuring Suchin Gururangan, Young Investigator at Allen Institute for Artificial Intelligence and Data Science Engineer at Appuri.

🚀 Don't miss…

This podcast episode features Dr. Mohamed Elhoseiny, a true luminary in the realm of computer vision with over a decade of groundbreaking research.

As an Assistant Professor at KAUST, Dr. Elhoseiny's work delves into the…

Our first guest with this new format is Kyle Lo, the most senior lead scientist in the Semantic Scholar team at Allen Institute for AI (AI2), who kindly agreed to share his perspective on #Science of #Science (#scisci) o…

Weird CommunicationIgnacio Escalante Meza, an assistant professor at the University of Illinois Chicago, joins us in this episode.

Ignacio gave an overview of a harvestman and the research around understanding their behavior.

He discussed how understanding biomechanics can help the field of robotics and bioinspired design.

Ignacio discussed the benefits of studying the animals’ behavior in the field compared to simulating them in the lab.

Ignacio spoke about the debate on whether these animals learn how to communicate or whether their communication ability is innate.

Akash VenkateshwaranI am Akash Venkateshwaran, a graduate student at UBC, Vancouver.

My research focuses on developing technologies to mitigate anthropogenic noise from marine activities.

Specifically, I work on smart decision support systems for ship navigation to provide sustainable and quiet operation conditions.

Utilizing data-driven modeling and machine learning techniques, we simulate the ocean environment, involving noise generation and propagation models, as well as ocean environment and mammal trajectory models.

My goal is to create innovative solutions that ensure eco-friendly and efficient maritime operations while protecting marine life from noise pollution.

Robbie MoonRobbie is an associate professor with research interests in financial accounting and auditing.

He works on a variety of research topics relating to firms' voluntary and mandatory disclosures, nontraditional disclosures such as social media, and the intersection of audit and financial accounting research.

Robbie's research often employs advanced analytical methods, such as machine learning and natural language processing techniques, and he has been published in all of the premier accounting journals.

Prior to entering academics, he worked for four years in the advisory practice at KPMG LLP in the firm's Atlanta office.

When not working, he enjoys spending time with his wife, Linds…

iNaturalistOur guest today is Scott Loarie, the executive director at iNaturalist.

iNaturalist is an open-source platform that connects people with nature.

Scott discussed how iNaturalist goes about their mission.

Scott discussed how iNaturalist collects data to answer questions from users.

He also discussed how he built AI models to return fine-grained answers to questions.

She discussed her journey from learning HTML and CSS to learning Python and R. She also shared applications of these languages for people in academia.

She also shared the Python packages useful for computer vision in crop cultivation.

She shared how LLMs are applied in the field of plant science.

John shared his journey to coding.

He also mentioned the challenge with Lean and some state-of-the-art tools, such as LeanDojo, that apply LLMs for theorem proving.

Animal Computer InteractionIn this episode, we are joined by Ilyena Hirskyj-Douglas, an Assistant Professor at the University of Glasgow’s Computer Science Department.

She also runs the Animal-Computer Interaction (ACI) research group.

Ilyena discussed how she designed a technology that allowed parrots to communicate virtually over a video call or through a video recording.

She revealed some interesting findings from that research, including factors that affect how the parrots use the technology.

She shared additional work needed to roll out the technology to the public.

Ape GesturesToday's guest is Cat Hobaiter, a primatologist at the University of St. Andrews in the Wild Minds Lab.

The lab researches what other species, particularly wild apes, are thinking about.

They do this by looking at how apes communicate through gestures.

In her work, she goes to the rain forest to capture video of apes' gestures.

She gave examples of some of the gestures ape species make and what the gestures mean.

His research focuses on unifying artificial and natural intelligence, specifically in understanding how the common sense capabilities of animals are determined.

He focused on common sense capabilities such as object permanence.

He also discussed how he set up such experiments using reinforcement learning.

He shared the deep learning and reinforcement learning libraries he used.

Kozzy gave his thoughts on the trajectory of reinforcement learning and possibilities for more capabilities in the field.

Théo discussed some of the methods for analyzing data in his field.

Théo explained why Hidden Markov Models (HMMs) are a better approach to modeling animal movement.

He shared the data variables they typically collect.

Théo discussed the analysis to check whether the model is accurate.

Paper in focusUnderstanding the ontogeny of foraging behavior: insights from combining marine predator bio-logging with satellite-derived oceanography in hidden Markov modelsFollow our guestXGoogle ScholarR PackagesmoveHMMmomentuHMMhmmTMB

His research revolves around bioinspired navigation and orientation.

He joins us to discuss his work on Bioinspired magnetoreception and navigation using magnetic signatures as waypoints.

He also discussed the parameters, such as position, orientation, etc., used to model the animals' navigation.

He discussed the strategy for understanding the animal’s navigation and replicating the process artificially.

Learn more about our guestTaylor’s labXLinkedInOther resourcePaper: A bioinspired navigation strategy that uses magnetic signatures to navigate without GPS in a linearized northern Atlantic ocean: a simulation study

Ben is interested in evolutionary biology and uses his experience as a software developer to build a software program called SLiM.

Ben discussed what SLiM does — running genetic simulations.

Ben also discussed the technical and biological background expected of a user to create models on SLiM.

He discussed the inputs (Eidos code) the model needs to create simulations.

Learn more about SLiMSLiM webpageSLiM Workshops

Behavioral GeneticsOur guest today is Jessica Hekman, the President of Functional Dog Collaborative and a teacher of behavioral biology at Virginia Tech.

She joins us to discuss her work on behavioral genetics, particularly in dogs.

Jessica gave background information about what the Functional Dog Collaborative does.

Jessica discussed how dog breeders can breed dogs with reduced risks of undesirable traits or diseases.

She also discussed how she and her coauthors got data to understand breed behaviors that are scientific or based on our perception.

She is interested in studying and understanding the neural mechanism of the honeybee waggle dance.

Barbara and Anna shared some breakthroughs in the field of animal communication.

Anna discussed how the honeybee uses the waggle dance to communicate.

Our guests explained how they captured the waggle dance of honeybees in their hives.

She also discussed how researchers use the neural socket to understand the workings of insects' brains.

Talmo PereiraDr. Talmo Pereira is a Principal Investigator at the Salk Institute for Biological Studies in San Diego, CA where he leads a research group as a Salk Fellow.

His lab (talmolab.org) focuses on the development of deep learning-based computational methods for recognition and modeling of complex biological systems, with applications ranging from neuroscience to cancer and plant biology.

His recent work has demonstrated how advances in deep learning and computer vision can enable quantitative phenotyping of complex behaviors through the development and application of approaches for markerless motion capture (sleap.ai).

This work has been published in Nature Methods and featured in T…

Solar power now provides 6% of the world's electricity, thanks to rapid growth. Host Jon Krohn discusses the factors driving this rise, the challenges ahead, and how AI and data science are optimizing solar technologies. Tune in for insights on the future of solar power, and don't forget to like, share, and subscribe! Additional materials: www.superdatascience.com/804 Interested in sponsoring a SuperDataScience Podcast episode? Email [email protected] for sponsorship information.

Daliana Liu is a big name in data science teaching, and she has always been generous in sharing everything she knows about getting a job in data science. In this episode, she continues to extend her generosity, helping listeners define their approach to achieving a fulfilling career in data science and tech. This episode is brought to you by AWS Inferentia (go.aws/3zWS0au) and AWS Trainium (go.aws/3ycV6K0), by Babbel (www.babbel.com/superdata), the science-backed language-learning platform, and by Gurobi (gurobi.com/sds), the Decision Intelligence Leader. Interested in sponsoring a SuperDataScience Podcast episode? Email [email protected] for sponsorship information. In this epis…

How to grab investor interest with your AI startup idea, revisiting algorithms, and helping practitioners ensure AI safety with regulatory frameworks and beyond: This month, you missed a whole bunch of great interviews.

How to grab investor interest with your AI startup idea, revisiting algorithms, and helping practitioners ensure AI safety with regulatory frameworks and beyond: This month, you missed a whole bunch of great interviews. But don’t worry, Jon Krohn is here to recap all the best bits for you! Additional materials: www.superdatascience.com/802 Interested in sponsoring a SuperDataScience Podcast episode? Email [email protected] for sponsorship information.

Merged LLMs are the future, and we’re exploring how with Mark McQuade and Charles Goddard from Arcee AI on this episode with Jon Krohn.

Learn how to combine multiple LLMs without adding bulk, train more efficiently, and …

Merged LLMs are the future, and we’re exploring how with Mark McQuade and Charles Goddard from Arcee AI on this episode with Jon Krohn. Learn how to combine multiple LLMs without adding bulk, train more efficiently, and dive into different expert approaches. Discover how smaller models can outperform larger ones and leverage open-source projects for big enterprise wins. This episode is packed with must-know insights for data scientists and ML engineers. Don’t miss out! Interested in sponsoring a SuperDataScience Podcast episode? Email [email protected] for sponsorship information. In this episode you will learn:

• Explanation of Charles' job title: Chief of Frontier Research [03:…

The SuperDataScience Podcast is celebrating its 800th episode!

Host Jon Krohn speaks to his grandmother, Annie, about growing up at a time when so many technologies we take for granted today were yet to be developed.

No-code games with GenAI, the creative possibilities of LLMs, and our proximity to AGI: In this episode, Jon Krohn talks to Andrey Kurenkov about what turned him from an AGI skeptic to a positivist.

You’ll also hear abou…

No-code games with GenAI, the creative possibilities of LLMs, and our proximity to AGI: In this episode, Jon Krohn talks to Andrey Kurenkov about what turned him from an AGI skeptic to a positivist. You’ll also hear about his wildly popular podcast “Last Week in AI” and how the NVIDIA-backed startup Astrocade is helping videogame enthusiasts to create their own games through generative AI. A must-listen! This episode is brought to you by AWS Inferentia and AWS Trainium. Interested in sponsoring a SuperDataScience Podcast episode? Email [email protected] for sponsorship information. In this episode you will learn:

• All about The Gradient and Last Week in AI [10:42]

• All about As…

Claude 3.5 Sonnet, Anthropic’s newest model, is making waves in the AI community.

This mid-size model outshines the larger Claude 3 Opus in tasks like code generation, content creation, and document summarization, and it…

Claude 3.5 Sonnet, Anthropic’s newest model, is making waves in the AI community. This mid-size model outshines the larger Claude 3 Opus in tasks like code generation, content creation, and document summarization, and it’s twice as fast. In this episode of The Super Data Science Podcast, Jon Krohn discusses its top-notch performance across benchmarks like MMLU, GPQA, and HumanEval, along with its improved machine vision capabilities. Plus, learn about the new Artifacts UI feature, which makes managing generated content easier by displaying outputs side-by-side with inputs. Tune in to find out why Claude 3.5 Sonnet is setting new standards in AI.

Additional materials: www.superdatascience.co…

Dr. Rosanne Liu, Research Scientist at Google DeepMind and co-founder of the ML Collective, shares her journey and the mission to democratize AI research.

She explains her pioneering work on intrinsic dimensions in deep …

Dr. Rosanne Liu, Research Scientist at Google DeepMind and co-founder of the ML Collective, shares her journey and the mission to democratize AI research. She explains her pioneering work on intrinsic dimensions in deep learning and the advantages of curiosity-driven research. Jon and Dr. Liu also explore the complexities of understanding powerful AI models, the specifics of character-aware text encoding, and the significant impact of diversity, equity, and inclusion in the ML community. With publications in NeurIPS, ICLR, ICML, and Science, Dr. Liu offers her expertise and vision for the future of machine learning. Interested in sponsoring a SuperDataScience Podcast episode? Email natalie@…

Want to feel optimistic about your day?

In this Friday episode, Simon Kuestenmacher talks to Jon Krohn about demography: What it is, why it’s so important, and why its forecasts should give us reason to hope for a better…

Want to feel optimistic about your day? In this Friday episode, Simon Kuestenmacher talks to Jon Krohn about demography: What it is, why it’s so important, and why its forecasts should give us reason to hope for a better future. In an increasingly globalized world, and with an aging population in countries with the biggest GDPs, demography is more valuable than ever. Additional materials: www.superdatascience.com/796 Interested in sponsoring a SuperDataScience Podcast episode? Email [email protected] for sponsorship information.

Join us in this exciting episode of the Data Science at Home podcast.

It’s all about GPUs.

We’ll take you on a journey through the inner workings of these powerful processors, explaining how they handle complex computations and drive everything from gaming graphics to scientific simulations.

Whether you’re a budding programmer or a tech enthusiast, understanding GPUs is key to unlocking new levels of performance and efficiency in your projects.

Tune in and get ready to turbocharge your tech knowledge!

In this episode, we sit down with Ryan Smith, Founder of QFunction LLC, to explore how AI and machine learning are revolutionizing cybersecurity.

With over 8 years of experience, including work at NASA’s Jet Propulsion Laboratory, Ryan shares insights on the future of threat detection and prevention, the challenges businesses face in maintaining effective cybersecurity, and the ethical considerations of AI implementation.

Learn about cost-effective strategies for small businesses, the importance of collaboration in combating cyber threats, and how QFunction tailors its AI solutions to meet diverse industry needs.

QFunction does cybersecurity differently.

By relying on scientific breakthroug…

In this last episode of the series “Rust in the Cosmos” we speak about what happens in space, what projects are currently active and what happened in the past that we can learn from?

What about Rust and space applications?

Build robotics applications in minutes, not months.

Amethix works to create and maximize the impact of the world’s leading corporations and startups, so they can create a better future for everyone they serve.

CommunitiesIntrepid AI, AeroRust, BytenookIntrepid AI Discord https://discord.gg/cSSzche6Cthttps://discord.gg/cSSzche6Ct Intrepid AI website: https://intrepid.aiAeroRust Discord invite: https://discord.com/invite/6jJyx5nEUqAeroRust website: AeroRust.orgReferences

In this episode of “Rust in the Cosmos” we delve into the challenges of building embedded applications for space.

Did you know that once you ship your app to space… you can’t get it back?

Build robotics applications in minutes, not months.

Amethix works to create and maximize the impact of the world’s leading corporations and startups, so they can create a better future for everyone they serve.

CommunitiesAeroRust, Intrepid, BytenookAeroRust Discord invite: https://discord.com/invite/6jJyx5nEUqAeroRust website: AeroRust.orgIntrepid AI Discord https://discord.gg/cSSzche6Cthttps://discord.gg/cSSzche6Ct Intrepid AI website: https://intrepid.aiReferences

In this episode of “Rust in the Cosmos” we delve into the challenge of testing software for… ehm … spaceHow can Rust help?

Build robotics applications in minutes, not months.

Amethix works to create and maximize the impact of the world’s leading corporations and startups, so they can create a better future for everyone they serve.

We provide solutions in AI/ML, Fintech, Defense, Robotics and Predictive maintenance.

CommunitiesAeroRust, Intrepid, BytenookAeroRust Discord invite: https://discord.com/invite/6jJyx5nEUqAeroRust website: AeroRust.orgIntrepid AI Discord https://discord.gg/cSSzche6Cthttps://discord.gg/cSSzche6Ct Intrepid AI website: https://intrepid.aiReferences

In this inaugural episode of “Rust in the Cosmos,” we delve into the intricacies of communication in space and some of the challenges in space application development.

In this episode we delve into the dynamic realm of game development and the transformative role of artificial intelligence (AI).

Join Frag, Jim and Mike as they explore the current landscape of game development processes, from initial creative ideation to the integration of AI-driven solutions.

With Mike’s expertise as a software executive and avid game developer, we uncover the potential of AI to revolutionize game design, streamline development cycles, and enhance player experiences.

SponsorsIntrepid AI is an AI assisted all-in-one platform for robotics teams.

Build robotics applications in minutes, not months.

In this episode, join me and the Kaggle Grand Master, Konrad Banachewicz, for a hilarious journey into the zany world of data science trends.

From algorithm acrobatics to AI, creativity, Hollywood movies, and music, we just can’t get enough.

It’s the typical episode with a dose of nerdy comedy you didn’t know you needed.

Buckle up, it’s a data disco, and we’re breaking down the binary!

SponsorsIntrepid AI is an AI assisted all-in-one platform for robotics teams.

In this episode of Data Science at Home, we explore the game-changing impact of low-code solutions in robotics development.

Discover how these tools bridge the coding gap, simplify integration, and enable trial-and-error development.

We’ll also uncover challenges with traditional coding methods using ROS.

Join us for a concise yet insightful discussion on the future of robotics!

Join us in a dynamic conversation with Yori Lavi, Field CTO at SQream, as we unravel the data analytics landscape.

From debunking the data lakehouse hype to SQream’s GPU-based magic, discover how extreme data challenges are met with agility.

Yori shares success stories, insights into SQream’s petabyte-scale capabilities, and a roadmap to breaking down organizational bottlenecks in data science.

Dive into the future of data analytics with SQream’s commitment to innovation, leaving legacy formats behind and leading the charge in large-scale, cost-effective data projects.

Tune in for a dose of GPU-powered revolution!

In this episode from a month ago, join me as we unravel the controversial CEO firing at OpenAI in December 2023.

I share my insights on the events, decode the intricacies, and explore what lies ahead for this influential organization.

Don’t miss this concise yet insightful take on the intersection of leadership and artificial intelligence innovation.

SponsorLearn what the new year holds for ransomware as a service, Active Directory, artificial intelligence and more when you download the 2024 Arctic Wolf Labs Predictions Report today at arcticwolf.com/datascience

!!WARNING!!

Due to some technical issues the volume is not always constant during the show.

I sincerely apologise for any inconvenienceFrancescoIn this episode, I speak with Richie Cotton, Data Evangelist at DataCamp, as he delves into the dynamic intersection of AI and education.

Richie, a seasoned expert in data science and the host of the podcast, brings together a wealth of knowledge and experience to explore the evolving landscape of AI careers, the skills essential for generative AI technologies, and the symbiosis of domain expertise and technical skills in the industry.

Dive into the world of Data Science at Home with our latest episode, where we explore the dynamic relationship between Artificial Intelligence and the redemption of open source software.

In this thought-provoking discussion, I share my insights on why now, more than ever, is the opportune moment for open source to leave an indelible mark on the field of AI.

Join me as I unpack my opinions and set expectations for the near future, discussing the pivotal role open source is set to play in shaping the landscape of data science and artificial intelligence.

Don’t miss out—tune in to gain a deeper understanding of this revolutionary intersection!

This episode is available as YouTube stream at htt…

In this captivating podcast episode, join renowned financial expert Chris Skinner as he delves into the fascinating realm of the future of money.

From cryptocurrencies to government currencies, the metaverse to artificial intelligence (AI), Skinner explores the intricate interplay between technology and humanity.

Gain valuable insights as he defines the future of money, examines the potential impact of cryptocurrencies on traditional government currencies, and addresses the advantages and disadvantages of digital currencies.

Brace yourself for an enlightening discussion on the integration of AI in the financial sector and its potential impact on humanity.

Once subscribed, you get full acces…