Your request has been blocked due to a network policy.

If you're running a script or application, please register or sign in with your developer credentials here.

Additionally make sure your User-Agent is not empty and is something unique and descriptive and try again.

if you're supplying an alternate User-Agent string, try changing back to default as that can sometimes result in a block.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please file a ticket here.

Your request has been blocked due to a network policy.

If you're running a script or application, please register or sign in with your developer credentials here.

Additionally make sure your User-Agent is not empty and is something unique and descriptive and try again.

if you're supplying an alternate User-Agent string, try changing back to default as that can sometimes result in a block.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please file a ticket here.

Your request has been blocked due to a network policy.

If you're running a script or application, please register or sign in with your developer credentials here.

Additionally make sure your User-Agent is not empty and is something unique and descriptive and try again.

if you're supplying an alternate User-Agent string, try changing back to default as that can sometimes result in a block.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please file a ticket here.

Your request has been blocked due to a network policy.

If you're running a script or application, please register or sign in with your developer credentials here.

Additionally make sure your User-Agent is not empty and is something unique and descriptive and try again.

if you're supplying an alternate User-Agent string, try changing back to default as that can sometimes result in a block.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please file a ticket here.

Your request has been blocked due to a network policy.

If you're running a script or application, please register or sign in with your developer credentials here.

Additionally make sure your User-Agent is not empty and is something unique and descriptive and try again.

if you're supplying an alternate User-Agent string, try changing back to default as that can sometimes result in a block.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please file a ticket here.

Your request has been blocked due to a network policy.

If you're running a script or application, please register or sign in with your developer credentials here.

Additionally make sure your User-Agent is not empty and is something unique and descriptive and try again.

if you're supplying an alternate User-Agent string, try changing back to default as that can sometimes result in a block.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please file a ticket here.

Your request has been blocked due to a network policy.

If you're running a script or application, please register or sign in with your developer credentials here.

Additionally make sure your User-Agent is not empty and is something unique and descriptive and try again.

if you're supplying an alternate User-Agent string, try changing back to default as that can sometimes result in a block.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please file a ticket here.

Your request has been blocked due to a network policy.

If you're running a script or application, please register or sign in with your developer credentials here.

Additionally make sure your User-Agent is not empty and is something unique and descriptive and try again.

if you're supplying an alternate User-Agent string, try changing back to default as that can sometimes result in a block.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please file a ticket here.

Your request has been blocked due to a network policy.

If you're running a script or application, please register or sign in with your developer credentials here.

Additionally make sure your User-Agent is not empty and is something unique and descriptive and try again.

if you're supplying an alternate User-Agent string, try changing back to default as that can sometimes result in a block.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please file a ticket here.

Your request has been blocked due to a network policy.

If you're running a script or application, please register or sign in with your developer credentials here.

Additionally make sure your User-Agent is not empty and is something unique and descriptive and try again.

if you're supplying an alternate User-Agent string, try changing back to default as that can sometimes result in a block.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please file a ticket here.

Your request has been blocked due to a network policy.

If you're running a script or application, please register or sign in with your developer credentials here.

Additionally make sure your User-Agent is not empty and is something unique and descriptive and try again.

if you're supplying an alternate User-Agent string, try changing back to default as that can sometimes result in a block.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please file a ticket here.

Your request has been blocked due to a network policy.

If you're running a script or application, please register or sign in with your developer credentials here.

Additionally make sure your User-Agent is not empty and is something unique and descriptive and try again.

if you're supplying an alternate User-Agent string, try changing back to default as that can sometimes result in a block.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please file a ticket here.

Your request has been blocked due to a network policy.

If you're running a script or application, please register or sign in with your developer credentials here.

Additionally make sure your User-Agent is not empty and is something unique and descriptive and try again.

if you're supplying an alternate User-Agent string, try changing back to default as that can sometimes result in a block.

if you think that we've incorrectly blocked you or you would like to discuss easier ways to get the data you want, please file a ticket here.

Effective ML with Limited Data: Where to StartA launch pad for tackling projects with small datasetsPhoto by Google DeepMind: https://www.pexels.com/photo/an-artist-s-illustration-of-artificial-intelligence-ai-this-image-depicts-how-ai-can-help-humans-to-understand-the-complexity-of-biology-it-was-created-by-artist-khyati-trehan-as-part-17484975/Machine Learning (ML) has driven remarkable breakthroughs in computer vision, natural language processing, and speech recognition, largely due to the abundance of data in these fields. However, many challenges — especially those tied to specific product features or scientific research — suffer from limited data quality and quantity. This guide provi…

Sebastian Raschka has helped demystify deep learning for thousands through his books, tutorials and teachingsSebastian Raschka has helped shape how thousands of data scientists and machine learning engineers learn their craft. As a passionate coder and proponent of open-source software, a contributor to scikit-learn and the creator of the mlxtend library, his code runs in production systems worldwide. But his greatest impact are through his teachings — his books Machine Learning with PyTorch and Scikit-Learn, Machine Learning Q and AI and Build a Large Language Model (From Scratch) have become essential guides for practitioners navigating the complex landscape of modern AI.Drawing from over…

A new cutting edge video generation tool, and the theory behind itContinue reading on Towards Data Science »

From Raw Data to Stunning Visuals: LLMs in ActionContinue reading on Towards Data Science »

I created a graph storage from dozens of annual reports (with tables)Continue reading on Towards Data Science »

Exploring the latest advancements in time seriesContinue reading on Towards Data Science »

What Did I Learn from Building LLM Applications in 2024? — Part 2An engineer’s journey to building LLM-powered applicationsIllustration of building AI application (image by author — generated using DALLE-3)In part 1 of this series, we discussed use case selection, building a team and the importance of creating a prototype early into your LLM-based product development journey. Let’s pick it up from there — if you are fairly satisfied with your prototype and ready to move forward, start with planning a development approach. It’s also crucial to decide on your productionizing strategy from an early phase.With recent advancements with new models and a handful of SDKs in market, it is easy to fe…

Practical insights for a data-driven approach to model optimizationContinue reading on Towards Data Science »

Practical insights for a data-driven approach to model optimizationContinue reading on Towards Data Science »

Practical insights for a data-driven approach to model optimizationContinue reading on Towards Data Science »

Questions that guide architectural decisions to balance functional requirements with non-functional ones, like latency and scalabilityContinue reading on Towards Data Science »

Data Analyst is probably the most underrated job in the data industryContinue reading on Towards Data Science »

Photo by julien Tromeur on UnsplashMAS Is All You Need: Supercharge Your Retrieval-Augmented Generation (RAG) with a Multi-Agent SystemHow to build a Multi-Agent RAG with AG2 and ChromaDBRetrieval-Augmented Generation (RAG) systems have improved rapidly in recent years. Ideally, we can distinguish their evolution into three phases: in the pre-LLM era, information retrieval systems primarily relied on traditional search algorithms and indexing techniques. These systems were limited in their ability to understand context and generate human-like responses. Then, LLMs entered the scene, resulting in a drastic paradigm shift. Now, there are agents and another paradigm shift is happening.But let’…

Mathematics and statistics, once the primary guides of machine learning research, now struggle to provide immediate insight into the latest breakthroughs.

This shift has prompted speculation about mathematics’ diminished role in machine learning research moving forward.

It is also the way that symmetries are usually leveraged when performing computations (for example, in machine learning).

One can reasonably argue that diagrammatic descriptions of well-known constructions, like products, are not useful for the machine learning researcher.

However, as we’ve demonstrated, while mathematics may not maintain the same role in machine learning research that it has held in the past, the success of…

Let's jump back to the 1970s, when Roger Schank introduced his "restaurant script" as a kind of dialogue system [1].

The minimum requirement we could have for a dialogue system is that it can stay on the task we gave them.

Concluding marksI have reviewed the making of current LLM dialogue systems, how and why it is insufficient.

The following are two research questions that I’m mostly excited about:(1) Better monitoring and control of dialogue systems with steering techniques.

CitationFor attribution of this in academic contexts or books, please cite this work as:Kenneth Li, "From prediction to purpose: a tutorial on LLM dialogue system", The Gradient, 2024.

This leads to our second conclusion: We need plausible positive visions of a society with capable AI, grounded in wellbeing.

The rest of this post describes in more detail (1) what we mean by AI that benefits our wellbeing, (2) the need for positive visions for AI grounded in wellbeing, and (3) concrete leverage points to aid in the development and deployment of AI in service of such positive visions.

In diving into the philosophy of flourishing, wellbeing economics, or psychological theories of human wellbeing, one encounters many interesting, compelling, but seemingly incompatible ideas.

The case so far is that we need positive visions for society with capable AI, grounded in individual a…

Looked at from another angle, there is much more noise than signal in financial data.

Another financial market application of LLMs might be synthetic data creation [4,8].

Then precious real market data could be employed to fine-tune the predictions and determine precisely the optimal speed to trade.

Financial market practitioners are often interested in extreme events, the times when trading strategies are more likely to experience significant gains or losses.

CitationFor attribution in academic contexts or books, please cite this work asRichard Dewey and Ciamac Moallemi, "Financial Market Applications of LLMs," The Gradient, 2024

All of these terms (“AI”, “gender”, and “bias”) can be somewhat overused and ambiguous.

A Short History of Studying Gender Bias in AIHere, I cover a very small sample of papers I’ve found influential studying gender bias in AI.

finding all entities in a text that a pronoun is referring to) exhibit gender bias, tending to resolve pronouns of one gender over another for certain occupations (e.g.

This article mainly focused on gender bias — and particularly, on binary gender.

AcknowledgementsThis post was originally posted on Art Fish IntelligenceCitationFor attribution in academic contexts or books, please cite this work asYennie Jun, "Gender Bias in AI," The Gradient, 2024@article{Jun2024bia…

As a general sequence model backbone, Mamba achieves state-of-the-art performance across several modalities such as language, audio, and genomics.

Here we’ll discuss:The advantages (and disadvantages) of Mamba (🐍) vs Transformers (🤖),Analogies and intuitions for thinking about Mamba, andWhat Mamba means for Interpretability, AI Safety and Applications.

The Mamba BlockLike a Transformer made up of stacked transformer blocks, Mamba is made up of stacked Mamba blocks as above.

The Mamba authors write, “the efficiency vs. effectiveness tradeoff of sequence models is characterised by how well they compress their state”.

Thanks to Gonçalo for reading an early draft, Jaden for the nnsight library …

This is a manifestation of a paradigm known as Open-Endedness AI that attempts emulate long term tasks.

Welcome to The Sequence Opinion where we debate some unique perspectives in the word of AI.

Open-Endedness AI represents a paradigm shift in artificial intelligence research, moving beyond task-specific systems towards AI capable of continuous learning, adaptation, and innovation.

This essay will explore the key principles, research methods, and challenges associated with Open-Endedness AI, as well as evaluate its potential for realization.

Key Principles of Open-Endedness AI

Created Using MidjourneyIn today’s edition of TheSequence Engineering, we are going to discuss one of my favorite AI engineering stacks that I have been actively using in the last few months.

llama.cpp is an open-source C/C++ library designed for efficient inference of large language models (LLMs), particularly those in the LLaMA family.

Developed by Georgi Gerganov, it implements Meta's LLaMA architecture with optimizations for various hardware configurations, including resource-constrained devices.

Architecture OverviewThe architecture of llama.cpp is built upon the foundation of the original LLaMA models, which are based on the transformer architecture.

However, llama.cpp incorporates se…

Created Using MidjourneyToday we will Discuss:An introduction to our new series about RAG.

💡 ML Concept of the Day: A New Series About Retrieval Augmented Generation(RAG)As our first new series of 2025, we would like to cover one of the simplest but most active areas in generative AI.

We are talking about retrieval augmented generation or how we often refer to it: RAG.

Conceptually, RAG is an architectural framework that enhances the functionality of large language models (LLMs) by incorporating external data retrieval mechanisms.

This capability is particularly beneficial in applications such as customer support and knowledge management, where timely and precise responses are critical.

You can subscribe to The Sequence below:📝 Editorial: NVIDIA AI Software Party at a Hardware ShowThe name NVIDIA is immediately associated with computing hardware and, in the world of AI, GPUs.

In several editions of this newsletter, we have highlighted NVIDIA’s rapidly growing AI software stack and aspirations.

NVIDIA NIM MicroservicesNVIDIA’s NIM (NVIDIA Inference Microservices) is a significant leap forward in the integration of AI into modern software systems.

Cosmos PlatformNVIDIA’s Cosmos Platform takes AI into the realm of robotics, autonomous vehicles, and vision AI applications.

AI Enterprise Software PlatformNVIDIA’s commitment to enterprise AI is reflected in its AI Enterprise Sof…

Created Using MidjourneyGiven the recent news about Microsoft open sourcing Phi-4, I thought it would be a good timing to dive into some of its technical details.

Microsoft Phi was been credited with starting the small language model(SLM) movement as an alternative to the “intelligence by scale” approach followed by the large AI labs.

Released a couple of years ago as part of the famous paper “Textbooks is All You Need”, every release of Phi brings new innovations in terms of data quality and training.

Not so small anymore, Phi-4 is a 14-billion parameter language model that emphasizes the importance of data quality in achieving performance comparable to, or even exceeding, much larger mode…

This is a manifestation of a paradigm known as Open-Endedness AI that attempts emulate long term tasks.

Open-Endedness AI has its root in evolutionary algoritms applied to long term interactions with a given environment .

Welcome to The Sequence Opinion where we debate some unique perspectives in the word of AI.

Open-Endedness AI represents a paradigm shift in artificial intelligence research, moving beyond task-specific systems towards AI capable of continuous learning, adaptation, and innovation.

This essay will explore the key principles, research methods, and challenges associated with Open-Endedness AI, as well as evaluate its potential for realization.

Created Using MidjourneyWelcome to The Sequence Engineering where we discuss core AI engineering topics, frameworks, platforms, implementation techniques etc.

The Sequence Knowledge: Continuing with educational topics and related research.

The Sequence Engineering: A standalone edition dedicated to engineering topics such as frameworks, platforms, and case studies.

I’ve started three AI companies in the last 18 months so have a lot of opinions about engineering topics.

The Sequence Research: Covering current research papers.

The Sequence Research: Covering current research papers.

In this series, we explored the fundamentals of knowledge distillations as well as its most important variations:TS Knowledge 445: Introduced the series and reviewed of one of the first papers about knowledge distillation.

TS Knowledge 453: Covers the principles of cross modal distillation including UC Berkeley’s paper about cross modal distillation for supervision transfer.

TS Knowledge 461: Discusses the challenges of knowledge distillation.

Advantages of Knowledge Distillation

The Sequence Engineering: A standalone edition dedicated to engineering topics such as frameworks, platforms, and case studies.

The Sequence Research: Covering current research papers.

Is Reasoning Exclusive to Massive Models or do Small Models Have a Chance?

This essay explores whether small language models (SLMs) can develop reasoning abilities comparable to their larger counterparts.

We will discuss the nature of reasoning in LLMs, the techniques that enhance reasoning in SLMs, and challenge the assumption that reasoning is exclusive to larger models.

Created Using MidjourneyPretraining foundation models is often perceived as a capability reserved for big AI labs.

Compute coordination ,data orchestration, constant experiments and the AI talent requirement are some of the challenges that make pretraining AI models prohibited for most organizations.

In that sense, lowering the bar for pretraining foundation models.

Fast-LLM is an open-source library specifically designed for training Large Language Models (LLMs) with a focus on speed, scalability, and cost-efficiency.

Developed by ServiceNow Research’s Foundation Models Lab, Fast-LLM aims to empower AI professionals, researchers, and enterprises in pushing the boundaries of generative AI.

Created Using MidjourneyIn this issue:An overview about the challenges of knowledge distillation.

💡 ML Concept of the Day: The Challenges of Knowledge DistillationThroughout this series, we have explored the different techniques and benefits of knowledge distillations for foundation models.

Knowledge distillation in foundation models presents several unique challenges that stem from the inherent complexity and scale of foundation models.

One of the primary difficulties lies in the substantial capacity gap between the teacher (foundation model) and the student model.

Foundation models often contain billions of parameters, while the goal of distillation is to create a much smaller, more effic…

Now onto today’s subject:In a recent essay in this newsletter we explored the transition from an emphasis in pretraining to post-training in foundation models.

One of those is the transition from preference tuning with methods such as the famous RLHF to reward modeling.

Initially, preference tuning served as the de facto approach to alignment, relying on human-annotated datasets to guide model behavior.

Although preference tuning yields significant benefits in terms of helpfulness and safety, it struggles to incorporate the full range of human intentions, values, and context-specific nuances.

The Rise of Foundation Models and Preference Tuning

Now onto today’s subject:Anthropic is widely known as OpenAI’s main rival and the creators of the Claude model.

However, in recent months, Anthropic has been expanding beyond its core model capabilities into areas such as agentic workflows and developer frameworks.

One of their most recent open source release sits precisely at the interception of these two areas.

The Model Context Protocol (MCP) is an open-source standard designed to connect AI assistants with various data sources, such as content repositories, business tools, and development environments.

This essay provides a technical exploration of MCP, focusing on its core architecture and key concepts.

In this issue:An overview of quantized distillation.

A review of Google DeepMind’s paper on model quantization and distillation.

💡 ML Concept of the Day: Understanding Quantized DistillationTo conclude our series about knowledge distillation, I would like to dive into one of the most sophisticated methods that combines distillation and quantization.

Quantized distillation has emerged as a powerful technique for compressing and optimizing deep neural networks, combining the benefits of knowledge distillation and quantization.

By leveraging the soft targets produced by the teacher model, quantized distillation can help mitigate the accuracy degradation typically associated with aggressive qua…

The recent Nobel Prize for groundbreaking advancements in protein discovery underscores the transformative potential of foundation models…Continue reading on SyncedReview »

Continue reading on SyncedReview »

Recent advancements in training large multimodal models have been driven by efforts to eliminate modeling constraints and unify…Continue reading on SyncedReview »

The Transformer architecture, introduced by Vaswani et al. in 2017, serves as the backbone of contemporary language models. Over the years…Continue reading on SyncedReview »

Large Language Models (LLMs) have become indispensable tools for diverse natural language processing (NLP) tasks. Traditional LLMs operate…Continue reading on SyncedReview »

Language models (LMs) based on transformers have become the gold standard in natural language processing, thanks to their exceptional…Continue reading on SyncedReview »

Continue reading on SyncedReview »

Navigation is a fundamental skill for any visually-capable organism, serving as a critical tool for survival. It enables agents to locate…Continue reading on SyncedReview »

The landscape of vision model pre-training has undergone significant evolution, especially with the rise of Large Language Models (LLMs)…Continue reading on SyncedReview »

A foundation model refers to a pre-trained model developed on extensive datasets, designed to be versatile and adaptable for a range of…Continue reading on SyncedReview »

Continue reading on SyncedReview »

While large language models (LLMs) dominate the AI landscape, Small-scale Large Language Models (SLMs) are gaining traction as…Continue reading on SyncedReview »

Consistency models (CMs) are a cutting-edge class of diffusion-based generative models designed for rapid and efficient sampling. However…Continue reading on SyncedReview »

The field of text-to-image synthesis has advanced rapidly, with state-of-the-art models now generating highly realistic and diverse images…Continue reading on SyncedReview »

In cognitive science, human thought processes are commonly divided into two systems: the fast, intuitive System 1 and the slower…Continue reading on SyncedReview »

Такие модели можно обучать с нуля и это дорого, нужен кластер с GPU (видеокарты) и много данных.

В домене текст-картинка бывают открытые модели, типа Stable Diffusion, Kandinsky и FLUX, бывают закрытые, типа DALL-E.Открытую модель можно дообучать разными способами.

Борис СтругацкийОсобенности: Для личностей типа Стругацких или Бродского, качественных фотографий крайне мало, но много и не надо.

Можно и фразу.

Владимир СурдинАлексей СемихатовВидео с их лекциями можно найти повсеместно, начать можно с канала Вселенная плюс на YouTube и в телеграм.

Один из них — поддержание стабильной орбиты пучка частиц (траектории, по которой происходит движение), которая критически важна для точности экспериментов.

Вот, кстати, наша статья про то, как сейсмические вибрации будут влиять на орбиту пучка в СКИФ: Beam Stability .

Классические подходы к стабилизации орбиты пучка в ускорителяхДля стабилизации орбиты используют датчики положения пучка (BPM) и магнитные корректоры.

По словам авторов получились следующие преимущества:Ускорение процесса коррекции орбиты и повышение точности по сравнению с классическими методами, такими как SVD.

Задача агента:Автоматическое восстановление орбиты пучка заряженных частиц за ограниченное время и минимальное коли…

В этой статье мы разберемся, чему научилась новая GPT o1, и как это повлияет на дальнейшую эволюцию ИИ.

Компания утверждает, что для них сброс счётчика линейки моделей к единичке знаменует собой переход к новой парадигме, и что эта нейросеть и вовсе демонстрирует новый уровень возможностей ИИ.

На форумах и в Твиттере была куча обсуждений, предвосхищений и хайпа, на фоне которых планка ожиданий некоторых людей взлетела до небес.

Издание Bloomberg рассказало, что в ходе внутренней демонстрации OpenAI показали концепцию из пяти уровней, помогающую отслеживать прогресс в создании ИИ.

Однако на уровне GPT-5 прирост в навыках может быть совсем другим (как в лучшую, так и в худшую сторону).

И в том, и в другом случаях объяснение действия не связано с реальным мотивом его сделать, и там, и там рождается поддельное (но правдоподобно звучащее) объяснение причин.

Просто сейчас это не воспринимается всерьёз, ведь LLM не распространены и не становятся ядром бизнес-процессов, включающих принятие решений.

Один и тот же текст запроса+ответа подаётся в модель, и производится оценка вероятности получить именно такой ответ при фиксированном запросе.

Это и желание продолжать существовать/жить, и нежелание умирать, и рассуждения об эмоциях и контроле.

Потому что абстракции, потому что обобщение, потому что это ровно то, за что мы ценим модели.

В ней авторы выделили 4 этапа зрелости, грубо можно разделить компании по частоте запусков экспериментов:на этапе Crawl компания проводит эксперимент раз в месяц (примерно 10 экспериментов в год);на этапе Walk – раз в неделю (примерно 50 экспериментов в год);на этапе Run – ежедневно (примерно 250 экспериментов в год);на этапе Fly – более 1000 экспериментов в год.

Более четкое разделение основывается на некоторых важных свойствах компаний, таких как: наличие команды платформы экспериментов, возможности автоматического расчета метрик, распространенности экспериментов в компании, самодостаточность продуктовых команд в проведении экспериментов, влияние экспериментов на компанию и другое.

Точка …

В ней авторы выделили 4 этапа зрелости, грубо можно разделить компании по частоте запусков экспериментов:на этапе Crawl компания проводит эксперимент раз в месяц (примерно 10 экспериментов в год);на этапе Walk – раз в неделю (примерно 50 экспериментов в год);на этапе Run – ежедневно (примерно 250 экспериментов в год);на этапе Fly – более 1000 экспериментов в год.

Более четкое разделение основывается на некоторых важных свойствах компаний, таких как: наличие команды платформы экспериментов, возможности автоматического расчета метрик, распространенности экспериментов в компании, самодостаточность продуктовых команд в проведении экспериментов, влияние экспериментов на компанию и другое.

Точка …

Mlflow Experiments and Mlflow RunsMLflow Experiments и MLflow Runs - это основные абстракции для структурирования проекта.

mlflow run mlproject --entry-point hyperparameters-tuning --env-manager conda --experiment-name Cancer_Classification --run-name Hyperparameters_Search -P n-trials=10Посмотрим на результаты в MLflow UI.

artifact_path: model flavors: python_function: data: model.xgb env: conda: conda.yaml virtualenv: python_env.yaml loader_module: mlflow.xgboost python_version: 3.11.4 xgboost: code: null data: model.xgb model_class: xgboost.sklearn.XGBClassifier model_format: xgb xgb_version: 2.0.3 mlflow_version: 2.14.2 model_size_bytes: 35040 model_uuid: 516954aae7c94e91adeed9df76cb405…

Как это определить и как предсказывать?

- Да... упс.. правда, они же уже определяют целевой признакВовремя не выкрутился, пришел фидбек, что я не понимаю разницу между обычными признаками (а.к.а.

Не, не делал, только DDP?

NVIDIA ищет единорогов, крутых и в рисече, и в инженерии.

Вакансия в Лондоне была, и в итоге они взяли моего бывшего коллегу: он уже в Лондоне и с чуть большим опытом в рекомендашках, явно лучше подходит.

ChatGPT 4 был значительно улучшен по сравнению с ChatGPT 3.5, что делает его более мощным инструментом.

Вам тоже надо учиться — учиться выстраивать взаимоотношение с ChatGPT, учиться общаться с ним.

Помните, вы всегда можете уточнить любую строчку в ответе ChatGPT и, в большинстве случаев, получить исчерпывающий ответ, который, вероятнее всего, обогатит вас новыми знаниями.

Вот несколько идей:Откройте с ChatGPT новый чат и в нём отправьте запрос в другой форме, желательно с новыми деталями.

И поэтому, когда с ChatGPT не удаётся что-то сделать с первого раза за 2–5 минут, возникает возмущение: “Ну, как так?!”.

In machine learning projects, achieving optimal model performance requires paying attention to various steps in the training process. But before focusing on the technical aspects of model training, it is important to define the problem, understand the context, and analyze the dataset in detail. Once you have a solid grasp of the problem and data, […]

The post Tips for Effectively Training Your Machine Learning Models appeared first on MachineLearningMastery.com.

Reinforcement Learning (RL) is a type of machine learning. It trains an agent to make decisions by interacting with an environment. This article covers the basic concepts of RL. These include states, actions, rewards, policies, and the Markov Decision Process (MDP). By the end, you will understand how RL works. You will also learn how […]

The post Principles of Reinforcement Learning: An Introduction with Python appeared first on MachineLearningMastery.com.

Deep learning is a subset of machine learning that has become a cornerstone in many technological breakthroughs. At the core of deep learning, it’s a model inspired by the human brain, which we call a neural network. Contrary to the traditional machine learning model, deep learning can automatically find feature representations from data. That’s why […]

The post 5 Tips for Getting Started with Deep Learning appeared first on MachineLearningMastery.com.

Feature engineering is an important step in the machine learning pipeline. It is the process of transforming data in its native format into meaningful features to help the machine learning model learn better from the data. If done right, feature engineering can significantly enhance the performance of machine learning algorithms. Beyond the basics of understanding […]

The post Tips for Effective Feature Engineering in Machine Learning appeared first on MachineLearningMastery.com.

Using machine learning to solve real-world problems is exciting. But most eager beginners jump straight to model building—overlooking the fundamentals—resulting in models that aren’t very helpful. From understanding the data to choosing the best machine learning model for the problem, there are some common mistakes that beginners often tend to make. But before we go […]

The post 5 Common Mistakes in Machine Learning and How to Avoid Them appeared first on MachineLearningMastery.com.

Photography has been around for more than a century. There are many old photos around, and probably your family has some, too. Limited by the camera and film of the time, you may have photos of low resolution, blurry, or with folds or scratches. Restoring these old photos and making them like new ones taken […]

The post Stable Diffusion Project: Reviving Old Photos appeared first on MachineLearningMastery.com.

Today’s digital landscape has never been so diverse. Every individual and company selects their preferred tools and operating systems, creating a diverse technological system. However, this diversity often leads to compatibility issues, making it hard to ensure application performance across different environments. This is where Docker plays a key role as an indispensable tool for […]

The post The Ultimate Beginner’s Guide to Docker appeared first on MachineLearningMastery.com.

Stable Diffusion has taken the AI art world by storm, empowering users to generate stunning and imaginative visuals with just a few text prompts. This opens exciting possibilities for creatives, including crafting impactful commercial posters. In this post, we’ll delve into using Stable Diffusion to design a compelling poster for a product. After finishing this […]

The post Stable Diffusion Project: Commercial Poster appeared first on MachineLearningMastery.com.

Introduction Here’s a something that new machine learning practitioners figure out almost immediately: not all datasets are created equal. It may now seem obvious to you, but had you considered this before undertaking machine learning projects on a real world dataset? As an example of a single class vastly outnumbering the rest, take for instance […]

The post 5 Effective Ways to Handle Imbalanced Data in Machine Learning appeared first on MachineLearningMastery.com.

Introduction Choosing the right machine learning model for your data is of major importance in any data science project. The model you select will have a significant impact on the insights you derive from your data, and ultimately determine the usefulness of a project. In this article, we aim to provide practical tips to help […]

The post Tips for Choosing the Right Machine Learning Model for Your Data appeared first on MachineLearningMastery.com.

Many people write in their jobs. Not everyone is a novel writer; some write technical documentation, business plans, news articles, and even blog posts. In those writings, illustrations are not essential but often good to have. They are decorations, interpretations, or visual explanations of the text. However, you probably do not want to spend too […]

The post Stable Diffusion Project: Creating Illustration appeared first on MachineLearningMastery.com.

If you are a machine learning student, researcher, or practitioner, it is crucial for your career growth to have a deep understanding of how each algorithm works and the various techniques to enhance model performance. Nowadays, many individuals tend to focus solely on the code, data, and pre-trained models, often without fully comprehending the machine […]

The post 5 Free Books on Machine Learning Algorithms You Must Read appeared first on MachineLearningMastery.com.

Stable Diffusion is a powerful tool that helps you generate pictures. It is fun to play with the generative AI tool. But it would be useful if the tool could help you in a real job. In this post, you will see how you can leverage the power of Stable Diffusion to work on something […]

The post Stable Diffusion Project: Word Art appeared first on MachineLearningMastery.com.

As a data professional, you should also know how to build predictive models with machine learning to solve business problems. And if you’re interested in machine learning, you’re probably also looking for the best resources to get going. Well, you can always choose a self-paced online course that best aligns with your learning preferences. But […]

The post 5 Free YouTube Channels Dedicated to Machine Learning Education appeared first on MachineLearningMastery.com.

If you’re looking to make a career in data science, you probably know that machine learning is one of the most in-demand skills. Whether you are a beginner looking to break into the field or an experienced professional aiming to level up your expertise, selecting the right machine learning course is super important. So how […]

The post Tips for Choosing the Right Machine Learning Course appeared first on MachineLearningMastery.com.

If AI can be superhuman at Go, surely AI can be slightly-worse-than-experts at Destruct-o-Match if we try?

Step 0: Is Making a Destruct-o-Match AI Against Neopets Rules?

I believe the precedent is in favor of a Destruct-o-Match AI being okay.

As long as I’m the one inputting moves the Destruct-o-Match AI recommends, I should be okay.

To write a game AI, we first need to implement the rules of the game in code.

I realize how late this is, but I didn’t get a post out while o1 was fresh, and still feel like writing one despite it being cold.

(Also, OpenAI just announced they’re going to ship new stuff starting tomorrow so it’s now or never to say something.)

OpenAI o1 is a model release widely believed (but not confirmed) to be a post-trained version of GPT-4o.

If true, that makes this video especially useful for understanding OpenAI o1.

Which I suppose is part of why I’m talking about o1 rather than building o1.

I expected to fill that void with more blog writing, but that’s not what happened.

The puzzles are great though, and if that’s good enough for you, I had fun with that.

Undertale YellowUndertale Yellow is a fantastic fan game, that’s been in development for 7 years and comes out feeling like a canon entry made by Toby Fox.

markdown 15837 2024 - 01 - 11 - ai - timelines - 2024. markdown 1939 2024 - 01 - 21 - mh - 2024. markdown 5076 2024 - 03 - 23 - crew - battle .

markdown 826 2024 - 04 - 30 - puzzlehunting - 201. markdown 8641 2024 - 07 - 08 - tragedies - of - reality .

There’s often a conflation between the research field of AI safety and the community of AI safety.

Me thinking AI safety is important is not an endorsement for or against anything else in the broader meme space it came from.

Historically, AI safety work did not appeal to me because of how theoretical it was.

I’m aware of the arguments that most AI safety work so far has either been useless or not that different from broader AI work.

Those who care about safety a lot call this safetywashing, the stapling of “safety” to work that does not advance safety.

I would extend it to reality is complicated, relative to code, and in robotics you’re often pushing a messy reality into an abstraction nice enough for code to act on it.

Robotics research relies on building new bridges between reality and software, but that happens outside of robotics too.

Any software that interfaces with reality will have imperfect knowledge of that reality.

However, that means all the messiness of reality is coming for a field that historically does a bad job at considering reality.

I consider the world of bits to be as much a part of reality as the world of atoms.

Most people will have more fun if they solve puzzles than if they don’t, but you don’t have to solve puzzles quickly to have fun.

I’m still going to explain the solving strategies I’ve learned, but puzzle solving is really an activity where you learn by doing.

Puzzle solving often involves relating two parts of the puzzle together.

Search everythingHonestly, a lot of puzzle solving is about taking random parts of the puzzle and throwing them into a search engine.

Bringing This TogetherTo showcase these strategies together, here is a puzzle I remember speedrunning especially quickly: The Three Little Pigs from Hunt 20 2.1 Puzzle Hunt.

This means we can reduce all the crew battle outcomes down to a single \(n \times n\) matrix, which I’ll call the crew battle matrix.

So I Wrote Some Python Code to Compute \(f\) for Arbitrary Crew BattlesLet’s consider the RPS crew battle again.

Here’s the matchup matrix:and here’s what my code outputs for the crew battle matrix, assuming optimal play.

But also, this suggests that crew battle strategy really isn’t that important in the first place!

If your crew loses, you don’t get to blame bad crew battle strategy.

From these excerpts, sociotechnical AI seems to have something to do with being people-centred, having social-purpose, understanding and reducing risks, building trust, widening opportunities, centring rights and values, being accountable for claims, and taking a system-wide view on AI.

To do this we will need more than technical AI research; and that is the role of sociotechnical AI research.

This expresses a hope for significant public benefit, but again, the burden of proof lies with AI designers, and so requires new forms of sociotechnical AI research.

Further developing all these varied forms for social and community involvement is another focus area for sociotechnical AI.

Sociotechnic…

At Youtube, the relationships between system parameters and metrics often seem simple — straight-line models sometimes fit our data well.

To find optimal values of two parameters experimentally, the obvious strategy would be to experiment with and update them in separate, sequential stages.

We address uncertainty in estimates $\widehat \beta$ of model coefficients with subsampling, our usual strategy for variance estimation in online experiments[12].

If our experiments were smaller then we might need to model our metrics with Poisson, Binomial or other GLMs instead.

We sometimes use $L^1$ regularization when we fit models to our live experiment data to select the most important non-linear c…

Superficially this looks fine, but of course the problem is hidden in how do we arrive with these equivalences?

Let's take a closer look: Kurzweil claims we should be seeing insect brain capability in $1000 computer in 2001.

Alright what would we expect from an insect brain?

It's 2024 at the time of writing this, and the best military technology that could benefit from an insect brain like capabilities is still on remote control.

That is of course if we actually get an insect brain this year.

Both of these assumptions seem reasonable and general and we can quickly see where Gaussian distribution should start popping out.

Since Gaussian distribution is fully defined by just two numbers - mean and variance, we can, by collecting enough data rather precisely estimate these values.

Below I simulate 400x200=80000 independent pixels, each taking a random value between zero and one [code available here].

These distributions we often refer to as "fat tails" [for putting much more "weight" in the tail of the distribution, as in outside of the "central" part] are really weird and problematic.

Let's get back to why we see Gaussian distribution anywhere at all?

Theory of Mind (ToM) is the ability to understand and reflect on the mental states of others.

Although this capability is crucial for human interaction, testing on Large Language Models (LLMs) reveals that they possess only a rudimentary understanding of it.

Although the most capable closed-source LLMs have come close to human performance on some ToM tasks, they still perform poorly on complex variations of the task that involve more structured reasoning.

In this work, we utilize the concept of "pretend-play", or ``Simulation Theory'' from cognitive psychology to propose ``Decompose-ToM'': an LLM-based inference algorithm that improves model performance on complex ToM tasks.

We recursively …

Semantic segmentation is essential for comprehending images, but the process necessitates a substantial amount of detailed annotations at the pixel level.

Unsupervised domain adaptation (UDA) for semantic segmentation is a technique that uses virtual data with labels to train a model and adapts it to real data without labels.

Some recent works use contrastive learning, which is a powerful method for self-supervised learning, to help with this technique.

We analyze the limitations of these works and propose a novel framework called Pseudo-label Guided Pixel Contrast (PGPC), which overcomes the disadvantages of previous methods.

Furthermore, our approach can enhance the performance of other U…

This paper proposes the ViT Token Constraint and Multi-scale Memory bank (TCMM) method to address the patch noises and feature inconsistency in unsupervised person re-identification works.

Many excellent methods use ViT features to obtain pseudo labels and clustering prototypes, then train the model with contrastive learning.

However, ViT processes images by performing patch embedding, which inevitably introduces noise in patches and may compromise the performance of the re-identification model.

This paper introduces the ViT Token Constraint to mitigate the damage caused by patch noises to the ViT architecture.

The proposed Multi-scale Memory enhances the exploration of outlier samples and …

Diffusion models represent a class of generative models that produce data by denoising a sample corrupted by white noise.

In physics, the renormalization group offers a fundamental framework for linking different scales and giving an accurate coarse-grained model.

Here we introduce a renormalization group-based diffusion model that leverages multiscale nature of data distributions for realizing a high-quality data generation.

Through reversing the renormalization group flows, our model is able to generate high-quality samples in a coarse-to-fine manner.

The proposed method alleviates the need for data-dependent tuning of hyperparameters in the generative diffusion models, showing promise fo…

Retrieval Augmented Generation (RAG) has emerged as a solution, enhancing LLMs by integrating real time data retrieval to provide contextually relevant and up-to-date responses.

Agentic Retrieval-Augmented Generation (Agentic RAG) transcends these limitations by embedding autonomous AI agents into the RAG pipeline.

This integration enables Agentic RAG systems to deliver unparalleled flexibility, scalability, and context awareness across diverse applications.

This survey provides a comprehensive exploration of Agentic RAG, beginning with its foundational principles and the evolution of RAG paradigms.

It presents a detailed taxonomy of Agentic RAG architectures, highlights key applications in…

To support future algorithm development, here we introduce our first-of-its-kind abdominal CT DIR benchmark dataset, comprising large numbers of highly accurate landmark pairs on matching blood vessel bifurcations.

Abdominal CT image pairs of 30 patients were acquired from several public repositories as well as the authors' institution with IRB approval.

2) Matching image patches were manually identified between two CTs of each image pair 3) Vessel bifurcation landmarks were labeled on one image of each image patch pair.

4) Image patches were deformably registered, and landmarks were projected onto the second image.

This dataset is a first-of-its-kind for abdominal DIR validation.

Understanding how misleading and outright false information enters news ecosystems remains a difficult challenge that requires tracking how narratives spread across thousands of fringe and mainstream news websites.

To do this, we introduce a system that utilizes encoder-based large language models and zero-shot stance detection to scalably identify and track news narratives and their attitudes across over 4,000 factually unreliable, mixed-reliability, and factually reliable English-language news websites.

Running our system over an 18 month period, we track the spread of 146K news stories.

Using network-based interference via the NETINF algorithm, we show that the paths of news narratives a…

Medical image anonymization aims to protect patient privacy by removing identifying information, while preserving the data utility to solve downstream tasks.

In this paper, we address the medical image anonymization problem with a two-stage solution: latent code projection and optimization.

In the projection stage, we design a streamlined encoder to project input images into a latent space and propose a co-training scheme to enhance the projection process.

In the optimization stage, we refine the latent code using two deep loss functions designed to address the trade-off between identity protection and data utility dedicated to medical images.

Source codes are available at https://github.co…

Large Language Models (LLMs) have shown strong promise as rerankers, especially in ``listwise'' settings where an LLM is prompted to rerank several search results at once.

However, this ``cascading'' retrieve-and-rerank approach is limited by the bounded recall problem: relevant documents not retrieved initially are permanently excluded from the final ranking.

Adaptive retrieval techniques address this problem, but do not work with listwise rerankers because they assume a document's score is computed independently from other documents.

In this paper, we propose an adaptation of an existing adaptive retrieval method that supports the listwise setting and helps guide the retrieval process its…

We present Mantis Shrimp, a multi-survey deep learning model for photometric redshift estimation that fuses ultra-violet (GALEX), optical (PanSTARRS), and infrared (UnWISE) imagery.

Machine learning is now an established approach for photometric redshift estimation, with generally acknowledged higher performance in areas with a high density of spectroscopically identified galaxies over template-based methods.

In comparison to tabular models, image models have additional design complexities: it is largely unknown how to fuse inputs from different instruments which have different resolutions or noise properties.

The Mantis Shrimp model estimates the conditional density estimate of redshift us…

Group theory has been used in machine learning to provide a theoretically grounded approach for incorporating known symmetry transformations in tasks from robotics to protein modeling.

In these applications, equivariant neural networks use known symmetry groups with predefined representations to learn over geometric input data.

We propose MatrixNet, a neural network architecture that learns matrix representations of group element inputs instead of using predefined representations.

MatrixNet achieves higher sample efficiency and generalization over several standard baselines in prediction tasks over the several finite groups and the Artin braid group.

We also show that MatrixNet respects gro…

Neural Radiance Fields (NeRF) often struggle with reconstructing and rendering highly reflective scenes.

Recent advancements have developed various reflection-aware appearance models to enhance NeRF's capability to render specular reflections.

However, the robust reconstruction of highly reflective scenes is still hindered by the inherent shape ambiguity on specular surfaces.

Observing the critical role of surface normals in parameterizing reflections, we introduce a transmittance-gradient-based normal estimation technique that remains robust even under ambiguous shape conditions.

Combined with a reflection-aware appearance model, our proposed method achieves robust reconstruction and high-…

Polysomnographic sleep analysis is the standard clinical method to accurately diagnose and treat sleep disorders.

It is an intricate process which involves the manual identification, classification, and location of multiple sleep event patterns.

In this paper we propose a multi-task deep-learning approach for the simultaneous detection of sleep events and hypnogram construction in one single pass.

Taking as reference state-of-the-art methodology for object-detection in the field of Computer Vision, we reformulate the problem for the analysis of multi-variate time sequences, and more specifically for pattern detection in the sleep analysis scenario.

We investigate the performance of the resu…

Class-incremental fault diagnosis requires a model to adapt to new fault classes while retaining previous knowledge.

Extracting discriminative features from few-shot fault data is challenging, and adding new fault classes often demands costly model retraining.

Moreover, incremental training of existing methods risks catastrophic forgetting, and severe class imbalance can bias the model's decisions toward normal classes.

To tackle these issues, we introduce a Supervised Contrastive knowledge distiLlation for class Incremental Fault Diagnosis (SCLIFD) framework proposing supervised contrastive knowledge distillation for improved representation learning capability and less forgetting, a novel …

For privacy and security concerns, the need to erase unwanted information from pre-trained vision models is becoming evident nowadays.

In real-world scenarios, erasure requests originate at any time from both users and model owners, and these requests usually form a sequence.

To address them, we first propose Group Sparse LoRA (GS-LoRA).

To further extend GS-LoRA to more practical scenarios, we incorporate prototype information as additional supervision and introduce a more practical approach, GS-LoRA++.

We conduct extensive experiments on face recognition, object detection and image classification and demonstrate that our method manages to forget specific classes with minimal impact on oth…

The meanings and relationships of words shift over time.

This phenomenon is referred to as semantic shift.Research focused on understanding how semantic shifts occur over multiple time periods is essential for gaining a detailed understanding of semantic shifts.However, detecting change points only between adjacent time periods is insufficient for analyzing detailed semantic shifts, and using BERT-based methods to examine word sense proportions incurs a high computational cost.To address those issues, we propose a simple yet intuitive framework for how semantic shifts occur over multiple time periods by leveraging a similarity matrix between the embeddings of the same word through time.We c…

Electroencephalogram (EEG) signals have emerged as a promising modality for biometric identification.

While previous studies have explored the use of imagined speech with semantically meaningful words for subject identification, most have relied on additional visual or auditory cues.

In this study, we introduce a cueless EEG-based imagined speech paradigm, where subjects imagine the pronunciation of semantically meaningful words without any external cues.

We assess a variety of classification methods, including traditional machine learning techniques such as Support Vector Machines (SVM) and XGBoost, as well as time-series foundation models and deep learning architectures specifically desig…

The rapid spread of fake news presents a significant global challenge, particularly in low-resource languages like Bangla, which lack adequate datasets and detection tools.

This version includes 11,700 additional, meticulously curated fake news articles validated from credible sources, creating a proportional dataset of 47,000 authentic and 13,000 fake news items across 13 categories.

In addition, we created a manually curated independent test set of 460 fake and 540 authentic news items for rigorous evaluation.

We invest efforts in collecting fake news from credible sources and manually verified while preserving the linguistic richness.

BanFakeNews-2.0 offers a valuable resource to advance…

In real-world sequential decision making tasks like autonomous driving, robotics, and healthcare, learning from observed state-action trajectories is critical for tasks like imitation, classification, and clustering.

Existing trajectory encoding methods often focus on specific tasks or rely on reward signals, limiting their ability to generalize across domains and tasks.

Our contributions are threefold: (1) We introduce a trajectory embedding approach that captures multiple abilities from state-action data.

(2) The learned embeddings exhibit strong representational power across downstream tasks, including imitation, classification, clustering, and regression.

(3) The embeddings demonstrate …

This study presents a comprehensive review of the potential of multimodal deep learning (DL) in medical diagnosis, using COVID-19 as a case example.

We explore the architecture of deep learning models, emphasising their data-specific structures and underlying algorithms.

We have implemented and analysed 11 deep learning models using COVID-19 image, text, and speech (ie, cough) data.

Our analysis revealed that the MobileNet model achieved the highest accuracy of 99.97% for COVID-19 image data and 93.73% for speech data (i.e., cough).

However, the BiGRU model demonstrated superior performance in COVID-19 text classification with an accuracy of 99.89%.

Convolutional neural networks (CNNs) are essential tools for computer vision tasks, but they lack traditionally desired properties of extracted features that could further improve model performance, e.g., rotational equivariance.

Such properties are ubiquitous in biomedical images, which often lack explicit orientation.

To overcome these challenges, we propose a novel and efficient implementation of the Symmetric Rotation-Equivariant (SRE) Convolution (SRE-Conv) kernel, designed to learn rotation-invariant features while simultaneously compressing the model size.

The SRE-Conv kernel can easily be incorporated into any CNN backbone.

We validate the ability of a deep SRE-CNN to capture equiva…

The multimodal language models (MLMs) based on generative pre-trained Transformer are considered powerful candidates for unifying various domains and tasks.

However, aerial detection has not been explored by existing RS MLMs because the autoregressive prediction mechanism of MLMs differs significantly from the detection outputs.

In this paper, we present a simple baseline for applying MLMs to aerial detection for the first time, named LMMRotate.

Specifically, we first introduce a normalization method to transform detection outputs into textual outputs to be compatible with the MLM framework.

Then, we propose a evaluation method, which ensures a fair comparison between MLMs and conventional …

In this paper, we propose Neural Honeytrace, a robust plug-and-play watermarking framework against model extraction attacks.

We first formulate a watermark transmission model from an information-theoretic perspective, providing an interpretable account of the principles and limitations of existing triggerable watermarking.

Guided by the model, we further introduce: (1) a similarity-based training-free watermarking method for plug-and-play and flexible watermarking, and (2) a distribution-based multi-step watermark information transmission strategy for robust watermarking.

Comprehensive experiments on four datasets demonstrate that Neural Honeytrace outperforms previous methods in efficiency…

Deep Learning Brain Computer Interfaces (BCIs) have recently successfully mapped neuronal activity to text contents in subjects who attempted to formulate speech.

One such model is Wav2Vec2 which has been trained in a self-supervised fashion to create meaningful representations of speech audio data.

In this study, we show that patterns learned by Wav2Vec2 are transferable to brain data.

We then execute full fine-tuning with pre-trained weights for Wav2Vec2, training ''from scratch'' without pre-trained weights as well as freezing a pre-trained Wav2Vec2 and training only the BFE each for 45 different BFE architectures.

These results indicate that knowledge transfer from audio speech recognit…

Effective chart summary can significantly reduce the time and effort decision makers spend interpreting charts, enabling precise and efficient communication of data insights.

Previous studies have faced challenges in generating accurate and semantically rich summaries of time-series data charts.

In this paper, we identify summary elements and common hallucination types in the generation of time-series chart summaries, which serve as our guidelines for automatic generation.

We introduce ChartInsighter, which automatically generates chart summaries of time-series data, effectively reducing hallucinations in chart summary generation.

Our evaluations using our benchmark show that our method sur…

Deep model merging represents an emerging research direction that combines multiple fine-tuned models to harness their specialized capabilities across different tasks and domains.

Current model merging techniques focus on merging all available models simultaneously, with weight interpolation-based methods being the predominant approaches.

In this study, we propose a training-free projection-based continual merging method that processes models sequentially through orthogonal projections of weight matrices and adaptive scaling mechanisms.

Our method operates by projecting new parameter updates onto subspaces orthogonal to existing merged parameter updates while using an adaptive scaling mecha…

In many real-world applications, agents must make sequential decisions in environments where conditions are subject to change due to various exogenous factors.

These non-stationary environments pose significant challenges to traditional decision-making models, which typically assume stationary dynamics.

Non-stationary Markov decision processes (NS-MDPs) offer a framework to model and solve decision problems under such changing conditions.

We present NS-Gym, the first simulation toolkit designed explicitly for NS-MDPs, integrated within the popular Gymnasium framework.

Our vision is that NS-Gym will enable researchers to assess the adaptability and robustness of their decision-making algorit…

Short text classification, as a research subtopic in natural language processing, is more challenging due to its semantic sparsity and insufficient labeled samples in practical scenarios.

We propose a novel model named MI-DELIGHT for short text classification in this work.

Specifically, it first performs multi-source information (i.e., statistical information, linguistic information, and factual information) exploration to alleviate the sparsity issues.

Moreover, we introduce a dual-level (i.e., instance-level and cluster-level) contrastive learning auxiliary task to effectively capture different-grained contrastive information within massive unlabeled data.

Meanwhile, previous models merel…

Automated audio captioning is a task that generates textual descriptions for audio content, and recent studies have explored using visual information to enhance captioning quality.

However, current methods often fail to effectively fuse audio and visual data, missing important semantic cues from each modality.

To address this, we introduce LAVCap, a large language model (LLM)-based audio-visual captioning framework that effectively integrates visual information with audio to improve audio captioning performance.

LAVCap employs an optimal transport-based alignment loss to bridge the modality gap between audio and visual features, enabling more effective semantic extraction.

Additionally, we …

They can “hallucinate” false information, particularly when given complex inputs.

Today, we’re introducing FACTS Grounding, a comprehensive benchmark for evaluating the ability of LLMs to generate responses that are not only factually accurate with respect to given inputs, but also sufficiently detailed to provide satisfactory answers to user queries.

We hope our benchmark will spur industry-wide progress on factuality and grounding.

To track progress, we’re also launching the FACTS leaderboard on Kaggle.

We’ve already tested leading LLMs using FACTS Grounding and have populated the initial leaderboard with their grounding scores.

Earlier this year, we introduced our video generation model, Veo, and our latest image generation model, Imagen 3.

Since then, it’s been exciting to watch people bring their ideas to life with help from these models: YouTube creators are exploring the creative possibilities of video backgrounds for their YouTube Shorts, enterprise customers are enhancing creative workflows on Vertex AI and creatives are using VideoFX and ImageFX to tell their stories.

Together with collaborators ranging from filmmakers to businesses, we’re continuing to develop and evolve these technologies.

Today we're introducing a new video model, Veo 2, and the latest version of Imagen 3, both of which achieve state-of-…

Today we’re excited to launch our next era of models built for this new agentic era: introducing Gemini 2.0, our most capable model yet.

Starting today our Gemini 2.0 Flash experimental model will be available to all Gemini users.

It's available in Gemini Advanced today.

TPUs powered 100% of Gemini 2.0 training and inference, and today Trillium is generally available to customers so they can build with it too.

If Gemini 1.0 was about organizing and understanding information, Gemini 2.0 is about making it much more useful.

Google Research Scientist David Warde and Google DeepMind Research Scientist Ian Goodfellow will present on Generative Adversarial Nets.

Teams across Google DeepMind will present more than 150 new papers on topics ranging from AI agents and generative media to innovative learning approaches.

Building adaptive, smart, and safe AI Agents LLM-based AI agents are showing promise in carrying out digital tasks via natural language commands.

Yet their success depends on precise interaction with complex user interfaces, which requires extensive training data.

AI agents trained using this dataset showed significant performance gains which we hope helps advance research into more general AI agents.

Because a perfect weather forecast is not possible, scientists and weather agencies use probabilistic ensemble forecasts, where the model predicts a range of likely weather scenarios.

The evolution of AI weather models GenCast marks a critical advance in AI-based weather prediction that builds on our previous weather model, which was deterministic, and provided a single, best estimate of future weather.

Better forecasts of extreme weather, such as heat waves or strong winds, enable timely and cost-effective preventative actions.

GenCast offers greater value than ENS when making decisions about preparations for extreme weather, across a wide range of decision-making scenarios.

Advanced forec…

Today we introduce Genie 2, a foundation world model capable of generating an endless variety of action-controllable, playable 3D environments for training and evaluating embodied agents.

Based on a single prompt image, it can be played by a human or AI agent using keyboard and mouse inputs.

Games play a key role in the world of artificial intelligence (AI) research.

However, training more general embodied agents has been traditionally bottlenecked by the availability of sufficiently rich and diverse training environments.

As we show, Genie 2 could enable future agents to be trained and evaluated in a limitless curriculum of novel worlds.

Quantum computers have the potential to revolutionize drug discovery, material design and fundamental physics — that is, if we can get them to work reliably.

Certain problems, which would take a conventional computer billions of years to solve, would take a quantum computer just hours.

If we want to make quantum computers more reliable, especially at scale, we need to accurately identify and correct these errors.

In a paper published today in Nature, we introduce AlphaQubit, an AI-based decoder that identifies quantum computing errors with state-of-the-art accuracy.

This collaborative work brought together Google DeepMind’s machine learning knowledge and Google Quantum AI’s error correction…

AI is revolutionizing the landscape of scientific research, enabling advancements at a pace that was once unimaginable — from accelerating drug discovery to designing new materials for clean energy technologies.

The AI for Science Forum — co-hosted by Google DeepMind and the Royal Society — brought together the scientific community, policymakers, and industry leaders to explore the transformative potential of AI to drive scientific breakthroughs, address the world's most pressing challenges, and lead to a new era of discovery.

Technologies Pushing the frontiers of audio generation ShareCopy link ×Our pioneering speech generation technologies are helping people around the world interact with more natural, conversational and intuitive digital assistants and AI tools.

Pioneering techniques for audio generation For years, we've been investing in audio generation research and exploring new ways for generating more natural dialogue in our products and experimental tools.

Download audio Audio clip of two speakers telling a funny story, with laughter at the punchline.

Download audio Audio clip of two speakers expressing excitement about a surprise birthday party.

Scaling our audio generation models Scaling our single-spe…

Technologies New generative AI tools open the doors of music creation ShareCopy link ×Our latest AI music technologies are now available in MusicFX DJ, Music AI Sandbox and YouTube Shorts For nearly a decade, our teams have been exploring how artificial intelligence (AI) can support the creative process, building tools that empower enthusiasts and professionals to discover new forms of creative expression.

Their input has been guiding our state-of-the-art generative music experiments, and helping us ensure that our new generative AI tools responsibly open the doors of music creation to everyone.

We’re also announcing updates to our music AI toolkit, called Music AI Sandbox, and highlighting…

This morning, Co-founder and CEO of Google DeepMind and Isomorphic Labs Sir Demis Hassabis, and Google DeepMind Senior Research Scientist Dr. John Jumper were co-awarded the 2024 Nobel Prize in Chemistry for their work developing AlphaFold, a groundbreaking AI system that predicts the 3D structure of proteins from their amino acid sequences.

AlphaFold’s predictions, made freely available through the AlphaFold Protein Structure Database, have given more than 2 million scientists and researchers from 190 countries a powerful tool for making new discoveries.

In a statement released after informed of the news, Demis Hassabis said:"Receiving the Nobel Prize is the honour of a lifetime.

After rec…

Research How AlphaChip transformed computer chip design ShareCopy link ×Our AI method has accelerated and optimized chip design, and its superhuman chip layouts are used in hardware around the world In 2020, we released a preprint introducing our novel reinforcement learning method for designing chip layouts, which we later published in Nature and open sourced.

Today, we’re publishing a Nature addendum that describes more about our method and its impact on the field of chip design.

Computer chips have fueled remarkable progress in artificial intelligence (AI), and AlphaChip returns the favor by using AI to accelerate and optimize chip design.

Using AI to design Google’s AI accelerator chips…

Developers can access our latest models for free via Google AI Studio and the Gemini API.

With the latest updates, 1.5 Pro and Flash are now better, faster, and more cost-efficient to build with in production.

For more details on migrating to the latest versions of Gemini 1.5 Pro and 1.5 Flash, check out the Gemini API models page.

Gemini 1.5 Pro We continue to be blown away with the creative and useful applications of Gemini 1.5 Pro’s 2 million token long context window and multimodal capabilities.

Increased rate limits To make it even easier for developers to build with Gemini, we are increasing the paid tier rate limits for 1.5 Flash to 2,000 RPM and increasing 1.5 Pro to 1,000 RPM, up f…

We’re changing that, and making these incredible technologies more easily accessible to millions of creators and billions of users around the world.

Over the next few months, we’re bringing our advanced generative AI models, Veo and Imagen 3 , to YouTube creators through Dream Screen .

Artificial intelligence (AI) technologies for generating creative content are improving rapidly, but seamless ways of using them still aren’t widely available.

New video generation technology in YouTube Shorts will help millions of people realize their creative visionCreators can soon generate video with Dream ScreenBy integrating Veo into Dream Screen, YouTube creators will soon be able to generate exciting …

Research Our latest advances in robot dexterity ShareCopy link ×Two new AI systems, ALOHA Unleashed and DemoStart, help robots learn to perform complex tasks that require dexterous movement People perform many tasks on a daily basis, like tying shoelaces or tightening a screw.

Today, we introduce two new papers featuring our latest artificial intelligence (AI) advances in robot dexterity research: ALOHA Unleashed which helps robots learn to perform complex and novel two-armed manipulation tasks; and DemoStart which uses simulations to improve real-world performance on a multi-fingered robotic hand.

This helps the robot learn from the data, so it can perform the same tasks on its own.

To ena…

A3 Ultra, with NVIDIA H200 GPUs is a new addition to the A3 family of NVIDIA Hopper GPU-accelerated VMs with twice the GPU-to-GPU network bandwidth and twice the high bandwidth memory (HBM) compared to A3 Mega with NVIDIA H100 GPUs.

A3 Ultra VMs offer the best performance in the A3 family.

Combined with our datacenter-wide 4-way rail-aligned network, A3 Ultra VMs deliver up to 3.2 Tbps of non-blocking GPU-to-GPU communication with RDMA over Converged Ethernet (RoCE).

A3 Ultra VMs are also available through GKE, which provides an open, portable, extensible, and highly scalable platform for training and serving AI workloads.

To try out A3 Ultra VMs, you can easily create a cluster with GKE or…

Digital markets demand new trading systemsToday, Deutsche Börse Group successfully operates high-volume/low-latency trading venues — such as Xetra, Börse Frankfurt, Eurex, and the European Energy Exchange, as well as partner exchanges — by using proven high-performance architectures.

The need for a new trading engine, and the desire to make it the cornerstone and first component of Deutsche Börse Group’s emerging Digital Asset Business Platform, stems from changing market structures.

Market participants also demand choice of market access, including internet connectivity to execute trades anytime, anywhere.

Finally, a new trading engine would have to meet not only these new requirements, bu…

Instead, we can leverage image embeddings and combine the search results with text data in Vertex AI Search.

Google Cloud's Vertex AI platform provides a comprehensive set of tools for building and deploying machine learning solutions, including powerful search capabilities:Vertex AI search: A highly scalable and feature-rich engine for many types of search.

Vertex AI multimodal embedding API: This is used to generate image embeddings (numerical representations of images).

Vertex AI Vector Search: This is used as the vector database to store the embeddings with metadata information for searching.

Our ensemble approach: Text + image powerTo create our multimodal search engine, we'll use an e…

How retailers can build an AI foundationRetailers can find assets to fuel their AI in all corners of the business.

Edge processing power decision point: CPU vs GPUNext, we’ll explore the critical decision on the right hardware to power your applications.

The two primary options are CPUs (Central Processing Units) and GPUs (Graphics Processing Units), each with its own strengths and weaknesses.

Consider this chart to guide your decision-making process, especially when choosing between deploying at a regional DC or at the edge.

This is less of a concern if deploying at a regional DC where power and cooling are centralized.

To provide customers with the most advanced ecosystem of solutions across industries, we’ve enabled these partners to easily build and scale products on our platform.

Many are deeply engaged with our AI technology to deliver new and novel AI solutions directly to our customers and theirs.

Generative AI has already had a significant impact on the retail industry by enabling businesses to run more personalized marketing campaigns, increase sales via improved search capabilities, and enhance customer service experiences through more accurate and tailored resolutions.

In many cases, AI agents are helping these businesses move beyond predictive capabilities to performing tasks autonomously.

At N…

Closing the gap between impressive model demos and real-world performance is crucial for successfully deploying generative AI for enterprise.

Despite the incredible capabilities of generative AI for enterprise, this perceived gap may be a barrier for many developers and enterprises to “productionize” AI.

This is where retrieval-augmented generation (RAG) becomes non-negotiable – it strengthens your enterprise applications by building trust in its AI outputs.

Today, we’re sharing the general availability of Vertex AI’s RAG Engine, a fully managed service that helps you build and deploy RAG implementations with your data and methods.

With our Vertex AI RAG Engine you will be able to:

Retailers have always moved quickly to connect and match the latest merchandise with customers' needs.

As retail organizations increasingly adopt AI foundation models and other AI technologies to improve the shopping journey, robust infrastructure becomes paramount.

Retailers need to be able to develop AI applications and services quickly, reliably, robustly, and affordably, and with support from Google Cloud and NVIDIA, leading companies are already accelerating their time to market and achieving scalable costs as they move AI from pilots into production.

Google Cloud has worked with NVIDIA to empower retailers to boost their customer engagements in exciting new ways, deliver more hyper-pe…

Fostering a more inclusive AI ecosystemAs AI continues to revolutionize industries, ensuring that diverse voices lead the way is critical for driving innovation that benefits everyone.

The Google for Startups Accelerator: Women Founders program is working to level the playing field, empowering women-led startups to bring fresh, diverse perspectives to the future of AI.

Margaryta Sivakova, the CEO of Legal Nodes, leveraged support from the program to scale her business:"Through Google for Startups Accelerator, we learned to build, improve, and scale AI solutions, focusing on production-grade AI, MLOps, and the right infrastructure for rapid scaling."

Maria Terzi, the CEO of Malloc Privacy, r…

Now, consider the scenario where a preprocessing task involves extracting multiple image URLs from each row of a large dataset and uploading the images to a Cloud Storage bucket.

It provides a simple API for distributing computations across multiple workers, making it a strong choice for implementing parallel data preprocessing pipelines.

Implementation detailsWe ran the data preprocessing on GKE using the accelerated platforms repository, which provides the code to build your GKE cluster and configure pre-requisites like running Ray on the cluster so you can run data preprocessing on the cluster as a container.

Each chunk is assigned to a Ray task, which is executed on a Ray worker.

Ray ta…

When fine-tuning Gemini, you have a couple models to choose from:Gemini 1.5 Pro: Google’s best model for general performance.

Efficiently improving the model with your data: Before fine-tuning a larger model like Gemini Pro, it's often beneficial to test your tuning data on a smaller, less expensive model like Gemini Flash first.

The success of your supervised fine-tuning depends significantly on the quality of your tuning data.

Crucially, ensure the prompts and instructions used in your fine-tuning dataset closely resemble those you plan to use in production.

Training-serving skewA critical factor influencing fine-tuning effectiveness is the alignment between your tuning data and productio…

The challenges of traditional ad break detectionTraditional ad break detection methods, designed primarily for structured television content with fade-outs and fixed commercial breaks, often struggle to identify ideal ad placement points in today's diverse video landscape.

Intelligent scene detection with Google’s Gemini modelsGemini’s multimodal capabilities can analyze video, audio, and text simultaneously, enabling a level of nuanced scene understanding that was previously impossible.

End of Tense Dialogue Scene Tense, dramatic Fade-out Scene of rising conflict Two characters arguing over a specific issue.

Busy Street to Quiet Cafe Neutral Hard cut, outdoor to indoor Scene transition A c…

Like many PyTorch users, you may have heard great things about JAX — its high performance, the elegance of its functional programming approach, and its powerful, built-in support for parallel computation.

Along the way, we introduce JAX by demonstrating how many things — from model definitions and instantiation to training — map to their PyTorch equivalents.

You can follow along with full code examples in the accompanying notebook: https://www.kaggle.com/code/anfalatgoogle/pytorch-developer-s-guide-to-jax-fundamentalsModularity with JAXAs a PyTorch user, you might initially find Jax’s highly modularized ecosystem to be quite different than what you are used to.

Unlike with PyTorch, it does …

Today, generative AI is giving organizations new ways to process and analyze data, discover hidden insights, increase productivity and build new applications.

However, data sovereignty, regulatory compliance, and low-latency requirements can be a challenge.

The need to keep sensitive data in certain locations, adhere to strict regulations, and respond swiftly can make it difficult to capitalize on the cloud's innovation, scalability, and cost-efficiency advantages.

Google Distributed Cloud (GDC) brings Google's AI services anywhere you need them — in your own data center or at the edge.

Designed with AI and data-intensive workloads in mind, GDC is a fully managed hardware and software solut…

This allows you to quickly generate baseline scores for the evaluation and refinement of your RAG system.

Common RAG evaluation frameworks include:RagasRagas is an open-source tool for evaluating RAG systems.

Vertex AI gen AI evaluation serviceThe Vertex AI gen AI evaluation service helps users test and compare generative models or applications based on custom metrics.

These questions should accurately reflect the main use cases the RAG system is intended to address.

Make use of the Vertex AI generative AI evaluation framework.

With the rapid advancement and adoption of AI, organizations face increasing pressure to ensure their AI systems are developed and used responsibly.

The ISO/IEC 42001:2023 standard provides a framework for addressing the unique challenges AI poses, and we're excited to announce that Google Cloud has achieved an accredited ISO/IEC 42001:2023 certification for our AI management system.

This certification helps demonstrate our commitment to developing and deploying AI responsibly.

In a landscape increasingly shaped by the advent of AI regulations, such as the EU AI Act, this certification is foundational upon which we continue to build and expand our responsible AI efforts.

As AI continues to …

“It recognises the value of our award-winning journalism and will give us early insights into how content is surfaced through AI.

“Apart from the benefits to the FT, there are broader implications for the industry.

It’s right, of course, that AI platforms pay publishers for the use of their material.

“We value the opportunity to be inside the development loop as people discover content in new ways.

As with any transformative technology, there is potential for significant advancements and major challenges, but what’s never possible is turning back time.

OpenAI, alongside industry leaders including Amazon, Anthropic, Civitai, Google, Meta, Metaphysic, Microsoft, Mistral AI, and Stability AI, has committed to implementing robust child safety measures in the development, deployment, and maintenance of generative AI technologies as articulated in the Safety by Design principles.

By adopting comprehensive Safety by Design principles, OpenAI and our peers are ensuring that child safety is prioritized at every stage in the development of AI.

Responsibly source our training datasets, detect and remove child sexual abuse material (CSAM) and child sexual exploitation material (CSEM) from training data, and report any confirmed CSAM to the relevant a…

Customers with a sustained level of tokens per minute (TPM) usage on GPT-4 or GPT-4 Turbo can request access to provisioned throughput to get discounts ranging from 10–50% based on the size of the commitment.

Reduced costs on asynchronous workloads: Customers can use our new Batch API to run non-urgent workloads asynchronously.

Batch API requests are priced at 50% off shared prices, offer much higher rate limits, and return results within 24 hours.

We plan to keep adding new features focused on enterprise-grade security, administrative controls, and cost management.

For more information on these launches, visit our API documentation or get in touch with our team to discuss custom solution…

Our new local presence also gets us closer to leading businesses like Daikin, Rakuten, and TOYOTA Connected who are using ChatGPT Enterprise to automate complex business processes, assist in data analysis, and optimize internal reporting.

ChatGPT also helps accelerate the efforts of local governments, such as Yokosuka City, which is leveraging the technology to improve the efficiency of public services in Japan.

Over the past year, the city has gradually provided ChatGPT access to almost all city employees, and 80% have reported increases in productivity.

Now Yokosuka City has formed a network with 21 local governments—including the Tokyo Metropolitan Government and the City of Kobe—to …

Assisted Fine-TuningAt DevDay last November, we announced a Custom Model program designed to train and optimize models for a specific domain, in partnership with a dedicated group of OpenAI researchers.

Since then, we've met with dozens of customers to assess their custom model needs and evolved our program to further maximize performance.

Today, we are formally announcing our assisted fine-tuning offering as part of the Custom Model program.

Fully custom-trained models imbue new knowledge from a specific domain by modifying key steps of the model training process using novel mid-training and post-training techniques.

Our team modified every step of the model training process, from domain-s…

We’ve also introduced additional content safeguards for this experience, such as blocking prompts and generations in a wider range of categories.

There are many benefits to creating an account including the ability to save and review your chat history, share chats, and unlock additional features like voice conversations and custom instructions.

For anyone that has been curious about AI’s potential but didn’t want to go through the steps to set-up an account, start using ChatGPT today.

We recognize that generating speech that resembles people's voices has serious risks, which are especially top of mind in an election year.

We are engaging with U.S. and international partners from across government, media, entertainment, education, civil society and beyond to ensure we are incorporating their feedback as we build.ÂThe partners testing Voice Engine today have agreed to our usage policies, which prohibit the impersonation of another individual or organization without consent or legal right.

In addition, our terms with these partners require explicit and informed consent from the original speaker and we don’t allow developers to build ways for individual users to create the…

Starting his career at DreamWorks Animation, Don Allen III is a multidisciplinary creator, speaker and consultant who collaborates with major tech and entertainment companies on mixed reality, virtual reality and AI applications.

“For a long time I've been making augmented reality hybrid creatures that I think would be fun combinations in my head.

Now I have a much easier way of prototyping the ideas before I fully build out the 3-D characters to place in spatial computers.” Don cites Sora’s “weirdness” as its greatest strength: “It’s not bound by traditional laws of physics or conventions of thought.” He says that working with Sora shifted his focus from “technical hurdle…

And now you can use this loop to design materials really quickly.

XIE: So you can really think about MatterSim and MatterGen accelerating different parts of materials discovery process.

They are also both foundation AI models, meaning they can both be used for a broad range of materials design problems.

Really, really a lot.

Yeah, I really, really like the example that Ziheng mentioned about the educational purposes.

MatterGen enables a new paradigm of generative AI-assisted materials design that allows for efficient exploration of materials, going beyond the limited set of known ones.

MatterGen can be fine-tuned to generate materials under different design requirements such as specific chemistry, crystal symmetry, or materials’ properties.

AI emulator and generator flywheelMatterGen presents a new opportunity for AI accelerated materials design, complementing our AI emulator MatterSim.

Looking aheadMatterGen represents a new paradigm of materials design enabled by generative AI technology.

That’s why we are interested in understanding the impact that MatterGen could have on materials discovery,” said C…

Modular and extensible : Users can easily customize systems with pluggable components, including custom agents, tools, memory, and models.

: Users can easily customize systems with pluggable components, including custom agents, tools, memory, and models.

Core: The foundational building blocks for an event-driven agentic system.

In addition to the framework, AutoGen 0.4 includes upgraded programming tools and applications, designed to support developers in building and experimenting with AutoGen.

Third-party component galleries: Import and use custom agents, tools, and workflows from external galleries to extend functionality.

To tackle these challenges, recent research on using AIOps agents for cloud operations—such as AI agents for incident root cause analysis (RCA) or triaging—has relied on proprietary services and datasets.

Users developing agents for cloud operations tasks with Azure AI Agent Service can evaluate and improve them using AIOpsLab.

This calls for a standardized and principled research framework for building, testing, comparing, and improving AIOps agents.

The AIOpsLab research paper has been accepted at SoCC’24 (the annual ACM Symposium on Cloud Computing).

Our approach integrates application and domain knowledge to create adaptable policies and “oracles” compatible with AIOps scenarios.

DAEPP: You know, we didn’t really think about the term fraud until we started prepping for this interview with you.

BADANES: Right, right.

One of the things that I get asked a lot is, why can’t we just build good AI to detect bad AI, right?

BADANES: So next time my kids are in a fight, I’m going to point them to Copilot and say, work with Copilot to mediate.

[LAUGHS] No, that’s really, really interesting.

Welcome to Research Focus, a series of blog posts that highlights notable publications, events, code/datasets, new hires and other milestones from across the research community at Microsoft.

Read the paperon-demand event Microsoft Research Forum Episode 4 Learn about the latest multimodal AI models, advanced benchmarks for AI evaluation and model self-improvement, and an entirely new kind of computer for AI inference and hard optimization.

In a recent paper: RedCode: Risky Code Execution and Generation Benchmark for Code Agents, published at NeurIPS 2024, researchers from Microsoft and external colleagues propose comprehensive and practical evaluations on the safety of code agents.

This res…

Earlier today, Lidong gave a keynote here at NeurIPS on the co-evolution of AI and systems engineering.

One dimension is that the scale of the AI systems that we have to support.

And the other dimension is if you look at AI systems, it’s actually a whole-stack kind of design.

STRICKLAND: Yeah, yeah.

ZHOU: Yeah, I think in terms of AI systems, I’m certainly pretty excited about what we can do together, you know, with a combination of AI and systems.

The question then becomes: How can we make prompt optimization faster, more accessible, and more adaptable across diverse tasks?

Download PromptWizardTo address this challenge, we developed PromptWizard (PW), a research framework that automates and streamlines the process of prompt optimization.

Joint optimization and synthesis of diverse examples: PW generates synthetic examples that are not only robust and diverse but also task-aware.

Stage 1: Refinement of prompt instructionThe first stage focuses on refining the task instructions of a prompt.

Through the critique-and-synthesis mechanism, PromptWizard ensures alignment between the prompt and examples, simultaneously synthesizing new exam…

Introducing GraphRAG 1.0Microsoft debuted (opens in new tab) the pre-release version of GraphRAG (opens in new tab) in July 2024 to advance AI use in complex domains.

The original CLI was intended as a starter demo for users to try GraphRAG on a sample dataset.

Our original data model stored all embeddings within the parquet output files after data ingestion and indexing.

This streamlining has also reduced the in-memory footprint of the pipeline, enabling users to index and analyze larger datasets with GraphRAG.

MigratingWe recommend users migrate to GraphRAG 1.0, which offers a streamlined experience including multiple improvements for both users and developers.

And then the second paradigm really emerged in the 17th century.

And so the third paradigm really began, I guess, sort of, in the ’50s and ’60s, the development of digital computers.

And when I think about AI for Science actually, the space of opportunity is colossal because science is, science is really just understanding more about the world around us.

And now the SMILES autoregressive model can now generate a molecule that’s an improvement on the starting molecule and knows about the protein binding.

But also if you think about [it], science is really about learning more about the world.

Today I’m talking to Jindong Wang, a senior researcher at Microsoft Research, and Steven Whang, a tenured associate professor at the Korea Advanced Institute of Science and Technology.

JINDONG WANG: OK, everybody knows that with the widespread usage of large language models, hallucination has become a crucial factor of concern.

So foreign key constraint basically requires that if there is some director mentioned in the movie table, it has to be one of the directors in the director table.

So now we can join the movie and director table and generate a bigger table.

HUIZINGA: Well, Jindong Wang and Steven Whang, thanks for joining us today, and to our listeners, thanks for tuning in.

The other one actually is some token actually is very, very hard to be predicted during the pretraining.

And the important thing for the data is about data filtering.

If we’re able to build a better model actually is able to benefit so many different kinds of application.

And definitely there’s a lot of things about how to build a better data [that] is unsolved yet in the literature.

And the other thing actually, we are working on something that’s very exciting.

The drawback of this approach is that it often misses the cultural nuances of local languages.

CHITALE: Now that we have created a benchmark, the next step is to evaluate how these multimodal models are performing on this benchmark.

So what we observed is there is a huge gap when it comes … in performance when we compare these proprietary offerings versus the open-source models.

These open-source models significantly lag behind the proprietary models.

CHITALE: CVQA is significant because it addresses a fundamental gap in how we evaluate vision-language and multimodal models today.

FOSTER: So this is a, kind of, a theoretical work on reinforcement learning, or RL.

FOSTER: Yeah, so if you look at these sort of RL problems with latent dynamics, this is something that’s actually received a lot of investigation in theory.

Like, can we take existing algorithms and use them to solve rich-observation RL problems in a modular fashion?

TINGLE: Dylan, I’d like to know—and I’m sure our audience would, too—what this work means when it comes to real-world application.

TINGLE: Well, Dylan Foster, thank you for joining us today to discuss your paper on reinforcement learning under latent dynamics.

A good example of Raghavan’s intention can be found in his exploration of the use AI in hiring.

Just before starting college, though, his love of math and computing called him to follow his family example into computer science.

The experience may be satisfying in the moment, but it can leave the user feeling slightly sick.

The model won the Exemplary Applied Modeling Track Paper Award at the 2022 Association for Computing Machinery Conference on Economics and Computation.

“Long-term satisfaction is ultimately important, even if all you care about is a company’s interests,” Raghavan says.

In the world of high-priced art, galleries usually act as gatekeepers.

“The gallery model has existed for such a long period of time that they are the tastemakers in the art world.

Determined to try selling his art, he collaborated with some prominent art galleries in London, Miami, and St. Moritz.

Later, he brought Nofal to Dubai to participate in World Art Dubai.

“Many people claim they want to help the art world, but in reality, they often fall back on the same outdated business models,” says Gulak.

For the past 150 years, researchers have had the benefit of the periodic table of elements to draw upon, which tells them that different elements have different properties, and one can’t magically transform into another.

What’s more, their neural network model can extract much more information about a molecule than just its energy.

“In previous work, people have used multiple different models to assess different properties,” says Hao Tang, an MIT PhD student in materials science and engineering.

After being trained on small molecules, the model can be generalized to bigger and bigger molecules.

“Previously, most calculations were limited to analyzing hundreds of atoms with DFT and just tens…

The open-access findings, reported Dec. 4 in the journal Nature Communications, show how machine learning can help neuroscientists understand how the brain uses auditory information in the real world.

Science of soundThe nervous system’s auditory signals are timed so precisely, researchers have long suspected that timing is important to our perception of sound.

To better understand the brain, Saddler and McDermott wanted to challenge a hearing model to do things that people use their hearing for in the real world, like recognizing words and voices.

The researchers showed that their model replicated human hearing well — better than any previous model of auditory behavior, McDermott says.

Tha…

We recently extended this idea to other generative AI tasks such as text summarization and found the same results.

Q: What can we do as consumers of generative AI to help mitigate its climate impact?

We can also make an effort to be more educated on generative AI emissions in general.

Many of us are familiar with vehicle emissions, and it can help to talk about generative AI emissions in comparative terms.

A: Mitigating the climate impact of generative AI is one of those problems that people all over the world are working on, and with a similar goal.

Vocal imitation is the sonic equivalent of doodling a quick picture to communicate something you saw — except that instead of using a pencil to illustrate an image, you use your vocal tract to express a sound.

To achieve this, the researchers engineered their system to produce and interpret sounds much like we do.

They started by building a model of the human vocal tract that simulates how vibrations from the voice box are shaped by the throat, tongue, and lips.

Then, they used a cognitively-inspired AI algorithm to control this vocal tract model and make it produce imitations, taking into consideration the context-specific ways that humans choose to communicate sound.

“In the same way that…

To overcome that limitation, MIT researchers have developed a computational technique that allows large language models to predict antibody structures more accurately.

This same approach can work for protein sequences — by learning which protein structures are most likely to be formed from different patterns of amino acids.

The resulting computational model, known as AbMap, can predict antibody structures and binding strength based on their amino acid sequences.

To demonstrate the usefulness of this model, the researchers used it to predict antibody structures that would strongly neutralize the spike protein of the SARS-CoV-2 virus.

Their model was able to identify antibody structures that …

Unlocking its secrets could thus enable advances in efficient energy production, electronics cooling, water desalination, medical diagnostics, and more.

Chief among those is a problem caused by bubbles forming so quickly they create a band of vapor across a surface that prevents further heat transfer.

One day his department head asked him to work on a problem in nuclear reactor safety known as transient boiling.

Today Bucci’s lab is developing new diagnostic techniques to study boiling and heat transfer along with new materials and coatings that could make heat transfer more efficient.

“The effectiveness of the boiling process on the surface of nuclear reactor cladding determines the effici…

You’d be better off with an automated research assistant — or perhaps artificial intelligence systems called multimodal vision language models (VLMs).

For straightforward search queries like “a reef with manmade structures and debris,” relatively large models like “SigLIP” found matching images, while smaller-sized CLIP models struggled.

Then, the researchers used the same search queries to see how well VLMs could retrieve iNaturalist images.

The annotators’ labels revealed when the models struggled to understand scientists’ keywords, as their results included images previously tagged as irrelevant to the search.

For example, VLMs’ results for “redwood trees with fire scars” sometimes inclu…

Corvus Robotics is addressing that problem with an inventory management platform that uses autonomous drones to scan the towering rows of pallets that fill most warehouses.

The company’s drones can work 24/7, whether warehouse lights are on or off, scanning barcodes alongside human workers to give them an unprecedented view of their products.

A new kind of inventory management solutionKabir has been working on drones since he was 14.

While working on Corvus, Kabir was also one of the founders of the MIT Driverless program that built North America’s first competition-winning driverless race cars.

Soon MSI was using Corvus every day across multiple facilities in its nationwide network.

Frida Polli, a neuroscientist, entrepreneur, investor, and inventor known for her leading-edge contributions at the crossroads of behavioral science and artificial intelligence, is MIT’s new visiting innovation scholar for the 2024-25 academic year.

She is the first visiting innovation scholar to be housed within the MIT Schwarzman College of Computing.

Polli began her career in academic neuroscience with a focus on multimodal brain imaging related to health and disease.

"I'm delighted to welcome Dr. Polli back to MIT.

Her entrepreneurial background makes her a terrific inaugural visiting innovation scholar,” says Dan Huttenlocher, dean of the MIT Schwarzman College of Computing and the Hen…

“By using multiple AI agents, we’re trying to simulate the process by which communities of scientists make discoveries,” says Buehler.

The foundation of their approach is an ontological knowledge graph, which organizes and makes connections between diverse scientific concepts.

To make the graphs, the researchers feed a set of scientific papers into a generative AI model.

This focuses AI models on developing a more principled way to understand concepts; it also allows them to generalize better across domains.

With the graph established, the researchers developed an AI system for scientific discovery, with multiple models specialized to play specific roles in the system.

Today, bulky silicon wafers serve as the main scaffold on which high-quality, single-crystalline semiconducting elements are grown.

Any stackable chip would have to include thick silicon “flooring” as part of each layer, slowing down any communication between functional semiconducting layers.

Without these thick silicon substrates, multiple semiconducting layers can be in more direct contact, leading to better and faster communication and computation between layers, the researchers say.

“You have to grow this single-crystalline material below 400 Celsius, otherwise the underlying circuitry is completely cooked and ruined,” Kim says.

“A product realized by our technique is not only a 3D logi…

She knew she wanted an undergraduate experience that encouraged her broad interests, a place where every field was within reach.

“NEET is about more than engineering,” says Amitava “Babi” Mitra, NEET founding executive director.

In 2023, she led an undergraduate team at the International Genetically Engineered Machine (iGEM) competition in Paris, France, where they presented a proof of concept for a therapy to treat cancer cachexia.

Her interdisciplinary drive took her to Merck over the summer, where Spivakovsky interned on the Modeling and Informatics team.

“My team continues to actively use the software I developed and the insights I gained through my work,” Spivakovsky says.

Virtual Personas for Language Models via an Anthology of BackstoriesWe introduce Anthology, a method for conditioning LLMs to representative, consistent, and diverse virtual personas by generating and utilizing naturalistic backstories with rich details of individual values and experience.

What does it mean for large language models (LLMs) to be trained on massive text corpora, collectively produced by millions and billions of distinctive human authors?

In this work, we introduce Anthology, an approach for steering LLMs to representative, consistent, and diverse virtual personas by providing richly detailed life narratives of individuals as conditioning context to models.

By grounding langu…

Linguistic Bias in ChatGPT: Language Models Reinforce Dialect DiscriminationSample language model responses to different varieties of English and native speaker reactions.

Over 1 billion people around the world speak varieties such as Indian English, Nigerian English, Irish English, and African-American English.

Then, we compared the language model responses to the “standard” varieties and the non-“standard” varieties.

Here, we included the original GPT-3.5 responses, plus responses from GPT-3.5 and GPT-4 where the models were told to imitate the style of the input.

That can reinforce barriers against speakers of non-“standard” varieties as AI models become increasingly used in …

How to Evaluate Jailbreak Methods: A Case Study with the StrongREJECT BenchmarkWhen we began studying jailbreak evaluations, we found a fascinating paper claiming that you could jailbreak frontier LLMs simply by translating forbidden prompts into obscure languages.

This blog post shows how to use a new, state-of-the art jailbreak benchmark - StrongREJECT - to accurately and robustly evaluate jailbreak methods.

PAP instructs an attacker model to persuade a victim model to give it harmful information using techniques like misrepresentation and logical appeals.

We conducted two experiments to test this hypothesis:We used StrongREJECT to evaluate 37 jailbreak methods on an unaligned model; Dolp…

Launching VHs: The Visual Haystacks Benchmark!

Humans excel at processing vast arrays of visual information, a skill that is crucial for achieving artificial general intelligence (AGI).

Visual Haystacks: the first "visual-centric" Needle-In-A-Haystack (NIAH) benchmark designed to rigorously evaluate Large Multimodal Models (LMMs) in processing long-context visual information.

The first NIAH benchmark for visual reasoning was introduced by Google in the Gemini-v1.5 technical report.

What is the Visual Haystacks (VHs) Benchmark?

TinyAgent: Function Calling at the EdgeThe ability of LLMs to execute commands through plain language (e.g.

The framework is open sourced and available at https://github.com/SqueezeAILab/TinyAgentTeaching LLMs to do Function CallingFigure 1: Overview of the LLMCompiler Function Calling Planner.

Once this function calling plan is generated, we can parse it and call each function based on the dependencies.

With our dataset in place, we can now proceed to fine-tune off-the-shelf SLMs to enhance their function calling capability.

Latency is the end-to-end latency of the function calling planner, including the prompt processing time and generation.

The following high-level steps show how to configure the Amazon Q Business – ServiceNow integration:Create a user in ServiceNow for Amazon Q Business to communicate with ServiceNow Create knowledge base articles in ServiceNow if they do not exist already Create an Amazon Q Business application and configure the ServiceNow data source and retriever in Amazon Q Business Synchronize the data source Create a ServiceNow plugin in Amazon Q BusinessPrerequisitesTo run this application, you must have an Amazon Web Services (AWS) account, an AWS Identity and Access Management (IAM) role, and a user that can create and manage the required resources.

Create an Amazon Q Business applicationAmazon Q off…

This post introduces HCLTech’s AutoWise Companion, a transformative generative AI solution designed to enhance customers’ vehicle purchasing journey.

Powered by generative AI services on AWS and large language models’ (LLMs’) multi-modal capabilities, HCLTech’s AutoWise Companion provides a seamless and impactful experience.

Responsible generative AI and security considerationsCustomers implementing generative AI projects with LLMs are increasingly prioritizing security and responsible AI practices.

He also holds AWS Solutions Architect Certification and has contributed to the research community by co-authoring papers and winning multiple AWS generative AI hackathons.

About HCLTechHCLTech i…

Applying GNN-based traffic prediction to the AWS backbone networkFor the backbone network traffic prediction application at AWS, we need to ingest a number of data sources into the GraphStorm framework.

Because this is an absolute metric, the model’s prediction can be either above or below the network traffic.

To still provide you with some code that makes it straightforward to get started solving your network prediction problems, we share a synthetic traffic prediction problem instead.

The second one, airport, traffic, airport , captures the amount of traffic sent between connected airports.

Using these artifacts, we can employ the graph construction tool to convert the air traffic graph d…

In this post, we cover two primary architectural patterns: fully local RAG and hybrid RAG.

To learn more about this hybrid RAG application or get hands-on with the cross-environment application, refer to Module 1 of our public AWS Workshop: Hands-on with Generative AI on AWS Hybrid & Edge Services.

To get started with our newly released workshop, see Hands-on with Generative AI on AWS Hybrid & Edge Services.

Additionally, check out other AWS hybrid cloud solutions or reach out to your local AWS account team to learn how to get started with Local Zones or Outposts.

About the AuthorsRobert Belson is a Developer Advocate in the AWS Worldwide Telecom Business Unit, specializing in AWS edge comp…

By combining the reasoning power of multiple intelligent specialized agents, multi-agent collaboration has emerged as a powerful approach to tackle more intricate, multistep workflows.

Later in this post, we provide evaluation data points to illustrate the benefits of multi-agent collaboration.

A hierarchical multi-agent collaboration frameworkThe MAC framework for Amazon Bedrock Agents starts from a hierarchical approach and expands to other mechanisms in the future.

Inter-agent communication – Communication is the key component of multi-agent collaboration, allowing agents to exchange information and coordinate their actions.

On Amazon Bedrock Agents, this can be achieved through collabor…

The Education and Training Quality Authority (BQA) plays a critical role in improving the quality of education and training services in the Kingdom Bahrain.

BQA oversees a comprehensive quality assurance process, which includes setting performance standards and conducting objective reviews of education and training institutions.

In this post, we explore how BQA used the power of Amazon Bedrock, Amazon SageMaker JumpStart, and other AWS services to streamline the overall reporting workflow.

Solution overviewThe proposed solution uses Amazon Bedrock and the Amazon Titan Express model to enable IDP functionalities.

The extracted text data is placed into another SQS queue for the next processin…

Amazon Q Business uses supported connectors such as Confluence, Amazon Relational Database Service (Amazon RDS), and web crawlers.

The Amazon Q Business web experience allows business users across various job titles and functions to interact with Amazon Q through the web browser.

The following demos are examples of what the Amazon Q Business web experience looks like.

PrerequisitesTo get started with your Amazon Q Business web experience, you need the following prerequisites:An AWS account that will contain your AWS resourcesAWS IAM Identity Center configured for an Amazon Q Business applicationAn Amazon Q Business subscription (Amazon Q Business Lite or Amazon Q Business Pro) and index (St…

This post demonstrates how you can gain a competitive advantage using Amazon Bedrock Agents based automation of a complex business process.

Solution overviewDigitalDhan, the proposed digital lending solution, is powered by Amazon Bedrock Agents.

Amazon Bedrock Agents deep diveBecause we used Amazon Bedrock Agents heavily in the DigitalDhan solution, let’s look at the overall functioning of Amazon Bedrock Agents.

Loan application: Collect all necessary details to create the loan application.

Based on your PAN {pan} and {aadhar}, your risk score is {riskScore} and your overall credit score is {cibilScore}.

The challenges of malware analysis – Malware analysis has become an increasingly critical and complex field.

Let’s explore some of the key challenges that make malware analysis demanding:Identifying malware – Modern malware has become incredibly sophisticated in its ability to disguise itself.

Deep Instinct, recognizing this need, has developed DIANNA (Deep Instinct’s Artificial Neural Network Assistant), the DSX Companion.

DIANNA is a sophisticated malware analysis tool that acts as a virtual team of malware analysts and incident response experts.

Prioritizing data security and compliance – Data security and compliance are paramount in the cybersecurity domain.

Amazon SES offers many email tools, including email sender configuration options, email deliverability tools, flexible email deployment options, sender and identity management, email security, email sending statistics, email reputation dashboard, and inbound email services.

This post explores how you can integrate Amazon Q Business with Amazon SES to email conversations to specified email addresses.

Add users to the Amazon Q Business applicationComplete the following steps to add users to the newly created Amazon Q business application:On the Amazon Q Business console, choose Applications in the navigation pane.

Sync Amazon Q data sourcesTo sync the data source, complete the following steps…

With Amazon Bedrock Custom Model Import, you can use new or existing models that have been trained or fine-tuned within SageMaker using Amazon SageMaker JumpStart.

PrerequisitesBefore you begin, verify that you have an AWS account with Amazon SageMaker Studio and Amazon Bedrock access.

If you don’t already have an instance of SageMaker Studio, see Launch Amazon SageMaker Studio for instructions to create one.

Under Model import settings, select Amazon SageMaker model and select the radio button next to your model.

To get more hands-on experience with Amazon Bedrock, check out our Building with Amazon Bedrock workshop.

To address this need, AWS generative AI best practices framework was launched within AWS Audit Manager, enabling auditing and monitoring of generative AI applications.

How to create your own assessment of the AWS generative AI best practices frameworkTo create an assessment using the generative AI best practices framework on Audit Manager, go to the AWS Management Console and navigate to AWS Audit Manager.

Principles of AWS generative AI best practices frameworkGenerative AI implementations can be evaluated based on eight principles in the AWS generative AI best practices framework.

To learn more on various security considerations for generative AI applications, see Securing generative AI: …

Amazon Q Business enables employees to become more creative, data-driven, efficient, organized, and productive.

In this blog post, we explore a client services agent assistant application developed by the London Stock Exchange Group (LSEG) using Amazon Q Business.

Following a review of available solutions, the LCH team decided to build a proof-of-concept around Amazon Q Business.

A ChatSync Lambda function is responsible for accessing the Amazon Q Business ChatSync API to start an Amazon Q Business conversation.

The Amazon Bedrock service uses Anthropic’s Claude v2 model to validate the Amazon Q Business application queries and responses against the golden answers stored in the S3 bucket.

This blog post with accompanying code presents a solution to experiment with real-time machine translation using foundation models (FMs) available in Amazon Bedrock.

The numbers are color-coded to represent two flows: the translation memory ingestion flow (orange) and the text translation flow (gray).

Strategy for TM knowledge baseThe LLM translation playground offers two options to incorporate the translation memory into the prompt.

Adding translation memoryLet’s test the impact of using a translation memory TMX file on the translation quality.

ConclusionThe LLM translation playground presented in this post enables you evaluate the use of LLMs for your machine translation needs.

Amazon Bedrock Flows provide a powerful, low-code solution for creating complex generative AI workflows with an intuitive visual interface and with a set of APIs in the Amazon Bedrock SDK.

Client email triageFor Parameta, every client email represents a critical touchpoint that demands both speed and accuracy.

Component integration – Seamless incorporation of other generative AI capabilities like Amazon Bedrock Agents or Amazon Bedrock Knowledge Bases, creating a comprehensive solution.

Amazon Bedrock Flows coordinates the sequence of operations, starting with the email from Amazon S3.

The low barrier to entry of Amazon Bedrock Flows allowed our team to quickly get up to speed and start del…

The model is designed to identify many of the roughly 3.7M oil and gas wells dug in the US since the mid-1800s.

But its primary purpose is to help find a particular subset of wells: undocumented orphaned wells (UOWs).

The only way to prevent potentially leaking wells from harming the environment is by sealing them—which is usually done with concrete.

Researchers identified UOWs by fine tuning a vision language model on digitized maps of California and Oklahoma counties (credit: Environ.

With those updated maps, the team fine-tuned their model on all the georeferenced maps of the two California counties.

In this blog post, we show how RAPIDS cuML can be used with skforecast to accelerate time series forecasting, allowing you to work with larger datasets and forecast windows.

Why time series forecasting?

In today’s data-driven world, enterprises rely on time series forecasting to make informed decisions, optimize processes, and mitigate risks.

Multistep forecastingOne popular technique used in time series forecasting is recursive multi-step forecasting, in which you train a single model and apply it recursively to predict the next n values in the series.

ConclusionTime series forecasting has been around for decades but remains incredibly important today.

NVIDIA NeMo Guardrails includes new NVIDIA NIM microservices to enhance accuracy, security and control for enterprises building AI across industries.

New NVIDIA NIM microservices for AI guardrails — part of the NVIDIA NeMo Guardrails collection of software tools — are portable, optimized inference microservices that help companies improve the safety, precision and scalability of their generative AI applications.

NeMo Guardrails helps developers integrate and manage AI guardrails in large language model (LLM) applications.

Industry leaders Amdocs, Cerence AI and Lowe’s are among those using NeMo Guardrails to safeguard AI applications.

AvailabilityNVIDIA NeMo Guardrails microservices, as wel…

The multiverse is about to get a whole lot cloudier as GeForce NOW opens a portal to the first season of hit game Marvel Rivals from NetEase Games.

Members can now game in a new dimension with expanded support for virtual- and mixed-reality devices.

The Fantastic Four will be playable in season one of the game.

Stream it all with a GeForce NOW membership across devices, from an underpowered laptop, Mac devices, a Steam Deck or the supported platform of virtual- and mixed-reality devices.

These newly supported devices will give members access to an extensive library of games to stream through GeForce NOW.

Troves of unwatched surgical video footage are finding new life, fueling AI tools that help make surgery safer and enhance surgical education.

The Surgical Data Science Collective (SDSC) is transforming global surgery through AI-driven video analysis, helping to close the gaps in surgical training and practice.

Learn more about SDSC, and hear more about the future of AI in healthcare by listening to the J.P. Morgan Healthcare Conference talk by Kimberly Powell, vice president of healthcare at NVIDIA.

CEO Aengus Tran emphasizes the importance of using AI in healthcare to reduce misdiagnoses and improve patient outcomes.

Subscribe to the AI PodcastGet the AI Podcast through Amazon Music, Appl…

Quantum computing is one of the most exciting areas in computer science, promising progress in accelerated computing beyond what’s considered possible today.

Quantum computing promises huge leaps forward for fields spanning drug discovery and materials development to financial forecasting.

NVIDIA is celebrating and exploring this remarkable progress in quantum computing by announcing its first Quantum Day at GTC 2025 on March 20.

A Quantum Day special address, unveiling the latest news and advances from NVIDIA in quantum computing shortening the timeline to useful applications.

Quantum Day at GTC 2025 is the destination for leaders and experts seeking to chart a course into the future of qu…

AI is making inroads across the entire healthcare industry — from genomic research to drug discovery, clinical trial workflows and patient care.

The four organizations at J.P. Morgan Healthcare announced partnerships with NVIDIA to advance drug discovery, accelerate pathology, enhance genomic research and augment healthcare with agentic AI, respectively.

“The ability for AI to now reason, plan and act is foundational to the way we’re going to go forward.”To support the development of these AI models, NVIDIA recently unveiled NVIDIA Cosmos, a physical AI platform that includes state-of-the art generative world foundation models.

For more from NVIDIA at the J.P. Morgan Healthcare Conference, …

In the webinar on January 28th, you'll get an inside look of the new GPU engine to learn how Polars' declarative API and query optimizer enable seamless GPU...

The recent NVIDIA webinar, Enhance Generative AI Model Accuracy with High-Quality Multimodal Data Processing, dove into the intricacies of data curation and processing, highlighting the capabilities of NVIDIA NeMo Curator.

NeMo Curator is available in multiple ways:NeMo Framework container/NVIDIA/NeMo-Curator GitHub repo/nemo-curator Pypi packageTo get started in production, create a NVIDIA AI Enterprise license and get production-ready branches, security updates, API stability, and support from NVIDIA AI experts.

Get Started with NeMoConclusionThe NVIDIA webinar underscored the significance of high-quality data in generative AI model development.

With NeMo Curator, you have access to power…

Using NVIDIA AI Foundry, the world’s leading clinical research and commercial services provider will offer its global life sciences customers AI agents to help accelerate complex workflows in drug research and development, evidence management and commercialization.

IQVIA, the world’s leading provider of clinical research services, commercial insights and healthcare intelligence, is working with NVIDIA to build custom foundation models and agentic AI workflows that can accelerate research, clinical development and access to new treatments.

IQVIA plans to use its unparalleled information assets, analytics and domain expertise — known as IQVIA Connected Intelligence — with the NVIDIA AI Foundr…

The NVIDIA BioNeMo Blueprint for generative protein binder design is a reference workflow for drug discovery platforms to help them use generative AI and GPU-accelerated microservices to intelligently navigate this immense search space.

Accelerate protein design with NVIDIA NIM and NVIDIA BlueprintsNVIDIA NIM microservices are modular, cloud-native components that accelerate AI model deployment and execution.

NVIDIA BioNeMo Blueprint for generative protein binder designThe NVIDIA BioNeMo Blueprint for generative protein binder design provides a comprehensive guide, showing how these microservices can optimize key stages of the protein design workflow.

A diagram showing the flow of informati…

The recently introduced SAFE-GPT model represented a paradigm shift in AI-driven molecular generation by introducing a chemically intuitive framework aligned with how medicinal chemists approach molecule design.

Comparing SAFE-GPT and GenMol for drug discovery tasksGenMol and SAFE-GPT represent two distinct approaches to AI-driven molecular generation, each with unique strengths and limitations (Table 1).

Molecular generation and exploration of chemical spaceSAFE-GPT uses a GPT architecture with sequential, autoregressive decoding, generating molecules fragment-by-fragment.

ConclusionThe importance of these molecular generation models goes beyond just how molecular generation is done.

Explo…

As a result, mainstream AI has become an integral part of every new application, driving economic growth, promoting U.S. interests and ensuring American leadership in cutting-edge technology.

Today, companies, startups and universities around the world are tapping mainstream AI to advance healthcare, agriculture, manufacturing, education and countless other fields, driving economic growth and unlocking the potential of nations.

The Biden Administration now seeks to restrict access to mainstream computing applications with its unprecedented and misguided “AI Diffusion” rule, which threatens to derail innovation and economic growth worldwide.

In its last days in office, the Biden Administrati…

NVIDIA’s second annual State of AI in Retail and CPG survey reveals rapid AI integration across business lines, especially advertising and marketing, customer engagement and supply chain operations.

This technological wave is simultaneously transforming advertising and marketing, customer engagement and supply chain operations.

By harnessing AI, retailers and CPG brands are not just adapting to change — they’re actively shaping the future of commerce.

It’s an in-depth look at the current ecosystem of AI in retail and CPG, and how it’s transforming the industries.

Consistent with last year’s survey, over 50% of retailers believe that generative AI is a strategic technology that will be a dif…

Driving the future of smart mobility, Hyundai Motor Group (the Group) is partnering with NVIDIA to develop the next generation of safe, secure mobility with AI and industrial digital twins.

Announced today at the CES trade show in Las Vegas, this latest work will elevate Hyundai Motor Group’s smart mobility innovation with NVIDIA accelerated computing, generative AI, digital twins and physical AI technologies.

“Hyundai Motor Group is exploring innovative approaches with AI technologies in various fields such as robotics, autonomous driving and smart factory,” said Heung-Soo Kim, executive vice president and head of the global strategy office at Hyundai Motor Group.

Hyundai Motor Group will …

Unlocking advertiser value through industry-leading ML innovationMeta Andromeda is a personalized ads retrieval engine that leverages the NVIDIA Grace Hopper Superchip, to enable cutting edge ML innovation in the Ads retrieval stage to drive efficiency and advertiser performance.

Its deployment across Instagram and Facebook applications has achieved +6% recall improvement to the retrieval system, delivering +8% ads quality improvement on selected segments.

Andromeda is designed to maximize ads performance by utilizing the exponential growth in volume of eligible ads available to the retrieval stage.

The design is optimized for AI hardware, minimizing memory bandwidth bottlenecks and enablin…

Meta’s ad recommendation engine, powered by deep learning recommendation models (DLRMs), has been instrumental in delivering personalized ads to people.

Learning from sequences: developing new sequence learning architectures to replace traditional DLRM neural network architectures.

A paradigm shift with learning from sequences for recommendation systemsMeta’s new system for ads recommendations uses sequence learning at its core.

Scaling the new sequence learning paradigmFollowing the redesign to shift from sparse feature learning to event-based sequence learning, the next focus was scaling across two domains — scaling the sequence learning architecture and scaling event sequences to be long…

At Open Compute Project Summit (OCP) 2024, we’re sharing details about our next-generation network fabric for our AI training clusters.

We’ve expanded our network hardware portfolio and are contributing two new disaggregated network fabrics and a new NIC to OCP.

At Meta, we believe that open hardware drives innovation.

At Meta, we envision a future of AI hardware systems that are not only scalable, but also open and collaborative.

We encourage anyone who wants to help advance the future of networking hardware for AI to engage with OCP and Meta to help share the future of AI infrastructure.

At the Open Compute Project (OCP) Global Summit 2024, we’re showcasing our latest open AI hardware designs with the OCP community.

These innovations include a new AI platform, cutting-edge open rack designs, and advanced network fabrics and components.

The open future of AI infraMeta is committed to open source AI.

We must also prioritize open and standardized models so we can leverage collective expertise, make AI more accessible, and work towards minimizing biases in our systems.​Just as important, we also need open AI hardware systems.

By addressing AI’s infrastructure needs together, we can unlock the true promise of open AI for everyone.​

Why we need better population mapsAccurate estimates of population are taken for granted in many countries.

As the world’s natural resource and energy demands scale, accurate population estimates also offer significant opportunities to improve sustainability efforts.

In addition to total population counts, Meta’s population maps also include demographic breakdowns for groups such as the number of children under five, women of reproductive age, youth, and the elderly.

AI-powered population estimates have been scientifically evaluated to be among the most accurate in the world for mapping population distribution for a variety of geographies and use-cases.

Please visit the Data for Good websit…

Meta is revamping its new data center design to optimize for artificial intelligence and the same methodology will be applicable for future data center optimizations as well.

As Meta is revamping its new data center design to optimize for artificial intelligence, the same methodology will be applicable for future data center optimizations as well to improve operational efficiency.

A reinforcement learning approach to data center coolingReinforcement learning (RL) is good at modeling control systems as sequential state machines.

There are also various RL approaches reported such as, transforming cooling optimization via deep reinforcement learning and data center cooling using model-predicti…

How PyTorch powers AI training and inferenceLearn about new PyTorch advancements for LLMs and how PyTorch is enhancing every aspect of the LLM lifecycle.

In this talk from AI Infra @ Scale 2024, software engineers Wanchao Liang and Evan Smothers are joined by Meta research scientist Kimish Patel to discuss our newest features and tools that enable large-scale training, memory efficient fine-tuning, and on-device LLM capabilities.

First, they cover the importance of memory-efficient fine-tuning and a few common architectural and algorithmic techniques to enable fine-tuning on consumer-grade hardware.

Then they discuss the challenges of deploying large models for on-device deployment and how …

In this talk from AI Infra @ Scale 2024, Joel Colburn, a software engineer at Meta, technical lead Junqiang Lan, and software engineer Jack Montgomery discuss the second generation of MTIA, Meta’s in-house training and inference accelerator.

They cover the co-design process behind building the second generation of Meta’s first-ever custom silicon for AI workloads, including the PyTorch software ecosystem, and the model architectures for Meta’s key applications.

They demonstrate how MTIA achieves the performance, efficiency, and developer experience to successfully launch models into production.

They also highlight several co-design examples where special silicon features are utilized to acc…

At AI Infra @ Scale 2024, Meta engineers discussed every step of how we built and brought Llama 3 to life, from data and training to inference.

Joe Spisak, Product Director and Head of Generative AI Open Source at Meta, talks about the history of Llama and Meta’s overarching vision for open source AI.

He’s joined by Delia David, a software engineer at Meta, to discuss all things data-related for GenAI.

Kaushik Veeraraghavan, a software engineer at Meta, discusses how Meta trains Llama at scale and delves into the data center, networking, and software investments that have enabled the development of Meta’s Llama 3 models.

Finally, Ye (Charlotte) Qia, a production engineer at Meta, discusses …

Delivering new AI technologies at scale also means rethinking every layer of our infrastructure – from silicon and software systems and even our data center designs.

For the second year in a row, Meta’s engineering and infrastructure teams returned for the AI Infra @ Scale conference, where they discussed the challenges of scaling up an infrastructure for AI as well as work being done on our large-scale GPU clusters, open hardware designs for next-generation data center hardware, and how Meta is building custom silicon like the Meta Training and Inference Accelerator (MTIA) to handle some of our AI training workloads.

Aparna Ramani, VP of Engineering at Meta, responsible for AI infrastructu…

Meta AI’s animate feature, which lets people generate a short animation of a generated image, carried unique challenges in this regard.

Here’s how we were able to deploy Meta AI’s animate feature using a combination of latency optimizations, traffic management, and other novel techniques.

We started by looking at the data for previous traffic on our AI-generated media both at their launches and over time.

With these changes, the preponderance of requests remained in region and latency dropped to roughly what we would expect.

The service tries to take a chunk of that region’s requests and offload them to a nearby region that can handle them without becoming more overloaded.

Our paper, “ RDMA over Ethernet for Distributed AI Training at Meta Scale ,” provides the details on how we design, implement, and operate one of the world’s largest AI networks at scale.

These RoCE clusters support an extensive range of production distributed GPU training jobs, including ranking, content recommendation, content understanding, natural language processing, and GenAI model training, among other workloads.

However, our experience with distributed AI training workloads provides a different perspective on tailoring the congestion control algorithms.

Moving forwardThe design and operation of large-scale RoCE networks for distributed AI training workloads have evolved to meet the …

What happens when a team of mechanical engineers get tired of looking at flat images of 3D models over Zoom?

Meet the team behind Caddy, a new CAD app for mixed reality.

They join Pascal Hartig (@passy) on the Meta Tech Podcast to talk about teaching themselves to code, disrupting the CAD software space, and how they integrated Caddy with Llama 3, and so much more!

Download or listen to the podcast episode below:You can also find the episode wherever you get your podcasts, including:The Meta Tech Podcast is a podcast, brought to you by Meta, where we highlight the work Meta’s engineers are doing at every level – from low-level frameworks to end-user features.

And if you’re interested in lea…

The key to developer velocity across AI lies in minimizing time to first batch (TTFB) for machine learning (ML) engineers.

AI Lab prevents TTFB regressions whilst enabling experimentation to develop improvements.

Optimizing TTFB helps ML engineers move fastThe overhead induced from TTFB is on the critical path for most ML development.

Here, we see the true utility of a framework like AI Lab and how it was used to facilitate this sweeping change.

O(Releases): Running a more holistic set of AI Lab tests prior to release and performing a bisect-like attribution process to find the root cause.

Improving tail utilization – the utilization level of the top 5% of the servers when ranked by utilization– within our infrastructure is imperative to operate our fleet efficiently and sustainably.

Challenges of load balancingThere are two approaches to load balancing:Routing load balancing – load balancing across replicas of a single model.

Placement load balancing – balancing load on hosts by moving replicas of a model across hosts.

It also exposed a deeper problem like spiky tail utilization, which was hidden behind the high tail utilization and was fixed once identified .

Optimizing tail utilization within IPnext thereby delivering these benefits to a broader range of expanding machine …

In this guide, we’ll show you how to build a RAG system using the LangChain framework, evaluate its performance using Ragas, and track your experiments with neptune.ai.

Part 1: Building a baseline RAG system with LangChainIn the first part of this guide, we’ll use LangChain to build a RAG system for the blog posts in the LLMOps category on Neptune’s blog.

Ragas works smoothly with LangChain, making it a great choice for evaluating our RAG system.

Step 1: Generate a RAG evaluation datasetAn evaluation set for RAG tasks is similar to a question-answering task dataset.

Step 2: Choose RAG evaluation metricsAs mentioned earlier, Ragas offers both LLM-based and non-LLM-based metrics for RAG syste…

In our paper, Understanding LLMs Requires More Than Statistical Generalization, we argue that current machine learning theory cannot explain the interesting emergent properties of Large Language Models, such as reasoning or in-context learning.

Inductive biases affect which solution the neural network converges to, such as the model architecture or the optimization algorithm.

How do language complexity and model architecture affect generalization ability?

showed how different neural network architectures generalize better for different language types.

Presumably, we’ll need to find different complexity measures for different model architectures that consider their specific inductive biases.

TL;DR At a large-scale model training (in huge models), anomalies are not rare events but problematic patterns that drive failure.

The Neptune Scale experiment tracker supports fault tolerance and is designed to maintain progress despite hardware failures, making it adaptable for enterprise teams tackling LLM fine-tuning, compliance, and building domain-specific models.

Experiment tracking back then was straightforward—dealing mostly with single models or small-scale distributed systems.

One of the biggest lessons we’ve learned is that experiment tracking has evolved into experiment monitoring.

That’s why we’re focusing on building intelligent alerts and anomaly detection right into our exp…

Key-value (KV) caching is a clever trick to do that: At inference time, key and value matrices are calculated for each generated token.

Implementing K-V caching in large-scale production systems requires careful cache management, including choosing an appropriate strategy for cache invalidation and exploring opportunities for cache reuse.

Key-value (KV) caching is a clever trick to do just that – let’s see how it works and when to use it.

Transformer architecture overviewBefore we dive into KV caching, we will need to take a short detour to the attention mechanism used in transformers.

Understanding how it works is required to spot and appreciate how KV caching optimizes transformer inferen…

In our paper, Learn from Failure: Fine-Tuning LLMs with Trial-and-Error Data for Intuitionistic Propositional Logic Proving, we explored this problem experimentally.

Our goal was to assess the influence of trial-and-error information in the training data on the performance of LLMs in theorem proving.

However, at the time we published our paper, current approaches to training LLMs for ATPs only utilized data on correct proof attempts.

We hope our work can raise the community’s awareness of the importance of trial-and-error data for automated theorem proving.

We believe this advancement is largely due to the substantial trial-and-error data included in the model’s training process.

We will explore these challenges and provide an example of fine-tuning the Llama 3 8B Instruct model utilizing the neptune.ai experiment tracker.

The Llama 3 training data is seven times larger than what Meta used for training Llama 2.

For pre-training, Meta combined four types of parallelization, an approach they dubbed “4D parallelism”: data, model, pipeline, and context.

Hands-on guide: resource-efficient fine-tuning of Llama 3 on Google ColabFine-tuning Llama 3 8B is challenging, as it requires considerable computational resources.

We’ll use the Llama 3 8B model, which is sufficient for this task despite being the smallest Llama 3 model.

This is the process we will be going through in the following section to answer the question: When do I decide to run LLMs locally?

PrivacyA very obvious argument in favor of running LLMs locally is that, in some cases, there is no alternative.

Related ML/AI Platform Build vs Buy Decision: What Factors to Consider Read moreWhat does it take to run LLMs locally?

However, recent advancements in optimization techniques, such as quantization and attention mechanism optimizations, have made it possible to run LLMs locally, even on a CPU.

LM StudioLM Studio is a user-friendly application designed to run LLMs locally.

Like Union, Neptune excels in scalability, making it the ideal tracking solution for teams working on large-scale model training.

The new Neptune Flyte plugin enables you to use Neptune to track, visualize, and manage your models.

In this blog post, you’ll learn how to use the Neptune plugin on Union.

Orchestrate and track your models with Flytekit’s Neptune PluginIn Union, data and compute are fundamental building blocks for developing all workflows.

With flytekit’s Neptune plugin, you can easily track your experiments, visualize results, and debug your models.

What are LLM hallucinations?

LLM hallucinations become a problem in LLM-based applicationsMost of the time, if you use an LLM, you probably won’t use a base LLM but an LLM-based assistant whose goal is to help with your requests and reliably answer your questions.

Before we dive into this further, I’d like to stress that when thinking about LLM hallucinations, it’s important to keep in mind the difference between a base LLM and an LLM-based assistant.

When we talk about LLM hallucinations as a problematic phenomenon, it’s in the context of an LLM-based assistant or system.

While it’s unlikely that this process introduces new hallucinations, hallucinations seeded upstream are amplified.

LLM guardrails prevent models from generating harmful, biased, or inappropriate content and ensure that they adhere to guidelines set by developers and stakeholders.

LLM guardrails are small programs that validate and correct the modes’ outputs to ensure they align with your application’s specific requirements and context.

We’ll approach the broad and constantly evolving field of LLM guardrails in three stages:First, we’ll talk about the key vulnerabilities threatening AI applications.

We’ll explore these different kinds of LLM guardrails using the Guardrails AI framework, an open-source tool for building reliable AI applications.

| SourceRule-based data validationThe simplest type of LLM g…

TL;DR Reinforcement Learning from Human Feedback (RLHF) unlocked the full potential of today’s large language models (LLMs).

Reinforcement Learning from Human Feedback (RLHF) has turned out to be the key to unlocking the full potential of today’s large language models (LLMs).

Related LLM Evaluation For Text Summarization Read moreThe RLHF processThe RLHF process consists of three steps:Collecting human feedback.

Collecting human feedbackThe first step in RLHF is to collect human feedback in the so-called preference dataset.

We analyzed the three steps of the RLHF training pipeline: collecting human feedback, training the reward model, and fine-tuning the LLM.

However, when we look for data in a specific domain or organization, we often end up finding structured data.

The most likely reason is that structured data is still the de facto standard for quantitative information.

Consequently, in the age of Large Language Models (LLM), structured data still is and will continue to be relevant—even Microsoft is working on adding Large Language Models (LLMs) to Excel!

LLMs are mostly used with unstructured data, particularly text, but with the proper tools, they can also help tackle tasks with structured data.

Use case 3: Synthetic structured data generationWhen working with structured datasets, it is common to need more data with the same characteristic…

In this article, I’ll explore the following questions:What are the basic and advanced strategies for prompt engineering, and when should they be used?

Basic prompt engineering strategiesWhile there is no universal way to design a prompt, there are strategies we can follow to create prompts that yield LLM outputs closer to what we expect.

The basic strategies we cover in this section are straightforward to implement in a lot of different tasks, and they involve minimal customization.

Use Cases: When to use prompt templatesFinally, I recommend trying this approach for your project if:You encounter repetitive tasks, prompt templates ensure uniformity and save a lot of time.

Lessons learnedAs w…

TL;DR Evaluating text summarization is difficult because there is no one correct solution, and summarization quality often depends on the summary’s context and purpose.

How does LLM text summarization work?

METEOR is recall-oriented and ensures that the generated text captures as much information from the reference text.

Example: Source Text: {{Document}} Summary: {{Summary}} Evaluation Form (scores ONLY): – Relevance: “””Different prompts for different evaluation criteria are available.

The extensive use of LLMs for text summarization, including the integration of summarization features in search engines, makes research in this field highly popular and relevant.

The demand for observability and the scale of the required infrastructure vary significantly along the LLMOps value chain.

The distributed structure of agentic networks adds another level of complexity, which is not yet addressed fully by LLM observability tools and practices.

The value chain of LLMOpsThe LLMOps value chain starts with training foundation models and subsequent task-specific finetuning.

Each step has different observability needs and requires different scales of observability tooling and infrastructure.

Full talk Observability in LLMOps: Different Levels of Scale Watch on Youtube

https://ykilcher.com/discord Links:

TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick

YouTube: https://www.youtube.com/c/yannickilcher

Twitter: https://twitter.com/ykilcher

Discord: https://discord.gg/4H8xxDF

BitChute: https://www.bitchute.com/channel/yannic-kilcher

Minds: https://www.minds.com/ykilcher

Parler: https://parler.com/profile/YannicKilcher

LinkedIn: https://www.linkedin.com/in/yannic-kilcher-488534136/

BiliBili: https://space.bilibili.com/1824646584 If you want to support me, the best thing to do is to share out the content :) If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):

SubscribeStar: https:/…

#tokenization #llm #meta This paper does away with tokenization and creates an LLM architecture that operates on dynamically sized "patches" instead of tokens. By controlling the patch size, they gain a level of control over the tradeoff between model size and FLOPs and use that to achieve more favorable scaling behavior than classically tokenized LLMs. Paper: https://ai.meta.com/research/publications/byte-latent-transformer-patches-scale-better-than-tokens/

Code: https://github.com/facebookresearch/blt Abstract:

We introduce the Byte Latent Transformer (BLT), a new byte-level LLM architecture that, for the first time, matches tokenization-based LLM performance at scale with significant imp…

This paper demonstrates in a series of experiments that current safety alignment techniques of LLMs, as well as corresponding jailbreaking attacks, are in large part focusing on modulating the distribution of the first few tokens of the LLM response. Paper: https://openreview.net/forum?id=6Mxhg9PtDE&s=09 Abstract:

The safety alignment of current Large Language Models (LLMs) is vulnerable. Simple attacks, or even benign fine-tuning, can jailbreak aligned models. We note that many of these vulnerabilities are related to a shared underlying issue: safety alignment can take shortcuts, wherein the alignment adapts a model's generative distribution primarily over only its very first few output to…

A deep dive into the TokenFormer and an opinion about its impact, novelty, and relation to prior work. Paper: https://arxiv.org/abs/2410.23168 Abstract:

Transformers have become the predominant architecture in foundation models due to their excellent performance across various domains. However, the substantial cost of scaling these models remains a significant concern. This problem arises primarily from their dependence on a fixed number of parameters within linear projections. When architectural modifications (e.g., channel dimensions) are introduced, the entire model typically requires retraining from scratch. As model sizes continue growing, this strategy results in increasingly high com…

This paper (by Apple) questions the mathematical reasoning abilities of current LLMs and designs a synthetic template-based dataset distribution to investigate various aspects around LLM performance of high-school level math questions. Paper: https://arxiv.org/abs/2410.05229 Abstract:

Recent advancements in Large Language Models (LLMs) have sparked interest in their formal reasoning capabilities, particularly in mathematics. The GSM8K benchmark is widely used to assess the mathematical reasoning of models on grade-school-level questions. While the performance of LLMs on GSM8K has significantly improved in recent years, it remains unclear whether their mathematical reasoning capabilities hav…

This paper posits the interesting question: How much of the performance of Mamba, S4, and other state-space-like models is actually just attributable to some very core concepts - rather than their elaborate architectures. The authors construct minimal versions of GRUs and LSTMs and report competitive performance. Paper: https://arxiv.org/abs/2410.01201 Abstract:

The scalability limitations of Transformers regarding sequence length have renewed interest in recurrent sequence models that are parallelizable during training. As a result, many novel recurrent architectures, such as S4, Mamba, and Aaren, have been proposed that achieve comparable performance. In this work, we revisit traditional …

How can one best use extra FLOPS at test time? Paper: https://arxiv.org/abs/2408.03314 Abstract:

Enabling LLMs to improve their outputs by using more test-time computation is a critical step towards building generally self-improving agents that can operate on open-ended natural language. In this paper, we study the scaling of inference-time computation in LLMs, with a focus on answering the question: if an LLM is allowed to use a fixed but non-trivial amount of inference-time compute, how much can it improve its performance on a challenging prompt? Answering this question has implications not only on the achievable performance of LLMs, but also on the future of LLM pretraining and how one s…

#llm #privacy #finetuning Can you tamper with a base model in such a way that it will exactly remember its fine-tuning data? This paper presents a method of doing exactly that, and implements it in modern transformers. OUTLINE:

0:00 - Intro & Overview

10:50 -Core idea: single-use data traps

44:30 - Backdoors in transformer models

58:00 - Additional numerical tricks

1:00:35 - Experimental results & conclusion Paper: https://arxiv.org/abs/2404.00473

Code: https://github.com/ShanglunFengatETHZ/PrivacyBackdoor Abstract:

Practitioners commonly download pretrained machine learning models from open repositories and finetune them to fit specific applications. We show that this practice introduces a…

Matrix multiplications (MatMuls) are pervasive throughout modern machine learning architectures. However, they are also very resource intensive and require special accelerators (GPUs). This paper explores architectures that do away with MatMuls and use quantization and recurrence to keep performance up. OUTLINE:

0:00 - Intro

2:30 - MatMul is everywhere

5:55 - Ternary accumulation as a substitute for matrix multiplication

16:35 - Replacing attention layers with recurrent layers

32:40 - Replacing dense layers with ternary channel mixing

38:30 - Language modelling results & scaling laws

45:00 - Other experimental results

48:20 - Conclusion Paper: https://arxiv.org/abs/2406.02528

Code: https://…

#rag #hallucinations #legaltech An in-depth look at a recent Stanford paper examining the degree of hallucinations in various LegalTech tools that incorporate LLMs. OUTLINE:

0:00 - Intro

1:58 - What are legal research tools and how are large language models used by them?

5:30 - Overview and abstract of the paper

9:29 - What is a hallucination and why do they occur?

15:45 - What is retrieval augmented generation (RAG)?

25:00 - Why LLMs are a bad choice when reasoning is involved

29:16 - The products that were tested

32:00 - Some shady practices by the researchers in the back and forth with the legal research companies

37:00 - Legal technology companies’ marketing claims to eliminate or solve…

xLSTM is an architecture that combines the recurrency and constant memory requirement of LSTMs with the large-scale training of transformers and achieves impressive results. Paper: https://arxiv.org/abs/2405.04517 Abstract:

In the 1990s, the constant error carousel and gating were introduced as the central ideas of the Long Short-Term Memory (LSTM). Since then, LSTMs have stood the test of time and contributed to numerous deep learning success stories, in particular they constituted the first Large Language Models (LLMs). However, the advent of the Transformer technology with parallelizable self-attention at its core marked the dawn of a new era, outpacing LSTMs at scale. We now raise a sim…

#gpt4o #sky #scarlettjohansson After the release of their flagship model GPT-4o, OpenAI finds itself in multiple controversies and an exodus of senior personnel - notably Ilya Sutskever References:

https://openai.com/index/gpt-4o-and-more-tools-to-chatgpt-free/

https://openai.com/index/hello-gpt-4o/

https://x.com/LiamFedus/status/1790064963966370209?t=rx2YBT9AdDdKPhI6dUH4zA&s=09

https://x.com/lmsysorg/status/1790097588399779991?t=rx2YBT9AdDdKPhI6dUH4zA&s=09

https://x.com/bindureddy/status/1790127425705120149?t=mMUBqFBRphx-bDuZ1j3mjQ&s=09

https://openai.com/index/improvements-to-data-analysis-in-chatgpt/

https://openai.com/index/openai-and-reddit-partnership/

https://archive.ph/jHlMm

https:/…

Paper: https://arxiv.org/abs/2403.07691 Abstract:

While recent preference alignment algorithms for language models have demonstrated promising results, supervised fine-tuning (SFT) remains imperative for achieving successful convergence. In this paper, we study the crucial role of SFT within the context of preference alignment, emphasizing that a minor penalty for the disfavored generation style is sufficient for preference-aligned SFT. Building on this foundation, we introduce a straightforward and innovative reference model-free monolithic odds ratio preference optimization algorithm, ORPO, eliminating the necessity for an additional preference alignment phase. We demonstrate, both empiri…

OUTLINE:

0:00 - Intro

0:19 - Our next-generation Meta Training and Inference Accelerator

01:39 - ALOHA Unleashed

03:10 - Apple Inks $50M Deal with Shutterstock for AI Training Data

04:28 - OpenAI Researchers, Including Ally of Sutskever, Fired for Alleged Leaking

05:01 - Adobe's Ethical Firefly AI was Trained on Midjourney Images

05:52 - Trudeau announces $2.4billion for AI-related investments

06:48 - RecurrentGemma: Moving Past Transformers for Efficient Open Language Models

07:15 - CodeGemma - an official Google release for code LLMs

07:24 - Mistral AI: Cheaper, Better, Faster, Stronger

08:08 - Vezora/Mistral-22B-v0.1

09:00 - WizardLM-2, next generation state-of-the-art-LLM

09:31 - Idefic…

Paper: https://arxiv.org/abs/2404.09173 Abstract:

While Transformers have revolutionized deep learning, their quadratic attention complexity hinders their ability to process infinitely long inputs. We propose Feedback Attention Memory (FAM), a novel Transformer architecture that leverages a feedback loop to enable the network to attend to its own latent representations. This design fosters the emergence of working memory within the Transformer, allowing it to process indefinitely long sequences. TransformerFAM requires no additional weights, enabling seamless integration with pre-trained models. Our experiments show that TransformerFAM significantly improves Transformer performance on long-…

Hey everyone! I am super excited to share a quick notebook on using Generative Feedback Loops to chunk code files and better structure how they are indexed in the Weaviate Vector Database! Chunking is one of the key topics in Vector Search. We need to break up long documents into smaller parts that we can encode with a pre-trained embedding model and index in a vector index, such as HNSW-PQ. Most solutions use some form of a rolling token window such as taking every 300 tokens as a chunk, with say 50 tokens overlapping between each window. Unfortunately, this solution doesn't work that well for code particularly. We don't want the chunk to cut off in the middle of a function or class defini…

Hey everyone! Thanks so much for watching this video exploring Gemini Pro 1.5 and Gemini Flash! Long Context LLMs!! This video covers 3 key tests, the classic "Lost in the Middle" exploration, using Long Context LLMs as Re-rankers in Search, and finally, testing Many-Shot In-Context Learning! I am really excited about the potential of Many-Shot In-Context Learning with DSPy's `BootstrapFewShot` and Gemini, curious to know what you think! Notebook: https://github.com/weaviate/recipes/blob/main/integrations/dspy/llms/Gemini-1.5-Pro-and-Flash.ipynb Gemini 1.5 Technical Report: https://storage.googleapis.com/deepmind-media/gemini/gemini_v1_5_report.pdf Chapters

0:00 Gemini 1.5!!

1:25 Setup and …

Hey everyone! Thank you so much for watching this overview of Llama 3 looking at the release notes and seeing a demo of how to integrate it with DSPy through Ollama and how to use DSPy's MIPRO to find the optimal prompt when using this new large language model for RAG! We are hosting an event in San Francisco on May 1st with Arize AI and Cohere, featuring a talk from Omar Khattab, the lead author of DSPy! Hope to see you there! https://lu.ma/dspy Introducing Meta Llama 3: https://ai.meta.com/blog/meta-llama-3/ Ollama Llama 3: https://ollama.com/library/llama3 Weaviate Recipes: https://github.com/weaviate/recipes/blob/main/integrations/dspy/llms/Llama3.ipynb Chapters

0:00 Llama3!!

1:28 Relea…

Hey everyone! Thank you so much for watching this overview of Command R+ showing you how you can use the new model in DSPy and a quick RAG demo, as well as walking through the details of the release post! Congratulations to the Cohere team! Super exciting times to be working with LLM systems! Introducing Command R+: A Scalable LLM Built for Business - https://txt.cohere.com/command-r-plus-microsoft-azure/ Link to demo notebook - https://github.com/weaviate/recipes/blob/main/integrations/dspy/llms/Command-R-Plus.ipynb Chapters

0:00 Welcome! Command R+!

1:12 Demo with Cohere, DSPy, and Weaviate

6:06 Command R+ Announcement Post

9:24 LLM Evals

Unfortunately, Large Language Models will not consistently follow the instructions that you give them. This is a massive problem when you are building AI systems that require a particular type of output from the previous step to feed into the next one! For example, imagine you are building a blog post writing system that first takes a question and retrieved context to output a list of topics. These topics have to be formatted in a particular way, such as a comma-separated list or a JSON of Topic objects, such that the system can continue writing the blog post! I am SUPER excited to share the 4th video in my DSPy series, diving into 3 solutions to structuring outputs in DSPy programs: (1) **…

Full video: https://youtu.be/IQqtsm-bBRU

Series exploring optics: https://www.youtube.com/watch?v=QCX62YJCmGk&list=PLZHQObOWTQDMKqfyUvG2kTlYt-QQ2x-ui

Full video: https://youtu.be/piJkuavhV50

Extracted from this video about holograms: https://youtu.be/EmKQsSDlaa4

Full video: https://youtu.be/IQqtsm-bBRU

A beautiful solution to the inscribed rectangle problem.

Playlist with more neat proofs: https://www.youtube.com/playlist?list=PLZHQObOWTQDPSKntUcMArGheySM4gL7wS

Instead of sponsored ad reads, these lessons are funded directly by viewers: https://3b1b.co/support

An equally valuable form of support is to simply share the videos. This argument was originally by Herbert Vaughan, appearing for examples in this issue of the Topology Proceedings.

https://topology.nipissingu.ca/tp/reprints/v06/tp06107.pdf 2020 Paper by Greene and Lobb:

https://arxiv.org/pdf/2005.09193 Nice Quanta article about this result:

https://www.quantamagazine.org/new-geometric-perspective-cracks-old-problem-about-rectangles…

The full video dives deeper into the field of math studying this called holomorphic dynamics: https://youtu.be/LqbZpur38nw

From this 7-minute LLM explainer: https://youtu.be/LPZh9BOjkQs

Dig deeper here: https://www.youtube.com/playlist?list=PLZHQObOWTQDNU6R1_67000Dx_ZCJB-3pi

Technical details as a talk: https://youtu.be/KJtZARuO3JY

Made for an exhibit at the Computer History Museum: https://computerhistory.org/

Instead of sponsored ad reads, these lessons are funded directly by viewers: https://3b1b.co/support Timestamps:

0:00 - Who this was made for

0:41 - What are large language models?

7:48 - Where to learn more No secret end-screen vlog for this one, the end-screen real estate was all full! ------------------ These animations are largely made using a custom Python library, manim. See the FAQ comments here:

https://3b1b.co/faq#manim

https://github.com/3b1b/manim

https:/…

From this full video: https://youtu.be/piJkuavhV50

Full video: https://youtu.be/GNcFjFmqEc8

Full video: https://youtu.be/-RdOwhmqP5s

The referenced stack exchange post by usename 2'5 9'2, http://math.stackexchange.com/questions/30046/alternative-notation-for-exponents-logs-and-roots

Full video: https://youtu.be/piJkuavhV50

I learned this from Matt Parker, who then made a full video: https://youtu.be/ga9Qk38FaHM

❤️ Check out Lambda here and sign up for their GPU Cloud: https://lambdalabs.com/papers Cosmos platform:

https://www.nvidia.com/en-us/ai/cosmos/

Hugging Face models: https://huggingface.co/collections/nvidia/cosmos-6751e884dc10e013a0a0d8e6

More: https://github.com/NVIDIA/Cosmos 📝 The paper "Cosmos World Foundation Model Platform for Physical AI" is available here:

https://research.nvidia.com/publication/2025-01_cosmos-world-foundation-model-platform-physical-ai 📝 My paper on simulations that look almost like reality is available for free here:

https://rdcu.be/cWPfD Or this is the orig. Nature Physics link with clickable citations:

https://www.nature.com/articles/s41567-022-01788-5 🙏 We woul…

❤️ Check out Weights & Biases and sign up for a free demo here: https://wandb.me/papers 📝 The paper is available here:

https://github.com/Genesis-Embodied-AI/DiffTactile

https://difftactile.github.io/ 📝 My paper on simulations that look almost like reality is available for free here:

https://rdcu.be/cWPfD Or this is the orig. Nature Physics link with clickable citations:

https://www.nature.com/articles/s41567-022-01788-5 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:

Alex Balfanz, Alex Haro, B Shang, Benji Rabhan, Gaston Ingaramo, Gordon Child, John Le, Juan Benet, Kyle Davis, Loyal Alchemist, Lukas Biewald, Martin, Michael Albrecht, Michael Te…

❤️ Check out Lambda here and sign up for their GPU Cloud: https://lambdalabs.com/papers 📝 The Edify 3D paper available here:

https://research.nvidia.com/labs/dir/edify-3d/

https://build.nvidia.com/shutterstock/edify-3d 📝 MeshGPT paper: https://nihalsid.github.io/mesh-gpt/ 📝 My paper on simulations that look almost like reality is available for free here:

https://rdcu.be/cWPfD Or this is the orig. Nature Physics link with clickable citations:

https://www.nature.com/articles/s41567-022-01788-5 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:

Alex Balfanz, Alex Haro, B Shang, Benji Rabhan, Gaston Ingaramo, Gordon Child, John Le, Juan Benet, Kyle Dav…

❤️ Check out Weights & Biases and sign up for a free demo here: https://wandb.me/papers 📝 The paper "Deep Fourier-based Arbitrary-scale Super-resolution for Real-time Rendering" is available here:

https://iamxym.github.io/DFASRR.github.io/ 📝 My paper on simulations that look almost like reality is available for free here:

https://rdcu.be/cWPfD Or this is the orig. Nature Physics link with clickable citations:

https://www.nature.com/articles/s41567-022-01788-5 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:

Alex Balfanz, Alex Haro, B Shang, Benji Rabhan, Gaston Ingaramo, Gordon Child, John Le, Juan Benet, Kyle Davis, Loyal Alchemist, Lukas Biewal…

❤️ Check out Lambda here and sign up for their GPU Cloud: https://lambdalabs.com/papers 📝 The papers are available here:

https://hover-versatile-humanoid.github.io/

https://blogs.nvidia.com/blog/robot-learning-humanoid-development/ 📝 My paper on simulations that look almost like reality is available for free here:

https://rdcu.be/cWPfD Or this is the orig. Nature Physics link with clickable citations:

https://www.nature.com/articles/s41567-022-01788-5 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:

Alex Balfanz, Alex Haro, B Shang, Benji Rabhan, Gaston Ingaramo, Gordon Child, John Le, Juan Benet, Kyle Davis, Loyal Alchemist, Lukas Biewald, Marti…

❤️ Check out Lambda here and sign up for their GPU Cloud: https://lambdalabs.com/papers Try Unreal Engine 5 here:

https://www.unrealengine.com/en-US/unreal-engine-5 My free course on ray tracing for you Fellow Scholars:

https://users.cg.tuwien.ac.at/zsolnai/gfx/rendering-course/ Our earlier paper with the spheres scene:

https://users.cg.tuwien.ac.at/zsolnai/gfx/adaptive_metropolis/ LuxCoreRender (free and open source): https://luxcorerender.org/ 📝 My paper on simulations that look almost like reality is available for free here:

https://rdcu.be/cWPfD Or this is the orig. Nature Physics link with clickable citations:

https://www.nature.com/articles/s41567-022-01788-5 🙏 We would like to thank …

❤️ Check out Lambda here and sign up for their GPU Cloud: https://lambdalabs.com/papers Sora is available here (for select countries, not in EU currently):

https://openai.com/sora/

https://sora.com/ 📝 My paper on simulations that look almost like reality is available for free here:

https://rdcu.be/cWPfD Or this is the orig. Nature Physics link with clickable citations:

https://www.nature.com/articles/s41567-022-01788-5 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:

Alex Balfanz, Alex Haro, B Shang, Benji Rabhan, Gaston Ingaramo, Gordon Child, John Le, Juan Benet, Kyle Davis, Loyal Alchemist, Lukas Biewald, Martin, Michael Albrecht, Michael Tedd…

❤️ Check out Lambda here and sign up for their GPU Cloud: https://lambdalabs.com/papers 📝 DeepMind's Genie 2 info is available here:

https://deepmind.google/discover/blog/genie-2-a-large-scale-foundation-world-model/ 📝 My paper on simulations that look almost like reality is available for free here:

https://rdcu.be/cWPfD Or this is the orig. Nature Physics link with clickable citations:

https://www.nature.com/articles/s41567-022-01788-5 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:

Alex Balfanz, Alex Haro, B Shang, Benji Rabhan, Gaston Ingaramo, Gordon Child, John Le, Juan Benet, Kyle Davis, Loyal Alchemist, Lukas Biewald, Martin, Michael Albr…

❤️ Check out Lambda here and sign up for their GPU Cloud: https://lambdalabs.com/papers 📝 The paper "Scintilla: Simulating Combustible Vegetation for Wildfires" is available here:

https://storage.googleapis.com/pirk.io/projects/scintilla/index.html 📝 My paper on simulations that look almost like reality is available for free here:

https://rdcu.be/cWPfD Or this is the orig. Nature Physics link with clickable citations:

https://www.nature.com/articles/s41567-022-01788-5 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:

Alex Balfanz, Alex Haro, B Shang, Benji Rabhan, Gaston Ingaramo, Gordon Child, John Le, Juan Benet, Kyle Davis, Loyal Alchemist, Luk…

❤️ Check out Weights & Biases and sign up for a free demo here: https://wandb.me/papersllm 📝 The blog post and paper are available here:

https://blogs.nvidia.com/blog/fugatto-gen-ai-sound-model/

https://d1qx31qr3h6wln.cloudfront.net/publications/FUGATTO.pdf 📝 My paper on simulations that look almost like reality is available for free here:

https://rdcu.be/cWPfD Or this is the orig. Nature Physics link with clickable citations:

https://www.nature.com/articles/s41567-022-01788-5 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:

Alex Balfanz, Alex Haro, B Shang, Benji Rabhan, Gaston Ingaramo, Gordon Child, John Le, Juan Benet, Kyle Davis, Loyal Alche…

❤️ Try Macro for free and supercharge your learning: https://macro.com/papers 📝 Blender 4.3 is available here:

https://www.blender.org/download/releases/4-3/ 📝 My procedural brush synthesis paper: https://users.cg.tuwien.ac.at/zsolnai/gfx/procedural-brush-synthesis-paper/ 📝 Showcased SLIM paper: https://igl.ethz.ch/projects/slim/ 📝 My paper on simulations that look almost like reality is available for free here:

https://rdcu.be/cWPfD Or this is the orig. Nature Physics link with clickable citations:

https://www.nature.com/articles/s41567-022-01788-5 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:

Alex Balfanz, Alex Haro, B Shang, Benji Rabhan, G…

❤️ Check out Lambda here and sign up for their GPU Cloud: https://lambdalabs.com/papers 📝 Unreal Engine 5.5 is available here:

https://www.unrealengine.com/en-US/blog/unreal-engine-5-5-is-now-available 📝 My paper on simulations that look almost like reality is available for free here:

https://rdcu.be/cWPfD Or this is the orig. Nature Physics link with clickable citations:

https://www.nature.com/articles/s41567-022-01788-5 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:

Alex Balfanz, Alex Haro, B Shang, Benji Rabhan, Gaston Ingaramo, Gordon Child, John Le, Juan Benet, Kyle Davis, Loyal Alchemist, Lukas Biewald, Martin, Michael Albrecht, Michael T…

❤️ Check out Lambda here and sign up for their GPU Cloud: https://lambdalabs.com/papers Oasis: A Universe in a Transformer - try it out now:

https://oasis.decart.ai/welcome More info:

https://oasis-model.github.io/ 📝 My paper on simulations that look almost like reality is available for free here:

https://rdcu.be/cWPfD Or this is the orig. Nature Physics link with clickable citations:

https://www.nature.com/articles/s41567-022-01788-5 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:

Alex Balfanz, Alex Haro, B Shang, Benji Rabhan, Gaston Ingaramo, Gordon Child, John Le, Juan Benet, Kyle Davis, Loyal Alchemist, Lukas Biewald, Martin, Michael Albrec…

❤️ Check out Lambda here and sign up for their GPU Cloud: https://lambdalabs.com/papers 📝 The paper is available here:

https://openai.com/index/introducing-simpleqa/ 📝 My paper on simulations that look almost like reality is available for free here:

https://rdcu.be/cWPfD Or this is the orig. Nature Physics link with clickable citations:

https://www.nature.com/articles/s41567-022-01788-5 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:

Alex Balfanz, Alex Haro, B Shang, Benji Rabhan, Gaston Ingaramo, Gordon Child, John Le, Juan Benet, Kyle Davis, Loyal Alchemist, Lukas Biewald, Martin, Michael Albrecht, Michael Tedder, Owen Skarpness, Richard Sundv…

02:00-05:38 What do you think were the most outstanding underestimated news and achievements in AI field in 2020?

05:41-11:28 What do you think about trends in ML like transformers trying to replace LSTMs in NLP?

11:29-16:06 Are you working on any new types of models right now?

16:07-20:41 What is your opinion on the most underestimated ML subfield like Reinforcement Learning?

20:42-22:17 Your best recommendation for our community is to look into AI in the real physical world, right?

22:18-33:10 Do you think it is possible to achieve great results in creative AI, particularly in subjective beauty?

33:17-35:50 What prevents chat bots from reaching more intelligent levels?

36:03-39:39 What is…

Mikita Shchutski | Lead Machine Learning Engineer, Quantori Training large medical models using electronic health records in order to create a highly informative medical embedding space

DataFest Online 2020

AI Hardware track https://ods.ai/tracks/ai-hardware-df2020 Register and get access to the tracks: https://ods.ai/events/datafest2020

Join the community: https://ods.ai/

Поиск давно перестал быть набором из 10 синих ссылок. Около половины пользовательских задач успешно и быстро решаются с помощью изображений, видео, фактовых ответов и других элементов выдачи. В Яндексе создали Блендер — инструмент на базе ML, который смешивает документы разной модальности и источники, чтобы оптимизировать пользовательский опыт. Как выбирали метрики, реализовывали Блендер и с какими особенностями столкнулись, рассказал Алексей Голиков, руководитель команды качественных вызовов в Яндексе Другие мероприятия Яндекса вы можете посмотреть здесь: https://events.yandex.ru/

Когда Алиса переезжала из телефона в колонку, у команды технологий голосового ввода появились новые вызовы. Например, впервые столкнулись с задачей AEC и направленного шумоподавления. Как удалось во всём разобраться и делать хорошие решения, почему голосовые помощники требуют особенного подхода и зачем колонке несколько микрофонов — рассказал Никита Рыжиков, руководитель Службы технологий голосового ввода в Яндексе Другие мероприятия Яндекса вы можете посмотреть здесь: https://events.yandex.ru/

Ранжирование — самая сложная ML-часть рекомендательной системы. Почему улучшать качество рекомендаций нужно нейросетевыми моделями, а не градиентными бустингами, рассказал Кирилл Хрыльченко, руководитель группы исследования перспективных рекомендательных технологий в Яндексе. Другие мероприятия Яндекса вы можете посмотреть здесь: https://events.yandex.ru/

Александр Воронцов, руководитель группы машинного обучения рекламной платформы в Яндекс Маркете, поделился кейсом автогенерации рекламных баннеров. Он рассказал, как строили работу от MVP на хакатоне до работающего решения в продакшене. А ещё показал, как с помощью YandexGPT и нескольких часов работы дизайнера делать рекламные креативы, которые не уступают в CTR ручным, и в два раза увеличить количество рекламодателей в баннерной рекламе Другие мероприятия Яндекса вы можете посмотреть здесь: https://events.yandex.ru/

Это фрагмент выступления руководителя управления качества в Яндекс Поиске Кати Серажим. На Practical ML Conf 2024 Катя рассказала об архитектуре и процессе обучения LLM для больших продуктовых проектов Яндекса. Посмотреть доклад целиком можно на нашем канале. Из него вы узнаете, что находится под капотом у Нейро и какие сложности решала команда на каждом из этапов его создания.

Это наши коллеги из YADRO: архитектор и эксперт по разработке ПО искусственного интеллекта Андрей Соколов и старший инженер по разработке ПО ИИ Юлия Камелина. На видео — запись из выступления на Practical ML Conf 2024. В докладе они рассмотрели простую методику для оптимизации подсистемы генеративного ридера, который является одним из ключевых компонентов вопросо-ответных систем на архитектуре RAG. Её особенности — это простота, скорость и минимальные требования к аппаратному обеспечению. Подписывайтесь на телеграм-канал Яндекса для ML-специалистов: https://t.me/yandexforml

Это доклад Петра Чуйкова, руководителя команды Data Science в HeadHunter, на Practical ML Conf 2024. В своём выступлении Пётр предложил проверенные на практике решения, рассказал о методах улучшения качества разметки с использованием кластеризации и подходах к согласованию ML- и бизнес-метрик, а также методах борьбы с пузырём рекомендаций. Особое внимание Пётр уделил реальному кейсу HeadHunter. Эта история показывает, как теоретические решения применяются на практике и какое влияние они оказывают на бизнес. Подписывайтесь на телеграм-канал Яндекса для ML-специалистов: https://t.me/yandexforml

Илья Дьяков, ML-разработчик из команды автономного транспорта Яндекса, прочитал этот доклад на Data Dojo. Илья рассказал, как на ML-треке Yandex Cup 2024 участникам предложили задачу: в рамках симуляции предсказать движение автомобилей с помощью ML-моделей. А ещё подробно разобрал подход, который дал наиболее точный результат. Подписывайтесь на нас в Telegram: https://t.me/yandexforml

Это доклад Александра Воронцова, руководителя службы качества рекламы в Яндекс Маркете, на Practical ML Conf 2024. В своём выступлении Александр рассмотрел устройство нативной рекламы в поиске Маркета. Как не терять рекламные товары среди миллионов других и как сделать эффективные алгоритмы кандидатогенерации с учётом рекламной ставки. Подписывайтесь на телеграм-канал Яндекса для ML-специалистов: https://t.me/yandexforml

Артём Вешкин, ML-разработчик из Яндекс Плюс Фантеха, прочитал этот доклад на Data Dojo. Артём рассказал, как на ML-треке Yandex Cup 2024 участникам предложили решить задачку от команды Музыки: найти кавер-версии песен в дата-сете из 350 тысяч треков. Артём разобрал лучшие решения и объяснил, почему здесь лучше всего подойдут методы компьютерного зрения. Подписывайтесь на нас в Telegram: https://t.me/yandexforml

Николай Карпачёв, руководитель группы базового качества перевода из Яндекс Переводчика, прочитал этот доклад на Data Dojo. В этом году его команда поучаствовала в контесте на главной конференции по машинному переводу WMT и попала в топ-5 по качеству решения. Николай объяснил, как они с ребятами сделали качественную модель с небольшим количеством параметров за счёт alignment-метода. А ещё рассказал, как LLM тюнят под перевод и кто пока выигрывает: человек или машина. Подписывайтесь на нас в Telegram: https://t.me/yandexforml

Это доклад Саввы Степурина, старшего разработчика команды рекомендаций в Яндекс Музыке, на Practical ML Conf 2024. В Яндекс Музыке есть специальная настройка «Незнакомое» в «Моей волне». В своём выступлении Савва разобрал её особенности по сравнению с обычным потоком рекомендаций. Рассказал, какие метрики важно растить и как переделать алгоритм рекомендаций под них. Внутри: особые подходы в отборе кандидатов и модели ранжирования, а ещё результаты их внедрения. Подписывайтесь на телеграм-канал Яндекса для ML-специалистов: https://t.me/yandexforml

Это доклад Кирилла Трофимова, ведущего ML-инженера в Яндекс Рекламе, на Practical ML Conf 2024. В своём выступлении Кирилл рассказал, как ребята улучшили понимание и контроль рекламной ML-инфраструктуры с помощью метасервиса, объединяющего все компоненты системы. А ещё о том, как это ускоряет тестирование, помогает разбираться во внутренних процессах и быстрее выкатывать новинки в прод. Подписывайтесь на телеграм-канал Яндекса для ML-специалистов: https://t.me/yandexforml

Это доклад Юрия Классена, руководителя группы разработки инфраструктуры машинного обучения в Купере, на Practical ML Conf 2024. В своём выступлении Юрий рассказал, как проходило внедрение Feast в качестве Online Feature Store. Внутри: подводные камни при работе с Feast, текущее техническое состояние проекта и результаты внедрения Feast в уже существующий workflow, а ещё много полезных советов. Подписывайтесь на телеграм-канал Яндекса для ML-специалистов: https://t.me/yandexforml

Спикер: Роман Дерунец Data Fest Siberia 5: https://ods.ai/events/datafestsiberia5

Трек NLP: https://ods.ai/tracks/sibfest5-nlp

_____

Наши соц.сети:

Telegram: https://t.me/datafest

Вконтакте: https://vk.com/datafest

Канал с вакансиями в telegram: https://t.me/odsjobs

Канал с апдейтами по курсам: https://t.me/odscourses

Как попасть в чат сообщества ODS Mattermost: https://ods.ai/tracks/mattermost

Спикер: Мария Мичурина Data Fest Siberia 5: https://ods.ai/events/datafestsiberia5

Трек NLP: https://ods.ai/tracks/sibfest5-nlp

_____

Наши соц.сети:

Telegram: https://t.me/datafest

Вконтакте: https://vk.com/datafest

Канал с вакансиями в telegram: https://t.me/odsjobs

Канал с апдейтами по курсам: https://t.me/odscourses

Как попасть в чат сообщества ODS Mattermost: https://ods.ai/tracks/mattermost

Спикер: Дари Батурова Data Fest Siberia 5: https://ods.ai/events/datafestsiberia5

Трек NLP: https://ods.ai/tracks/sibfest5-nlp

_____

Наши соц.сети:

Telegram: https://t.me/datafest

Вконтакте: https://vk.com/datafest

Канал с вакансиями в telegram: https://t.me/odsjobs

Канал с апдейтами по курсам: https://t.me/odscourses

Как попасть в чат сообщества ODS Mattermost: https://ods.ai/tracks/mattermost

Спикер: Ислам Умаров Data Fest Siberia 5: https://ods.ai/events/datafestsiberia5

Трек: https://ods.ai/tracks/sibfest5-ml-security

_____

Наши соц.сети:

Telegram: https://t.me/datafest

Вконтакте: https://vk.com/datafest

Канал с вакансиями в telegram: https://t.me/odsjobs

Канал с апдейтами по курсам: https://t.me/odscourses

Как попасть в чат сообщества ODS Mattermost: https://ods.ai/tracks/mattermost

Спикер: Сиракан Багдасарян, MLOps-инженер, OZON Банк Data Halloween 2024: https://ods.ai/events/halloween2024_spb

Трек: https://ods.ai/tracks/halloween2024-spb

_____

Наши соц.сети:

Telegram: https://t.me/datafest

Вконтакте: https://vk.com/datafest

Канал с вакансиями в telegram: https://t.me/odsjobs

Канал с апдейтами по курсам: https://t.me/odscourses

Как попасть в чат сообщества ODS Mattermost: https://ods.ai/tracks/mattermost

Спикер: Никита Венедиктов, NLP Researcher, RAFT Data Halloween 2024: https://ods.ai/events/halloween2024_spb

Трек: https://ods.ai/tracks/halloween2024-spb

_____

Наши соц.сети:

Telegram: https://t.me/datafest

Вконтакте: https://vk.com/datafest

Канал с вакансиями в telegram: https://t.me/odsjobs

Канал с апдейтами по курсам: https://t.me/odscourses

Как попасть в чат сообщества ODS Mattermost: https://ods.ai/tracks/mattermost

Спикер: Алексей Козлов, Евгений Тайчинов, ML-инженер, Работа.ру Data Halloween 2024: https://ods.ai/events/halloween2024_spb

Трек: https://ods.ai/tracks/halloween2024-spb

_____

Наши соц.сети:

Telegram: https://t.me/datafest

Вконтакте: https://vk.com/datafest

Канал с вакансиями в telegram: https://t.me/odsjobs

Канал с апдейтами по курсам: https://t.me/odscourses

Как попасть в чат сообщества ODS Mattermost: https://ods.ai/tracks/mattermost

Спикер: Марк Паненко, CDS, OZON Банк Data Halloween 2024: https://ods.ai/events/halloween2024_spb

Трек: https://ods.ai/tracks/halloween2024-spb

_____

Наши соц.сети:

Telegram: https://t.me/datafest

Вконтакте: https://vk.com/datafest

Канал с вакансиями в telegram: https://t.me/odsjobs

Канал с апдейтами по курсам: https://t.me/odscourses

Как попасть в чат сообщества ODS Mattermost: https://ods.ai/tracks/mattermost

Data Fest Siberia 5: https://ods.ai/events/datafestsiberia5

Трек: https://ods.ai/tracks/sibfest5-ml-infrastructure

_____

Наши соц.сети:

Telegram: https://t.me/datafest

Вконтакте: https://vk.com/datafest

Канал с вакансиями в telegram: https://t.me/odsjobs

Канал с апдейтами по курсам: https://t.me/odscourses

Как попасть в чат сообщества ODS Mattermost: https://ods.ai/tracks/mattermost

Спикер: Маршалова Аня, Тихобаева Оля

Название: Вспомнить всё по короткой подсказке разбор статьи Rethinking LLM Memorization through the Lens of Adversarial Compression Data Fest Siberia 5: https://ods.ai/events/datafestsiberia5

Трек: https://ods.ai/tracks/sibfest5-ds-talks

_____

Наши соц.сети:

Telegram: https://t.me/datafest

Вконтакте: https://vk.com/datafest

Канал с вакансиями в telegram: https://t.me/odsjobs

Канал с апдейтами по курсам: https://t.me/odscourses

Как попасть в чат сообщества ODS Mattermost: https://ods.ai/tracks/mattermost

Спикер: Мурашкина Анна Data Fest Siberia 5: https://ods.ai/events/datafestsiberia5

Трек: https://ods.ai/tracks/sibfest5-ds-talks

_____

Наши соц.сети:

Telegram: https://t.me/datafest

Вконтакте: https://vk.com/datafest

Канал с вакансиями в telegram: https://t.me/odsjobs

Канал с апдейтами по курсам: https://t.me/odscourses

Как попасть в чат сообщества ODS Mattermost: https://ods.ai/tracks/mattermost

Спикер: Шубин Вадим Data Fest Siberia 5: https://ods.ai/events/datafestsiberia5

Трек: https://ods.ai/tracks/sibfest5-ds-talks

_____

Наши соц.сети:

Telegram: https://t.me/datafest

Вконтакте: https://vk.com/datafest

Канал с вакансиями в telegram: https://t.me/odsjobs

Канал с апдейтами по курсам: https://t.me/odscourses

Как попасть в чат сообщества ODS Mattermost: https://ods.ai/tracks/mattermost

Спикер: Марина Платонова

Название: Новые горизонты лесной отрасли: применение ML для распознавания пород и состояния деревьев Data Fest Siberia 5: https://ods.ai/events/datafestsiberia5

Трек CV: https://ods.ai/tracks/sibfest5-cv

_____

Наши соц.сети:

Telegram: https://t.me/datafest

Вконтакте: https://vk.com/datafest

Канал с вакансиями в telegram: https://t.me/odsjobs

Канал с апдейтами по курсам: https://t.me/odscourses

Как попасть в чат сообщества ODS Mattermost: https://ods.ai/tracks/mattermost

Спикер: Тимур Смирнов

Data Fest Siberia 5: https://ods.ai/events/datafestsiberia5

Трек: https://ods.ai/tracks/sibfest5-ml-security

_____

Наши соц.сети:

Telegram: https://t.me/datafest

Вконтакте: https://vk.com/datafest

Канал с вакансиями в telegram: https://t.me/odsjobs

Канал с апдейтами по курсам: https://t.me/odscourses

Как попасть в чат сообщества ODS Mattermost: https://ods.ai/tracks/mattermost

Спикер: Артём Карасюк

Data Fest Siberia 5: https://ods.ai/events/datafestsiberia5

Трек: https://ods.ai/tracks/sibfest5-ml-infrastructure

_____

Наши соц.сети:

Telegram: https://t.me/datafest

Вконтакте: https://vk.com/datafest

Канал с вакансиями в telegram: https://t.me/odsjobs

Канал с апдейтами по курсам: https://t.me/odscourses

Как попасть в чат сообщества ODS Mattermost: https://ods.ai/tracks/mattermost

Twitch: https://www.twitch.tv/justin_helps

Discord: https://discord.gg/NbruaNW

Store: https://store.dftba.com/collections/primer

Patreon: https://www.patreon.com/primerlearning Source and further reading on the common side-blotched lizard:

Sinervo, B.; C.M. Lively (1996). "The rock–paper–scissors game and the evolution of alternative male strategies". Nature. 380 (6571): 240–243.

https://en.wikipedia.org/wiki/Common_side-blotched_lizard Made with Godot

Github: https://github.com/Primer-Learning/PrimerTools Made possible by support from these wonderful Patrons:

abledbody

Alba Caparros-Roissard

Andrew Lang

Anthony Eufemio

Brian Cloutier

Captain Chinchilla

Christoph Grabo (@asaaki)

Christy Ser…

Volodymyr Zelenskyy is the President of Ukraine.

On YouTube this episode is available in English, Ukrainian, and Russian.

Captions and voice-over audio tracks are provided in English, Ukrainian, Russian, and the original mixed-language version, with subtitles available in your preferred language.

To listen to the original mixed language version, please select the English (UK) audio track audio track.

Thank you for listening ❤ Check out our sponsors: https://lexfridman.com/sponsors/ep456-scSee below for timestamps, transcript, and to give feedback, submit questions, contact Lex, etc.

Adam Frank is an astrophysicist studying star systems and the search for extraterrestrial life and alien civilizations.

Thank you for listening ❤ Check out our sponsors: https://lexfridman.com/sponsors/ep455-scSee below for timestamps, and to give feedback, submit questions, contact Lex, etc.

Go to https://encord.com/lexEight Sleep: Temp-controlled smart mattress cover.

Go to https://betterhelp.com/lexNotion: Note-taking and team collaboration.

Go to https://drinkLMNT.com/lexAG1: All-in-one daily nutrition drinks.

Saagar Enjeti is a political journalist & commentator, co-host of Breaking Points with Krystal and Saagar and The Realignment Podcast.

He is exceptionally well-read, and the books he recommends are always fascinating and eye-opening.

You can check out all the books he mentions in this episode here: https://lexfridman.com/saagar-booksThank you for listening ❤ Check out our sponsors: https://lexfridman.com/sponsors/ep454-scSee below for timestamps, transcript, and to give feedback, submit questions, contact Lex, etc.

Go to https://eightsleep.com/lexAG1: All-in-one daily nutrition drinks.

Go to https://drinkLMNT.com/lexBetterHelp: Online therapy and counseling.

Javier Milei is the President of Argentina.

This episode is available in both English and Spanish.

Thank you for listening ❤ Check out our sponsors: https://lexfridman.com/sponsors/ep453-scSee below for timestamps, transcript, and to give feedback, submit questions, contact Lex, etc.

Go to http://netsuite.com/lexBetterHelp: Online therapy and counseling.

Go to https://betterhelp.com/lexAG1: All-in-one daily nutrition drinks.

Dario Amodei is the CEO of Anthropic, the company that created Claude.

Amanda Askell is an AI researcher working on Claude’s character and personality.

Chris Olah is an AI researcher working on mechanistic interpretability.

Thank you for listening ❤ Check out our sponsors: https://lexfridman.com/sponsors/ep452-scSee below for timestamps, transcript, and to give feedback, submit questions, contact Lex, etc.

(3:49:02) – Character training(3:50:01) – Nature of truth(3:54:38) – Optimal rate of failure(4:01:49) – AI consciousness(4:16:20) – AGI(4:24:58) – Chris Olah – Mechanistic Interpretability(4:29:49) – Features, Circuits, Universality(4:47:23) – Superposition(4:58:22) – Monosemanticity(5:05…

Rick Spence is a historian specializing in the history of intelligence agencies, espionage, secret societies, conspiracies, the occult, and military history.

Thank you for listening ❤ Check out our sponsors: https://lexfridman.com/sponsors/ep451-scSee below for timestamps, transcript, and to give feedback, submit questions, contact Lex, etc.

Go to http://netsuite.com/lexBetterHelp: Online therapy and counseling.

Go to https://betterhelp.com/lexMasterClass: Online classes from world-class experts.

Go to https://masterclass.com/lexpodShopify: Sell stuff online.

Bernie Sanders is a US Senator from Vermont and a two-time presidential candidate.

Thank you for listening ❤ Check out our sponsors: https://lexfridman.com/sponsors/ep450-scSee below for timestamps, and to give feedback, submit questions, contact Lex, etc.

Go to https://eightsleep.com/lexSaily: An eSIM for international travel.

Go to https://saily.com/lexGround News: Unbiased news source.

Go to https://drinkLMNT.com/lexOUTLINE:(00:00) – Introduction(08:51) – MLK Jr(11:43) – Corruption in politics(23:00) – Healthcare in US(31:33) – 2016 election(37:32) – Barack Obama(43:26) – Capitalism(51:35) – Response to attacks(56:32) – AOC and progressive politics(1:04:24) – Mortality(1:06:30) – Hope fo…

Graham Hancock a journalist and author who for over 30 years has explored the controversial possibility that there existed a lost civilization during the last Ice Age, and that it was destroyed in a global cataclysm some 12,000 years ago.

He is the presenter of the Netflix documentary series “Ancient Apocalypse”, the 2nd season of which has just been released.

Thank you for listening ❤ Check out our sponsors: https://lexfridman.com/sponsors/ep449-scSee below for timestamps, and to give feedback, submit questions, contact Lex, etc.

Go to https://notion.com/lexRiverside: Platform for recording podcasts and videos from everywhere.

Go to https://creators.riverside.fm/LEXLMNT: Zero-sugar electro…

Jordan Peterson is a psychologist, author, lecturer, and podcast host.

Thank you for listening ❤ Check out our sponsors: https://lexfridman.com/sponsors/ep448-scSee below for timestamps, transcript, and to give feedback, submit questions, contact Lex, etc.

Go to https://eightsleep.com/lexGround News: Unbiased news source.

Go to https://betterhelp.com/lexLMNT: Zero-sugar electrolyte drink mix.

Go to https://drinkLMNT.com/lexOUTLINE:(00:00) – Introduction(07:07) – Nietzsche(14:48) – Power and propaganda(19:54) – Nazism(24:54) – Religion(41:18) – Communism(47:03) – Hero myth(49:12) – Belief in God(59:24) – Advice for young people(1:12:02) – Sex(1:32:00) – Good and evil(1:44:46) – Psychopathy(1…

Aman Sanger, Arvid Lunnemark, Michael Truell, and Sualeh Asif are creators of Cursor, a popular code editor that specializes in AI-assisted programming.

Thank you for listening ❤ Check out our sponsors: https://lexfridman.com/sponsors/ep447-scSee below for timestamps, transcript, and to give feedback, submit questions, contact Lex, etc.

Go to https://encord.com/lexMasterClass: Online classes from world-class experts.

Go to https://masterclass.com/lexpodShopify: Sell stuff online.

Go to http://netsuite.com/lexAG1: All-in-one daily nutrition drinks.

Ed Barnhart is an archaeologist and explorer specializing in ancient civilizations of the Americas.

He is the Director of the Maya Exploration Center, host of the ArchaeoEd Podcast, and lecturer on the ancient history of North, Central, and South America.

Ed is in part known for his groundbreaking work on ancient astronomy, mathematics, and calendar systems.

Thank you for listening ❤ Check out our sponsors: https://lexfridman.com/sponsors/ep446-scSee below for timestamps, and to give feedback, submit questions, contact Lex, etc.

Go to https://drinkag1.com/lexNotion: Note-taking and team collaboration.

Vivek Ramaswamy is a conservative politician, entrepreneur, and author of many books on politics, including his latest titled Truths: The Future of America First.

Thank you for listening ❤ Check out our sponsors: https://lexfridman.com/sponsors/ep445-scSee below for timestamps, transcript, and to give feedback, submit questions, contact Lex, etc.

Go to http://netsuite.com/lexGround News: Unbiased news source.

Go to https://eightsleep.com/lexOUTLINE:(00:00) – Introduction(12:50) – Conservatism(16:06) – Progressivism(21:41) – DEI(26:33) – Bureaucracy(33:25) – Government efficiency(48:34) – Education(1:02:59) – Military Industrial Complex(1:25:18) – Illegal immigration(1:46:53) – Donald Trump(…

Vejas Liulevicius is a historian specializing in Germany and Eastern Europe, who has lectured extensively on Marxism and the rise, the reign, and the fall of Communism.

Thank you for listening ❤ Check out our sponsors: https://lexfridman.com/sponsors/ep444-scSee below for timestamps, and to give feedback, submit questions, contact Lex, etc.

Go to https://drinkag1.com/lexBetterHelp: Online therapy and counseling.

Go to https://betterhelp.com/lexNotion: Note-taking and team collaboration.

Go to https://notion.com/lexLMNT: Zero-sugar electrolyte drink mix.

Gregory Aldrete is a historian specializing in ancient Rome and military history.

Thank you for listening ❤ Check out our sponsors: https://lexfridman.com/sponsors/ep443-scSee below for timestamps, transcript, and to give feedback, submit questions, contact Lex, etc.

Go to https://drinkLMNT.com/lexShopify: Sell stuff online.

Go to https://shopify.com/lexAG1: All-in-one daily nutrition drinks.

Go to https://drinkag1.com/lexBetterHelp: Online therapy and counseling.

Donald Trump is the 45th President of the United States and the Republican candidate in the 2024 US Presidential Election.

Thank you for listening ❤ Check out our sponsors: https://lexfridman.com/sponsors/ep442-scSee below for timestamps, transcript, and to give feedback, submit questions, contact Lex, etc.

Go to https://ground.news/lexEncord: AI tooling for annotation & data management.

Go to https://encord.com/lexEight Sleep: Temp-controlled smart mattress.

Go to https://eightsleep.com/lexNetSuite: Business management software.

DAEPP: You know, we didn’t really think about the term fraud until we started prepping for this interview with you.

BADANES: Right, right.

One of the things that I get asked a lot is, why can’t we just build good AI to detect bad AI, right?

BADANES: So next time my kids are in a fight, I’m going to point them to Copilot and say, work with Copilot to mediate.

[LAUGHS] No, that’s really, really interesting.

Earlier today, Lidong gave a keynote here at NeurIPS on the co-evolution of AI and systems engineering.

One dimension is that the scale of the AI systems that we have to support.

And the other dimension is if you look at AI systems, it’s actually a whole-stack kind of design.

STRICKLAND: Yeah, yeah.

ZHOU: Yeah, I think in terms of AI systems, I’m certainly pretty excited about what we can do together, you know, with a combination of AI and systems.

And then the second paradigm really emerged in the 17th century.

And so the third paradigm really began, I guess, sort of, in the ’50s and ’60s, the development of digital computers.

And when I think about AI for Science actually, the space of opportunity is colossal because science is, science is really just understanding more about the world around us.

And now the SMILES autoregressive model can now generate a molecule that’s an improvement on the starting molecule and knows about the protein binding.

But also if you think about [it], science is really about learning more about the world.

Today I’m talking to Jindong Wang, a senior researcher at Microsoft Research, and Steven Whang, a tenured associate professor at the Korea Advanced Institute of Science and Technology.

JINDONG WANG: OK, everybody knows that with the widespread usage of large language models, hallucination has become a crucial factor of concern.

So foreign key constraint basically requires that if there is some director mentioned in the movie table, it has to be one of the directors in the director table.

So now we can join the movie and director table and generate a bigger table.

HUIZINGA: Well, Jindong Wang and Steven Whang, thanks for joining us today, and to our listeners, thanks for tuning in.

The other one actually is some token actually is very, very hard to be predicted during the pretraining.

And the important thing for the data is about data filtering.

If we’re able to build a better model actually is able to benefit so many different kinds of application.

And definitely there’s a lot of things about how to build a better data [that] is unsolved yet in the literature.

And the other thing actually, we are working on something that’s very exciting.

FOSTER: So this is a, kind of, a theoretical work on reinforcement learning, or RL.

FOSTER: Yeah, so if you look at these sort of RL problems with latent dynamics, this is something that’s actually received a lot of investigation in theory.

Like, can we take existing algorithms and use them to solve rich-observation RL problems in a modular fashion?

TINGLE: Dylan, I’d like to know—and I’m sure our audience would, too—what this work means when it comes to real-world application.

TINGLE: Well, Dylan Foster, thank you for joining us today to discuss your paper on reinforcement learning under latent dynamics.

The drawback of this approach is that it often misses the cultural nuances of local languages.

CHITALE: Now that we have created a benchmark, the next step is to evaluate how these multimodal models are performing on this benchmark.

So what we observed is there is a huge gap when it comes … in performance when we compare these proprietary offerings versus the open-source models.

These open-source models significantly lag behind the proprietary models.

CHITALE: CVQA is significant because it addresses a fundamental gap in how we evaluate vision-language and multimodal models today.

We are sorry, the page you requested cannot be found.

The page you are looking for could not be found or is no longer available.

Joining me today to discuss the state of DNA data storage and some of our contributions are several members of the DNA Data Storage Project at Microsoft Research: Principal Researcher Bichlien Nguyen, Senior Researcher Jake Smith, and Partner Research Manager Sergey Yekhanin.

Once we do this, we return to our original data and we’ve completed, let’s call it, one DNA data storage cycle.

So, like, I mean, coding is an important aspect of this whole idea of DNA data storage because we have to deal with errors—it’s a new medium—but talking about error-correcting codes in the context of DNA data storage, so, I mean, usually, like … what are error-correcting codes about?

In DNA data storage, the …

We are sorry, the page you requested cannot be found.

The page you are looking for could not be found or is no longer available.

Researcher Siddharth Suri and professor David Holtz give a brief history of prompt engineering, discuss the debate behind their recent collaboration, and share what they found from studying how people’s approaches to prompting change as models advance.

Learn more:

Chris and Jay, thank you for joining us today for Abstracts and congratulations!

HAWBLITZEL: Yeah, so I think, you know, traditionally verification or this formal software verification that we’re doing has been considered a little bit of a pie-in-the-sky research agenda.

They’ve discovered that you can get the high performance that you want for systems code without having to sacrifice the ability to reason about ownership and lifetimes, concurrency.

TINGLE: Well, finally, Chris, what are some of the open questions or future opportunities for formal software verification research, and what might you and your collaborators tackle next?

I’m Amber Tingle, and we hope you’ll join us again for Ab…

And at that time, large language model was really in its infancy and people just started exploring what large language model can help us in terms of improving software reliability.

But of course, you know, large language model is a field that is moving so fast.

And that’s one of the reasons we used large language models, because traditional static analysis or traditional program analysis cannot capture this.

I’m actually working on how to leverage large language model to verify the correctness of code, code that may be generated by large language model itself.

STOICA: So we’re thinking of, as Shan mentioned, exploring what large language models can do in this bug-finding/testing arena furth…

Today, I’m speaking with my intern, Angela Busheska, about her work this summer and her experience as a Microsoft Research intern.

And I think I’m really grateful that we took this direction because we had a chance to understand at a granular level what is actually happening.

RANGANATHAN: And I think we realized this and you can chime in, Angela, but I feel that the traceability is a vehicle for data.

BUSHESKA: Yeah, so this will be a fall really spent with a lot of applications because of, like, graduate school.

As all the listeners can probably see, I’m a person who really speaks a lot, even more than needed sometimes.

Our guest in this episode is David Tench, a Grace Hopper postdoctoral fellow at Lawrence Berkeley National Labs, who specializes in scalable graph algorithms and compression techniques to tackle massive datasets. In this episode, we will learn how his techniques enable real-time analysis of large datasets, such as particle tracking in physics experiments or social network analysis, by reducing storage requirements while preserving critical structural properties. David also challenges the common belief that giant graphs are sparse by pointing to a potential bias: Maybe because of the challenges that exist in analyzing large dense graphs, we only see datasets of sparse graphs? The truth is ou…

In this episode, Dave Bechberger, principal Graph Architect at AWS and author of "Graph Databases in Action", brings deep insights into the field of graph databases and their applications. Together we delve into specific scenarios in which Graph Databases provide unique solutions, such as in the fraud industry, and learn how to optimize our DB for questions around connections, such as "How are these entities related?" or "What patterns of interaction indicate anomalies?" This discussion sheds light on when organizations should consider adopting graph databases, particularly for cases that require scalable analysis of highly interconnected data and provides practical insights into leveraging…

In this episode, Adam Machowczyk, a PhD student at the University of Leicester, specializes in graph rewriting and its intersection with machine learning, particularly Graph Neural Networks. Adam explains how graph rewriting provides a formalized method to modify graphs using rule-based transformations, allowing for tasks like graph completion, attribute prediction, and structural evolution. Bridging the worlds of graph rewriting and machine learning, Adam's work aspire to open new possibilities for creating adaptive, scalable models capable of solving challenges that traditional methods struggle with, such as handling heterogeneous graphs or incorporating incremental updates efficiently. R…

In this episode, the data scientist Wentao Su shares his experience in AB testing on social media platforms like LinkedIn and TikTok. We talk about how network science can enhance AB testing by accounting for complex social interactions, especially in environments where users are both viewers and content creators. These interactions might cause a "spillover effect" meaning a possible influence across experimental groups, which can distort results. To mitigate this effect, our guest presents heuristics and algorithms they developed ("one-degree label propagation”) to allow for good results on big data with minimal running time and so optimize user experience and advertiser performance in soc…

Alex Bisberg, a PhD candidate at the University of Southern California, specializes in network science and game analytics, with a focus on understanding social and competitive success in multiplayer online games. In this episode, listeners can expect to learn from a network perspective about players interactions and patterns of behavior. Through his research on games, Alex sheds light on how network analysis and statistical tests might explain positive contagious behaviors, such as generosity, and explore the dynamics of collaboration and competition in gaming environments. These insights offer valuable lessons not only for game developers in enhancing player experience, engagement and rete…

In this episode we discuss the GitHub Collaboration Network with Behnaz Moradi-Jamei, assistant professor at James Madison University. As a network scientist, Behnaz created and analyzed a network of about 700,000 contributors to Github's repository. The network of collaborators on GitHub was created by identifying developers (nodes) and linking them with edges based on shared contributions to the same repositories. This means that if two developers contributed to the same project, an edge (connection) was formed between them, representing a collaborative relationship network consisting of 32 million such connections. By using algorithms for Community Detection, Behnaz's analysis reveals in…

In this episode we discuss the GitHub Collaboration Network with Behnaz Moradi-Jamei, assistant professor at James Madison University. As a network scientist, Behnaz created and analyzed a network of about 700,000 contributors to Github's repository. The network of collaborators on GitHub was created by identifying developers (nodes) and linking them with edges based on shared contributions to the same repositories. This means that if two developers contributed to the same project, an edge (connection) was formed between them, representing a collaborative relationship network consisting of 32 million such connections. By using algorithms for Community Detection, Behnaz's analysis reveals in…

We are joined by Abhishek Paudel, a PhD Student at George Mason University with a research focus on robotics, machine learning, and planning under uncertainty, using graph-based methods to enhance robot behavior. He explains how graph-based approaches can model environments, capture spatial relationships, and provide a framework for integrating multiple levels of planning and decision-making.

We are joined by Maciej Besta, a senior researcher of sparse graph computations and large language models at the Scalable Parallel Computing Lab (SPCL). In this episode, we explore the intersection of graph theory and high-performance computing (HPC), Graph Neural Networks (GNNs) and LLMs.

In this episode, we sit down with Yuanyuan Tian, a principal scientist manager at Microsoft Gray Systems Lab, to discuss the evolving role of graph databases in various industries such as fraud detection in finance and insurance, security, healthcare, and supply chain optimization.

Our new season "Graphs and Networks" begins here! We are joined by new co-host Asaf Shapira, a network analysis consultant and the podcaster of NETfrix – the network science podcast. Kyle and Asaf discuss ideas to cover in the season and explore Asaf's work in the field.

Join us for our capstone episode on the Animal Intelligence season. We recap what we loved, what we learned, and things we wish we had gotten to spend more time on. This is a great episode to see how the podcast is produced. Now that the season is ending, our current co-host, Becky, is moving to emeritus status. In this last installment we got to spend a little more time getting to know Becky and where her work will take her after this. Did Data Skeptic inspire her to learn more about machine learning? Tune in and find out.

David Obembe, a recent University of Tartu graduate, discussed his Masters thesis on integrating LLMs with process mining tools. He explained how process mining uses event logs to create maps that identify inefficiencies in business processes. David shared his research on LLMs' potential to enhance process mining, including experiments evaluating their performance and future improvements using Retrieval Augmented Generation (RAG).

Our guest today is Risa Shinoda, a PhD student at Kyoto University Agricultural Systems Engineering Lab, where she applies computer vision techniques. She talked about the OpenAnimalTracks dataset and what it was used for. The dataset helps researchers predict animal footprint. She also discussed how she built a model for predicting tracks of animals. She shared the algorithms used and the accuracy they achieved. She also discussed further improvement opportunities for the model.

In today's episode, Kirill and Hadelin detail what generative AI models, like large language models, are and how they fit within the broader category of foundation models.

So, now, with that example in hand and with a good understanding of the lifecycle of foundation models in general, there are a lot of foundation models out there.

But that's going to mean doing steps one, two, and three in the lifecycle, and that's going to cost a lot of money.

It's somewhere in between where you do get access to the foundation models and you can choose your foundation models, you can customize them.

So remember, Jon, when you were saying that actually LLMs are included in foundation models, because in fa…

Podcast TranscriptJon Krohn: 00:00:00 This is episode number 852 our In Case You Missed It in December episode.

I don't think we ever put that in production, but it gave us something we could use and measure against our outcomes.

So, you don't need to be worried about spending dollars or tens of dollars by switching to GPT-4o mini.

00:11:17You have data that's sensitive and that you are not comfortable sending that data to a third party.

And for some that's a good, that's going to be fun and pleasurable, and for others that's going to be utter doom and disaster, right?

Florian is chief product officer and member of the board at Terra Quantum, a leading quantum computing startup headquartered in Switzerland and Germany.

You gave a great presentation on quantum computing, quantum applications.

How does Terra Quantum address its... And this is a quote from Terra Quantum themselves, "Terra Quantum has a commitment to responsible innovation in the ethical implementation of quantum technology."

These are things that we would not have to do if we were only to focus on quantum computing and other security aspects of protecting against quantum computing.

Otherwise, share this episode with folks who would love to learn about quantum computing or quantum ML.

For Jon Krohn, the continuous calendar gives him a realistic and uninterrupted overview of his time.

Well, it’s the start of another year, which means it's time for another continuous calendar from us here at SuperDataScience!

In order to do that efficiently, we love the continuous calendar format.

So if you’d like to get started today with your own super efficient continuous calendar in 2025, simply head to jonkrohn.com/cal25.

The good news for you listeners is that, in 2025, the SuperDataScience Podcast will be a bigger priority for me than ever before.

And so I think that's something just to think about.

Sadie Lawrence: 00:36:05... particularly if you want me to get specific, but I really think it's us as consumers.

I could speak for a long time about this, but you are the guest, so I'm going to just let you go first, please.

I'm going to quickly preempt what you're going to say with a couple of recent episodes that I released on this topic.

Agentic AI, AI integration to everyday devices.

Podcast Transcript(00:05):This is Five Minute Friday with a holiday greeting from all of us at the SuperDataScience Podcast.

(00:27):Welcome back to the Super Data Science Podcast.

in depth over the course of the year through our podcast episodes, allowing you to hear directly from leading experts and practitioners like Andrew Ng, Bernard Marr and Sol Rashidi.

(02:38):From all of us here at the SuperDataScience Podcast, happy holidays!

Until next time, keep on rockin’ it out there and I’m looking forward to enjoying another round of the SuperDataScience podcast with you very soon.

Then there's one small recent delta on agentic AI that's very similar, which is the emergence of reasoning frameworks, which is perhaps just a way of applying agentic AI.

Ed Donner: 00:46:44Well, if you're going to open that door, let me see.

They have leaderboards for medical models, models that are specialized in medical domain.

So, we built up this startup, which was about taking AI models and applying them to the field of talent.

You're really amazing at this and it's something that I've seen you do over the years and I've been so amazed by it.

In this Five-Minute Friday, Jon Krohn speaks to Anu Jain, CEO of Nexus Cognitive, and Mahesh Kumar, CMO of Acceldata. They talk about the importance of updating data, especially for predictive models that make key financial decisions for a company, as well as the current state of data governance and why it’s overdue its own update. Additional materials: www.superdatascience.com/846 Interested in sponsoring a SuperDataScience Podcast episode? Email [email protected] for sponsorship information.

"I'm going to give a talk about the most powerful word in the world."

So yeah, so your book Tech Agnostic in it, you describe technology use as a ritual that offers both relief and entrapment.

01:03:08So your book Tech Agnostic delves into the implications of portraying tech as a messianic force like this.

And for some that's a good, that's going to be fun and pleasurable, and for others that's going to be utter doom and disaster, right?

Jon Krohn: 01:32:30We've also-Greg Epstein: 01:32:32It was really, really a thorough conversation, and in many wonderful ways.

I think you're spot on 100%.

But an AI system can scale way, way, way, way more than us.

Jon Krohn: 00:00:00This is episode number 844, our "In Case You Missed It in November” episode.00:00:19Welcome to the Super Data Science Podcast, I'm your host, Jon Krohn.

I totally agree with that, but then now you're going to have to jump through another hoop.

So for me, it was very much like an educational goal that I had, and it was obviously a very structured way of learning Data Analytics.

The idea is one of utilizing essentially the fundamentals of machine learning models, and the fact that machine learning models live in fairly large often representational spaces.

Now, Stained Glass Transform in effect is itself a very small machine learning model.

Eiman Ebrahimi: 00:22:33I think data security is one of those topics that a lot of folks that like myself come from high-performance backgrounds.

Everything in between those three to five tiers of data, that's where most of the really interesting use cases that people talk about tend to be.

It's generally focused on whatever system is the most efficient for either a large language model or any other machine learning model for tha…

In this Five-Minute Friday, Jon interviews Chris Bennett and Joseph Balsamo on the importance of flexibility in the way we deploy AI models, Dell’s brand positioning in the AI space, and whether GenAI’s business applications stand up to the hype. Additional materials: www.superdatascience.com/842 Interested in sponsoring a SuperDataScience Podcast episode? Email [email protected] for sponsorship information.

Andrew Ng 00:10:12You know, that's interesting question.

Can you elaborate on why planning, using multiple tools, and code generation are so crucial for building effective vision AI applications?

And I see, I actually think there's a lot of media companies with a lot of, well, this is raising my hand, right?

If I click somewhere else, you know, there's no gray wolf, right, elsewhere.

Yeah, that's, that's very cool, Andrew to see.

What do AI, robotics, and premium wine grapes have in common? Everything, as it turns out. In this episode, we explore viticultural robotics a revolutionary project combining machine learning, spectroscopic sensors, and VR-controlled robotics to tackle one of agriculture’s trickiest challenges: harvesting delicate wine grapes worth over $6,000 per tonne. From vineyards in the UK to cutting-edge labs, discover how these innovations could transform not just viticulture but the entire future of precision agriculture. Additional materials: www.superdatascience.com/840 Interested in sponsoring a SuperDataScience Podcast episode? Email [email protected] for sponsorship information.

In this episode, we dive into the transformative world of AI, data analytics, and cloud infrastructure with Josh Miramant, CEO of Blue Orange Digital.

As a seasoned entrepreneur with over $25 million raised across ventures and two successful exits, Josh shares invaluable insights on scaling data-driven businesses, integrating machine learning frameworks, and navigating the rapidly evolving landscape of cloud data architecture.

From generative AI to large language models, Josh explores cutting-edge trends shaping financial services, real estate, and consumer goods.

Tune in for a masterclass in leveraging data for impact and innovation!

Linkshttps://blueorange.digital/https://blueorange.digit…

Here’s the updated text with links to the websites included:AI is revolutionizing the military with autonomous drones, surveillance tech, and decision-making systems.

In this episode of Data Science at Home, we expose the cutting-edge tech reshaping defense—and the chilling ethical questions that follow.

🐦 Twitter: @DataScienceAtHome📘 LinkedIn: Francesco Gad📷 Instagram: https://www.instagram.com/datascienceathome/📘 Facebook: https://www.facebook.com/datascienceAH💼 LinkedIn: https://www.linkedin.com/company/data-science-at-home-podcast💬 Discord Channel: https://discord.gg/4UNKGf3NEW TO DATA SCIENCE AT HOME?

Data Science at Home explores the latest in AI, data science, and machine learning.

S…

In this episode of Data Science at Home, we’re diving deep into the powerful strategies that top AI companies, like OpenAI, use to scale their systems to handle millions of requests every minute!

From stateless services and caching to the secrets of async processing, discover 8 essential strategies to make your AI and machine learning systems unstoppable.

Instagram: https://www.instagram.com/datascienceathome/Twitter: @datascienceathomeFacebook: https://www.facebook.com/datascienceAHLinkedIn: https://www.linkedin.com/company/data-science-at-home-podcastDiscord Channel: https://discord.gg/4UNKGf3NEW TO DATA SCIENCE AT HOME?

Data Science at Home explores the latest in AI, data science, and ma…

Together, they explore the growing importance of distinguishing human-written from AI-generated text, discussing real-world examples from social media to news.

How reliable are current detection tools like DetectGPT?

What are the ethical and technical challenges ahead as AI continues to advance?

And is the balance between innovation and regulation tipping in the right direction?

Tune in for insights on the future of AI text detection and the broader implications for media, academia, and policy.

Welcome to Data Science at Home, where we don’t just drink the AI Kool-Aid.

Today, we’re dissecting Sam Altman’s “AI manifesto”—a magical journey where, apparently, AI will fix everything from climate change to your grandma’s back pain.

In this episode, I’ll break down the bold (and often bizarre) claims in Altman’s grand speech for the Intelligence Age.

I’ll give you the real scoop on what’s realistic, what’s nonsense, and why some tech billionaires just can’t resist overselling.

Chapters00:00 – Intro00:18 – CEO of Baidu Statement on AI Bubble03:47 – News On Sam Altman Open AI06:43 – Online Manifesto “The Intelleigent Age”13:14 – Deep Learning16:26 – AI gets Better With Scale17:45 – Conclu…

In this episode of Data Science at Home, we dive into the hidden costs of AI’s rapid growth — specifically, its massive energy consumption.

With tools like ChatGPT reaching 200 million weekly active users, the environmental impact of AI is becoming impossible to ignore.

Each query, every training session, and every breakthrough come with a price in kilowatt-hours, raising questions about AI’s sustainability.

Join us, as we uncovers the staggering figures behind AI’s energy demands and explores practical solutions for the future.

From efficiency-focused algorithms and specialized hardware to decentralized learning, this episode examines how we can balance AI’s advancements with our planet’s …

Subscribe to our new channel https://www.youtube.com/@DataScienceatHomeIn this episode of Data Science at Home, we confront a tragic story highlighting the ethical and emotional complexities of AI technology.

This devastating event has sparked urgent discussions on the mental health risks, ethical responsibilities, and potential regulations surrounding AI chatbots, especially as they become increasingly lifelike.

🎙️ Topics Covered:AI & Emotional Attachment: How hyper-realistic AI chatbots can foster intense emotional bonds with users, especially vulnerable groups like adolescents.

Mental Health Risks: The potential for AI to unintentionally contribute to mental health issues, and the challe…

Ever feel like VC advice is all over the place?

That’s because it is.

In this episode, I expose the madness behind the money and how to navigate their confusing advice!

Watch the video at https://youtu.be/IBrPFyRMG1QSubscribe to our new Youtube channel https://www.youtube.com/@DataScienceatHome00:00 – Introduction00:16 – The Wild World of VC Advice02:01 – Grow Fast vs. Grow Slow05:00 – Listen to Customers or Innovate Ahead09:51 – Raise Big or Stay Lean?

14:20 – The Real VC Secret: Focus on Your Team and Vision17:03 – Outro

In this episode of Data Science at Home, Frag dives deep into the wild claims that Large Language Models (LLMs) like Chinchilla 70B are beating traditional lossless compression algorithms.

🧠💥But before you toss out your FLAC collection, let’s break down Shannon’s Source Coding Theorem and why entropy sets the ultimate limit on lossless compression.

We explore: ⚙️ How LLMs leverage probabilistic patterns for compression 📉 Why compression efficiency doesn’t equal general intelligence 🚀 The practical (and ridiculous) challenges of using AI for compression 💡 Can AI actually BREAK Shannon’s limit—or is it just an illusion?

If you love AI, algorithms, or just enjoy some good old myth-busting, thi…

Are AI giants really building trustworthy systems?

A groundbreaking transparency report by Stanford, MIT, and Princeton says no.

In this episode, we expose the shocking lack of transparency in AI development and how it impacts bias, safety, and trust in the technology.

We’ll break down Gary Marcus’s demands for more openness and what consumers should know about the AI products shaping their lives.

Check our new YouTube channel https://www.youtube.com/@DataScienceatHome and Subscribe!

We’re revisiting one of our most popular episodes from last year, where renowned financial expert Chris Skinner explores the future of money.

In this fascinating discussion, Skinner dives deep into cryptocurrencies, digital currencies, AI, and even the metaverse.

He touches on government regulations, the role of tech in finance, and what these innovations mean for humanity.

Now, one year later, we encourage you to listen again and reflect—how much has changed?

Are Chris Skinner’s predictions still holding up, or has the financial landscape evolved in unexpected ways?

In this episode, join me and the Kaggle Grand Master, Konrad Banachewicz, for a hilarious journey into the zany world of data science trends.

From algorithm acrobatics to AI, creativity, Hollywood movies, and music, we just can’t get enough.

It’s the typical episode with a dose of nerdy comedy you didn’t know you needed.

Buckle up, it’s a data disco, and we’re breaking down the binary!

SponsorsIntrepid AI is an AI assisted all-in-one platform for robotics teams.

In this episode we delve into the dynamic realm of game development and the transformative role of artificial intelligence (AI).

Join Frag, Jim and Mike as they explore the current landscape of game development processes, from initial creative ideation to the integration of AI-driven solutions.

With Mike’s expertise as a software executive and avid game developer, we uncover the potential of AI to revolutionize game design, streamline development cycles, and enhance player experiences.

SponsorsIntrepid AI ( https://intrepid.ai ) is an AI assisted all-in-one platform for robotics teams.

Build robotics applications in minutes, not months.

In this episode, we dive into the wild world of Large Language Models (LLMs) and their knack for… making things up.

Can they really generalize without throwing in some fictional facts?

Or is hallucination just part of their charm?

Let’s separate the genius from the guesswork in this insightful breakdown of AI’s creativity problem.

TL;DR;LLM Generalisation without hallucinations.

The hype around Generative AI is real, but is the bubble about to burst?

Join me as we dissect the recent downturn in AI investments and what it means for the tech giants like OpenAI and Nvidia.

Could this be the end of the AI gold rush, or just a bump in the road?