Open in app

Sign in

Write

Sign in

Freedom Preetham
Freedom Preetham

4.3K Followers

Home

About

8 hours ago

Part 8 — Mathematical Explanation of Why It’s Hard for LLMs to Memorize

From the beginning of this blog series we have seen how the development of transformer models like GPT-4 represents a paradigm shift in natural language processing, demonstrating remarkable abilities in generating and understanding text. Contrary to popular belief, LLMs (or transformers in general) cannot be tuned for memorization, a task…

Llm

16 min read

Part 8 — Mathematical Explanation of Why It’s Hard for LLMs to Memorize
Part 8 — Mathematical Explanation of Why It’s Hard for LLMs to Memorize
Llm

16 min read


Published in

The Simulacrum

·2 days ago

Don Reed’s East 14th St

I cried the next day as the hot shower caressed my back. I did not know how not to cry. Don has resonated with me so deep, that it was cathartic. I was not sure what I was getting into when I got an invite to watch the biographical, comical…

Don Reed

3 min read

Don Reed’s East 14th St
Don Reed’s East 14th St
Don Reed

3 min read


Published in

Autonomous Agents

·3 days ago

Part 7 — Strategies for Enhancing LLM Safety: Mathematical and Ethical Frameworks

The quest to enhance the safety of Large Language Models (LLMs) is a sophisticated interplay of technical innovation, ethical considerations, and practical applications. …

Artificial Intelligence

14 min read

Part 7 — Strategies for Enhancing LLM Safety: Mathematical and Ethical Frameworks
Part 7 — Strategies for Enhancing LLM Safety: Mathematical and Ethical Frameworks
Artificial Intelligence

14 min read


Published in

Autonomous Agents

·4 days ago

Part 6 — Adversarial Attacks on LLM. A Mathematical and Strategic Analysis

Adversarial attacks on Large Language Models (LLMs) represent a sophisticated area of concern in AI safety, requiring an intricate blend of mathematical rigor and strategic foresight. These attacks, aimed at manipulating LLMs to produce unintended outputs, range from subtle input alterations to exploiting systemic vulnerabilities. In this blog I provide…

Artificial Intelligence

8 min read

Part 6 — Adversarial Attacks on LLM. A Mathematical and Strategic Analysis
Part 6 — Adversarial Attacks on LLM. A Mathematical and Strategic Analysis
Artificial Intelligence

8 min read


Published in

Meta Multiomics

·5 days ago

Complexities of Allelic Expression in the Human Genome

The study of allelic expression in heterozygous loci of the human genome is a central theme in understanding the intricacies of human genetics and genomics. It unveils the complexities underlying genetic diversity, disease predisposition, and phenotypic variation. …

Biology

4 min read

Complexities of Allelic Expression in the Human Genome
Complexities of Allelic Expression in the Human Genome
Biology

4 min read


Published in

The Simulacrum

·6 days ago

Building Better Children: The Key to Emotional Resilience

In the journey of parenting, one of the most profound questions we often face is: “How do we build better children?” While I don’t claim to be an expert, my experience raising two successful boys has given me some valuable insights. Particularly, I believe that from the tender age of…

Children

5 min read

Building Better Children: The Key to Emotional Resilience
Building Better Children: The Key to Emotional Resilience
Children

5 min read


Published in

Autonomous Agents

·6 days ago

Deep Dive into Rank Collapse in LLMs

Transformers, central to advancements in machine learning, leverage the self-attention mechanism for tasks across various domains, including natural language processing and computer vision. However, the underlying dynamics of these models, particularly concerning self-attention networks (SANs), present challenges like rank collapse. A recent study provides a mathematical analysis of this phenomenon…

Llm

10 min read

Deep Dive into Rank Collapse in LLMs
Deep Dive into Rank Collapse in LLMs
Llm

10 min read


Published in

Autonomous Agents

·Nov 29

Brilliant Evolution of Transformer Blocks — A Mathematical Deep Dive

Let me start with saying that this is one of the best evolutions I have come across ever since the inception of Transformers! Large language models (LLMs) can expand their capabilities through various scaling strategies. The more straightforward approach involves amplifying the computational resources — this is a matter of…

Llm

16 min read

Simplifying Transformer Blocks — A Detailed Mathematical Explanation
Simplifying Transformer Blocks — A Detailed Mathematical Explanation
Llm

16 min read


Published in

Autonomous Agents

·Nov 26

Part 5 — In-Depth Analysis of Red Teaming in LLMs: A Mathematical and Empirical Approach

The field of Large Language Models (LLMs) is rapidly advancing, necessitating robust red teaming strategies to ensure their safety and reliability. In part 4, I covered “Enhancing Safety in LLMs: A Rigorous Examination of Jailbreaking” Red teaming, a method of simulating adversarial attacks to identify vulnerabilities, requires a deep understanding…

Llm

12 min read

Part 5 — In-Depth Analysis of Red Teaming in LLMs: A Mathematical and Empirical Approach
Part 5 — In-Depth Analysis of Red Teaming in LLMs: A Mathematical and Empirical Approach
Llm

12 min read


Published in

Autonomous Agents

·Nov 26

Part 4 — Enhancing Safety in LLMs: A Rigorous Mathematical Examination of Jailbreaking

The concept of jailbreaking Large Language Models (LLMs) such as GPT-4 represents a formidable challenge within the domain of artificial intelligence. This process entails the strategic manipulation of these advanced models to operate beyond their predefined ethical guidelines or operational boundaries. In the previous blog, I covered “Mathematically Assessing Closed-LLMs…

Llm

14 min read

Part 4 — Enhancing Safety in LLMs: A Rigorous Examination of Jailbreaking
Part 4 — Enhancing Safety in LLMs: A Rigorous Examination of Jailbreaking
Llm

14 min read

Freedom Preetham

Freedom Preetham

4.3K Followers

AI Research | Math | Genomics | Quantum Physics

Following
  • Barack Obama

    Barack Obama

  • Cassie Kozyrkov

    Cassie Kozyrkov

See all (9)

Help

Status

About

Careers

Blog

Privacy

Terms

Text to speech

Teams