One Year Into The Gradient Ascent Adventure

A convenient index of the best articles & reflections from a year of writing about AI

It’s edition #52, folks. 

It's been just over one year since Gradient Ascent first hit inboxes, and boy has it been an incredible journey of learning and growth. 

I started this newsletter with one goal–to create the most accessible and fun resource to learn Machine Learning (ML) and AI.

To achieve this, I'd have to get better at writing and sharpen my thinking. But, most importantly, I'd have to show up every week. Like any ML researcher worth their salt, I wanted to test if this would work before scaling up.

Could I reach an audience of 500 readers by the end of the year? 

Humbling stats. Source: Substack

From a whimsical idea to a community of 6000 readers spanning 48 US states and 124 countries, this tiny labor of love has surpassed my wildest expectations.

None of this would be possible without your readership and support. You've been with me through thick and thin, from algorithm breakdowns to poorly drawn doodles. Before I go any further, thank you from the bottom of my heart.

Thanks, Gradient Ascent Fam!

💡Beyond the Stats

Writing has made me think clearly, teach effectively, and be a better leader at work. It's also opened some of the most incredible opportunities in my life:

• Gradient Ascent was featured on wired.com.

• Collaborations with incredible writers like

  Cameron R. Wolfe, Ph.D.
  .

• Speaking engagements and workshops at various startups and communities like Write of Passage, The Hamburg Data Science Community, and Small Bets.

The feedback and questions I got each week from writing the newsletter inspired me to write and illustrate "AI for The Rest of Us" (Wink Wink: Looking for a great holiday gift for your friends and family who want to learn AI?).

Writing online has changed my life and connected me with a global community of kindred spirits.

✍️ "Gradient Ascent" Wrapped Year One

I've combed through every article I wrote since starting this newsletter and curated the best for you. This post serves as an index to find topics you're interested in quickly.  

Language Models and NLP

1. Exponentially Faster Language Modeling

2. Retrieval Augmented Generation (RAG)

3. LIMA: Less Is More for Alignment

4. What does an LLM look at?

5. Large Language Models Explained

6. Harry Potter and the Chamber of Embeddings

7. ChatGPT

Computer Vision

1. Vision Transformers: Part I

2. Vision Transformers: Part III

3. Vision Transformers: Part V

4. Building your own Panorama Generator

5. Autoencoders

6. Variational Autoencoders

   1. Concept

   2. Code

7. GANs

   1. Concept

   2. Code

8. Text-to-Image Diffusion Models: Part II

9. Text-to-Image Diffusion Models: Part III

10. Dalle-3 Decoded

Deep Learning Techniques

1. Speeding up Neural Networks

2. Dataset distillation

3. Machine Unlearning

Interviewing, Tips and Tricks for ML Practitioners

1. Machine Learning Interviews

2. Keeping up with AI in 2023

3. How to Deconstruct ML Papers

4. One of The Best Ways to Improve Your ML Skills

Doodles and Visual Explainers

1. Skip Connections

2. The Importance of Data

3. Weight Initialization

4. Learning Rate

5. Gradient Clipping

6. Gradient Boosting

7. CLIP

8. Few-Shot Learning

9. Quantization

10. Q* Conspiracy

💭 Lessons Learned

• Consistency is key. Perfectionism can be a trap. The best way to get better at writing and growing is to show up consistently. As James Clear says in Atomic Habits, try to be 1% better each time. When you put in the reps, the quality will improve over time. Plus, you'll enrich the connection with your audience each week.

• Listen to the Reader. This newsletter is here to serve the reader. I ask myself, "Will this piece help my readers?" before writing any article. It's also why I conducted the reader survey a couple of weeks back. One of my favorite quotes on this topic comes from the book "Write a Must-Read" by A.J. Harper:

A book is not about something. A book is for someone.

• Building a System. We welcomed a second child into our lives this year. Balancing a 9-5 and two small humans while writing a weekly newsletter is really hard. I owe my wife an insurmountable debt of gratitude for letting me do this, but it's not sustainable or healthy in the long run. So, I'm building a writing system that doesn't involve me editing a piece into the wee hours of the night before it's due. The author of Not Boring,

  Packy McCormick
  said it best in his essay:

  She wakes up fresh as a daisy and wonderful, and I wake up feeling like an extra from The Last of Us.

• Experiment Often and Fail Fast. Part of finding the "product-reader" fit is to try many things out and learn from the results. This is something I continue to do today. Doing this also gives me creative energy. After all, if I want my newsletter to be fun and not a dense resource, that won't happen by doing the same thing week-in-week-out.

🤝 Appreciation

I'd be remiss if I didn't appreciate a few folks who've been incredible for me this year.

• Write of Passage: The course changed the way I write and connected me with some of the best and brightest creative minds on earth. Thank you for the push¹!

• Christin Chong, PhD
  : For your invaluable support with the book. It was one amazing adventure.

• Cameron R. Wolfe, Ph.D.
  and Zach Mueller for reviewing the book and giving me insightful technical feedback. 

• The amazing beta readers who volunteered their time and energy to make the book more readable.

• Louie Bacaj
  and
  Chris Wong
  ² (and the Small Bets community) for the amazing Newsletter Launchpad course that helped me learn to ship a valuable newsletter. 

• Sarah Fay
  for her insightful feedback to improve my Substack (and writing).

• Michael Sklar
  for seeing things others miss.

• My best half and two tiny humans for tolerating this pet project of mine. 

• The amazing Substack newsletters that support my work and helped me grow. I owe you all a lot!

I'm sure I'm missing a ton of people, so please forgive my lapse. I plead a complete lack of sleep and raccoon eyes. You know who you are and please know that I appreciate you and can't thank you enough.

🦾 The Present and Future of Gradient Ascent

I've got quite a few ideas planned for Gradient Ascent, but here are the things on the top of my mind:

• Growth: Triple the size of the Gradient Ascent community by the end of 2024

• Value: Bring even more value to you through an infusion of fresh, carefully crafted content (look out for announcements) 

• Healthy Consistency: Build a sustainable writing system and get better at writing

• Mentoring: I'm also going to open my calendar up in Q1 2024 to help a few selected folks level up in their AI journey. If you're interested in working with me 1:1, send me a message.

A massive thank you for being the most awesome readers I could ask for.

Thank you for every open, read, and share. Here's to making the complex simple, the boring fun, and the impossible possible. The AI train isn't stopping anytime soon. I said on day one that we're in this together. That’ll always be my north star.

Happy holidays to you and yours.

1. Special mention to

John Nicholas

,

Rick Lewis

,

Steven Foster

,

Michelle Elisabeth Varghese

,

Malavika Mudit

,

Ishan Shanavas

,

Camilo Moreno-Salamanca

,

Alex Michael

,

Latham Turner

,

Charlie Becker

,

Charlie Bleecker

,

Silvio Castelletti

,

Chris Coffman

,

Terri Lonier, PhD

,

Karena

,

Alexandra Allen

,

Janahan Sivaraman

,

Chao Lam

and

Haley Brengartner

for your amazing support during and after the cohort.

2. Chris also gets props for editing a messy draft of the book 🙂