≡

How do I keep up?

How do I keep up with the daily onslaught of AI news? The short answer is: you don't. There's no need to drink from a firehose of information every day, to keep up. My single most important advice is to let time and your gut dictate what is worth paying attention to. Time is one of the greatest natural filters, and you don't need a fancy subscription for it.

💡

Note: This guide isn't really related to interviews per se, but it's a question I often get. Feel free to skip this if you're in a rush, but if you're feeling overwhelmed with all the AI news, this guide may help.

Time has already filtered out a lot

To give you a rough idea of how effective time is, here are a few buzz-worthy AI concepts and terms that floated around at the start of the ChatGPT craze — which are no longer really relevant. If you've been struggling to stay afloat since ChatGPT's release — as I have — then you should feel some deja vu reading the below.

  • Prompt engineering: There are entire online textbooks dedicated to prompt engineering, but at the end of the day, it just boils down to being clear in your prompt. Just like how you would communicate with a genie that does exactly as you say in precisely the way you didn't want.

    • Many of the tips that were hot on social media — such as giving examples, using special formatting, and asking the model to reason step by step — are now no longer necessary for cutting edge models. The first two are obsolete due to instruction tuning, and the last one is now baked into popular models via post-training.
    • Beyond being hilarious, hacks like promising to tip ChatGPT or asking ChatGPT to take on a persona will become less and less necessary over time. They're still funny, years later, but there's no particular need to stay abreast of social media humor.
    • One common trick was to ask ChatGPT to refine a prompt you provided, before feeding the refined prompt straight back into ChatGPT. This technique has slowly phased out over time, since post-training has also made this technique unnecessary.

  • AutoGPT: AutoGPT was insanely popular for a brief period of time in 2023, but since then, has largely fallen out of the public eye.

    • You can see Google Trends here comparing three of the most popular agentic libraries, with AutoGPT trailing far behind similar libraries. Granted, AutoGPT serves a different purpose — regardless, I haven't heard it mentioned since or used in any production capacity.
    • Time also crowns a clear winner. Both LlamaIndex and Langchain began to overlap in purpose and function quite early on, and now, between LlamaIndex and LangChain, there's a clear winner: You can see that LangChain has 9x as much search interest as LlamaIndex, almost from the get-go and lasting until today.

  • Everyone's Large Language Models: After the alpaca instruction-tuned models were released, there was a huge flurry of companies that released new instruction-tuned or pretrained models. There was a massive number of these, ranging from Dolly to Pythia to Cerebras-GPT. Since then, these models have been forgotten, superseded by Llama's own instruction-tuned models or other newly-pretrained models like DeepSeek-V3. The open-source world is today led by Llama and DeepSeek.

What to look out for?

Say that you've accepted that time is a great filter, and now, you'd like to know what past news is worth paying attention to. This depends on what you're looking for, but for me personally, I look for new abilities, clever application to new domains, and any technique that makes the model cheaper to train or run inference on.

  • New capabilities: One recent key capability was tool use for Large Language Models, which is prolific in both open-source and proprietary models today. At its core, the method focused on teaching the model how to use APIs, while hallucinating less. The core insight was clever: Namely, there is very clearcut signal for the code that the Large Language Model returns — whether or not it runs, and whether or not the API is called correctly. Given how common tool use is now, I would say this was and is a capability worthy paying attention to.
  • New (interesting) application: Although this is old news now, Copilot at the time of release was very intriguing. Since then, I've been convinced that smart "autocomplete" for coding like Cursor is a concrete, very powerful application of Large Language Models. The hope is that in the near future, we'll have less of the mundane coding to do — and more interesting engineering challenges to have the time to tackle. The number products spawning in this space is testimony to the coding application's significance — there's now Void, Zed, and many others.
  • New optimization: This is a personal interest, since I research architectures and techniques that speed up training or inference. You may have heard of Flash Attention by now, which was a clever technique to reduce the number of memory reads and writes; I broke down Flash Attention in more detail, in How Flash Attention works. DeepSeek also recently made waves for how cheap their pricing is, so this is clearly a topic that is critical to the future of AI adoption.

You'll notice that I don't prioritize new datasets, new architectures etc. We're running out of human-generated versions of the former, and I don't personally believe architectures are the key to the next "level" in AI development. These are personal preferences, but regardless of what you decide to look out for, use public signals to validate what you believe is important.

Here are some weak signals for the value of a particular topic. It's worth noting I myself don't use these metrics daily — they're just a temperature check.

  • Citation counts. Say you're looking at alternatives to the transformer architecture: This is not too hard. Mamba has been cited 2000+ times in the last 2 years, and in the same amount of time, RWKV has been cited "only" 400+ times. Both are clearly well-cited and popular papers, but one is clearly more well-received. After diving into the pair of papers, you'd find that Mamba improves on RWKV by simply making model parameters be input-dependent. That's a reasonable innovation, and it also explains why Mamba is more popular — it's an improvement over the other.
  • Google trends. We can compare the popularity of several major AI products, ChatGPT vs Claude vs Gemini vs DeepSeek. ChatGPT is clearly dominating and continuing to grow, with DeepSeek enjoying a brief period of intense popularity. ChatGPT has 8x the traffic of Gemini, which has 2x the traffic of Claude and DeepSeek — as of time of writing. Search popularity isn't necessarily an indication of product use, but it's correlated with public discussion of the topic on social media. Funnily enough, the use of Google itself is possibly declining, so this may become less and less reliable of a metric over time.

All in all, use the topic's "performance" in the public domain to judge whether or not it's worth learning in detail. Certain pieces of news, like DeepSeek, are immediately worth picking up, and you won't miss those kinds of rare moments — mostly because it's too popular to ignore, and everyone will be talking about it. Beyond that, just see how often you hear about a topic. The third time you hear about it, it's time to sit down with it.