Types of Questions

Here are several categories of interview questions. Use these categories to organize your practice — for example, practice at least one question from each category each week.

Translate concepts into code

These questions test your ability to connect high-level concepts with code. To a limited degree, it's also just a test generally if you know the concept to begin with. The concepts here are the building blocks of AI you should be familiar with: what a convolution is, what attention is etc.

• Build from scratch. Build a minimal version of a commonly-used operation from scratch. Typically, the operations are reduced to the simplest form, so any commonly-used bells-and-whistles are either part of follow up questions or skipped entirely¹. Alternatively, using pre-built operations, build a commonly-used combination of operations. This might be a small neural network for example.
• Debug models. Debugging a system even without AI is already decently challenging. Luckily, many of the same techniques carry over, but unlike before, AI is truly a black box. There are techniques you can employ to mitigate this unknown. See Tips for Debugging.

Here are some sample questions in this category.

• Convolutions from scratch
• Attention from scratch

Work with machine learning frameworks

There are multiple problems designed to mimic challenges you may face on the job, involving an existing codebase. Within this general category, there are two topics to consider. Neither of these involve AI knowledge per se, but these are common optimizations that anyone working with the model will need to know.

• Vectorize a program. "Vectorizing" just means "removing loops". Instead of looping manually, execute a single, large batched operation using your framework of choice — whether it be numpy, PyTorch, JAX etc. In short, it's much more efficient to have your underlying C program, CUDA kernel, or Metal shader run the for loop for you. See Tips for Vectorization.
• "Graphifying" a program. This is just the process of converting a dynamically executed program into a static graph. This is more common than you might think — the framework itself, such as JAX, may require it. Even if you're using PyTorch, you'll need to do this eventually when exporting a model. For libraries other than JAX, you may need to vectorize a program before doing this step. See Tips for "graphifying".

Here are some sample questions in this category.

• Preparing argmax for training

Hold analytical discussions

• Discuss a paper. You're asked to read a paper in advance of the interview, and in the interview, you'll be asked about the paper. The rough idea is: Given time, can you understand what is going on in this paper, deeply?
• Analyze. Reason through a problem possibly using code or math to answer questions. You may propose experiments, do some napkin math, or make a few plots to understand how to assess a new algorithm or fix a problem. The crux of this question is to understand: If you're faced with a decision to make on the job, can you come up with the right metrics, sanity checks, and experiments to make an educated choice?

Here are some sample questions in this category.

• Sparse dot product
• Streaming from Flash Attention

Miscellaneous

There are two more types of questions that you'll encounter.

• Build infrastructure. Complete a fairly standard coding interview question for a related topic. For example, you may work on a data structure or algorithm that is often used in training. You can complete these questions "standalone," without understanding its relation to AI.
• AI Design. This is the AI-equivalent of a systems design question, but instead of designing the architecture for compute and storage infrastructure, design a series of experiments, data collection strategies, and task formulation. Rarely is architecture that important. Here are some sample questions in this category.

Takeways

Don't memorize the categories above — that won't help you very much. The categories below are only useful for understanding your strengths and weaknesses at a macro scale. For example, maybe you find that you're performing worse at "coding from scratch" problems, and you're performing better at "AI design" problems. This can help you calibrate your future practice, to cover all of your bases. If anything, pay attention to the categories of questions you struggle with or excel at.