Advice on Reading Research Papers (by Prof. Andrew Ng)

Here we’ll summarize two major recommendations given by Prof. Andrew Ng in his CS230 Deep Learning course:

Reading research papers

Compile a list of papers
Try to create a list of research papers, medium posts and whatever text or learning resource you have
Skip around the list
- Read research papers in a parallel fashion; meaning try to tackle more than one paper at a time
- Concretely, try to quickly skim and understand each of these paper and do not read it all, maybe you read 10–20% of each one and probably that will be enough to give you a high-level understanding of the paper in hand.
- After that, you may decide to eliminate some of these papers or just go over one or two them and read them fully.
Amount of papers
- 5–20 papers
  - Probably enough basic knowledge of the specific domain
  - But maybe not enough to research or be at the cutting-edge.
- 50–100 papers
  - Probably have a very good understanding of the domain application

Do NOT start reading the paper from the first to the last word. Instead, take multiple passes through the paper

Read the Title, the abstract and the figures
- By reading the title, abstract, the key network architecture figure, and maybe the experiments section, you will be able to get a general sense of the concepts in the paper.
- In deep learning, there are a lot of research papers where the entire paper is summarized in one or two figures without the need to go hardly through the text.
Read the introduction + conclusions + figures + skim the rest
- The introduction, the conclusions and the abstract are the places where the author(s) try to summarize their work carefully to clarify for the reviewer why their paper should be accepted for publication.
- Skim the related work section (if possible), this section aims to highlight work done by others that somehow ties in with the author(s) work.
Read the paper but skip the math
Read the whole thing but skip the parts that don’t make sense

When reading a paper, try to answer the following questions:

If you can answer these questions, hopefully, that will reflect that you have a good understanding of the paper.

It turns out as you read more papers, with practice you get faster. Because a lot of authors use common formats when writing papers.

Try to rederive it from scratch. Although, it takes some time but it’s a very good practice.

Lightweight: Download open-source code (if you can find it) and run it.
Deeper: Reimplement from scratch. if you can do this, that’s a strong sign that you have really understood the algorithm in hand.

The most important thing to keep on learning and getting better is to learn more steadily rather than having a focus-intensive activity.

It’s better to read two papers a week for the next year than cramming everything over a short period of time.

Just focus on doing important work and consider your job as a tactic and a chance to do useful work.

A very common pattern for successful machine learning engineers, strong job candidates, is to develop a T-shaped knowledge base

To build the horizontal piece

A very efficient way to build foundational skills in these domains is through courses and reading research papers.

To build the vertical piece

You can build it by doing related projects, open-source contributions, research and internships.

Learn the most
tend to choose things to work on that allow you to learn the most.
Do important work
work on worthy projects that moves the world forward.
Try to take machine learning to traditional industries

Last updated on Apr 6, 2022