In this blog post, I’ll share with you my learning process so that you can learn anything faster.
Today, the ability to constantly learn and reinvent ourselves is essential. For many years, we lived in a life model that had two moments: a period of learning and a period of working. We lived these moments in a sequential way. We would spend some time learning some skill and then, in the second moment of life, we would use those skills to earn a living. Well, those days are over. The social acceleration is killing this two-moments model and now we need to learn how to learn if we want to stay relevant.
In this blog post, I’ll share with you how I learn. It’s a seven steps process that I’ve been improving over time and helped me to succeed in my Ph.D., consulting jobs, and personal businesses. It goes as follows:
- Define a learning goal
- Research your topic
- (to release on November 18, 2018)
- (to release on November 25, 2018)
- (to release on December 2, 2018)
- (to release on December 9, 2018)
- (to release on December 16, 2018)
Every week I’ll be publishing a different chapter. As a result, the entire blog post will be finished by the middle of December. It will be like a Christmas gift.
1. Define a learning goal
The first step in this process is to define a learning goal. You can think about your learnings as a journey. When you travel, you have a destination. By knowing where you want to get, you can successfully define what you’ll need for the journey and where you will be passing by. You make a plan, you execute it, and you arrive at your destination. Learning is no different.
Goals keep you focused. As we get older, time becomes a scarce resource. Work, family, gym, kids, friends, hobbies, Benfica… Our lives are like elephants. They just don’t fit into 24 hours matchboxes. Accordingly, we need to find ways to make the best use of time. Defining goals is a way to do so. Goals discipline your mind and help you to focus on the important things. They are blinkers that prevent us to see sideways and force us to look ahead.
The SMART framework is useful for goal definition. This framework is so popular that it sounds like a cliché. As you probably know, SMART stands for Specific, Measurable, Achievable, Relevant, and Timely. The meaning of these words may vary from one person to another. Consequently, instead of defining them, I usually prefer to associate questions with each of these words. A nice exercise that you can do to define your learning goals is to answer the questions in Figure 1.
You can hack goal definition using reverse engineering. For example, I often see people doing learning projects related to the Udacity nanodegree on Self Driving Car Engineer. If you notice, these projects are a great learning resource because they provide you with specific tasks and clear outcomes. In other words, these projects are a great learning resource because they have well-defined learning goals. They make it easy to understand what you want to achieve (specific) and if you achieved it or not (measurable). Moreover, the projects are divided into small tasks supported by tutorial videos to facilitate learning (attainable), have a promise of a better professional future (relevant), and have deadlines associated (timely). In short, the science is all there. We just need to take this example into account and use it as a mental model when defining our learning goals.
Practical example: I’ve always been curious about Recurrent Neural Networks (RNN). However, I’ve never had a specific reason to learn more about it. Recently, because of my Ph.D., I started working with time-series. I knew that RNN could be used to solve time-series problems so I took this opportunity to explore RNN. Therefore, I took 30 minutes and, on paper, I started answering the questions presented in Figure 1. Here’s the result:
- Specific. I want to learn how to use RNN for time series predictions.
- Measurable. I’ll have two measures: a progress measure and an outcome measure. The progress measure will be to work a minimum of 3 hours per day, five days a week. I’ll monitor this measure to ensure that I’m consistently working on my learning goal. The outcome measure will be to publish a kernel on Kaggle with an RNN solution for the Web Traffic Time Series Forecasting competition. This will show that I’m able to make predictions on time-series problems using RNN.
- Attainable. I think this goal is attainable because I already know the mechanics of Neural Networks and there are a lot of learning resources online.
- Relevant. It is a relevant goal for me because the application of RNN will correspond to a chapter of my Ph.D. thesis.
- Timely. I will complete my learning goal by 30/11/2018.
2. Research your topic
Now that we know where we want to go, it’s time to research the topic. Keeping the journey analogy, the goal now is to gather all the resources that we need for the trip. The difference is that instead of collecting water, food, and clothes, we will be collecting scientific papers, blog posts, and videos. Welcome to the Information Age.
There are two types of research. One is focused on the ‘big picture’. In this case, we will be collecting resources that will introduce us to the topic. For example, all the books with the word ‘introduction’ are good candidates for this type of research. The other type of research is focused on ‘specifics’. Here we look for practical stuff that is directly related to what we want to learn. The more the resources resemble our learning task, the better. For example, if our goal is to learn how to build an algorithm that identifies cats and dogs in a picture, the best resource we can find is a step-by-step tutorial that does exactly what we are looking for. You can see this pattern in my Medium post.
At this stage of the 7 Steps to Learning, I’m more concerned with the big picture research. Why? Well, because that’s how I learn. Knowledge is built the same way sculptures are made: you do it part by part, starting by the basic shapes and adding the details later. So, don’t be shy. Release the Michelangelo that lives inside you and start creating knowledge masterpieces.
One research tip that I can give you now for free is to define, right at the outset, how much time you’ll spend on research. It’s easy to spend days on the net, collecting resources, and thinking how cool it will be to know more about X or Y. Daydreaming it’s great. Lots of psychological studies support the use of daydreaming as a way to fuel your motivation. However, there’s a difference between dreams and reality: action. Accordingly, my advice is to impose limits on yourself so that you can be productive and reach your goals.
I have a general framework for research. This framework has three steps, which I perform in an iterative way. They go as follows:
- Keywords. In Google Era, keywords are… key. I usually start with general keywords and I tune them as I progress in my research. It’s the same logic that I stated before. You with something general and then, as you progress and learn, you dive into the details.
- Tabs. After the search, I open some of the resulting links in different tabs. The idea is to open all the links that have a slight chance of being useful. Selection comes later. If you’re not using Google. Chrome, you may consider it. It is known as one of the best browsers to use if you keep a dozen tabs open at once.
- Zotero (optional). Zotero is a free tool that helps you to collect and organize information. After filtering some of the tabs, those whose content I want to keep, I import through Zotero. If you’re using Google Chrome, you can use the Zotero add-on. It will simplify the importing process. By default, I don’t add keywords to the documents at an initial stage. I prefer to do it later when I know that a document is really important, and I know which keywords I should use to refer to it. Note that I use Zotero when I’m doing a serious research. If it’s something simpler, I’ll probably just keep track of my research on Google Docs.
To be a research ninja using this framework, you need to:
- Master speed reading. Don’t understand speed reading in the sense of being able to read 1000 words per minute. Speed reading here is about skimming the materials fast to understand if its content is useful or not. If you’re dealing with text, some quick tips are: read the introduction and the conclusion first; pay special attention to titles, tables, and figures; and read only the first sentence in each paragraph. If you’re dealing with videos, it’s more challenging. You can zap through the video, clicking on different places of the video progress bar. Another option is to roll over the progress bar to preview the video (works on some YouTube videos).
- Know your sources. In my opinion, this is the secret sauce. Nowadays, we have more and more information. However, our time and our processing capacity are limited. Accordingly, we need mechanisms to select information. Most of these mechanisms are based on reputation. We choose information that has already been filtered, evaluated, and/or commented by others. An interesting discussion about reputation and its influence on the Information Age can be found in the book ‘Reputation: What Is and Why It Matters’.
It’s hard to help you developing speed reading because it requires practice (and I can’t do that for you), but I can share with you my main research sources.
Google needs no introduction. My fifty cents here are: pay attention to the suggestions given by the autocomplete tool; and learn how to use the advanced search operators.
Advanced search operators are important because they can make your search more specific. You can access the full list of advanced search operators here. My favorites are:
- “search term” – forces an exact-match search. Example: “bidirectional rnn”.
- OR – searches for X or Y. Example: “bidirectional rnn” OR “sequence transfer learning”.
- AND – searches for X and Y. Example: “bidirectional rnn” AND “sequence transfer learning”.
- * – acts as a wildcard and will match any word or phrase. Example: “bidirectional rnn” * “advantages”.
- filetype: – restricts results to those of a certain filetype (I use often for PDF and PPT). Example: “bidirectional rnn” filetype:pdf.
- site: – limits results to those from a specific website. Example: “bidirectional rnn” site:pmarcelino.com.
Google Scholar, the best friend of the scientific researcher. You can use Google Scholar to find scientific papers and scientists. Let me show you how through an example.
Imagine that you want to learn about ‘reinforcement learning’. The first thing to do is to go to Google Scholar and guess what? Write ‘reinforcement learning’ (Figure 2).
Note that Google automatically suggests you a set of keyword combinations, based on popular searches. This can be useful for future searches.
Ok, so once you search for ‘reinforcement learning’, you’ll get something like what is shown in Figure 3.
As we can see, there’s a popular reference from Sutton and Barto (Reinforcement Learning: An Introduction). It has been cited 28326 times and it was published by the MIT Press. Looks solid. Let’s take a look at the authors to know a little bit more about them.
After clicking on RS Sutton, we access his profile (Figure 4). It seems that he’s currently working at DeepMind and at the University of Alberta. Since 2016, he has been more and more cited. Apart from the ‘Reinforcement Learning: An Introduction’, he has two other works that have been cited often: ‘Learning to Predict by the Methods of Temporal Differences’, and ‘Neuronlike Adaptive Elements that can Solve Difficult Learning Control Problems’. These works are a bit old: they are from 1988 and 1983, respectively. We can see his recent works by clicking on ‘YEAR’ (Figure 5).
Sutton has been active in 2018. He keeps working on Reinforcement Learning and, according to the titles of his works, he’s particularly on Temporal-Difference Learning (whatever that means).
The take-aways of this research are that Sutton is working actively on Reinforcement Learning, at least since the 80s; and he has several popular works published and he’s working for DeepMind, the company that created AlphaGo. I don’t your opinion, but I believe that Sutton may know something about Reinforcement Learning.
Assuming that Sutton is a credible source, we have two different paths that we can follow to extend our research:
- Via authors
- Check who’s the top people in Sutton’s field of research.
- See who’s working with Sutton and do the same exercise we did for him.
- Via articles
- Check the works who cited ‘Reinforcement Learning: An Introduction’.
- Look for articles related to ‘Reinforcement Learning: An Introduction’.
As you can imagine, it’s easy to spend hours and hours in this cycle. That’s why it is important to define a time limit for research at the outset.
YouTube has been awarded as the Top Tool for Learning in 2018. I used to prefer books to videos as a learning material, but that’s changing.
Videos have some advantages over books. In particular, they have audio and image transitions that enrich the learning experience. Moreover, you can find great on YouTube nowadays. It’s up to you to choose if you want to watch the MIT course on Reinforcement Learning, attend a Reinforcement Learning lecture given by Richard Sutton (Figure 6), or watch the videos from the favorite of the crowd, Siraj Raval.
GitHub is a place where people share code. On GitHub, you can find pieces of code on everything: cutting-edge algorithms, official repositories, and all the kind of open source projects. I often use GitHub to search for specific algorithms. The chances that someone already implemented and shared them on GitHub are high.
Another important pointer is the Awesome Manifesto. There are many ‘awesome lists’ on GitHub. These lists have curated content on specific topics, which can be helpful to complement your research or even to save you some research time. For example, there’s an awesome list on Reinforcement Learning. I don’t know if it’s a good one, but they cite Sutton. That’s promising.
As far as I know, Reddit has the best online communities on everything. When you’re an engineer, it’s easy to fall in love with Reddit. Their minimalist design immediately suggests that you’re in a place where content is king.
There are two communities that I follow on Reddit. One is r/datascience. It has about 85k subscribers and it discusses a wide range of topics related to data science. The other community is the r/machinelearning. This one is bigger (493k) and it goes more into technical details. Both communities are good, but if I had to pick just one, I’d pick the r/machinelearning.
But now you ask, ’how can I use Reddit for research purposes?’ Easy. One option is to throw a question to the middle of the crowd (read the rules before!). Another option is to add the word ‘wiki’ at the end of the Reddit link. For example, to read the r/machinelearning wiki, we need to access the following link: https://www.reddit.com/r/MachineLearning/wiki. Not all Reddit communities have Wikis, but when they have, they tend to be very good.
arXiv is a repository of scientific papers that were approved for publication after moderation. The papers are on STEM topics and you can access all the papers for free. In general, the papers have high quality.
If you want to explore arXiv, you might be interested in arXiv Sanity Preserver. This website was built by Andrej Karpathy, a Deep Learning God. The aim of arXiv Sanity Preserver is to provide a better interface for interacting with the body of work published in the repository.
You may also want to take a look at GitXiv. Although incomplete, GitXiv has an interesting purpose. GitXiv links arXiv research to open-source implementations published on GitHub (hence the name). I use GitXiv mainly to explore new ideas so from times to times I go there to see what’s going on. When I’m looking for implementations of a specific algorithm, I still prefer to use the GitHub search engine.
There are other resources that I also use, although not that often. These resources are:
- Twitter. To limit the number of characters was the best invention ever on social networks. Follow the right people and you’ll see how useful Twitter can be. Twitter it’s not suited for quick researches on a specific topic, but it’s great to be aware of what’s happening.
- Amazon. The biggest store on Earth. Name a book and you’ll find it on Amazon. In addition to an infinite catalog, Amazon has a helpful review system (although it works better for entertainment products than for technical books). Some of the hidden gems that I found on Amazon are Deep Learning with Python, Python Machine Learning: Machine Learning and Deep Learning with Python, scikit-learn, and TensorFlow, and Automate the Boring Stuff with Python: Practical Programming for Total Beginners.
- Medium. I use Medium to find tutorials. Usually, I hunt tutorials on Towards Data Science, Hacker Noon, and freeCodeCamp. Medium is great because most of its tutorials are written by practitioners, who are more concerned with showing their work than with academic vanities. The great caveat of Medium is that it is so well-designed that the most useless content looks credible.
Ok, there’s an Easter Egg for the brave readers who survived so far. The Easter Egg is the link to my own search engine. Searches made by this engine are restricted to a list of websites that I regard as great content sources. I have been using this search engine for the last year. It greatly improved the quality of my research because it searches deeper into my favorite websites. Remember, I am a research scientist. It may happen that my list does not fit your needs (especially if they are more practical than mine, which is not difficult). Therefore, I recommend you to create your own search engine. You can do it here.
Practical example: For the sake of completeness, I exhaustively described my research sources. In practice, I’m usually much more agile, especially when I’m doing for research for a personal learning project on topics that I already know a little bit about.
In the case of the RNN project that I have been using as an example, my research was quite straightforward. After defining a limit of one hour for the research step, I went through the general research framework that I mentioned before:
- Keywords. I started with a search on ‘online course recurrent neural networks’. My goal was to find a general RNN course that I could use to learn the basics and get the big picture.
- Tabs. I opened the 10 first links that Google suggested me and scrolled to the end of the page to see if anything else was worthy of my click. Some more tabs resulted from this merciful action. I quickly checked each tab to see if there was any tab that I could discard. Then, I ordered the tabs so that those who were about people reviewing courses (e.g. Quora pages) were first. Tabs referring to online courses (e.g. Udemy) went to the end. After, I read the content of the first group of tabs to find out which were the most popular courses. Only after this step, I started looking to the list of courses given by Google. Fortunately, I found a course from Andrew Ng. I know Andrew from previous courses and he’s simply the best. It’s easy to learn from him. Consequently, I decided to go for his course and finish it as fast as I could.