1. Why I Chose "Understanding Deep Learning"
Using AI every day, I'm excited to see how much more it will change the world. At times, it seemed so difficult that I had my doubts about whether I could do it. But my love for computers, the fact that math seemed manageable, and the appeal of being able to create things I want drogasve me to study AI in earnest.
After finishing a Python study group, I considered what to study next. We discussed options like Huggingface's LLM Course and Pytorch's Tutorials, but I felt I lacked a foundational understanding of deep learning and wanted to study it from a more mathematical perspective. I found out about UDL, a fairly recent book that covers everything from fundamental models like CNNs and RNNs to the latest concepts like Transformers and Diffusion. It also uses PyTorch, is available for free, and its code is publicly accessible. That's why I've decided to dive into this book.
In this blog post series, due to the book's license, I won't be providing detailed translations or summaries. Instead, I'll share the impressions and key takeaways that I and my study group have gathered.
2. The Types of AI
I had a rough idea of what Supervised Learning, Unsupervised Learning, and Reinforcement Learning were, and that Deep Learning is a part of Machine Learning. However, I wasn't clear on their exact relationship, but now I understand it for sure. A diagram in the book's first chapter showed me this relationship at a glance: both ML and DL encompass Supervised, Unsupervised, and Reinforcement learning.
The difference between ML and DL, the book explains, is that DL uses deep neural networks to create much more complex, diverse, and variable-length mappings between inputs and outputs. I had never really thought about how DL fits within ML, but understanding it simply as the ability to perform more complex mappings makes it seem much simpler.
Supervised learning can be divided into Regression and Classification problems, which in turn have sub-categories like multivariate regression and multiclass classification, depending on whether they predict a single value or multiple values. These are basic concepts, but reviewing them made things clearer. This led me to wonder if regression and classification exist in unsupervised or reinforcement learning. However, after a little thought, it immediately made sense that they wouldn't, since unsupervised learning is about learning patterns from input data, and reinforcement learning is about mapping an agent's actions to the real world.
Meanwhile, the concept of "Latent Variables" was new to me and quite interesting. It's the idea that data is often lower-dimensional than the number of theoretically possible values. For example, while the number of 100-character strings you can create by randomly arranging UTF-8 characters is vast, the number of those that form a coherent Korean sentence is extremely small. This is why these low-dimensional variables are used in generative models. This makes me wonder if it would be possible to start with latent variables and then increase the dimensionality later for more detailed predictions.
3. AI Ethics
The book discusses several ethical considerations, emphasizing that researchers who study and create AI must be clearly aware of its risks. These include bias, explainability, weaponization, concentration of power, and existential risk. The pace of AI development is accelerating, and it seems these risks grow larger as it advances. These problems seem difficult to solve with simple regulations alone, and seeing them laid out all at once made me realize that I need to think not only about AI's performance but also about how we can solve these societal problems.
4. Closing Thoughts
The next chapter marks the start of the core content, but it was fascinating to get a high-level overview of how AI is structured. I feel I've chosen the right material and I'm eager to study it all the way through. Last year, I built an Android smartphone control agent called YeYa, and I still have a passion for researching AI that can perform on-screen tasks well. The definition of reinforcement learning in the book seems to be a perfect fit for what I have in mind, which makes me even more eager to learn about it. Unfortunately, it's in the very last chapter, which I guess is because you need to understand supervised and unsupervised learning first.
Reference
[1] Prince, S. J. D. (2023). Understanding Deep Learning. The MIT Press. Retrieved from http://udlbook.com