Motivations

This post is to document my learning journey on GANs or Generative Adversarial Networks. GAN technology is considered to be one of the most promising advancements in the field of AI. I have worked in facial recognition previously and I have a decent background in ML/deep learning. Anyone who has trained deep learning models would understand the importance of training data, which is scarce in many cases.

My learning goals are the following:

  1. Obviously, for the pleasure of learning! First.
  2. To keep myself up-to-date with deep learning advancements.
  3. Hopefully, I would like to explore how much GANs can actually help in synthetic data generation for training deep learning models.
  4. Perhaps see if I can do something creative with these GANs.

Rather than scouring the web for resources randomly, I want to take a more principled approach. I decide to follow the GAN Specialization on Coursera.

I completed the wonderful Machine Learning course taught by Prof. Andew Ng a few years ago and I loved it. I did all the exercises sincerely and learnt a lot from it. I even had the Ecole Doctorale (the PhD program office at my French university) to accept this toward my course requirements. I hope to derive the same pleasure and value from this course.

To really motivate myself, I have taken the paid option with Coursera which costs 40 EUR/month. I asked if my company can take care of this budget but I did not get a nod. Actually I don’t care and I think this is a good investment.

Coursera GAN specialization

There are a total of 3 courses in this specialization. Each course has a number of programming assignments to be completed. I like this do-and-learn approach in general which for me is quite effective strategy.

I hope to review each of the 3 courses here as and when I progress through the course. This is an additional motivator for me.

Course1 : Build Basic Generative Adversarial Networks

  • The lectures were sequenced nicely with crisp and clear explanations.

  • The coding assignments could have been a little more harder. They did not challenge me at all. Probably, this was intended by the course creators since too hard assignments might result in students dropping out of the course or the specialization itself. It is well-known that the dropout rates are extremely high in MOOC courses. Perhaps, another reason is that ML/deep learning is not something very new to me.

Course2 : Build Better Generative Adversarial Networks

  • A lot of optional python notebooks were provided which I found positive.

  • Yet again, the programming assignment was boring. I felt as though this was conceived in a hurry. I could simply fill up the missing code snippets as if putting back a puzzle together and get away scot-free!

  • I completely agree that ethics and fairness in machine learning are extremely important and relevant. Unfortunately, the explanations provided for fairness concepts including the ones in the Bias quiz on Week2 were inadequate. I can go a step further and say that the clarity and fluidity in the progress of Course 1 was missing in Course 2.

  • In the quiz, instead of flying elephants and flamingoes when discussing equality of opportunity, we could have introduced a real-world example, like gender bias in hiring decisions or loan application acceptances or college admissions. In one sentence, your intentions were noble but the execution could have been really better.

My general comments to the course creators

  1. Sacrifice depth for width - There are so many additional optional readings (like in week3) where you have simply suggested papers to read. In my opinion, this could be replaced with in-depth discussions. As example is to discuss about the actual training in the assignment notebooks.

  2. My suggestion would be to add a project or a list of projects - this should be a real problem which can be solved by GANs. In the courses, you can build those concepts which will be eventually necessary to solve this problem and complete the project.

  3. As ML engineers and practitioners, we are interested in knowing how to solve problems which show up in practice during the training. For example, how to diagnose why a training has failed, how to log experiments (online or offline wandb integration) and what are the remedies for it.

  4. Please publish the statistics on number of people who took the course and the average time spent by them to complete the videos and assignments in each week for each course. This might help us to be organized and plan our schedules accordingly.