6
$\begingroup$

Pretty soon I will be finishing up Understanding Machine Learning by Shai Ben-David and Shai Shalev-Shwartz. I absolutely love the subject and want to learn more, the only issue is I'm having trouble finding a book that could come after this. Ultimately, my goal is to read papers in JMLR's COLT.

  1. Is there a book similar to Understanding Machine Learning that would progress my knowledge further and would go well after reading UML?
  2. Is there any other materials (not a book) that could allow me to learn more or prepare me for reading a journal like the one mentioned above?

(also taking courses in this is not really an option so this will be for self-study)

This question was also asked here on AI SE from suggestion of comments.

Improve this question
$\endgroup$
12
  • 2
    $\begingroup$ Although TCS SE is one of the best places to ask these questions, Artificial Intelligence SE also has some users interested in this topic and that may be able to answer this question. Of course, learning theory is a central in AI. $\endgroup$
    – user34637
    Apr 17, 2020 at 3:48
  • $\begingroup$ See e.g., this: cstheory.stackexchange.com/a/46574/13319 (the Kearns-Vazirani book is a bit old, but still very relevant). $\endgroup$
    – Clement C.
    Apr 17, 2020 at 16:51
  • $\begingroup$ These recent lecture notes on Statistical Learning Theory (organized as a single, organized file) by Bruce Hajek and Maxim Raginsky may also be helpful. $\endgroup$
    – Clement C.
    Apr 17, 2020 at 16:53
  • $\begingroup$ This question was also asked here: https://ai.stackexchange.com/q/20355/2444. $\endgroup$
    – user34637
    Apr 17, 2020 at 17:53
  • 1
    $\begingroup$ @nbro, as a moderator, you should know that different stackexchange sites have their own site-specific rules. You should probably also not go to other stackexchanges encouraging people to ignore their policies because you "disagree" with them. Our policy is spelled out here: cstheory.meta.stackexchange.com/questions/225/…. I will not close this post because we already have upvoted answers and because PMaynard is a new user here. $\endgroup$
    – Lev Reyzin ♦
    Apr 19, 2020 at 1:03

3 Answers 3

Reset to default
6
$\begingroup$

Another good introductory book is "Foundations of Machine Learning" by Mohri et al.: https://www.amazon.com/Foundations-Machine-Learning-Mehryar-Mohri/dp/0262039400/. It has a large overlap with the Shai and Shai book, but also quite a bit of content that they don't cover.

There are also good books and surveys on more advanced or specialized topics:

There are more, but I hope this is a good start.

Improve this answer
$\endgroup$
5
  • 1
    $\begingroup$ Thank you this seems like a great list! I'm thinking it would be best to additionally work on computational complexity, high dimensional probability, and convex optimization. This also seems to confirm my suspicion that there are no more 'general' books on the subject after this. Please let me know if you have anymore tips on learning this subject. Additionally it would be nice to know about how far I am from being able to read and understand journals like COLT. $\endgroup$
    – PMaynard
    Apr 17, 2020 at 18:19
  • $\begingroup$ For complexity, you should look up Arora/Barak. For various results in high dimensional probability, the "data science" book by Blum, Hopcroft, and Kannan is good: cs.cornell.edu/jeh/book.pdf. For convex optimization, books and surveys by Vempala are good. As for reading COLT/ALT/JMLR papers, some are easier than others, and you can learn some topics before others. There's no reason not to try reading the papers as soon as you want and to consult books and references as needed. $\endgroup$
    – Lev Reyzin ♦
    Apr 17, 2020 at 18:23
  • $\begingroup$ Good idea to maybe start with some more approachable papers earlier on. Thanks again for the book recommendations (funny you should mention it Arora/Barak is on my nightstand right now). $\endgroup$
    – PMaynard
    Apr 17, 2020 at 18:29
  • 2
    $\begingroup$ For high-dimensional probability, the... "High-Dimensional Probability" book by Vershynin is really good. math.uci.edu/~rvershyn/papers/HDP-book/HDP-book.html @PMaynard $\endgroup$
    – Clement C.
    Apr 17, 2020 at 18:55
  • $\begingroup$ Foundations is excellent. I was about to comment with this and then immediately saw your answer. Its treatment is quite mathematically mature, in my opinion. $\endgroup$
    – Max von Hippel
    Apr 21, 2020 at 3:25
4
$\begingroup$

I have a list of references (incomplete) that may interest you: https://kiranvodrahalli.github.io/links/#resources-notes-textbooks-monographs-classes-etc

(second all the existing suggestions).

Improve this answer
$\endgroup$
2
$\begingroup$

I would recommend the book:

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.