External Resources for ML

Links, blogs, courses, people, etc. in the fields of AI, ML, CV, NLP, etc. that I want to keep at one place (for my reference).

Websites

Learning ML

1. Learn ML from TensorFlow

2. ML Crash Course from Google

Daily papers

1. HuggingFace Papers

2. PapersWithCode (also see other portals)

3. Scholar Inbox: More personalized list of papers daily

4. arxiv-sanity

5. AIModels.fyi blog notes (paper summaries)

Paper search

1. Litmaps

2. Semantic Scholar

3. Google Scholar

Conferences

1. AI Deadlines: A PapersWithCode Repository that keeps track of upcoming conferences in AI related subject areas

AI Summaries

Warning

AI generated summaries might not always be factual (or have high utility).

1. Bing Chat on Microsoft Edge: Open PDF in Edge, click Bing Chat, ask it questions about the PDF

2. arxiv-summary

3. scisummary

Blogs

1. The AISummer: Getting started with AI and many concepts pages as articles

   • See page on attention, NLP transformers, and ViT transformer

   • See page on NeRF and InstantNGP

   • See page on Diffusion models

2. Lilian Weng’s Blog

   • See Transformer version family

   • See Prompt engineering

   • See overview of LLM powered autonomous agents

Podcasts

1. Lex Fridman’s Podcast

Projects

Interesting projects in the wild

1. HuggingFace Projects Documentation: A collection of awesome community projects

   • Transformers: Different transformer implementations

   • timm: SOTA computer vision implementations

   • Hub: Models, datasets, etc. at one place

   • Tokenizers: Tokenizers for production and research

   • Diffusers: Diffusion algorithms

2. Kornia: Computer vision algorithms (AI centric)

3. PyG - PyTorch Geometric: Geometric deep learning on PyTorch

4. PyTorch Points 3D: Implementation of various 3D algorithms in one repository

5. PyTorch Metric Learning: Implementation of distance-based data mining, losses, trainers, testers, etc. for metric learning

6. Numba: JIT compilation for making python code faster (even has CUDA acceleration and parallel for loops)

7. RAPIDS Ecosystem: Many libraries for accelerated pandas dataframe, Scikit-learn like API, analytics, clustering, solvers, etc.

8. CuPy and PyCUDA: NVIDIA CUDA in Python

9. HMMLearn: Unsupervised learning and inference of Hidden Markov Models

10. XGBoost: Optimized gradient boosting library that is efficient, flexible, and portable

11. LightGBM: Gradient boosting framework that uses tree-based learning algorithms

12. Ray: Scaling AI workloads (data, training, hyperparameter tuning, RL, serving, etc.). From AnyScale Platform team.

13. Projects on large language models

    • Lightning-AI/lit-gpt: Implementation of SOTA open-source LLMs with quantization and LoRA like enhancements

    • LLaMA Index: LLMs on your own data

    • LangChain: Build applications powered by language models

    • tiktoken: OpenAI’s BPE tokenizer

    • Guardrails: Adding guardrails to LLM outputs (prompting). Also see this blog on inspecting the under-the-hood prompting.

14. Collection of interesting research in the wild

    • Microsoft’s microsoft/unilm: Collection of Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities. Also see their website.

    • Google’s massive google-research repository: contains all publicly released code by Google Research.

    • Facebook’s Detectron2 repository: contains SOTA detection and segmentation algorithms. Also see their xformers repository for transformer optimization hacks.

    • Google’s Scenic Library: has many computer-vision research tools (coded in JAX).

15. spaCy: NLP tool

16. ESPnet: End-to-End Speech Processing Toolkit. Also see docs.

17. Acme: RL components and agents by Google DeepMind

18. Some PyTorch frameworks for (numerical) optimizers: torchimize (API), pytorch-minimize

19. Collection of parallel (distributed/multi-GPU and node) training resources

    • HF Accelerate: Use PyTorch models on any device and distributed configuration (by HuggingFace)

    • Horovod: Distributed DL framework for Tensorflow, PyTorch, and Keras

    • LambdaLabs Blog: Distributed training guide for PyTorch (using MPI)

20. Some more interesting ML projects in the wild

    • Teachable Machine (GitHub): Train a model to recognize images, sounds, and poses online and export TensorFlow model.

    • Quick Draw: Dataset and model to recognize hand drawings (doodling dataset).

    • ONNX: Interoperability framework for AI models (GitHub)

    • ONNX Runtime: Deploy an ONNX model on multiple platforms (GitHub)

21. MLHub CLI: Command line framework for various ML models (not related to this project)

22. AutoML: Neural architecture search (NAS) and hyperparameter selection/optimization

23. Radiant Earth: Earth observation data (geo-spatial informatics)

24. ICESat-2: Ice, Cloud and land Elevation Satellite-2 (geo-spatial informatics)

Startups

1. ArtPark Ignite: Venture-building program for AI and Robotics from ARTPARK@IISc

Books

1. Ian Goodfellow - Deep Learning book

Courses

AI and Machine Learning

1. Stanford CS229 - Machine Learning - Prof. Anand Avati

   • Stanford’s Machine Learning course. There are five modules; supervised learning: linear and logistic regression, classification, linear models, generative learning, kernel methods, and support vector machines (SVMs); deep learning: neural networks and back propagation; generalisation and regularisation: complexity bounds and model selection; unsupervised learning: clustering, expectation maximisation (EM) algorithms (ELBO), VAEs, PCA, Independent Component Analysis, self-supervised learning (SSL) and foundation models; reinforcement learning: decision processes, policies, linear quadratic regulation (LQR), differential dynamic programming (DDP), linear quadratic gaussians (LQG), policy gradients. Main course design by Andrew Ng.

   • Related:

     • Stanford CS230 - Deep Learning - Andrew Ng: YouTube playlist - Autumn 2018

   • Links: Website (SEE Page, Stanford page), CS229 Fall 2023-24 Syllabus, Course Notes by Andrew Ng, YouTube Playlist - Spring 2023, YouTube Playlist - Autumn 2018

2. NYU - Deep Learning - SP21

   • Deep learning course at NYU from Yann LeCun and Alfredo Canziani

   • Links: Course Docs - Spring 2020 (major release, other didactics), YouTube Playlist - Spring 2020, GitHub - Spring 2021

3. Stanford CS231n - Deep Learning for Computer VIsion - Fei Fei Li

   • Links: YouTube Playlist, Course website

4. CMU - 11-785 Introduction to Deep Learning

5. CMU - 16-825 - Learning for 3D Vision - Spring 2023

   • Course GitHub (Assignments), GitHub (Submissions)

6. Cornell Tech CS 5785 - Applied Machine Learning

   • Links: YouTube playlist - Fall 2020, GitHub - Notebooks and Slides

7. MIT 6.S192: Deep Learning for Art, Aesthetics, and Creativity

   • Links: YouTube Playlist

8. MIT - Introduction to Deep Learning

   • Links: YouTube Playlist

9. Stanford CS25 - Transformers United

   • Links: YouTube Playlist - Cases

10. UC Berkeley - Full Stack Deep Learning

    • Links: YouTube Playlist - Spring 2021

11. University of Tubingen - Statistical Machine Learning - Summer 2020

    • Links: YouTube Playlist

12. University of Tubingen - Introduction to Machine Learning - Winter 2020/21

    • Links: Dmitry Kobak’s Blog - Slides

13. UC Berkeley - CS294-158-SP20 - Deep Unsupervised Learning Spring 2020

    • Links: YouTube Playlist

14. Michigan - EECS 498.008 / 598.008 - Deep Learning for Computer Vision - Winter 2022

    • Links: YouTube Playlist

Geometric Deep Learning

1. UvA - An Introduction to Group Equivariant Deep Learning

   • Part of Geometric Deep Learning series from University of Amsterdam. Contains lecture videos on group theory, steerable group convolutions, and equivariant graph neural networks. Also has Colab assignments.

2. UPenn - Graph Neural Networks - ESE 5140

   • GNNs (lectures and labs/assignments). Overview of GNNs from NVIDIA, distill

Reinforcement Learning

1. Stanford CS234 - Reinforcement Learning - Emma Brunskill

   • Links: YouTube Playlist

2. UC Berkeley CS 285 - Deep Reinforcement Learning

   • Links: YouTube Playlist

3. UC Berkeley CS 294 - Deep Reinforcement Learning (Fall 2015)

   • Links: YouTube Playlist - Foundations of Deep RL - Pieter Abbeel

Computer Vision

1. University of Tubingen - Computer Vision - Prof. Dr. Andreas Geiger

   • Introduction and history of computer vision. Photogrammetry, image sensing pipeline, structure-from-motion, bundle adjustment, stereo reconstruction, probabilistic graphical models, optical flow, shape from shading, stereo, coordinate based networks, image recognition, semantic segmentation, object detection, self-supervised learning, and other advanced topics (compositional models, human body models, deepfakes, etc.). University of Tubingen Computer Vision course by Prof. Dr. Andreas Geiger.

   • Links: YouTube Playlist, Public Material: Slides and exercises

Natural Language Processing

1. CMU - CS 11-737 Multilingual NLP - Spring 2022

   • Links: YouTube Playlist

2. CMU - CS 11-711 - Advanced NLP - Fall 2022

   • Links: YouTube Playlist

3. Stanford CS224U: Natural Language Understanding

   • Links: GitHub, YouTube Playlist

4. UMass - CS685 - Advanced Natural Language Processing - Spring 2023

   • Links: YouTube Playlist - Fall 2020

YouTube Playlists

1. 3Blue1Brown Course - Neural Networks: Awesome theory and explanation, from basic neural networks and back propagation to GPT-3 (transformers and tokenizers).

2. Andrej Karpathy - Neural Networks: Zero to Hero

3. Samuel Albanie - Foundation Models

4. GCP - Making Friends with Machine Learning

5. HuggingFace Course YouTube Playlist

   • Links: All HF Courses, HF NLP Course, HF Audio Course, HF Deep RL Course

6. Jeremy Howard - Practical Deep Learning for Coders 2022

7. MLOps - Machine Learning Engineering for Production

Communities

Some communities you can follow

1. ML Collective: ML research opportunities, collaboration, and mentorship

People

1. Geoffrey E. Hinton, Yann LeCun, and Yoshua Bengio: Founders of modern deep learning (received the turing award for it in 2018)

2. Jurgen Schmidhuber (IDSAI, Swiss): LSTM

3. Jitendra Malik (UC Berkeley, Meta): Computer vision and AI

4. Leonidas J Guibas (Stanford): 3D computer vision backbones (PointNet).

5. Abhinav Gupta (CMU RI): Computer Vision and AI

6. Sergey Levine (UC Berkeley): Reinforcement Learning for Robotics

7. Dhruv Batra (Georgia Tech, Meta): Embodied AI Agents, Robotics

8. Michael Bronstein (CS Univ. of Oxford): Geometric deep learning and graph neural networks.

9. Max Welling (Qualcomm UvA): VAEs, graph CNNs

10. Luca Carlone (MIT): SPARK Lab; SLAM and robust perception.

11. Saurabh Gupta (UIUC, Meta): Computer vision, robotics, and AI

Follow these folks on social media (for new research)

1. Dmytro Mishkin: Kornia (CV+AI framework), tweets papers

2. Phil Wang a.k.a. Lucidrains: Open source contributions on GitHub

3. Ahsen Khaliq a.k.a. AK a.k.a. akhaliq: Tweets and HuggingFace papers, Gradio

4. Aran Komatsuzaki: Tweets papers, LAION and EleutherAI

5. Mike Young: Paper summaries

6. Ryohei Sasaki: Research on autonomous driving (LiDAR)

7. Dr Ronald Clark (CS, Oxford): Real time SLAM, bundle adjustment, scene understanding, and motion tracking

8. Devendra Singh Chaplot (CMU, FAIR): Visual navigation, object goal navigation, exploration, embodied AI

9. Dhruv Shah (UC Berkeley): Robotics & AI