Deep Learning

To tackle problems in robotics/self-driving with deep learning, we must delve deep into fundamental theories of deep learning. Particularly speaking unsupervised, and semi-supervised methods. Moreover, ground breaking works from other application fields such as Computer Vision (CV) and Natural Language Processing (NLP) are also of vital importance in providing inspirations.

Generator Nets

The problem of generating human-like behaviour relies significantly on generative models. VAE family and GAN family are some majorly used Generator Net frameworks. (Many more others to be surveyed, we need to find at least one more general survey on generator nets/generative models)

  - Generative Adversarial Nets
  - Conditional Variational Autoencoders
  - Diffusion
  1. [GAN] Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., … & Bengio, Y. (2014). Generative adversarial nets. Advances in neural information processing systems, 27. [pdf]
  2. [VAE] Kingma, D. P., & Welling, M. (2013). Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114. [pdf]
  3. [CVAE] Kingma, D. P., Mohamed, S., Jimenez Rezende, D., & Welling, M. (2014). Semi-supervised learning with deep generative models. Advances in neural information processing systems, 27. [pdf]
  4. [VAE-related] Lopez, R., Regier, J., Jordan, M. I., & Yosef, N. (2018). Information constraints on auto-encoding variational bayes. Advances in neural information processing systems, 31. [pdf]
  5. [VAE-related] Lopez, R., Boyeau, P., Yosef, N., Jordan, M., & Regier, J. (2020). Decision-making with auto-encoding variational Bayes. Advances in Neural Information Processing Systems, 33, 5081-5092. [pdf]
  6. [VAE-related] Zhi-Han, Y. (2022). Training Latent Variable Models with Auto-encoding Variational Bayes: A Tutorial. arXiv preprint arXiv:2208.07818.

Baye Networks/Bayes Statistics/Bayes Inference

  1. Ghahramani, Z. (2015). Probabilistic machine learning and artificial intelligence. Nature, 521(7553), 452-459.

CV Milestones

  1. [AlexNet]

NLP Milestones

  1. [Attention]

Attention Mechanism

Others

  1. [Dropout]
  2. [MAE]
  3. [ResNet]

Behaviour Prediction

Most existing generative BP literatures combines a type of seq-to-seq model (lstm/transformer etc.) and a generator net framework (CVAE/GAN etc.). We summarize proceedings in both deterministic and generative BP works here, as our baselines for comparison.

Deterministic Method Baselines

  • Surveys

  1. Rudenko, A., Palmieri, L., Herman, M., Kitani, K. M., Gavrila, D. M., & Arras, K. O. (2020). Human motion trajectory prediction: A survey. The International Journal of Robotics Research, 39(8), 895-935. [pdf]

    • This is widely recognized as a very thorough survey of deterministic methods.
  • RNN-based deterministic models
  1. [Social-lstm] Alahi, A., Goel, K., Ramanathan, V., Robicquet, A., Fei-Fei, L., Savarese, S.: Social LSTM: Human trajectory prediction in crowded spaces. In: IEEE Conf. on Computer Vision and Pattern Recognition (2016) [pdf]
  2. [Social-Attention] Vemula, A., Muelling, K., Oh, J.: Social attention: Modeling attention in human crowds. In: Proc. IEEE Conf. on Robotics and Automation (2018) [pdf]
  3. Morton, J., Wheeler, T. A., & Kochenderfer, M. J. (2016). Analysis of recurrent neural networks for probabilistic modeling of driver behavior. IEEE Transactions on Intelligent Transportation Systems, 18(5), 1289-1298. [pdf]
  • Some other paradigms that are deterministic
  1. [Gaussian process] Wang, J. M., Fleet, D. J., & Hertzmann, A. (2007). Gaussian process dynamical models for human motion. IEEE transactions on pattern analysis and machine intelligence, 30(2), 283-298. [pdf]
  2. [social forces] Helbing, D., & Molnar, P. (1995). Social force model for pedestrian dynamics. Physical review E, 51(5), 4282. [pdf]
    • I also see several paper improving this ‘social force’ model. Fetch a few more if necessary.

Generative Method Baselines

  • The GAN family
  1. [Social-GAN(SGAN)]
  2. [SoPhie]
  3. []

  • The CVAE family

  • Normalizing flows

Hints from CV

Can we gain insight or improvement from the idea of MAE (masked autoencoder in BP problem?)

Hints from NLP

Game theory

Finding Nash equilibrium can be regarded as solving a sequence of optimization problems for each agent. (?) Can we leverage this thought to improve the training process of BP models? We need to learn more about this topic…

Cognitive Prior Knowledge

  1. Brod, G., Werkle-Bergner, M., & Shing, Y. L. (2013). The influence of prior knowledge on memory: a developmental cognitive neuroscience perspective. Frontiers in behavioral neuroscience, 7, 139.
  2. Britton, B. K., & Tesser, A. (1982). Effects of prior knowledge on use of cognitive capacity in three complex cognitive tasks. Journal of verbal learning and verbal behavior, 21(4), 421-436.
  3. Hollingsworth, S. (1989). Prior beliefs and cognitive change in learning to teach. American educational research journal, 26(2), 160-189.
  4. Hayes-Roth, B., & Hayes-Roth, F. (1979). A cognitive model of planning. Cognitive science, 3(4), 275-310.
  5. Urey, Z. C. U. (2019). The cognitive use of prior knowledge in design cognition: the role of types and precedents in architectural design. Journal of Contemporary Urban Affairs, 3(3), 39-50.
  6. Hollingsworth, S. (1989). Prior beliefs and cognitive change in learning to teach. American educational research journal, 26(2), 160-189.
  7. Marsh, R. L., Ward, T. B., & Landau, J. D. (1999). The inadvertent use of prior knowledge in a generative cognitive task.Memory & cognition, 27, 94-105.
  8. Joram, E., & Gabriele, A. J. (1998). Preservice teachers’ prior beliefs: Transforming obstacles into opportunities. Teaching and teacher education, 14(2), 175-191.
  9. Ma, I., Westhoff, B., & Van Duijvenvoorde, A. C. K. (2020). The cognitive mechanisms that drive social belief updates during adolescence. BioRxiv, 2020-05.
  10. Moorman, N. M., Gopalan, N., Singh, A., Hedlund-Botti, E., Schrum, M. L., Yang, C., … & Gombolay, M. (2023, June). Investigating the Impact of Experience on a User’s Ability to Perform Hierarchical Abstraction. In RSS 2023 Workshop on Learning for Task and Motion Planning. [pdf]

Reinforcement Learning

  • Deep Q Learning (Deep Q Network)
  1. Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., … & Hassabis, D. (2015). Human-level control through deep reinforcement learning. nature, 518(7540), 529-533.[pdf]
  2. Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013). Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602. [pdf]
  3. Wolf, P., Hubschneider, C., Weber, M., Bauer, A., Härtl, J., Dürr, F., & Zöllner, J. M. (2017, June). Learning how to drive in a real world simulation with deep q-networks. In 2017 IEEE Intelligent Vehicles Symposium (IV) (pp. 244-250). IEEE. [pdf]
  4. Sorokin, I., Seleznev, A., Pavlov, M., Fedorov, A., & Ignateva, A. (2015). Deep attention recurrent Q-network. arXiv preprint arXiv:1512.01693. [pdf]

Control theory

Can we integrate a behaviour prediction model to something like a model predictive control framework?

Mathmatics

Optimization

Optimization algorithms is one of the most important module in deep learning frameworks. It is important to get to know some important works regarding it.

Other AI

Dynamic Programming

Evolutionary Algorithm

  1. Eiben, A. E., & Smith, J. (2015). From evolutionary computation to the evolution of things. Nature, 521(7553), 476-482.
  2. Mitchell, M., & Taylor, C. E. (1999). Evolutionary computation: an overview. Annual Review of Ecology and Systematics, 30(1), 593-616.
  3. Bäck, T., Fogel, D. B., & Michalewicz, Z. (1997). Handbook of evolutionary computation. Release, 97(1), B1.
  4. Fogel, D. B. (2000). Introduction to evolutionary computation. Evolutionary computation, 1.
  5. Eiben, A. E., & Smith, J. E. (2015). Introduction to evolutionary computing. Springer-Verlag Berlin Heidelberg.