Short description:

This book approaches robotics from a deep learning perspective. Artificial intelligence (AI) has transformed many fields, including robotics. This book shows you how to reimagine decades-old robotics problems as AI problems and is a handbook for solving problems using modern techniques in an era of large foundation models. 

The book begins with an introduction to general-purpose robotics, how robots are modeled, and how physical intelligence relates to the movement of building artificial general intelligence, while giving you an overview of the current state of the field, its challenges, and where we are headed. The first half of this book delves into defining what the problems in robotics are, how to frame them as AI problems, and the details of how to solve them using modern AI techniques. First, we look at robot perception and sensing to understand how robots perceive their environment, and discuss convolutional networks and vision transformers to solve robotics problems such as segmentation, classification, and detection in two and three dimensions. The book then details how to apply large language and multimodal models for robotics, and how to adapt them to solve reasoning and robot control. Simulation, localization, and mapping and navigation are framed as deep learning problems and discussed with recent research. Lastly, the first part of this book discusses reinforcement learning and control and how robots learn via trial and error and self-play.

The second part of this book is concerned with applications of robotics in specialized contexts. You will develop full stack knowledge by applying the techniques discussed in the first part to real-world use cases. Individual chapters discuss the details of building robots for self-driving, industrial manipulation, and humanoid robots. For each application, you will learn how to design these systems, the prevalent algorithms in research and industry, and how to assess trade-offs for performance and reliability. The book concludes with thoughts on operations, infrastructure, and safety for data-driven robotics, and outlooks for the future of robotics and machine learning.

In summary, this book offers insights into cutting-edge machine learning techniques applied in robotics, along with the challenges encountered during their implementation and practical strategies for overcoming them.

What You Will Learn

• Explore ML applications in robotics, covering perception, control, localization, planning, and end-to-end learning


• Delve into system design, and algorithmic and hardware considerations for building efficient ML-integrated robotics systems


• Discover robotics applications in self-driving, manufacturing, and humanoids and their practical implementations


• Understand how machine learning and robotics benefit current research and organizations

Long description:

This book approaches robotics from a deep learning perspective. Artificial intelligence (AI) has transformed many fields, including robotics. This book shows you how to reimagine decades-old robotics problems as AI problems and is a handbook for solving problems using modern techniques in an era of large foundation models. 

The book begins with an introduction to general-purpose robotics, how robots are modeled, and how physical intelligence relates to the movement of building artificial general intelligence, while giving you an overview of the current state of the field, its challenges, and where we are headed. The first half of this book delves into defining what the problems in robotics are, how to frame them as AI problems, and the details of how to solve them using modern AI techniques. First, we look at robot perception and sensing to understand how robots perceive their environment, and discuss convolutional networks and vision transformers to solve robotics problems such as segmentation, classification, and detection in two and three dimensions. The book then details how to apply large language and multimodal models for robotics, and how to adapt them to solve reasoning and robot control. Simulation, localization, and mapping and navigation are framed as deep learning problems and discussed with recent research. Lastly, the first part of this book discusses reinforcement learning and control and how robots learn via trial and error and self-play.

The second part of this book is concerned with applications of robotics in specialized contexts. You will develop full stack knowledge by applying the techniques discussed in the first part to real-world use cases. Individual chapters discuss the details of building robots for self-driving, industrial manipulation, and humanoid robots. For each application, you will learn how to design these systems, the prevalent algorithms in research and industry, and how to assess trade-offs for performance and reliability. The book concludes with thoughts on operations, infrastructure, and safety for data-driven robotics, and outlooks for the future of robotics and machine learning.

In summary, this book offers insights into cutting-edge machine learning techniques applied in robotics, along with the challenges encountered during their implementation and practical strategies for overcoming them.

What You Will Learn

• Explore ML applications in robotics, covering perception, control, localization, planning, and end-to-end learning


• Delve into system design, and algorithmic and hardware considerations for building efficient ML-integrated robotics systems


• Discover robotics applications in self-driving, manufacturing, and humanoids and their practical implementations


• Understand how machine learning and robotics benefit current research and organizations

Who This Book Is For

Software and AI engineers eager to learn about robotics, seasoned robotics and mechanical engineers looking to stay at the cutting edge by integrating modern AI, and investors, executives or decision makers seeking insights into this dynamic field

Table of Contents:

Chapter 1: Introduction: General Purpose Robotics.- Chapter 2: Robot Perception: Sensors and Image Processing.- Chapter 3: Robot Perception: 3D Data and Sensor Fusion.- Chapter 4: Foundation Models in Robotics.- Chapter 5: Simulation.- Chapter 6: Mapping, Localization, and Navigation.- Chapter 7: Reinforcement Learning and Control.- Chapter 8: Self Driving Cars.- Chapter 9: Industrial Robotics.- Chapter 10: Humanoid Robotics.- Chapter 11: Data-Driven Robotics in Practice.