News/Tech News

Microsoft Wants to Use ChatGPT to Control Robots through Language

Published on feb 27, 2023

Microsoft Autonomous Systems and Robotics Group researchers have demonstrated how OpenAI’s ChatGPT can be used for robotics applications, including how to design prompts and how to instruct ChatGPT to use specific robotic libraries.

Microsoft’s engineers explain that current robotics is based on a feedback loop between the robot and an engineer who is responsible for coding, observing, and correcting the robot’s behavior.

Microsoft vision can be used to translate a human-language description of the task to be accomplished into code for the robot by using ChatGPT. By doing so, the engineer (in the loop) would be replaced by a non-technical user (on the loop), whose only responsibility would be to provide the original task description in human language, observe the robot, and provide any feedback about its behavior, which ChatGPT would also translate into code for improving it.

Using their experimental approach, Microsoft researchers created a number of case studies that included zero-shot task planning to instruct a drone to inspect the content of a shelf; manipulating objects with robotic arms; searching for a specific object in an environment using object detection and object distance APIs; and others.

All of those cases, ChatGPT was able to generate the code to control the robot as well as to request clarifications when it found user input to be ambiguous, according to Microsoft.

In its efforts to make ChatGPT suitable for robotic applications, Microsoft investigated three main areas: how prompts can be designed to guide ChatGPT, how to utilize APIs and develop new high-level APIs, and how to provide human feedback via text. As a result, these three areas represent the cornerstones of a methodology for utilizing ChatGPT for robotic applications.

By using a simulation or inspecting the code, the user evaluates ChatGPT’s code and provides feedback for ChatGPT to improve.

Using the generated code, a robot can be programmed when the outcome is satisfactory to the user.

As part of the launch, Microsoft is launching a collaborative open-source platform that allows users to share prompting strategies for a variety of robot categories, which currently includes all the prompts and conversations the Microsoft team used during their research. Additionally, they plan to add robotic simulators and interfaces for testing ChatGPT-generated algorithms.

Tech News

Windows Forms Binding Improvements in .NET 7 for MVVM Support img

Windows Forms Binding Improvements in .NET 7 for MVVM Support

Command binding preview features are included in the .NET 7 framework in order to modernize Windows Forms applications…

MicroStream becomes a member of the Eclipse Foundation img

MicroStream becomes a member of the Eclipse Foundation

A Java object-graph persistence framework, MicroStream, has announced its participation in the Eclipse…

Our Latest Blog

Mastering Full Stack Python Development with Django A Comprehensive Guide

Mastering Full Stack Python Development with Django: A Comprehensive Guide

Python is a powerful programming language that has taken the world of web development by...
Read More
Mastering Machine Learning A Beginner's Guide to Python

Mastering Machine Learning: A Beginner’s Guide to Python

Welcome to the world of machine learning! With the ever-increasing demand for artificial intelligence and...
Read More
Unlocking the Power of Data Science with Python A Beginner's Guide

Unlocking the Power of Data Science with Python: A Beginner’s Guide

Data science has become an essential part of many industries today, and Python has become...
Read More

Follow Us


Browse LSET presentations to understand interesting…

Explore Now

Get complete guides to empower yourself academically…

Explore Now

Learn about information technology and business…

Explore Now