Nov 8, 2023

PX4 Simulator

Introduction

Unless you have infinite money and resources, you will need to utilize simulation to avoid costly crashes and speed up your development cycle. Thankfully, the PX4 ecosystem offers multiple routes to leverage the power of simulation while developing your drone applications or hardware. In a recent talk at the PX4 Developer Summit 2023, Godfrey Nolan, President of RIIS LLC. gave attendees a world tour of PX4 simulators and provided a getting-started guide for each of the available options.

An Overview of PX4 Simulators

Why Simulation Matters

Replicating real-world conditions

Whether it’s navigating through challenging terrains, simulating adverse weather conditions, or testing the drone’s response to unforeseen obstacles, simulation offers a safe and cost-effective means of replicating real-world challenges without the need for physical drones. In essence, drone simulation bridges the gap between theory and practice, allowing for comprehensive testing and validation before deployment in the field.

Accurate testing and debugging

The process ensures that drones are fine-tuned for optimal performance, enhancing safety, reliability, and efficiency. Additionally, debugging in a simulated environment allows for precise and controlled examination of potential problems

Customizable environment

Simulations can be tailored to your specific application’s use case and even further allow for the safe testing of otherwise harsh or inaccessible environmental conditions or scenarios.

Development efficiency

Time and $$$$$

Available Simulators

jMavSim
Gazebo
AirSim

SITL and HITL

Software in the Loop (SITL)

SITL refers to a testing pattern that simulates everything, including the hardware components, virtually. Some advantages SITL provides are a faster iteration loop, less complexity to get started, and the ability to isolate software issues without being concerned about false data from hardware failures. SITL tests can potentially fall prey to incorrectly simulated hardware that will fail a real-world test.

Hardware in the Loop (HITL)

HITL puts you one step closer to a real-world implementation by including actual hardware like a flight controller or a sensor. This form of testing is more complex but also more comprehensive and can sometimes surface issues that wouldn’t be seen or can’t be modeled in a SITL test.

A Deeper Look at SITL

This diagram looks complicated, but essentially, what it’s showing is that we are running PX4 and a simulator on our computer. For software development, we usually aren’t using QGroundControl, but instead are using a MAVSDK Android client. Below is a picture of the setup.

Pretty basic, right?

An Example Using MAVSDK for SITL Testing

We’re going to dive into an actual simulator setup later, but let’s get a bird’s eye view of the SITL workflow first.

If you crack open the MAVSDK github repo you will see a couple of examples, but we’re going to focus on the Android client. We’re going to take the Android client and point it at PX4 and jMavSim. Below is an example of a very simple mission. The drone goes from waypoint 1 to waypoint 2 and returns safely to land at waypoint 1.

To get this running, we are going to need the IP address of the phone running the application. Make sure your laptop is on the same WiFi, and use the following MAVLink start command:

HEADLESS=1 make px4_sitl jmavsim

If your laptop and application are connected, you will receive a BUILD SUCCESFUL message, and at the bottom, you can check the IPs to verify.

If You’re on a Mac

The format to get things to work is not all that different and maybe even a little easier.

docker run --rm -it jonasvautherin/px4-gazebo-headless:1.13.3

A Deeper Look at HITL

Hardware in the Loop can be more complex, but it doesn’t have to be hard to get started. Below you can see we’ve utilized The Cube Orange Pixhawk in a very simple setup with our laptop and phone.

TIP: We recommend using MAVSDK client on a physical phone. Make sure the laptop and phone are on the same network.

Trying out a PX4 Simulator

Ensure you have the following:
- Hardware: Windows 11 PC, Android or iOS device, compatible graphics card.
- Software: Windows Subsystem for Linux (WSL), PX4 SDK, Android Studio or Xcode (optional).

Installing Windows Subsystem for Linux (WSL)

Install WSL on your Windows 11 PC:
1. Open PowerShell as an administrator.
2. Run the command wsl --install.
3. Follow the on-screen instructions to complete the installation.

Installing X11 for Linux GUI Support

Set up X11 for Linux graphical support:

Open WSL and update your package list
Install X11
Verify X11

sudo apt update
sudo apt install x11-apps -y
xclock

Setting up jMavSim

Now we are getting closer to the fun part. In order to get jMavSim up and running, first you need to clone the PX4-Autopilot repository.

git clone https://github.com/PX4/PX4-Autopilot.git
bash ./PX4-Autopilot/Tools/setup/ubuntu.sh

Make sure to create an inbound rule for UDP port 14550 and an outbound rule for TCP 4560. To do so, go into the Windows Defender Firewall under Advanced Settings, you can create custom inbound and outbound rules.

Setting up the MAVSDK

Clone the MAVSDK Java repository for Android development:
1. Clone the MAVSDK repository with: git clone https://github.com/mavlink/MAVSDK-Java.
2. Open Android Client in Android Studio
3. Point the client at the jMavSim MAVLink server (“udp://14550“)

Running PX4 Simulator with JMAVSim

Run the following command in your Powershell terminal now:

make px4_sitl jmavsim
pxh> mavlink start -x -u 14550 -f -p -t {{phone IP address}}

We’ve Reached the Simulator!

Options

In our example, we used a MAVSDK app, but you can also use QGroundControl.

Gazebo

Those in the field of robotics will probably know about Gazebo. As a popular robotics simulation platform, it actually serves quite nicely as a drone simulation platform too. It’s robust, flexible, and can even handle complex simulations like drone swarms.

Let’s get Gazebo running now to what it’s like. You’ll probably notice some similarities in the process we used to get jMavSim running.

make px4_sitl gz_x500
pxh> mavlink start -x -u 14550 -f -p -t {{phone IP address}}

Run mavlink start with the above parameters and Gazebo will pop up just like jMavSim did.

Many of the people we’ve worked with have expressed a desire to use an older version of Gazebo. If that’s what you are looking to do there are few more steps.

sudo apt-get remove gz-garden
sudo apt install aptitude
sudo apt autoremove
sudo aptitude install gazebo libgazebo11 libgazebo-dev
make px4_sitl gazebo-classic

AirSim

Microsoft AirSim was a formidable option for simulation in the field, however, it looks like this awesome tool will be sunsetting soon. If you’re still interested in using AirSim, here’s what you’ll have to do.

> ipconfig
export PX4_SIM_HOST_ADDR=172.31.x.x
cd PX4-Autopilot
make px4_sitl_default none_iris
> AirSimNH -settings .\settings.json

Unfortunately, the documentation has not been well maintained, so there are some important issues to note. To get it working, you need to get your WSL and Windows 11 talking. You need to find out what your IP address is for Windows and then export that through the WSL to tell it where it’s supposed to be looking.

Tips and Tricks

To review, here are the main points and pitfalls you need to focus on to get up and running with a PX4 simulator

Update graphics driver for X11
Firewall settings
sudo hwclock-hctosys
mavlink start -x -u 14550 -f -p -t 192.168.7.140
Android Sutdio Eel
$ sudo apt autoremove
AirSim needs Visual C++ & DirectX runtime
IP address show

Onward to Mobile Development

Leveraging Continuous Integration (CI) with Jenkins

Jenkins is an open-source automation server. We are going to use it to set up a modern mobile app development environment for our drone application now that we have our simulators up and running.

Building our App

There are three main tasks we need to complete to build our app.

Start jMavSim
Build APK
Run Espresso Tests

1. Start jMavSim

You may think we can just start up jMavSim like we did before, but it would be running in the wrong context. It is running in the Windows 11 context, and we need to get it to run in the WSL 2 context. To overcome this, we are going to create a simple Jenkins script that runs the same make px4_sitl jMavSim, but now in the right context.

Tip: Avoid using the plug ins within Jenkins. Most times working with the command line will make it easier to debug.

2.Build APK

This is the 1/10 times that we use the plugin. It’s well tried and tested. If you wish though, you can also run the build command from the command line using gradlew build.

3. Espresso Tests

Espresso is an automated UI testing tool provided by Google. It’s relatively simple to use and can allow you to test apps with hundreds of views and branching paths. It does not eliminate the need for user testing but can surface issues in complex apps that pure user testing may not discover. Let’s use Espresso in Jenkins to run some tests on our drone application.

To get this going, use Invoke Gradle script with Use Gradle Wrapper and Make gradlew executable checked.In Android you will have your Main, which is where your source code goes, Android Tests which is where all the Espresso tests will go, and Tests, where all the unit tests will go. Under Tasks, put in connectedCheck, to make sure it completes all the Android Tests, otherwise it will only complete the unit tests.