Resources

In this application, a DEMO SENSE2GOL PULSE board is used to make a blind spot detector for a bicycle that can be mounted on the back of a bicycle to alert the rider about an approaching vehicle, without having the rider to lose concentration on the road in front of him

1. Hardware Setup

XMC4700 Pin Out for Arduino

 

Connecting power source to Arduino

 

 

The pin 4.6 on the XMC board is attached to digital pin 2 of the Arduino and the ground of the XMC™ to the ground of Arduino MKR Wi-Fi 1010. The circuit is complete. The Arduino is powered with a lithium polymer battery of at least 1024 mAh range, you can also power it by connecting it to the JST PH-2 connection area which is more secure. make sure that the lipo has a JST-PH 2 connector attached to it or if you have a Lipo battery of a similar mAh range you can solder the PH-2 connectors to the terminals and use it to power Arduino. You can find JST connectors in online stores like Amazon, Conrad for a cheap price.

JST PH2 Connection

 

JST PH2 Lipo Battery Connection

 

You can also connect the piezoelectric sensor to the digital pin 4 and add extra lines of code so that it will buzz to warn you. This can be used as a redundant warning system in case the notification to the cell phone can not detected by you.

Piezo Circuit

 

tone(buzzer, 1000); // Send 1KHz sound signal...

delay(1000); //...for 1 sec

noTone(buzzer); // Stop sound...

add the above code in line 038 of the IDE code and the buzzer will notify you by buzzing for 1 second, if you want more time, increase the delay from 1000 to 2000 for 2 seconds and 3000 for 3 seconds, and so on.

2. Software Setup

This section describes the software setup for the Arduino IDE and firmware update for Arduino MKR Wi-Fi 1010.

2.1. Arduino IDE

To program the Arduino we need the IDE from Arduino which can be downloaded here https://www.arduino.cc/en/main/software

Arduino IDE Download

 

2.2. Arduino Packages

2.2.1. Boards packages

The board package can be downloaded by going into Arduino IDE and tools--> boards--> board manager and install the SAMD board’s package.

Arduino boards manager

 

2.2.2. Libraries

Arduino boards require several libraries that enable them to communicate with other devices. Go to -- >tools-->manage libraries

Library manager

 

Also install BLE Library, BLYNK, and WIFI NINA

BLE, BLYNK, and WIFI NINA libraries

 

2.2.3. Update Firmware

• We can install the latest firmware to the Arduino board using the updater sketch.
• After selecting the board -->updater sketch-->upload the sketch-->test connection-->update firmware -- >upload Wi-Fi certificates.

Firmware updater

 

Updating the Firmware

 

2.3. Mobile App Configuration

The BLYNK mobile app can be used in this case to communicate with Arduino to retrieve the current state of the sensors. The app can be found on the app stores of android and IOS Here is a link to help you set it up

https://blynk.io/en/getting-started/

Notification widget in the project

 

Widget Box

 

Select the notification widget from the box and place it on the project, take the project key, and mail it to yourself from the project settings.

Project settings

Project start button

 

Backstory

I don't know about you, but my childhood was all about RC cars. As I grew older they were somewhere along the road replaced with video games, where I had the chance to actually (for once in my life mom!) go over the speed limit. Now, instead of just reminiscing on some old memories, in this project we're taking this matter into our own hands. To cut to the chase: in this project, we are going to combine the controls known from video games with my childhood's greatest hobby. (*Drumroll*): we are going to control an RC rover with a Dualshock 4 (PS4) controller.

Of course, this doesn't have to be a rover as well; this project is perfect for reviving your old toy car (the dusty one laying in your basement for more than 3 years because you forgot where you kept its remote control). The main concept revolves around wirelessly controlling a DC motor so feel free to unleash the boundaries to your creativity.

 Fast and Furious 10 Spoiler Alert!

Looking at the Hardware

Let's talk about the hardware we're dealing with. First of all, we have the Raspberry Pi model B. This model has built-in Bluetooth support, which will be useful in connecting with the Playstation controller. Moreover, we will be using the Infineon TLE 94112 #Pi HAT to control our DC motors (ThePi HAT is compatible with any 40 pin layout Raspberry Pi version). This could also be used for LED control. However we will only be controlling motors in this project. For the human component of control, we will use a Dualshock 4 controller. For the rover's skeleton, we will use a custom 3 wheeled lego set using omniwheels and three DC Motors with all their respective wires out ready to be connected.

Taking a closer look at the Pi HAT

Figure 1. TLE94112ES HAT for Raspberry Pi

 

This Pi HAT is essentially a multi-half-bridge circuit, where to keep this as simple as possible we are having control over the direction and amount of power the DC motor receives; in other words: we have the capacity to change the motor's speed and direction. (For further details, please refer to the theory section in the TLE94112 Hackster Protip)

To use the Pi HAT: all you have to do is supply it with power (recommended range between 5.5V and 20V) and connect it to your motors.

To keep this project as simple as possible we will be using the Raspberrypi's GPIO pins to power the Pi HAT, however, if speed is what you desire then I would definitely recommend powering the Pi HAT with something that has more juice.

Pi HAT basic test

Enough introductions let's drive straight in! This section is where we get a hint of how we're actually going to program our project. Mount the Pi HAT onto the Raspberry Pi and connect the HATs power connections as well as a DC motor to outputs 1 and 5.

Figure 2. Basic Test Hardware Setup

 

Onto the raspberry pi itself through a command line:

• Make sure you have the package manager for Python packages by entering:

sudo apt-get install python3-pip

• Install the Infineon multi half-bridge library by entering:

sudo pip3 install multi-half-bridge

• clone Infineon's multi-half-bridge repository by entering:

git clone https://github.com/Infineon/multi-half-bridge.git

At this point, you have everything you need to control the Pi HAT (i.e control the motors), installed onto your Pi. Fortunately, the multi-half-bridge circuit library comes with some pre-written code examples to showcase how to use the product. You can find these examples in the directory:

multi-half-bridge/src/framework/raspberrypi/examples_py/

The same examples are also written in c++ but in this project, I will just stick to python. Let's try running the basicTest.py script by navigating to the examples_py directory and in the command line entering:

sudo python3 basicTest.py

If you followed the previous instructions, then your motor should start as soon as you run the script for three seconds and then stop. Congratulations you're officially done with 30% percent of the project! I know it was hard getting here but this was legitimately the hardest part. What remains is just understanding the code used in the test and then writing your own program.

Basic Test Code Analysis

Before getting into any details on the code specialties, I want you first to think about the code as a compound structure made up of a skeleton and an active component. In the skeleton component: you basically explain your setup to the program, i.e you tell the program which motors are connected to which pins and how do you want these connections to operate. This component involves no active control whatsoever to the motors (i.e it has nothing to do with accelerating, decelerating, moving, or stopping); it is just you basically telling the software how the hardware is assembled. Now onto the active component; this is where all the bells and whistles lie: this is the part where you instruct the motors exactly what to do and when.

"""
# name        basicTest
# author      Infineon Technologies AG
# copyright   2021 Infineon Technologies AG
# brief       This example shows how to switch two half bridge outputs with minimal code.
# details
    It will switch on two outputs (one to Vsup and one to GND), wait 3 seconds
    and switch off both outputs (both to floating state).

# SPDX-License-Identifier: MIT

"""

import multi_half_bridge_py as mhb
from time import sleep

# Create a Tle94112Rpi instance for each motor controller
controller = mhb.Tle94112Rpi()

# Create a Tle94112Motor instance for each connected load
motor = mhb.Tle94112Motor(controller)

# Enable MotorController
# Note: Required to be done before starting to configure the motor
# controller is set to default CS0 pin
controller.begin()

# Clear all errors to start clean
controller.clearErrors()

# Let the library know that a load is connected to HB1 (high side)
# and HB5 (low side).
motor.connect(motor.HIGHSIDE, controller.TLE_HB1)
motor.connect(motor.LOWSIDE, controller.TLE_HB5)
motor.setPwm(motor.LOWSIDE, controller.TLE_NOPWM)

# Initialize the motor
motor.begin()

# Switch the load on
motor.start(255)

sleep(3)

# Switch the load off (set outputs to floating state)
motor.coast()

This is the code for the basic test (it would occupy only 12 lines if it wasn't for the comments). Let's start dissecting!

import multi_half_bridge_py as mhb
from time import sleep

The code starts with importing the Infineon multi-half-bridge library that we have already installed. It also imports the function from the time module (we will see why shortly).

Skeleton component

controller = mhb.Tle94112Rpi()
motor = mhb.Tle94112Motor(controller)

Thereafter a controller structure is instantiated and a motor structure is also instantiated under this controller structure. (This is the first part of the skeleton component that we were talking about earlier.)

Remember, we are just telling the software how the hardware is assembled.

controller.begin()
controller.clearErrors()

the controller is enabled through the begin function and the error Registers which are used for debugging purposes are cleared using the clear errors function. (we will not be using the Pi HAT's debugging features in this project but if you want to have a deeper look into them refer to section 5 of the TLE94112 Hackster Protip).

motor.connect(motor.HIGHSIDE, controller.TLE_HB1)
motor.connect(motor.LOWSIDE, controller.TLE_HB5)
motor.setPwm(motor.LOWSIDE, controller.TLE_NOPWM)
motor.begin()

To conclude the skeleton component, the motor connections are specified in the motor.connect() function, where we specify to the program the terminals that the DC motor is connected to. I personally do not give the HIGHSIDE/LOWSIDE pin selection much thought; all I need to do is have one connection on HIGHSIDE and the other on LOWSIDE because I know that I can change the direction of the motor's motion from the program to my preference. After specifying the motor's connections, all you have to do is tell the program, whether or not you want to use Pulse width modulation (if you are not familiar with this term read the next CRASH COURSE subsection).

CRASH COURSE: PULSEWIDTH MODULATION

 

Duty Cycle Examples (Source: https://dccwiki.com/Pulse_Width_Modulation)

 

Pulse width modulation is switching power ON and OFF with a high frequency with the aim of controlling the power delivered to the devices. The ratio of ON and OFF time is called duty-cycle and this indicates the amount of power delivered to the device (in our case motors). To keep everything simple: it is just a way of quantifying the power that we are giving to the motors, where at 100% duty-cycle the motor is getting full power (we could use this when we want the motor to move at maximum speed), if we want to decelerate then we would reduce the duty-cycle and if we want to accelerate we would increase it.

Back to the basic test: in the basic test the motor is not assigned a pulse width modulation channel: i.e it will either move at full power or it won't move at all (regardless of the direction). If we would want to change that we would change the setPwm function to:

motor.setPwm(motor.LOWSIDE, controller.TLE_PWM1)

Where we would be assigning a pulse width modulation channel, named PWM1 to this motor.

Note: You could assign multiple motors to the same PWM channel.

motor.begin()

Last but not least we end the skeleton component with the begin function, which essentially enables the motor structure and initializes it.

Active Control Component

It's action time.

motor.start(255)

And now for the MVP of this script: we have the start function, which is the function that actually runs the motor. If you're using pulse width modulation then you can set the duty cycle between the range of 0 to 255. Where at 0 the motor is not running at all and 255 is the motor running at full power. To change the direction of the motor all you have to do is add a negative sign right before the value, so if we would want the motor in this example code to take the other direction we would change the command to:

motor.start(-255)
sleep(3)

Remember the sleep function? This line is the one in charge of keeping the motor working for 3 seconds before stopping. We will not be using it in our project but you can always use it in any application, where you want to power your motors differently depending on the amount of time that goes by.

motor.coast()

The coast() function is in charge of stopping the delivery of power to the motor in order to stop the motor, however without any form of active braking.

A closer look at the active control programming component

In the past section, we saw how simple it was to write a program, that runs a motor at full power for three seconds and then coasts it. Let's get a deeper look into other active control functionalities besides moving and coasting.

The active control component consists mainly of the following functions:

Figure 3. Main active control functions

Connecting the Dualshock 4 controller to the Raspberrypi

Now that we're pretty familiar with coding the Pi HAT we could now move forward in our project. Time to start using the PS4 controller; so let's begin by firstly pairing it with our Raspberry Pi. Connecting the PS4 controller with the Pi could be done in multiple ways. To keep it simple here we will choose the easiest one; we will pair with it through the Raspbian graphical user interface.

Initially, we will have to put the controller in pairing mode by simply holding the Playstation button and the share button at the same time. Keep holding until the LED at the back of the controller starts blinking.

Now that the controller is running in pairing mode let's hop into the Raspberry Pi side. Head to the right corner and left-click on the Bluetooth icon. Click on add device and scroll through the list until you find the controller. Click on it and pair. And voila now you're connected!

Using the pyPS4 library

The pyPS4 library allows us to use a paired Dualshock 4 controller as an input device to a python script. To install the library; in a command line enter:

sudo pip3 install pyPS4Controller

Let's take a look at how to use it. Thankfully there is a well-written guide on how to use it but to make a long story short all we have to do is copy this code from the repository and under the new Class define functions that resemble the response we want to be associated with certain events.

from pyPS4Controller.controller import Controller

class MyController(Controller):

def __init__(self, **kwargs):
    Controller.__init__(self, **kwargs)

def on_x_press(self):
    print("Hello world")

def on_x_release(self):
    print("Goodbye world")

controller = MyController(interface="/dev/input/js0", connecting_using_ds4drv=False)
# you can start listening before controller is paired, as long as you pair it within the timeout window
controller.listen(timeout=60)

for example, this code snippet prints Hello world when we press the x button on the controller and prints Goodbye world upon releasing it. A list of all events is also available at the repository itself:

on_x_press
on_x_release
on_triangle_press
on_triangle_release
on_circle_press
on_circle_release
on_square_press
on_square_release
on_L1_press
on_L1_release
on_L2_press
on_L2_release
on_R1_press
on_R1_release
on_R2_press
on_R2_release
on_up_arrow_press
on_up_down_arrow_release
on_down_arrow_press
on_left_arrow_press
on_left_right_arrow_release
on_right_arrow_press
on_L3_up
on_L3_down
on_L3_left
on_L3_right
on_L3_x_at_rest  # L3 joystick is at rest after the joystick was moved and let go off on x axis
on_L3_y_at_rest  # L3 joystick is at rest after the joystick was moved and let go off on y axis
on_L3_press  # L3 joystick is clicked. This event is only detected when connecting without ds4drv
on_L3_release  # L3 joystick is released after the click. This event is only detected when connecting without ds4drv
on_R3_up
on_R3_down
on_R3_left
on_R3_right
on_R3_x_at_rest  # R3 joystick is at rest after the joystick was moved and let go off on x axis
on_R3_y_at_rest  # R3 joystick is at rest after the joystick was moved and let go off on y axis
on_R3_press  # R3 joystick is clicked. This event is only detected when connecting without ds4drv
on_R3_release  # R3 joystick is released after the click. This event is only detected when connecting without ds4drv
on_options_press
on_options_release
on_share_press  # this event is only detected when connecting without ds4drv
on_share_release  # this event is only detected when connecting without ds4drv
on_playstation_button_press  # this event is only detected when connecting without ds4drv
on_playstation_button_release  # this event is only detected when connecting without ds4drv

Let's try and use both code snippets to come up with our own code in order to have a better grasp of how to use the library and tailor it to our own application. Let's say we want to print some messages on the screen if we press some buttons on the controller. For example, let's write a script that prints "going up" if we press the forwards' arrow key and prints "not going up anymore" if we release the upwards arrow key:

• The first step would be to copy the hello world code from above and then remove the x button functions like this:

from pyPS4Controller.controller import Controller

class MyController(Controller):

def __init__(self, **kwargs):
    Controller.__init__(self, **kwargs)

controller = MyController(interface="/dev/input/js0", connecting_using_ds4drv=False)
# you can start listening before controller is paired, as long as you pair it within the timeout window
controller.listen(timeout=60)

• The second step would be to copy (from the code snippet above) the events that we want our program to respond to:

first being the up arrow button being pressed on_up_arrow_press we then use it to create our own function as follows

from pyPS4Controller.controller import Controller

class MyController(Controller):

def __init__(self, **kwargs):
    Controller.__init__(self, **kwargs)

def on_up_arrow_press(self):
    print("going up")

controller = MyController(interface="/dev/input/js0", connecting_using_ds4drv=False)
controller.listen(timeout=60)

Now let's add the other function for releasing the up-arrow: we will do the exact same process by copying the event from the snippet and then using it to write the function in your code

Event: on_up_down_arrow_release

from pyPS4Controller.controller import Controller

class MyController(Controller):

    def __init__(self, **kwargs):
        Controller.__init__(self, **kwargs)

    def on_up_arrow_press(self):
        print("going up")

    def on_up_down_arrow_release(self):
        print("not going up anymore")

controller = MyController(interface="/dev/input/js0", connecting_using_ds4drv=False)
controller.listen(timeout=60)

To run the script you just wrote, simply navigate in a command line to the directory containing your script and then enter

sudo python3 "nameOfYourScript".py

Rover Software Breakdown

In the following part we use the 12 line basicTest code and tailor it to our application. (You most probably do not have the exact same setup I am having, this should not matter at all at the end of the day you will just modify, add or remove functionalities to the code, the structure is essentially the same)

import multi_half_bridge_py as mhb
from pyPS4Controller.controller import Controller

The only two libraries we need for this particular project are the ones that we installed, which are the Infineon multi-half-bridge library and the pyPS4Controller.

Skeleton Code Component:

Instances:

controller = mhb.Tle94112Rpi()
motor1 = mhb.Tle94112Motor(controller)
motor2 = mhb.Tle94112Motor(controller)
motor3 = mhb.Tle94112Motor(controller)

Analog to the basic test we first instantiate a controller and then a motor In my setup, the lego kit came with three motors so I will make three motor instances and name them motor1, motor2, and motor3.

Controller configuration:

controller.begin()
controller.clearErrors()

Before doing any motor configuration it is critical to enable the controller and clear its error registers.

Motor configuration:

I am using three different motors in this setup so I will be configuring each one of them on their own

Motor1 configuration:

motor1.connect(motor1.HIGHSIDE, controller.TLE_HB1)
motor1.connect(motor1.LOWSIDE,  controller.TLE_HB5)
motor1.setPwm(motor1.HIGHSIDE, controller.TLE_PWM1)
motor1.begin()

This motor is physically connected to outputs 1 and 5 so all I did was specify that in the connect function. The choice of which side to be used as HIGHSIDE and which is to be used for LOWSIDE was completely arbitrary (because I get to control the motor's direction through code so I did not really give it that much thought). I am not using pulse width modulation because I want to give the motor as much power possible, however, I did assign the motor a pulse width modulation channel just in case I change my mind in the future. The last step in the motor configuration is to enable it.

Motor 2 Configuration:

motor2.connect(motor2.HIGHSIDE, controller.TLE_HB7)
motor2.connect(motor2.LOWSIDE,  controller.TLE_HB9)
motor2.setPwm(motor2.HIGHSIDE, controller.TLE_PWM2)
motor2.begin()

The second and third motor configurations were completely analog to the configuration of the first, the only difference are the outputs these motors are connected to.

Motor 3 Configuration:

motor3.connect(motor2.HIGHSIDE, controller.TLE_HB6)
motor3.connect(motor2.LOWSIDE,  controller.TLE_HB4)
motor2.setPwm(motor3.HIGHSIDE, controller.TLE_PWM3)
motor3.begin()

This concludes the skeleton component of the code.

Active Control Code Component:

I try to keep my code as organized and as readable as possible through the use of functions. For example, instead of asking motors 1 and 3 to start every time I want to ask the program to make the rover move forward. I will simply write a function that does that and name it move_forward(). The next subsection is purely dedicated to making such functions.

Custom Functions:

Before getting into the function I want to give a hint on how I want to control my setup. My setup consists of three motors, I want to dedicate two of them to moving forward and backward (namely motors 1 and 3). Motor2 will be in charge of turning left or right.

def motors_stop():
    motor3.stop(255)
    motor2.stop(255)
    motor1.stop(255)

As the name of the function suggests this custom function breaks all three motors to stop the rover's motion. The braking is performed with maximum force. ( If you want to implement softer braking feel free to reduce the stop functions input to something below 255)

def move_forward():
    motor1.start(255)
    motor3.start(-255)

To move my rover forwards all I have to do is move motor1 and motor3. (If I did not want motor3 to have a contradicting direction to the motion of motor 1 I could have switched the physical pin connections of motor3 and that would have done the job).

def move_backward():
    motor1.start(-255)
    motor3.start(255)

Analog to the move_forward() function, the move_backwards() is the exact same with just different motor motion directions.

def turn_left():
    motor2.start(255)

Motor2 is the only motor in charge of turning the rover. Note that the rover is not a finite state machine: i.e it could turn left while moving forward or backward.

def turn_right():
    motor2.start(-255)

The turn_right() function does exactly what the turn_left function does, but with a different direction of motor motion.

The rest of the functions are kind of intuitive.

def coast_vertical():
    motor1.coast()
    motor3.coast()
def coast_horizontal():
    motor2.coast()
def rotate_clockwise():
    motor1.start(-255)
    motor2.start(-255)
    motor3.start(-255)
def rotate_anticlockwise():
    motor1.start(255)
    motor2.start(255)
    motor3.start(255)
def motors_coast():
    motor1.coast()
    motor3.coast()
    motor2.coast()

Dualshock 4 Controller Component:

Now for the final part of this project's code. We will copy the "hello world" code snippet and add the events we want from the list above and associate them with the functions we just created.

class MyController(Controller):

    def __init__(self, **kwargs):
        Controller.__init__(self, **kwargs)
    def on_x_press(self):
        motors_stop()
    def on_up_arrow_press(self):
        move_forward()
    def on_left_arrow_press(self):
        turn_left()
    def on_right_arrow_press(self):
        turn_right()
    def on_down_arrow_press(self):
        move_backward()
    def on_up_down_arrow_release(self):
        coast_vertical()
    def on_left_right_arrow_release(self):
        coast_horizontal()
    def on_R1_press(self):
        rotate_clockwise()
    def on_R1_release(self):
        motors_coast()
    def on_L1_press(self):
        rotate_anticlockwise()
    def on_L1_release(self):
        motors_coast()
    def on_R2_press(self, value):
        move_forward()
        return super().on_R2_press(value)
    def on_R2_release(self):
        coast_vertical()
        return super().on_R2_release()
    def on_L2_press(self, value):
        move_backward()
        return super().on_L2_press(value)
    def on_L2_release(self):
        coast_vertical()
        return super().on_L2_release()

controller = MyController(interface="/dev/input/js0", connecting_using_ds4drv=False)
controller.listen(timeout=60)

Annnnnd Voila! Now we're done!

For my rover to work I need to run this python script, but let's say we want to experiment with my driving skills outside. I can't go everywhere with my monitor, keyboard, and mouse. It is more realistic to make the Raspberry Pi run our script on startup.

Running our script on Startup (Autostart)

If you've reached this stage, you have a functioning.py file.

Open a command line and enter:

sudo nano /etc/profile

scroll to the end of the window and enter

sudo python3 "the directories leading to your file"/"yourfile".py

in my case it was

sudo python3 /home/pi/Desktop/project_christmas.py

Press ctrl+o and enter to save and then ctrl+x to exit and then you're done. Now your Raspberry Pi will run your script as soon as it starts. Your DualShock 4 will connect automatically to the Raspberry Pi if it is paired as well.

And that's all there is to the project. I hope that you now possess all the knowledge you need to revive that old toy car of yours. Feel free to check out the other projects as well if you liked this one. In the meantime drive safe.

We built this self-stabilizing robot as a base setup to show how robots will interact with their surroundings based on different sensor types.

In this example you can see the robot equipped with a 24GHz radar and an ultrasonic sensor/

The XMC4700 offers several unused communication interfaces to add different kinds of sensors to your project.

Interested in building your own robot? Find instructions on how to build it HERE.

1) Mechanics

Start your build with the Top PlateXMC4700 or Top Plate XMC4700 electric wire. The electric wire version offers a surrounding notch for an electroluminescent wire.

Top Plate XMC4700 electric wire

 

This part fits a DCDC voltage converter (WE 173010578) from 12V battery voltage to 5V for our controller and a possible electroluminescence inverter. Further more, it keeps your cables in place and holds the robots main switch.

 

Top Plate XMC4700 electric wire assembled

 

Screw the XMC4700 Relax Kit on top of it and stack the Motorshield on top of the microcontroller. The Motorshield offers an open area which is great to solder the IMU(inertial measurement unit) sensor right on top.

Controller, sensor and power board

 

Continue with printing and assembly of the lower part - the chassis. It holds the two motors as well as the optional electroluminescence inverter.

 

Chassis

 

 

Chassis mounted

 

Mount the top plate to the chassis using M3 spacers that match your batterie height.

 

 

To connect wheels to the motors we used M5 spacers which we drilled out and added a thread to fix the wheel axle to the motor axle.

Wheel axle

 

To attach the wheels to the wheel axle we printed another adaptor:

Wheel adaptor

That's it - these are all parts you need to assemble your self-stabilized robot:

 

 

Full assembly

2) Electronics

This section gives you a short intro into the used electronic components. For a more detailed look check the supplier webpages.

• Stepper Motors (Nema17)

Steppers are important whenever you need to set an exact position. For such kind of robots that's not a must have but we used them to drive the robot back to its "home position" which is the starting point after you pushed it away.

Stepper Motor Nema17

 

• Adafruit Motor/Stepper/Servo Shield for Arduino v2 

This is the power stage to drive the motors. It's using TB6612 MOSFET drivers with 1.2A current capability per channel and comes with a handy Arduino library to drive stepper motors.

Adafruit Motor Control Shield

 

• Elwire 

We use an electroluminescence (EL) wire to show the mode of our robot. It is a wire that you feed with AC voltage of about 100V and 2500Hz. This will let it glow all over its surface.

 

EL wire

 

• ELInverter 

To create the AC voltage for our EL wire an EL inverter is needed: Input 5V DC -> Output 100V AC

EL inverter

 

 

• IMUBNO055 

This sensor will help us to know the position of our robot and if it is about to fall to one side. For this we use: the Absolute Orientation (Euler Vector, 100Hz) Three axis orientation data based on a 360° sphere and the Angular Velocity Vector (100Hz) Three axis of 'rotation speed' in rad/s.

BNO055 breakout board

 

• XMC4700RelaxKit 
  

This is the main controller and therefore the brain of our robot. It's featuring the Arduino Uno pinout and is enabled to be used in the Arduino IDE:

XMC 4700 Relax Kit lite

3) Motor control

You can run the motors in four modes: Microstep (1/8), Interleave, Single and Double.

Microstep (1/8) performs a smooth handover from one coil of the stepper to the other. This is the smoothest but also the slowest mode:

Microstepping

 

Interleave is only introducing a half step (1:2) during the handover from one coil to the other. This mode is faster than microstepping but not that smooth.

 

Double and Singlestep(1:1) are fulfilling a full step. While "single" mode is using only one coil as active one, "double" is using two active coils and has therefore more torque. These are the fastest modes but are also a bit rough at low speed.

The different speed ranges can be better seen in this graph:

 

 

To make full use of the stepper's capabilities in the different modes we use all of them by handing over from one to another:

 

Unfortunately this is a nightmare for any following control algorithm as for a different change of the control signal (x-axis) you see a different increase of speed value (y-axis).Thats why we linearized the motor control to one get a linear relation of control signal to speed increase:

 

In Arduino we could use this mapping function to do so:

if(Regelausgang_Lage >= -160 && Regelausgang_Lage <= 160){
        Schrittart=1; // Microschritt (beide Richtungen)
        Speed = map (Regelausgang_Lage, -160,160, -400,400);
      }
      else if (Regelausgang_Lage >=161 && Regelausgang_Lage <= 650)  {
        Schrittart=2; //Interleave (positiv)
        Speed = map (Regelausgang_Lage, 161,650, 90,400);
      }
        else if ( Regelausgang_Lage >= -650 && Regelausgang_Lage <=-161) {
        Schrittart=2; //Interleave (negativ)
        Speed = map (Regelausgang_Lage, -161,-650, -90,-400); 
       }
      if (Regelausgang_Lage >=651)  {
        Schrittart=3; //Double  (positiv)
        Speed = map (Regelausgang_Lage, 651,1300, 200,400);
       }                    
       if ( Regelausgang_Lage <= -651) {
        Schrittart=3; //Double (negativ)
        Speed = map (Regelausgang_Lage, -651,-1300, -200,-400); 
       }                    
    }

4) Control algorithms

A cascade control is used to control the system. This means we use four subordinate control loops. This topology offers the advantage of simplifying a complex system using several subsystems. Each subsystem is regulated with a specific controller. Disturbances that act on an internal subsystem are adjusted before they can influence the external control loops. Since each subsystem requires a certain amount of time to implement manipulated variables, the higher-level control systems must be slower by a factor of 4. The system can be divided into four subsystems:

• Lageregelung adjusts the angle of the robot and prevents the robot from falling over. We use a PID controller manipulating the wheelspeed for this.
• Geschwindigkeitsregelung controls the speed of the robot. We use another PID controller manipulating the angle of the robot for this.
• Positionsregelung controls the position of the robot in relation to its starting point. We use a P controller to manipulate the speed of the robot.
• Orientierungsregelung controls the direction in which the robot moves. We use another P controller for this which acts on the wheelspeeds.

 

As it's time critical to run these control algorithms they are triggered by timer interrupts:

 

// Interrupt Setup
//------------------------------------------------------------------------------------------------
// Setup Postition Interrupt
//------------------------------------------------------------------------------------------------
  XMC_CCU4_SLICE_COMPARE_CONFIG_t pwm_config = {0};
            pwm_config.passive_level = XMC_CCU4_SLICE_OUTPUT_PASSIVE_LEVEL_HIGH;
            pwm_config.prescaler_initval = XMC_CCU4_SLICE_PRESCALER_32768;
            XMC_CCU4_Init(CCU41, XMC_CCU4_SLICE_MCMS_ACTION_TRANSFER_PR_CR);
            XMC_CCU4_SLICE_CompareInit(CCU41_CC43, &pwm_config);
            XMC_CCU4_EnableClock(CCU41, 3);
            XMC_CCU4_SLICE_SetTimerPeriodMatch(CCU41_CC43, 704); // Adjust last Value or Prescaler
 /* Enable compare match and period match events */
  XMC_CCU4_SLICE_EnableEvent(CCU41_CC43, XMC_CCU4_SLICE_IRQ_ID_PERIOD_MATCH);
/* Connect period match event to SR0 */
  XMC_CCU4_SLICE_SetInterruptNode(CCU41_CC43, XMC_CCU4_SLICE_IRQ_ID_PERIOD_MATCH, XMC_CCU4_SLICE_SR_ID_0);
  /* Configure NVIC */
  /* Set priority */
  NVIC_SetPriority(CCU41_0_IRQn, 10U);
  /* Enable IRQ */
  NVIC_EnableIRQ(CCU41_0_IRQn); 
        XMC_CCU4_EnableShadowTransfer(CCU41, (CCU4_GCSS_S0SE_Msk << (4 * 3)));
        XMC_CCU4_SLICE_StartTimer(CCU41_CC43);

//------------------------------------------------------------------------------------------------
// Setup Geschwindigkeit Interrupt
//------------------------------------------------------------------------------------------------
XMC_CCU4_Init(CCU42, XMC_CCU4_SLICE_MCMS_ACTION_TRANSFER_PR_CR);
            XMC_CCU4_SLICE_CompareInit(CCU42_CC43, &pwm_config);
            XMC_CCU4_EnableClock(CCU42, 3);
            XMC_CCU4_SLICE_SetTimerPeriodMatch(CCU42_CC43, 176); // Adjust last Value or Prescaler 176->25Hz 220->20Hz
 /* Enable compare match and period match events */
  XMC_CCU4_SLICE_EnableEvent(CCU42_CC43, XMC_CCU4_SLICE_IRQ_ID_PERIOD_MATCH);
/* Connect period match event to SR0 */
  XMC_CCU4_SLICE_SetInterruptNode(CCU42_CC43, XMC_CCU4_SLICE_IRQ_ID_PERIOD_MATCH, XMC_CCU4_SLICE_SR_ID_0);
  /* Configure NVIC */
  /* Set priority */
  NVIC_SetPriority(CCU42_0_IRQn, 10U);
  /* Enable IRQ */
  NVIC_EnableIRQ(CCU42_0_IRQn); 
        XMC_CCU4_EnableShadowTransfer(CCU42, (CCU4_GCSS_S0SE_Msk << (4 * 3)));
        XMC_CCU4_SLICE_StartTimer(CCU42_CC43);
        
//------------------------------------------------------------------------------------------------
// Setup Sensor Interrupt
//------------------------------------------------------------------------------------------------
XMC_CCU4_Init(CCU43, XMC_CCU4_SLICE_MCMS_ACTION_TRANSFER_PR_CR);
            XMC_CCU4_SLICE_CompareInit(CCU43_CC43, &pwm_config);
            XMC_CCU4_EnableClock(CCU43, 3);
            XMC_CCU4_SLICE_SetTimerPeriodMatch(CCU43_CC43, 43); // Adjust last Value or Prescaler 
 /* Enable compare match and period match events */
  XMC_CCU4_SLICE_EnableEvent(CCU43_CC43, XMC_CCU4_SLICE_IRQ_ID_PERIOD_MATCH);
/* Connect period match event to SR0 */
  XMC_CCU4_SLICE_SetInterruptNode(CCU43_CC43, XMC_CCU4_SLICE_IRQ_ID_PERIOD_MATCH, XMC_CCU4_SLICE_SR_ID_0);
  /* Configure NVIC */
  /* Set priority */
  NVIC_SetPriority(CCU43_0_IRQn, 10U);
  /* Enable IRQ */
  NVIC_EnableIRQ(CCU43_0_IRQn); 
       XMC_CCU4_EnableShadowTransfer(CCU43, (CCU4_GCSS_S0SE_Msk << (4 * 3)));
       XMC_CCU4_SLICE_StartTimer(CCU43_CC43

4.1 Position control - " Lageregelung"

Remember - we adjust the angle of the robot and prevent the robot from falling over. In order to set a certain angle, the motor axis is moved in such a way that it is either below the center of gravity (robot is standing) or a moment equilibrium is established from accelerating moments (robot moves). The position control loop is called via a timer interrupt with 100 Hz. It can be described as following:

PID controller:

 

 

P - the proportional share is multiplying our angle offset with a constant factor "Kp"You can imagine it as a lenght of an lever pushing against the offset

I - the integral part term "Ki" increases action in relation not only to the error but also the time for which it has persisted. We simply add the current offset to an "integrator" value.

D- the derivative term "Kd" does not consider the error, but the rate of change of error. Luckily our sensor is already measuring this by measuring the Angular Velocity Vector. The D part is acting like a damper to the system.

//------------------------------------------------------------------------------------------------
// PID Lageregler 
//------------------------------------------------------------------------------------------------
    Winkelabweichung  = (double(euler.z())-sollWinkel);  
    integrator_Lage   = constrain (integrator_Lage + Winkelabweichung,-integrator_LageMAX, integrator_LageMAX); // I Anteil der Lageregelung wird begrenzt
    Regelausgang_Lage = constrain (KP_Lage*Winkelabweichung\
                                 + KI_Lage * integrator_Lage\
                                 + KD_Lage*double(gyroscope.x()),-1300, 1300);                                  // Begrenzung auf Schrittmotorkennlinie

4.1.1 Tuning the position controller

We tune the controller by adjusting the parameters Kp, Ki, Kd of the PID control algorithm. The method used is called "Ziegler–Nichols tuning method". Keep Ki and Kd at 0 while increasing Kp until the system reached the limit of stability and thus gets in a harmonic oscillation. The value found in out case this way is referred to as Kp_krit=200 and showed a period of 300ms.

Harminic oscillation

 

Use the following table to calculate Kd = Kp*Tv and Ki = Kp/Tn

Ziegler–Nichols table

 

Note: Ki would be 800 in our case but as we add 100 times per second we bring it down to 8.

 

4.2 Speed control - " Geschwindigkeitsregelung "

The speed controller controls the speed of the robot. We use another PID controller manipulating the angle of the robot for this. The position control loop is called via a timer interrupt with 25Hz. It can be described as following:

Speed control

The wheelspeed of the robot is given by our "Lageregelung" control loop. We additionaly smooth this value with a low pass filter.Then we also calculate the acceleration by taking the actual and last speed value as well as the time difference between measuring them of 40ms (25Hz).

 

PID controller:

PID speed control

 

 

void get_sollsollWinkel() {
//--------------------------------------------------------------------------------------------------------------------------------------------------------------
// Tiefpassfilter
//--------------------------------------------------------------------------------------------------------------------------------------------------------------
      Regelausgang_Lage_5 = Regelausgang_Lage_4;
      Regelausgang_Lage_4 = Regelausgang_Lage_3;
      Regelausgang_Lage_3 = Regelausgang_Lage_2;
      Regelausgang_Lage_2 = Regelausgang_Lage_1;
      Regelausgang_Lage_1 = Regelausgang_Lage;

      Regelausgang_Lage_filter = (Regelausgang_Lage_1 \
                                + Regelausgang_Lage_2 \
                                + Regelausgang_Lage_3 \
                                + Regelausgang_Lage_4 \
                                + Regelausgang_Lage_5)/5;   
        
//--------------------------------------------------------------------------------------------------------------------------------------------------------------
// Bilden der Geschwindigkeitsableitung
//--------------------------------------------------------------------------------------------------------------------------------------------------------------
    
      d_Geschwindigkeit = (Regelausgang_Lage_filter - Regelausgang_Lage_filter_alt)\
                          /Ta_Geschwindigkeitsregler;
      Regelausgang_Lage_filter_alt = Regelausgang_Lage_filter;
        
//--------------------------------------------------------------------------------------------------------------------------------------------------------------
// Geschwindigkeitsregler -> Ausgang sollWinkel
//--------------------------------------------------------------------------------------------------------------------------------------------------------------

     Geschwindigkeitsabweichung = sollGeschwindigkeit - Regelausgang_Lage_filter;

     if (abs(Positionsabweichung) <= 15 ) {  //Regelparameter bei geringer Positionsabweichung
           KP_geschw = 0.0045;               // Proportionalanteil des Geschwindigkeitsreglers
           KI_geschw = 0.0009;               // Integralanteil des Geschwindigkeitsreglers  
           KD_geschw = 0.000;                // Differentianteil des Geschwindigkeitsreglers  
     }     
     else{                                   //Regelparameter bei größerer Positionsabweichung
           KP_geschw = 0.0090;               // Proportionalanteil des Geschwindigkeitsreglers
           KI_geschw = 0.0007;               // Integralanteil des Geschwindigkeitsreglers  
           KD_geschw = 0.0002;               // Differentianteil des Geschwindigkeitsreglers  
     }
           
           integrator_geschw = constrain(integrator_geschw + Geschwindigkeitsabweichung,\
                               -sollWinkelMAX/KI_geschw, sollWinkelMAX/KI_geschw);   //I Anteil wird begrenzt
           sollWinkel        = constrain(KP_geschw * Geschwindigkeitsabweichung \   
                                       + KI_geschw * integrator_geschw \                           
                                       + KD_geschw * d_Geschwindigkeit,\                                  
                                       -sollWinkelMAX, sollWinkelMAX);               //Winkelbegrenzung
  }

4.2.1 Tuning the speed controller

Tuning of the speed controller is the same as for the position controller.With PID we still see oscillations for low speed offsets. Switching to an PI controller for low speed offsets solved this issue. In this we decreased Kp and increased Ki a bit. This is a bit of trying out 

4.3 Position control - " Positionsregelung"

This controller controls the position of the robot in regard to its initial position. The position is known by counting the rotation steps of the steppers. A simple P controller acting on the speed of the robot is sufficient for this:

 

We tried different values for Kp. At Kp=4 the best controller performance was shown.

 

void get_sollGeschwindigkeit() {
//--------------------------------------------------------------------------------------------------------------------------------------------------------------
// Positionsregler -> Ausgang sollGeschwindigkeit
//--------------------------------------------------------------------------------------------------------------------------------------------------------------      
 Positionsabweichung  = (sollPosition - pos_aktuell);
 
   //+++Ausweichen+++    
       if (Distanz_X1 <= 15 && Distanz_X1 > 0) {      //Einschaltschwelle "Hand folgen"
        KP_dist_X1 = 75;
        KP_pos = 0;
        EL_threshold = 1;                             //EL-Wire leuchtet
       }                                              
   //+++Positionsregler+++                            
       else if (Distanz_X1 > 30 || Distanz_X1 == 0) { //Ausschaltschwelle "Hand folgen"
         KP_dist_X1 = 0;
         KP_pos = 4;
         EL_threshold = constrain(abs(Positionsabweichung),0,300)* -0.15 + 50;
                                                     //EL-Wire blinkt(2-10Hz)              
       }
       else {/*Hysterese*/}

        sollGeschwindigkeit  =  constrain (KP_pos*Positionsabweichung  \
                                         + KP_dist_X1*(-15+Distanz_X1),\
                                -GeschwindigkeitMAX, GeschwindigkeitMAX);      

}

Motivation

I don't know about you, but as a gamer, I grew up around multiple versions of Joysticks. And if you're a curious person like me, you're probably wondering how these Joysticks work. How does a controller sense that I moved the Joystick in a certain direction?

Not a gamer? Not interested in how most Joysticks work? Your loss. But at least you drive cars or are interested in them? Do you sometimes feel that you need to crank the volume up or change the track with help of the infotainment system or maybe tilt the side-view mirrors? Don't you wanna know a possible way how the car could sense, that you rotated any of those Knobs clockwise or anti-clockwise and act accordingly?

You got lost in the music and forgot which gear you were on? What do you do? You look in front of you and the number is there on the dashboard (well, assuming you drive a manual car, if it's automatic and you can't tell if you're on R or D or N, you might have a bigger problem) But how does the car sense the position of the gear-stick?

Most likely you find one of our Infineon 3D magnetic sensors:

• TLV493D-A1B6
• TLE493D-W2B6
• TLI493D-W2BW (crazy small)
• TLE493D-P2B6

1) "Say my Name."

The following picture showcases what the name of the IC stands for. For example:

Fig 1.1) Meaning of the names

 

I know you people love numbers and specs, so instead of reading the whole data sheet, have a look at the tables below underlining some important specs of the three 3D-Sensors.

 

1 / 2 • Fig 1.2) Industrial and Consumer: Notice the difference in Update rates between the two?

 

2/2 Fig 1.3) Automotive

Are these specs not enough for you? Why? Are you doing a Ph.D. in 3D-Sensors? Anyway, here are the links to their datasheets, if you need some more info:

• TLI493D-W2BW
• TLE493D-W2B6
• TLV493D-A1B6
• TLE493D-P2B6

2) Crash Course Hall Effect

Most Hall Effect Sensors are devices that are activated by an external magnetic field. We know that a magnetic field has two important characteristics flux density, (B) and polarity (North and South Poles).

The output signal from a Hall effect sensor is the function of magnetic field density around the device. When the magnetic flux density around the sensor exceeds a certain pre-set threshold, the sensor detects it and generates an output voltage called the Hall Voltage, V_H. Consider the diagram below.

Fig 2.1) Hall Effect ; src:https://www.electronics-tutorials.ws/electromagnetism/hall-effect.html

 

When the device is placed within a magnetic field, the magnetic flux lines exert a force on the semiconductor material which deflects the charge carriers, electrons and holes, to either side of the semiconductor slab. This movement of charge carriers is a result of the magnetic force they experience passing through the semiconductor material.

As these electrons and holes move sideward a potential difference is produced between the two sides of the semiconductor material by the build-up of these charge carriers.

This is the basic concept of how most Hall effect sensors work. For the wisecrackers who want to know more about how the Hall Effect works, check out this article.

The 3D magnetic sensors use three of these Hall effect sensors in one package - thus being able to detect magnetic fields in all three dimensions.

 

3) Fast prototyping with Infineon Shield2Go

You find TLI493D-W2BW, TLE493D-W2B6, and TLV493D-A1B6 on Infineon's 2Go Shields. TLE493D-P2B6 is currently only available as Kit2Go, therefore, this board is Plug & Play ready and no additional Microcontroller is needed.

Shield2Go boards feature a sensor on a breakout part followed by a standardized form factor and pinout defined by Infineon. All Shields2GOs are compatible with Arduino and the Arduino IDE. They also follow a Plug & Play principle i.e you can mount them on an XMC2Go microcontroller board and you're ready2Go. If you're using an Arduino Board or an XMC Board (using the Arduino Uno Pinout) you can use the MyIoT Adapter. (Click here to find out how to use the MyIoT Adapter)

Find the Shield2Go board layouts and pinouts below:

1 / 3

 

2/3    S2GO_3D_TLE493DW2B6-A0

3/3 

CAUTION: DO NOT ATTEMPT TO SOMEHOW CONNECT THESE SHIELDS2GO DIRECTLY WITH AN ARDUINO BOARD OR ANOTHER BOARD THAT OPERATES WITH 5V.THIS WILL CAUSE DAMAGE TO THE SHIELDS2GO, BECAUSE THEY HAVE A 3V3 OPERATIONAL VOLTAGE. SO MAKE SURE TO APPLY PROPER LEVEL-SHIFTING. (TheMyIoT Adapter does that automatically).

4) Available Add-ons

To show you the functions of the 3D Sense Shields2Go, while also using their dedicated libraries in the Arduino IDE later on in section 5, Infineon gets you five Add-ons to test and showcase the various, potential, typical applications of these 3D-Sensors:

• Rotating Knob with push function
• Joystick
• Small Joystick for the W2BW package
• Linear Slider
• Direction Indicator
• Power Drill trigger
• Tilting knob
• Out Of Shaft

 

1 / 8 • ROTATE KNOB 3D 2 GO KIT

 

2/8  JOYSTICK FOR 3D 2 GO KIT

3/8 3D PLAY2GO KIT

4/8  LINEAR-SLIDER 2GO

5/8   DIR_INDICATOR2GO

6/8  POWER_DRILL2GO

7/8   MINI_CONTROL2GO

8/8  OUT OF SHAFT FOR 3D 2 GO

Now if you want to save some money clean up that dusty, rusty 3D printer of yours. You can also print 3 of these add-ons yourself. Find out how in the next section.

4) Customizable and DIY Add-ons

Do you choose to print them out or integrate them into your custom design?

Head over to the attachments section and download the STL files:

• Joystick
• Rotating Knob
• Linear Slider

You can also find these here.

Then you can pick a slicer of your choice. I used PrusaSlicer but it doesn't really matter which one you choose. If you're using PrusaSlicer then you can also use the following printing presets; they turned out to work fine:

Fig 4.1) PrusaSlicer printing presets - linear slider as an example

 

As you can see Fig 4.1 was for the Linear slider parts, but all the other parts for the other add-ons had basically the same printing presets.

Finished with printing? Great, you're basically done. Now onto the assembly.

Starting off with the linear slider, because it's a bit more complicated than the other ones.

1 / 3 • Fig 4.1) Glue the magnet here

 

2/3   Fig 4.2) Sliding the bar into place

3/3   Fig 4.3) Assembly of linear slider

• You need to superglue the magnet (5x5mm neodym) into the hole of 3d_Linear_2_9. (take a look at Fig 4.1)
• Next, slide the slidingbar into the rectangular hole of 3d_Linear_2_9 as shown in Fig4.2.
• Then use an M1 Screw (length: 10mm) to screw the slidingbar to the 3d_Linear_1_6 as shown in Fig 4.3.
• Finally, slide the 3d_linear_4 into the 3d_linear_1_6 (See Fig 4.3)
• Optionally, you can also put the limiters (3d_Linear_3_1) on top of your sliding bar.

After the assembly, you can screw it on top of your 3DSense Shield2Go with an M2 Screw (length: 10mm):

 

1 / 2 • DIY Linear Slider

 

2/2  DIY Linear Slider on top of a 3DSense Shield2Go plugged into an XMC2Go

The other add-ons are pretty easy, you just have to:

• Glue the magnet for the joystick into the hole of the 3d_KugelstabSTL or the magnet for the rotating knob into the hole of the 3D_Drehknopf.STL (follow the red arrow below).
• For the pushdown function of the Knob, you'd have to insert a pogo pin first inside its dedicated hole of the 3D_Halter.STL as shown in Fig 4.4 below.
• After the magnets are in place, you take these models and just push them into Kugelhalter_3D_810_2.STL or 3D_Halter.STL respectively until it's in a stable state (follow the blue line below).

Use an M2 screw (length: 10mm) to attach the add-on on top of your Shield2Go.

 

1 / 3 • Fig 4.4) Pogo pin insertion

 

2/3  Fig 4.5) Joystick assembly

3/3  Fig 4.6) Rotating Knob assembly

Et voilà, you have your other two own Addons2Go:

 

1 / 4 • DIY Joystick Addon2Go

 

2/4  DIY Rotation Knob Addon2Go

3/4 DIY Joystick Addon2Go on top of a 3DSense Shield2Go mounted on an XMC2Go

4/4   DIY Rotation Knob Addon2Go on top of a 3DSense Shield2Go mounted on an XMC2Go

5) "We want the numbers, Mason!"

I am going to use the MyIoT Adapter stacked on an Arduino Mega 2560 with the Shields2Go plugged into the MyIoT Adapter:

My Setup

 

 

HINT: If you're using an XMC2Go or the Kit2Go variation of the 3D sensors, you might also want to try out this awesome app Infineon has developed (Take a look at the GIF-teasers below). Head over to the Attachments and you'll find a .zip folder named "3D Magnetic Sensors 2Go - GUI", download and install the software. This is basically a GUI for the add-ons mentioned in the previous section which gives you a quick demo about how to turn the coordinates extracted from the sensors into useful data for a specific application. When using a Shield2Go, all you have to do is mount the Shield2Go - with the add-on on top of it - onto the XMC2Go, then connect it to your PC via Micro-USB, start the application and choose which add-on you have on top of your 3DSense Shield2Go.

 

 1 / 3 • Fig 5.3) Joystick Add-On GUI Teaser

 2/3   Fig 5.4) Rotating Knob Add-On GUI Teaser

3/3    Linear Slider Add-On GUI Teaser

First of all, if you haven't got the Arduino IDE installed on your PC, please go ahead and install it from here. Also if you have not yet programmed an Arduino-compatible XMC Board e.g XMC2Go or XMC1100 BootKit etc., then please visit Infineon's GitHub page (click here) and follow their instructions, don't worry it's pretty simple.

Now after you've set up the Board so that Arduino IDE can recognize it, you can install the libraries you need. Hover over Sketch->IncludeLibraries and click on ManageLibraries and then type in the search bar the name of the Sensor you're using. For Example:

Fig 5.5) installing library for TLV493D-A1B6 Shield2Go

 

Now we have our working environment ready and we're good2Go -see what I did there?- Anyways, let me take you through the basic functions you'll most probably need when dealing with any of the three Shields.

First of all, you want to initialize the Sensor. That's done by globally defining an object and calling its constructor, depending on the sensor you are using, after including the libraries and before the setup(). The following table summarizes the basic methods you will probably use when dealing with any of the three sensors:

 

1 / 2 • Fig. 5.6) Methods table (to be continued...) 

 

2/2 Fig. 5.7) Methods Table

The cartesian coordinates are defined as follows:

These sensors also have Access modes which you can set in the setup()-function using the method:

setAccessMode(mode);

This is not mentioned in the table, because the access modes are not the same for every sensor, plus, their description is a bit long. So here are the available modes if you want to set them:

Fig. 5.9) Accessmodes of the sensors - use them wisely

 

NOTE: You don't have to set the Access mode, it is set to a default mode, MASTERCONTROLLEDMODE (for TLE493D and TLI493D) and POWERDOWNMODEinitially (for TLV493D).

5.1) Wake up!...and maybe consume less!

No, I'm not talking to you, I'm talking to the Sensors.

The three 3D Sensors generate an interrupt signal to the microcontroller to signal a finished measurement cycle. Now, the TLV493D-A1B6 does that always i.e whenever a measurement is ready it sends out an interrupt signal to the µC, that's because it doesn't belong to the WakeUp family.

Now for the Shields with a Wake-Up ability, the TLE493D-W2B6,and TLI4W2BW -the Sensors of the WakeUp Family- generate interrupts when the measurement passes a certain threshold defined by the user, so in the end, it consumes less power.

You can check if the WakeUp mode is enabled or not by the following method:

HINT: For the TLE493D-W2B6, it is enabled by default.

wakeUpEnabled();

and set the threshold by:

setWakeUpThreshold(x_high,x_low,y_high,y_low,z_high,z_low);

where the parameters are of the type float and represent the high and low boundaries of x, y and z coordinates respectively.

Here the adjustable range can be set with a ratio of size [-1, 1]. When all the measurement values Bx, ByandBz are within this range, the Interrupt is disabled. If the arguments are out of range or any upper threshold is smaller than the lower one, the function returns without taking effect.

If you chose to consume less power by setting the access mode of the Sensor to LOWPOWERMODE, then you can also set the update rate which determines the frequency of the updateData() by using:

setUpdateRate(rate) // parameter of type uint8_t, 
                                            // ranges from [0,7],
                                            // 0 being the fastest

REMEMBER to define an object of the sensor class you are using, before you call these methods.

Do you know how the newborn of the family gets all the attention and gets extra spoiled? That's how we treat our baby, TLI493D-W2BW. (Weird name for a newborn? I thought too till I saw Elon Musk's newborn baby's name)

Seriously, when dealing with the TLI493D-W2BW, you have someextra functionality e.g you can set the measurement range with:

Tli493d.setMeasurementRange(range);

whereas the parameter, range, can be:

• FULL (-160mT to 160mT)
• SHORT(-100mT to 100mT)

EXTRA SHORT (-50mT to 50mT)

This sets the magnetic range that can be measured. The smaller the range, the higher the sensitivity.

PARTY-POOPER: If you choose to use the EXTRA SHORT range, you will have to disable the WakeUp mode first for this sensor. Which brings me to the next point.

For the TLI493D-W2BW you can enable or disable the WakeUp mode with:

Tli493d.disableWakeUp(); 
// or
Tli493d.enableWakeUp();

and you can set the threshold like the , TLI493D-W2BW A0 but also in milli Tesla (mT) or in LSB-format respectively with

Tli493d.setWakeUpThresholdMT(x_high,x_low,y_high,y_low,z_high,z_low);
// or
Tli493d.setWakeUpThresholdLSB(x_high,x_low,y_high,y_low,z_high,z_low); 
                                //of type int16_t, ranges from [-2048,2047]

Wow! This kid is really spoiled, huh?!

5.2) Simulation Tool

Infineon has also provided a cool online simulation tool to give you a good overview or an expectation of the outcome when using the 3D Sensors for some applications e.g linear position measurement, angle measurement, and for the Joystick. It also gives an approximation of the error caused by e.g mechanical misalignment.

First, you'd have to give the tool some info as an input like magnet type, dimension, or application-specific parameters to allow it to set up your specific model. After providing the tool with the necessary info, hit calculate and wait for the simulation results. These results serve as appropriate information to optimize and accelerate further design activities

 

1 / 2 • Online Simulation Tool

 

Simulation results for Linear Position Measurement as an example

After the calculation, you have the option to request a summary as a .pdf document or .tab file (you can open it with excel). The summary returns the calculation results while also telling you the input you've provided for the simulation tool. In some cases, the .pdf version offers you a supplier link to the magnet you've specified.

6) Farewell

You can check out the examples Infineon has integrated into the libraries for each sensor. You can do this by opening up the Arduino IDE and hovering over File->Examples->... under Examplescustomlibraries. There you'll find some pretty simple examples for the Shield of your choice, which you can Upload immediately on your Board. This way you can easily test out the different methods and access modes as well and monitor the output on Arduino's Serial Monitor.

So there you have it folks, TheThree3DSense Shields2Go. You might wanna check out the Window Opener. It features the TLE493D-W2B6 and showcases how it was used to help lock or unlock a Window and aid in opening, closing or tilting it wirelessly with your Smartphone. Also don't miss out on the 3DSense Kits2Go variation of our 3D magnetic sensors.

When you hear the word 'Joystick', the next thing that pops up in most minds is 'gaming' ! The same popped up in our minds too. We had Infineon's  TLE493D-W2B6 MS2GO  and the 3D printed Joystick-add on and in hand and we started day-dreaming playing all the games with it. And soon we had the vision of a smart joystick which could figure out shapes that are drew using that joystick - opening up a new level of gaming experiences!

You can extend this to all kind of applications that somehow involves a change in movement that can be attributed to change in magnetic field or modify this application to a different level such as using this for character recognition or assisting blind people by notifying them about their indicated movement by clubbing in a speech aspect and so on!!

Let me break the suspense and take you on the journey of how we trained a PC so that it could tell you the shape that you create using the Joystick!

Basic Know-How

There is a magnet embedded on the bottom of the joystick whose movement will cause a change in the magnetic field which will be detected by the TLE493D-W2B6 magnetic sensor present in the TLE493D-W2B6 S2Go kit.

Magnet embedded Joystick placed on top of TLE493D-W2B6 magnetic sensor

 

Make-it-Work

Follow the steps below and your joystick will become smart !

1. Setup

The hardware is pretty simple! Just place the Shield2Go Adapter for Raspberry Pi on top of the Raspberry Pi and on top of it, place the sensor and joystick assembly and then you connect the four lines SDA, SCL, GND, VDD of the 3D magnetic sensor to the Shield2Go adapter and you're ready to go !

Joystick and Sensor placed on the RPI using the IOT Adapter

2. Let things communicate

Next step would be to initiate the communication between the RPi and Magnetic Sensor using I2C. Let's go deep on the communication part!

The User Manual for TLE-493D-W2B6 sensor describe the sensors’ register-map, I2C communication and steps involved very efficiently. On the par, let's walk through on doing it with the RPi!

First, we scan the I2C bus on the RPi using i2cdetect -y 1. We see the 7-bit address of our sensor as 35H. This we will use to start the I2C communication handle.

Scan the I2C bus and get the address of the sensor

 

We use PIGPIO library in python to communicate via I2C. Generally, PIGPIO library is installed by default but if not, one could pip install, via pip install pigpio.

In order to use pigpio module in Python (remote GPIO for Raspberry Pi), pigpio Daemon (pigpiod) has to be loaded to memory on RPi. This also requires sudo privileges, thus, you have to enter following command in your terminal at every startup.

After this, we are ready to speak to our sensor ! We will be using jupyter notebook to ease process visualization of setting up the I2C communication between the sensor and the RPi.

 

In first block we import the pigpio library, then open the channel for I2C communication via handle, using following block of code.

For further writing and reading we just have to use this handle (h).

 

|Also, do keep in mind that there are certain changes we need to make on the register level to initiate the I2C communication. The steps for the same are as listed below :

Config and Mod1 register for setting up I2C communication

 

• Set PR bit in Mod1 register as High to enable 1-byte read mode. Failing to set this bit results in the sensor giving NACK for read requests on I2C or the register values are read as FFH.
• Enable clock stretching and disable /INT and collision avoidance by resetting the CA bit to Low and setting INT bit to High.
• Now set the mode to Master Controlled Mode by writing 01b to MODE bits
• IICAdr bits are left as 00b since the product type is A0.
• The FP bit value is set/reset according to the odd parity of the Mod1 register and the PRD bits of the Mod2 register, which in this case comes out to be 0b.

With these steps, the value to be written to Mod1 register becomes 15H.

• Now, set the TRIG in Config register as 10b, for we need ADC trigger on after reading the 05H register.
• Other bits in Config register are left as default as those enable the Temperature and Bz measurement.

With these steps, the value to be written to Config register becomes 20H.

We write these values to register of 3D magnetic sensor using write_byte api of the pigpio library as follows:

After this you can use an update function to keep on reading from the sensor, as follows:

Now we are set to draw and play using the obtained raw data from the sensor!

 

3. Make it 'Smart' with Machine Learning

We generated an image dataset from a polar plot using the magnetic field coordinates from the sensor as written on the code. Our dataset consisted of around 4000 images on each category of shape (Circle [C], Square [S], Triangle [T] and None of the mentioned[N]).

Images from the Generated Dataset

 

 

Using this dataset a Neural Network was trained on https://studio.edgeimpulse.com/. You can use Edge impulse to train your own dataset or even the same dataset as shown below :

• Click on the above link which will navigate you to Edge Impulse Studio. You'll have to create a free account on the same for starting your machine learning feat !

 

• Now create a new 'developer' project as shown :

 

 

• A window as shown below will pop up once you create the project. Since you are going to upload the generated images, you have to select 'images' in the list. Also, select 'Classify a single image' in the next window that pops up !

 

 

• Now, we see a pop up that asks for the data that is to be used for training. Since we already have the data available, we will use the uploader to upload it from the PC.

 

• Upload the category of images one after the other giving them respective labels by choosing the files and then click on 'Begin upload'.

 

 

• Click on 'Create Impulse' and set the dimensions of image as 32x32 (This can be any size but we are limiting the size here to reduce the processing requirement as the 'free developer mode' allows only as much ! ). Then add the processing block (select for Image) and the learning blocks (select transfer learning) and save the impulse !

 

 

 

 

• After you save the impulse, more options will open up for processing under the heading 'Impulse design' on the left. Select the image tab and on the color depth, select 'Grayscale' and click on 'Save parameters'. A new window will open up for generating features from the data where you just have to click on 'Generate features'.

 

 

Visualization of generated features

 

 

• Now, select 'transfer learning' under the heading 'Impulse design' and alter the training parameters as required to train the model for best results and click on 'Start training'.

 

• Once the training is done, a corresponding confusion matrix will be generated giving an indication of how well the model was trained.

 

• Now, Click on the 'Model testing' heading and test the model on the created test dataset by clicking on 'classify all' tab. The resulting accuracy will indicate how well the model learned !

 

 

 

• After testing, if you select the heading 'Dashboard', you'll be able to download the model in the desired format.

 

This is as much of Edge Impulse that you'll need for this project to work but if you need to know more, do checkout : https://docs.edgeimpulse.com/docs !

We found converting the trained Keras network model to tensorflow lite model worked better than using the tensorflow lite model itself. The converted tensorflow lite model along with the dataset we used is attached for your use. You can also use custom trained model by generating your own dataset, train it and use it!

4. Get-Set --> Ready -->Go!

To deep dive into our workaround, you just have to download the attached Requirements.txt and the 'Smart Shape Stick' zipped folder in the code section which contains the python Code for Smart Shape Stick, Trained Keras Model and the used dataset. Once downloaded on your Raspberry pi, you initially have to install the packages mentioned on the Requirements.txt. The links for guiding the package installations is also provided in the Requirements.txt. Now, you can make a folder containing the keras model (which is converted to tensorflow lite model) and the python code. You can open the python code on any of the platforms that support python such as Google Colab, Jupyter Notebook, Notepad++ or Visual Studio Code. We have used Visual Studio Code at our end. Now, just run the program and draw the shape to see the shapes getting predicted by our Smart Shape Stick.

P.S. You need to be real fast while drawing the shape once the code starts running as the current version of the code is capturing only 1000 samples i.e., if you aren't fast the code will have only part of the image and not the whole. Modifying the number of samples that are captured is another option if you want to be slow as a snail.

Code snippet for increasing the number of samples

 

What to Expect?

Predicts a circle as it is !

 

Predicts a square as it is !

 

Intelligent enough to recognize a shape that it's not taught !!

 

Have fun extending this application to other characters!

Race on the TRACK !

• If you are new to Raspberry pi, then do have a look through this for getting you started :

• To know more on the Infineon sensors : https://www.infineon.com/cms/en/product/sensor/magnetic-sensors/
• For skimming through I2C implementation of Rpi : https://learn.adafruit.com/adafruits-raspberry-pi-lesson-4-gpio-setup/configuring-i2c