Maze Solver - LeJOS

In this maze solver the robot follows a left hand rule as given in https://embedjournal.com/shortest-path-line-follower-robot-logic-revealed/. The program works in two phases. In the first phase, the robot samples the entire path, stores the x and y co-ordinates of the path and creates a map of maze in the memory. Given two consecutive points, the program also calculates and stores the direction of the robot at each point.  Then the program calculates the shortest path by cancelling out the U turn samples and the samples in opposite directions. In the second phase, the robot actually follows the shortest path. During path following, the robot samples the path again and whenever the direction of two consecutive points falls apart from the expected direction (which we got in the first phase) the robot aligns its direction accordingly.

Let me go through the program logic for shortest-path-finding and shortest-path-following in detail. However, before going through it, it would be better to read the article about Limited Maze Solver in order to understand the high level code structure that I have followed. Let us go through the robot path below and mark the path with directions and corresponding sample numbers in those directions. 

Let us assume that the robot path has the following directions and samples along each of them, when the robot traverses it in the first phase: (In real life, the number of samples may differ when the robot traverses, but the high level logic remains the same)

[North (20), West (10), South (30), East (30), North (35), South (20), East (15), North (40), West (20), East (20), North (20), West (20), East (20), North (25), West (20), East (20), South (25 + 20 + 40 + 20 = 105)]

According to the left-hand-rule in the reference site above, we will further reduce the samples where we have East-West and North-South directions in pairs. This is quite simple to understand, as when the robot traverses to North and then comes to South along the same path, the efforts of traversing eventually cancel out. Same for the directions East-West. After cancelling the opposite direction efforts in the first iteration, the path and the samples look like below. 

[North (20), West (10), South (30), East (30), North (15), East (15), North (40), North (20), North (25), South (25 + 20 + 40 + 20 = 105)]

We will then perform the second iteration of cancellation of East-West and North-South. Below is the result after second iteration.  

[North (20), West (10), South (30), East (30), North (15), East (15), North (40), North (20), South (20 + 40 + 20 = 80)]

We will then perform the third iteration of cancellation of East-West and North-South. Below is the result after third iteration.  

[North (20), West (10), South (30), East (30), North (15), East (15), North (40), South (40 + 20 = 60)]

We will then perform the fourth iteration of cancellation of East-West and North-South. Below is the result after fourth and final iteration. 

[North (20), West (10), South (30), East (30), North (15), East (15), South (20)]

When we reach the above iteration and the shortest path does not reduce any further with the "cancellation of samples in opposite direction strategy", we will break the logic. Let us go through few important methods which I have used in the Cruiser class to come up with the shortest path. 

run(): This is the run() method of the Cruiser thread which has the important logic to find and follow the shortest path. It calls three main methods to complete our goal.

populateFirstPathSamples()

findShortedPath()

followShortestPath()

summarize(): While explaining this method I am assuming that you have gone through Limited Maze Solver article in detail. In addition to my explanation there, I want to mention that in this method we take into account all the consecutive direction changes. With each direction change, we keep a record of the number of samples in the same direction. Let us assume that while populating the First Path samples, we get the direction sequence as follows (as known from the direction property of the summary objects):

[4, 4, 2, 2, 2, 2, 3, 3, 3, 3, 3, 1, 1, 1, 1, 1]

This sequence will get recorded into our directionOutput objects as follows:

4 (2), 2 (4), 3 (5), 1 (5)

Here, the figures in the bracket represent the number of samples in the given direction. Each directionOutput object has four properties like summary objects, X, Y, direction and removal. In addition to that I have introduced one more property, numberOfSamples.  

Plotted TachoPoseProvider Readings

I realized that the Maze Solver was going little ahead of the path that was expected. To look for a reason, we decided to visualize the way the robot traverses the path. We exported the data which the pose provider gathered, to the computer. And we plotted the way the path looks like. The above image represents the "actual path" that is visualized by the robot. At the end, the robot diverges from the path that it is expected to travel. 

findShortedPath(): Calls the methods removeOppositeDirections42, removeOppositeDirections24, removeOppositeDirections13, removeOppositeDirections31 and removeUTurn iteratively until the shortest path size cannot be further reduced. Let us take a look at removeOppositeDirections42 method in detail. We will also take a look at removeUTurn method in detail. 

removeOppositeDirections42():  

Follow entire shortest path and check whether the directions 4 and direction 2 appear consecutively. Check the corresponding number of samples with each direction as explained above. Then call the decideWhichOneToRemove() method by passing the entire shortest path and the index of direction 2. Extract the number of samples for both, the direction 2 and its previous direction 4 in the direction array. Find the difference between the samples of direction 2 and direction 4. If the difference is less than DIFFERENCE_BETWEEN_EQUAL_SAMPLES, treat the distance between the two directions 2 and 4 as equal and nullify them by removing both of them from the direction array. 

NOTE: I have created the constant DIFFERENCE_BETWEEN_EQUAL_SAMPLES = 11, because due to technical/processing lags, the difference between two equal and opposite directions, may not be exactly zero. So, I have created a threshold of 11 samples to consider the equal and opposite number of samples.    

If the difference between the samples of equal and opposite direction is greater than 11, that means there is unequal distance travelled in the directions. In such case, subtract the longer distance from the shorter distance and set the difference appropriately in either direction 2 or direction 4 (based on the direction which is travelled more)

removeUTurn(): 

Follow the entire path and check if any of the directions 1, 2, 3 and 4 has number of samples less than 11, we remove all those directions from the shortest path, considering either it is a U-Turn or it is noise.  

followShortestPath(): The algorithm for following shortest path goes as follows:

1. Extract the number of samples in the heading direction of the robot from the shortest path array
2. Follow the path with PID controller pilot for the above number of samples   
3. Get two more samples from the pilot
4. Check if the direction property of those two samples match with the next direction in the shortest path array
5. If the directions match, move the shortest path array pointer to the next direction and continue with step 1
6. If the directions do not match, align the direction by calling the alignDirection() method, then move the shortest path array pointer to the next direction. Add the STABILIZATION_ SAMPLES to the number of samples in a given direction. Then continue with step 1

NOTE: I have added the STABILIZATION_SAMPLES in order to give the robot some time, to align in a given direction appropriately. However, from the performance of the robot, it looks like it is NOT required to add them. Because due to addition of them, the effect is little different. Whenever the robot is expected to the change the direction, considerable delay is present. Like the robot should turn much earlier before taking U-Turn number 1. However, it almost reaches till the end. This is because of accumulation of all previous stabilization_samples during direction changes. Same behavior is evident at U-Turn number 5. The robot is expected to stop immediately after taking the turn, however it goes ahead much further.   

alignDirection(): This method is used to turn and align the robot in different directions based on the required direction changes in the shortest path. I call the steer API of the robot which turns it onto a circular path. Please note that the entire turning will not be in the first stretch, because I have used angle 60, -60, -90 to turn. This method may need to be called twice to get the robot at the appropriate angle. For example: we need to call the pilot1.steer(-180, 60) twice to get a turn reach 90 degrees. We may need to call the pilot1.steer(-200, -90) twice to get a turn of 180 degrees. If we leave the robot at an intermediate angle, Line-Follower behavior will come to help, to automatically bring the robot on the black stripe from the white surface. 

NOTE: The performance of the robot is dependent on tuning the following properties appropriately:

• STABILIZATION_SAMPLES
• DIFFERENCE_BETWEEN_EQUAL_SAMPLES
• U_TURN_SAMPLES
• TOTAL_SAMPLES

The TOTAL_SAMPLES constant works in the first phase of robot traversal of the path. These are the samples required to complete the traversal across complete trajectory. This number is coined with a bit of trial and error and observations.

 

Limited Maze Solver With DifferentialPilot and PIDController

Maze solver being a complex topic for me, I decided to start with a simplest maze first (check the video). The program is inspired by the video here. The main reference for finding the shortest path is here. The code for this program is given below. The program works with DifferentialPilot and PIDController classes supported by leJOS. The program works in two phases. In the first phase, it samples the entire path for the direction in which the robot is moving. It then applies the shortest path algorithm over the direction. In the second phase robot follows the shortest path. I will explain all the methods in the program below. 

Cruiser(): This is the Cruiser class constructor which internally calls pidInitialize(), pilotInitialize(), pid1Initialize(), pilot1Initialize(). 

run(): This is the method over the Cruiser thread which implements the shortest path following algorithm (from the left), by calling following methods in sequence:

• populateFirstPathSamples()
• findShortedPath()
• followShortestPath()

populateFirstPathSamples(): It populates the summary array. This array contains the instantaneous values of X and Y coordinate of the robot's pose and also the Direction of the robot's pose while traversing the path for the first time. I have introduced a variable removal in order to use it for plotting the trajectory of the shortest path. This method executes the following steps in order to populate the summary array:

• Call getASample(), which return the X and Y coordinates for the robot, based on its pose while collecting the sample
• Populate X, Y, Direction and Removal properties for the sample in a Direction object and create a single entry in the summary array. Please note that the Direction object has a direction property too. 
• Repeat the above steps for 180 times (The figure 180 samples collection, is coined through a bit of trial and error). You guessed it right, there will be 180 entries in the summary array at the end of the traversing the entire path for the first time. Each entry will be of type Direction object 

getASample(): Steers the entire path with the PID controller and samples the path after each 10 milli seconds. It does such sampling 10 times, gets the X and Y position at each sample. It averages X and Y over these 10 samples. It puts the X and Y values in the sample array. So, by this time you should know that each sample consists of only 2 elements, the X and Y coordinates of robot's pose (averaged over 10 samples). 

getDirection(): This method decides the direction in which the robot is moving as per the following figure. 

This method assigns the following numbers with the directions:

North = 1

East = 2

South = 3

West = 4

Below is the algorithm for setting up the direction variable

• The input to this method is the X and Y coordinates of the current pose and its previous pose. 
• dx is difference between the X coordinate of the current pose and X coordinate of previous pose
• dy is the difference between Y coordinate of current pose and Y coordinate of previous pose
• I have set direction of movement from South to North to be when the line passing through the two poses of the robot has a slope less than 1 and dx is greater than 0
• I have set direction of movement from North to South to be when the line passing through the two poses of the robot has a slope less than 1 and dx is less than 0
• I have set direction of movement from East to West to be when the line passing through the two poses of the robot has a slope greater than 1 and dy is greater than 0
• I have set direction of movement from West to East to be when the line passing through the two poses of the robot has a slope greater than 1 and dy is less than 0

summarize(): This method executes the following crucial steps:

• Out of 180 summary samples, remove all the summary samples where direction = 0. That means neither of East, West, North, South is associated with it
• Out of the remaining summary samples, consider only those where consecutive samples does NOT have the same direction. That means I am considering only those consecutive summary elements with changes in the direction to be on the shortest path. 

findShortedPath(): As mentioned in the summarize() method above, the shortest path list takes all the summary samples which represent change in the direction, as an input. Then it removes all the samples where the previous three direction changes look like 4-1-2 in a sequence corresponding to West-North-East. This sequence represent the U turn (which is little delayed turn) which the robot takes at the dead end. I have come up with this sequence removal after observing the logs which I print in the program. 

followShortestPath(): As we know now, that the shortest path constitutes the changes in the direction of the robot. To follow the path, consider the following algorithm. 

• Collect the first three samples from pilot1 for the second phase. 
• Don't take any action until the three samples are aligned in the same direction as shortest path. Just keep following the path. 
• The moment the direction of the three samples get misaligned, start taking a turn. Then stabilize until the stabilization count gets decremented to zero. 
• After mis-aligning and consequent stabilization, if the three consecutive samples get aligned with the next index of the shortest path, increment the shortest path index to point to the next direction.
• Then start with the next three samples of pilot1 and repeat the same procedure again!   

NOTE: The number three for three consecutive samples while following the shortest path is considered after some trial and error. We can even go for comparing two samples or four samples. However, two samples will make the algorithm more error prone. And four samples will create some delay in processing.  

Line Follower with LeJOS DifferentialPilot and PIDController classes

Below is the program which makes use of LeJOS DifferentialPilot and PIDController classes to make a line follower. The underlying PID Controller concept is the same as explained in the article Line Follower with PID Controller. The appropriate values of Kp, Ki and Kd are set in the program. Higher values of Ki makes the robot unstable in line following (makes it go round and round around itself). Lower values of Kd gives a delayed response when following sharp runs (like a right/left/U turn). Higher values of Kp makes the robot follow the line in a shaky manner. The video clip shows the performance of the program.

Let me explain the Cruiser class of the LeJOS program below. The core classes, their attributes and methods used in the Cruiser class are explained in the LeJOS APIs at the below links:

• DifferentialPilot
• OdometryPoseProvider
• PIDController

The program starts with setting the parameters - wheel diameter, track width, left motor, right motor and reverse setting which are required for DifferentialPilot to function. The OdometryPoseProvider is then initialized with DifferentialPilot class object passed to its constructor, in order to track the pose of the robot in the 2D-space. The pose represents the current location and the heading of the robot. For the pose, the distance travelled by the robot is measured in Pilot units (that means if we have set the wheel diameter and track width in inches, we will get the distance travelled in inches. If we have set them in centimeters, we will get the distance in centimeters). The heading angle is measured in degrees. The pose readings are required if we want to plot the trajectory of the robot. An example of expected trajectory is given below. 

     

After feeding the instantaneous pose readings into a graph plotter (like plotly), we get to see the actual trajectory as follows:

To take a look at good in-place U-Turns at the dead ends, please take a look at below video. 

In our example of Line Follower, the pose readings do not exactly match with the actuals after the robot performs U-turns at the dead-ends. If we apply certain correction for in-place U-Turns, we may be able to get the exact trajectory. However, I have not applied any correction for this plot but we do get an idea of the trajectory that the robot is following. More concrete use of the pose, will be explained in the Maze Solver article. For now, in the Cruiser, we need to set the travel speed and the rotate speed of the robot. Travel speed is in the wheel-diameter unit per second and the rotate speed is in degrees per second. Further, we initialize the PIDController constructor with the set-point (in our case, the threshold value), set the required PID parameters and pass the instantaneous value of the light intensity to the doPID () method. The Kp, Ki and Kd needs to be tuned appropriately in order for the robot to follow the black line over white surface. 

NOTE: It would serve great help if you read the below article before going through code given in current article. That will make you familiar with the calibration concept. 

Line Follower with PID Controller

Line Follower with PID Controller

I am sharing the leJOS line follower program with PID controller. Here is the reference literature for PID controller. As we all know the PID Controller works based on continuous feedback from the system relative to a given set-point. If the system moves away from the set-point, the feedback is provided such that, it will return back to the set-point. The PID algorithm works so as to minimize the error between the actual and the set-point. In case of the Line-Follower program below, the system is the Line-Follower itself. The set-point is the light intensity which the system has to follow in order to continue moving on the black line and should not lose the track. In our case, we define the set-point as the average light intensity of black and white colors. We call the set-point as threshold. The way of configuring the threshold being average = (black + white)/2, the line-follower follows the edge of the black stripe placed on the white paper. I have put two perpendicular stripes for the line follower as a challenge, to know how precisely it can follow the turn. (It is relatively easy for the line follower to follow smooth curves. But most of the time in real-life scenario, we do not get smooth turns. We encounter perpendicular sharp turns). The other challenge was to find out how effectively the line-follower faces a dead-end. Hence in the below small experiment, I have incorporated two perpendicular turns and two dead-ends with black stripes. You can imagine it as a road, on which the car is moving with a relatively slow speed. The speed is kept slow in order for the Line-Follower not to lose its track. We can even increase the speed and try the same experiment, but let us keep that for some other time. 

Now let us look at the digitized-PID controller algorithm which I have incorporated in the LeJOS program below. The set-point is the threshold as described in the above paragraph. The feedback is provided in the form of error, which is the difference between the actual-light-intensity at any given instance and the threshold value. If the actual-light-intensity of the robot is more towards black or more towards white, the system will pull it back towards the threshold, i.e. the edge of the line. In the below digitized-PID algorithm, we convert the error, its integral and its derivative into power value which will be supplied to the right and left motors. The power will be supplied to the motors such that, if the car is too much on the black surface, it will be pulled on to the white surface and vice-versa. In the digitized version of the integral error, the past error value is added to the current error value iteratively. While the digitized version of derivative, the last error is subtracted from the current error iteratively. Please take a look into the following three lines of code for illustration, where color represents the current light intensity based on the robot's position on the black or white surface.

  

The correction variable represents the calculated value of power, where error term gets multiplied by Kp (the proportional coefficient), the integral term gets multiplied by Ki (the integral coefficient) and the derivative term gets multiplied by Kd (the derivative coefficient). The Kp, Ki and Kd need to be tuned in order for the Line-Follower to follow the line appropriately with a certain speed. In my case the default power of the Line-Follower is set to 20. This value is based on trial and error in order for the speed to be relatively slow. Below is the video clip to show the performance of the program for given kp, ki and kd parameters.  

NOTE - In order for the Calibration details of Black and White intensities, it would better to take a look at my Line Follower program first.

Observations: The program seems to be following the perpendicular turn well with the current Kp, Kd and Ki coefficients and the given speed. However, it does not take the U turn at the dead-end very well. For taking an in-place U turn there is a need for more fine-tuning of PID coefficients. To know about how a better in-place U-turn looks like, please take a look at my video In-Place U turn. I have not yet concluded anything about the 'effect of tuning the parameters' over the line follower. However, that would be reserved for the future. 

I hope you liked this article!

Omni Biped

Below is the Java program for the Omni Biped robot (one creation from Daniele Benedettelli's book 'Creating Cool Mindstorm's NXT Robots'). The main design of the program is inspired by the author's Single Task 'C' program given in the book. The robot walks forward until it encounters an obstacle in front of it. As soon as it encounters an object, it takes a turn until there is no further obstacle on its way. 

Let me explain about the program a bit. The program has two classes, OmniBiped and NXTUtils. The OmniBiped program has the main method which begins with calling the init() method. The init() method resets the tachometer count and realigns the legs. In Lego Mindstorms NXT, units of measurements used are in tachometer counts rather than rotation or degrees. One pulse of tachometer is equal to one degree. To complete one full step, the robot requires 3 complete rotations of the motor. That means it requires 1080 tacho-counts (or 1080 degress). Whereas to complete half a step, the robot requires 540 tacho-counts (or 540 degrees). The realignLegs() method aligns the legs such that the initial position of the legs will be as shown in the following picture. As you should see, the 

After the initial position, the feet take half step which look like below. 

After half a step, the feet take full step which look like below. 

While realigning, the program calculates the right_count and the left_count. These counts represent the number of degrees each leg require to complete a full step, in case the robot stops in an intermediate position of legs. If this count is less than half a step distance, the robot prefers to take the respective leg backwards (because that is much quicker to achieve the realignment). In case if this count is more than half a step distance, the robot prefers to take the respective leg forward. While moving the motors forward or backwards, the tachometer count gets updated. When the right_count as well as the left_count become less than zero, the motor stops that means the re-alignment process stops. 

 

Now let us come back to the main() method. It calls the init() method and realigns the legs. Then the robot begins walking, with both the legs moving forward. That means, the motors moving the respective legs are driving forward. They keep driving forward, until an obstacle occurs at less than 24 cm distance (as defined by the constant NEAR). Once, the obstacle occurs on its way, both the motors stop, the legs are realigned again and the robot moves either left or right randomly based on the direction variable. The direction variable can be either -1 or 0 or +1 (representative of left/right direction). The robot can move in any one of these two directions until there is no obstacle within the range of 50 cm (as defined by the constant FAR). 

The robot measures the distance to the obstacle by the getAvgUltrasonicSensorReading() method in the NXTUtils class. This method averages the distance to the obstacle over 5 samples. I calculated the average in order to smoothen out the spikes in the ultrasonic sensor readings. 

Below is a small clip showing the behavior of the robot. 

  

NOTE: Please look at the commented part of the code carefully and do not consider it. 

If we analyze the movement of robot, it moves straight quite well in the beginning. It also stops when it encounters an object at less than 24 cm. Once the object encounters the robot takes a turn quite well. However the robot appears quite stuck when it approaches the cable and the wall at an angle (not exactly in front of it). When object approaches at an angle, the robot has to take multiple turning movements, which appears quite slow.  

I hope you liked this article!

Reference Sites:

Using the Tacho-Motor Class (ev3dev.org)

LeJOS installation

Robotic Arm

Below is the simple LeJOS program for making the Robotic Arm work. The LeJOS firmware for NXT can be downloaded and installed on the NXT brick from LeJOS NXJ 0.9.1. After installation of LeJOS, please load my below program on NXT brick, press the LEFT, RIGHT, UP and DOWN buttons and the Touch Sensor. See the performance. The LEFT and RIGHT buttons should move the arm to the left and right. The ENTER and ESCAPE button should move the arm up and down. When the touch sensor is pressed and if the Claw is in open condition in the past (as defined by the toggle flag), the Claw gets closed. And if the Claw is in closed condition in the past, the Claw gets opened. As you guessed it correctly, the toggle flag stores the past condition in which the Claw was and reverses it. 

Line Follower

Here is a small clip showing the Line Follower performance and the LeJOS code associated with it. The robot here is the small car which is built using small booklet provided with NXT kit. The LeJOS firmware for NXT can be downloaded and installed on the NXT brick from LeJOS NXJ 0.9.1. The design of my Java program is heavily influenced by the Line Follower in the book "Programming Lego Mindstorms with Java" by Giulio Ferrai et all. The actual algorithm to follow the line is influenced by Jacek Fedorynski's Line Follower NXC program. The LeJOS program has four classes: LFJfedor, Cruiser, LFUtils and LineValueHolder. The LFJfedor class contains the main method and is the starting point of the program. This class initializes the light sensor and calibrates the sensor for Black and White values in its constructor. The black and white values are stored inside a Java Bean called LineValueHolder. LFJfedor class also starts the thread Cruiser.  The Cruiser class has the algorithm to follow the line, given the calibrated black and white values and the light intensity at any given point in time. LFUtils class calculates the average light intensity over 20 samples, at any given point of time. The average over 20 points is calculated in order to smoothen out the spikes in the light intensity (if they exist). Let me explain some of the program details below.  

In order to calibrate the threshold for White, the NXT brick beeps twice. It prints the message "white" on the screen of the NXT brick. The user is supposed to keep the light sensor on the White part of the line follower chart and press the ESCAPE button. This process will record the White light intensity in the Java Bean  LineValueHolder. Similarly, record the Black value when the NXT beeps twice again. It prints the message "black" on the NXT screen. The user is supposed to keep the light sensor on the black strip of the line follower chart and press the ESCAPE button. This process will record the Black light intensity in the Java Bean LineValueHolder. Once the White and Black intensities are recorded, the Line Follower program works as follows:

1. The Line follower follows the edge of the line where the threshold value is (Black + White)/2
2. The instantaneous value of the light is calculated by averaging over 20 light samples
3. The B Moter and C Moter is powered in proportion to the difference between the threshold and the instantaneous value of light (defined by the variable "color")
4.  The initial value of power = 22 and the constant multiplier 50 requires few attempts to be set correctly in order to follow the line, in a certain speed

I wish you enjoyed the article!