Smart Laboratory
Table of Contents
Introduction
Project Team Members
- Kevin Chen
- Kunye Chen
- Binghan He
- Melissa Lee
- Lijia Liu
- Rachel Schlossman
Project Definition
The Smart Laboratory project is a continuation of the Cloud-Based Shared Lab Initiative, which seeks to provide high-quality laboratory experiences to distance-learning students. Our group will be implementing concepts covered in the Decision and Control of Humanoid Robotics (ME 396D) course at the University of Texas at Austin to synthesize a controller that provides a safe and high-quality learning experience to the student.
Project Platform
The intended proof-of-concept platform for the Cloud-Based Shared Lab Initiative is the Apptronik P170 Orion, shown in Figure 1.
Figure 1. Apptronik P170 Orion series elastic actuator.
Our project will be developed for the University of Texas at Austin Series Elastic Actuator (UT-SEA) platform, shown in Figure 2.
Figure 2. UT series elastic actuator.
Project Deliverables
High-Level Controller
The high-level controller will serve as a planner for control system design to guide students through the process of developing a controller. Instead of specifying rigid sequences for controller design processes, we will implement formal methods - such as linear temporal logic - to specify controller interactions as a series of relationships between environmental inputs and system outputs. This planner will automatically generate the design process sequence and assist students in designing a controller based on experiment requirements.
Low-Level Controller
The low-level controller will assess the input received from a student and passed down from the high-level controller to ensure that the actuator system operates within safety limitations. Rather than forcing the system to shut down when operating in an unsafe manner, such a controller will drive modifications to the actuator's behavior prior to unsafe execution. Behaviors considered unsafe will be determined by the stability and mechanical limitations of the actuator. Some characteristics to be considered include system stability determined via the phase margin, maximum actuator arm speed, and maximum force output.
High-Level Controller
Motivation
The concept of a system that switches between modes is not new and the process of selecting a control scheme from a list is relatively uninteresting and does not provide the flexibility for a system to react to unexpected inputs. As a result, we wish to create a controller that can react to unpredictable behavior to perform tasks on demand. We propose a methodology to specify system logic to enable a robot to determine controller structure on its own. That is, rather than defining the controller structure a priori, we specify the logic to synthesize the controller structure needed for each task.
Due to the complexity of generating such an advanced controller, we defined the scope of this project to only encompass a high-level controller for experiments developed in sequential modes. These concepts will be extended to the free-play modes in the future.
Design Concept
Finite State Machine
In our discussions regarding the forward path of this project, we found that experimental procedures are already sequential and rigid, which removes the need to use an adaptive system, such as Linear Temporal Logic (LTL). Since the work we produce is intended not only to improve the Cloud-Based Shared Lab Initiative, but to also demonstrate the feasibility of an adaptive system based on LTL in robotics, we chose to widen our scope. We propose a system governed by LTL that enables students not only to execute the experimental procedures for controls systems education but allows a form of "free-play" that guides students in the process of developing a controller without knowing a priori what a student will choose to do.
Two-Player's Game
The system interaction between the high-level controller and a student and the system can be considered to be a two-player's game, in which the system receives some input from the environment (i.e. the student and lower-level controller) and produces a reaction in response. We propose a system that enables students to build free-form control systems on their own. With each input, such as system identification or PID constants, the high-level controller reacts by suggesting various other fields for the student to manipulate to create a controller for the system.
Connection to Sequential Laboratory Procedures
Since each action will prompt a response from the high-level controller, sequential laboratory procedures are still possible by ensuring that the proper path to developing one particular type of controller for one particular control objective can be reached via the suggestions from the high-level system.
Logic Grammar and Atomic Propositions
To create such a system, we must first create a grammar to specify these control policies. The grammar is composed of atomic propositions that can be used in combination with linear temporal logic to describe missions and tasks. A significant component of our work lies in researching prior work in this field and defining a structure for the grammar and atomic propositions for mission specification.
Atomic Propositions
We define an atomic proposition to be a statement that cannot be simplified any further. Atomic propositions are Boolean statements that describe sensor data from the environment (e.g. "input current received") or actions conducted by the system (e.g. "request an input current").
Linear Temporal Logic (LTL) Specifications
Linear temporal logic is built from a finite set of propositional variables (such as atomic propositions), logical operators (such as ¬ and ∨), and temporal modal operators (such as G and F). The temporal modal operators are shown in Table 1.
Table 1. Temporal Modal Operators
| Symbol | Definition |
|---|---|
| G | Globally (Always) |
| F | Future (Eventually |
| U | Until |
| X | Next |
| R | Release |
Some LTL examples that follow these specification properties are shown in Table 2.
Table 2. LTL Examples
| Name | LTL Expression | Definition |
|---|---|---|
| Obstacle Avoidance |  | Avoid all obstacles before eventually reaching a goal area |
| Sequencing |  | Reach specific areas in a specific order |
| Coverage |  | Eventually visit all areas |
Application of Linear Temporal Logic in the Smart Lab System
In designing the Smart Lab system, we intend for the high level controller to be an medium of communication between the student and the low-level controller. That is, the student and the low-level controller communicate with each other through the high-level controller rather than directly. This allows us to help the student interpret lower-level system responses, to protect the lower-level system from student error, and to guide the student in the process of developing a controller to manipulate the lower-level system. A schematic of the intended system interaction is shown in Figure 3.
Figure 3. System Interaction Schematic
Syntax
Linear temporal logic formulas are interpreted over all sequences of reactions to inputs from the student and low level controller, starting from some initial state x0. The atomic propositions of the logic are labels representing system reactions in the environment, such as "Force Control Experiment Selected" or "PID Method Selected."
Let
be a set of such propositions. Then the LTL formulas are defined according to the following grammar:
Semantics
We can also define the continuous semantics of LTL formulas over sequences of system reactions. Let x(t) for t ≥ 0 denote the state of the lab system at time t and let xt be a possible sequence of reactions starting at x(t). That is, xt denotes the flow of x(τ) under the inputs u(τ) for τ ≥ t.
For example, suppose that after t = 5, the proportional gain, Kp, and derivative gain, Kd, cannot be entered until the PD method is selected. Then for x5, the LTL specification is:
where 
is "PD Method Selected", 
is "Proportional Constant (Kp) Entered", and 
is "Derivative Constant (Kd) Entered".
We generated a LTL specification table (link) that includes the LTL propositions as well as the safety and liveness properties that characterize the environment and system.
State Machine Generation
Temporal Logic Planning Toolbox (TuLiP) Implementation for the System Identification Experiment
To generate the state machine, we used the Python package TuLiP (link). By inputting the LTL specifications we determined previously in GR(1) format, TuLiP allowed us to automatically generate a complex state machine.
Atomic Propositions
The first component of formulating LTL specifications in TuLip is to specify the atomic propositions that are manipulated by the LTL logic statements. These propositions include environmental input and system actuation atomic propositions, shown in Figures 4 and 5, respectively. (
)
Figure 4. Sample environmental input atomic propositions in TuLiP.
Figure 5. Sample system actuation atomic propositions in TuLiP.
These atomic propositions are those pulled from the LTL specification table presented in the previous section. While we had originally designated 4-character codes to represent each proposition, we found them difficult to understand and chose to describe the variables in more detail instead.
Linear Temporal Logic Statement Formulation
In developing the LTL specifications for the controller, we specify assumptions and requirements for safety and liveness characteristics of the system. The high-level controller must guarantee that the human user successfully completes the experiments and that the low-level system executes the experiments in a proper manner. In order to achieve these capabilities, the high-level controller is synthesized base on the following assumptions:
(i) Only one action is allowed from either the human user or the low-level system during one time step,
(ii) the low-level system is guaranteed to provide feedback (completed or failed experiment) when the high-level controller commands it to execute the experiment, and
(iii) the human user is able to provide acceptable experiment inputs in a finite number of trials.
Assumption (i) and (ii) allow us to provide these statements using a global (G) declaration, as shown in Figures 6 and 7, respectively.
Figure 6. Sample mutual exclusion statement to guarantee that conflicting states do not occur simultaneously
Figure 7. Sample environmental reaction statement to guarantee a response to an action that requires a response.
Assumption (iii) implies an environment liveness specification which indicates always eventually the experiment is completed. In order to make this liveness specification realizable, we need to add one additional environment safety specification which requires that the human user only inputs parameter when the high-level asks him/her to, and one additional environment liveness specification which indicates that always eventually there is parameter entered by the human user., as shown in Figure 8.
Figure 8. Sample liveness assumption to guarantee that the student will enter data at some point during program execution.
Just as the environmental propositions feature safety and liveness statements, so do the system propositions. Safety requirements place constraints on system behavior to guarantee or prevent certain behaviors. As in environmental statements, these statements are implemented with a global (G) declaration, as shown in Figure 9.
Figure 9. Sample system reaction statement to constrain system behavior.
Finite-StateMachine
After providing TuLiP with all of the LTL specifications needed to constrain the system, TuLiP creates a state machine for program execution. Figure 10 shows one iteration of our TuLiP generated state machines that has fewer states, for better readability. TuLiP also provides a Python file specifying all of the state transitions for the state machine, which we were able to use to create a finite state machine to interact with the student. In order to interact with the lower-level controller, we used predefined Python libraries to interface with MATLAB.
Figure 10. Sample TuLiP generated state machine for 6 states and all possible transitions between them given by the LTL specifications.
Low-Level Controller Integration
Since the lower level controller and experiments were developed in Matlab and the hig-level controller was developed in Python, we integrated those two controllers using the MATLAB Engine API for Python, which allows the developer to call Matlab files in Python.
Smart Lab Demonstration Video
Students can access the Smart Lab and select options using the terminal. Below is a demo which shows how the Smart Lab works:
Low-Level Controller
The first priority of the low-level controller is to ensure the safety of any action given by the high-level controller. Such a strategy draws a parallel to the guarantee of safety in motion planning, as discussed in [3]. Errors will be communicated to the high level controller via the passing of integer values, when necessary.. For example, if student inputs current values are not within the range of safe values, a value of 1 will be passed to the high level controller. The low-level controller also performs basic learning checks. If the student is performing a sequential experiment for the first time, the low-level controller will check that the student has entered a sufficient range of values to guarantee that the student can obtain experimental output that supports learning objectives.
First the high-level controller must inform the low level of which experiment is to be conducted. The table below outlines the communication structure via the passing of values from the high-level to the low-level.
Integer Passed to Low-Level | Associated Experiment |
1 | System Identification – Sequential |
2 | System Identification – Free Play |
3 | System Identification – Simulation |
4 | Force Control – Sequential |
5 | Force Control – Free Play |
6 | Position Control – Sequential |
7 | Position Control – Free Play |
Once the low-level is aware of which experiment it will conduct, it can begin to perform safety and learning checks. The high-level control structure code can be found here.
System Identification Experiment - Sequential
The purpose of this lab is to allow the student to gain familiarity with the series elastic actuator and gain an understanding of the system identification process. He/she will understand the relationship between input current and output force via the exploration of the associated open loop transfer function. Via the input of a selected chirp signal, he/she will obtain time response data to and the associated Bode plot, and then will be able to identify system parameters 
, mk, and beff.
The post-lab exercise will be to vary system parameters, considering the given Bode plot and given transfer function, to find the correct three values.
For the experiment, the high level controller will instruct the student to enter two current values between 0.5 and 1.0 amps, and with a difference of at least 0.3 amps between the two values when the student is performing the experiment for the first time. The low-level controller is responsible for first ensuring that the input current values are within the bounds of safe values. Once safety is guaranteed, if the student is performing the experiment for the first time, then the low-level controller will ensure a sufficient gap between the two current values (learning check). If either of these constraints are not met, the low level controller will send a signal to the high level controller in order to instruct the student to modify current values. Once the current constraints are met, the two experiments will be conducted. The low-level will pass up the associated Bode plots and the output force vs. current data. The diagram below outlines this scheme:
Sample output data is below:
System Identification Experiment - Free Play
In the free play mode, the student will perform a single experiment and the low level controller will verify that a safe current value is entered.
System Identification Code
Matlab code which drives the Simulink model simulation and is called by both the low-level sequential and low-level free play code
Simulation which allows the student to vary system identification parameters
Force Control Experiment - Sequential
The purpose of this lab is to allow students to understand how optimal selection of controller gains allows for accurate force control. Specifically, a key learning takeaway is that as the proportional gain, kp, increases, bandwidth (performance) increases but phase margin (stability) decreases.
As the students explore how the proportional gain and derivative gains, kp and kd , respectively, impact system behavior, the low-level controller will ensure both safety and that the students achieve the experiment learning goal. Hence, there are two checks that must be performed when the student performs the experiment for the first time. First, the safety checker will ensure that the kp values are within an acceptable safety range, 0 to 0.7. Once this is satisfied, the low level controller will ensure that there are sufficient differences between the three kp values. It was determined that there should be a difference of at least 0.2 between each kp value in order for students to observe changes in actuator behavior as kpvaries. If the student would like to repeat the experiment, there is no restriction on the kp values other than that they lie within the safe range of values. The figure below shows the interaction between the high-level and low-level controller for this experiment. Low level control is within the dashed green box. The low level controller will provide the Bode plots and the output force vs. time data to the high level controller.
Sample output data is below:
Force Control Experiment - Free Play
In free play mode, the student will have more liberty to play with two parameters- proportional gain and the damping ratio. As can be seen in the flow diagram below, the low level controller will first verify a sufficient phase margin (via utilization of the associated system's open loop transfer function). Next, there are checks that the gain and damping ratio (ζ) are within defined bounds provided to the student. Once these checks are performed, the force control experiment is executed. If a requirement is not met, feedback will be provided to the student. The figure below shows the interaction between the high-level and low-level controller for this experiment.
Force Control Code
Matlab code which drives the Simulink model simulation and is called by both the low-level sequential and low-level free play code
Position Control Experiment - Sequential
The position control experiment will allow students to observe settling time and maximum overshoot performance parameters of the actuator. To do so, he/she will be able to select three desired arm angles for three experiments while the proportional gain remains fixed at 0.05, the damping ratio remains fixed at 0.9, and the cutoff frequency equals 10 Hz. The actuator will move its arm from the starting position of 60 degrees to the desired arm angle, and back to 60 degrees shortly thereafter. Since we have placed a constraint on the maximum speed of the actuator on the low level (less than 300 degrees/s) there is not a risk of the actuator reaching its maximum power output capabilities. Thus, the safety check will be to ensure that the entered desired arm angles are less than or equal to 120 degrees. The learning check is that there are at least 10 degrees of separation between each entered desired arm angle. When the students repeat the experiment, the learning check constraint will be removed. The figure below shows the UT-SEA arm in two different positions.
Sample output data is below:
The figure below shows the interaction between the high-level and low-level controller for this experiment.
Position Control Experiment - Free Play
In free play mode, the student will have have more freedom to vary numerous parameters - the desired arm angle, the proportional gain, the cutoff frequency, and the damping ratio. The bounds of these four parameters have been determined to allow students to explore a wide range of parameter values without compromising system safety. Through simulation it was determined that the upper kp bound should be 0.6 with position control implemented, rather than 0.7, the upper kp bound for the force control experiments. The safety check flow diagram is shown below:
Position Control Code
Matlab code which drives the Simulink model simulation and is called by both the low-level sequential and low-level free play code
Experiment Examples (Available on the Shared Lab Initiative Page)
(Note: Experiment 3: Disturbance Observer is outside the scope of this project.)
Other Explored Technologies
Hidden Markov Models (HMM)
Motivation
Our initial approach in developing a high-level controller to guide a student through the experiment procedures was to implement a finite state machine that interacts with the low-level controller at each state. The basic transitions for the finite state machine would be the steps of the experimental procedures. In forming the preliminary transition system, we explored the possibility of using a HMM to recover the most likely error states of the system given the information from the sensors and the student input. Since some transitions depend solely on student input rather than actions from the system, a HMM seemed to be a reasonable approach.
How it Works
Hidden Markov Models, like other Markov models, require that the states follow the Markov property - that is, the current state does not depend on other states or transition history. Not all of the states in a HMM are completely observable and complete control over the probabilistic state transitions is not possible, as shown in Figure 3.
Comparison of different types of Markov models. [6]
Issues with Implementation of Hidden Markov Models
A significantly large amount of data would be required to determine the transition probabilities for a HMM. Although these data can be obtained by running the physical system a large number of times, running the experiment procedure on the test bed would require a large number of students to test the system, since this is intended to be a platform for education. As a result, generating the transition probabilities would be infeasible and out of the scope of this project. Instead, we determined that a simple finite state machine with manually identified error states would be a better approach.
Signal Temporal Logic (STL)
Linear temporal logic deals with system models in discrete states and an abstract definition of time. In order to use the concept of temporal logic in the continuous, real-time domain, we considered the use of signal temporal logic. A comparison of the various types of temporal logic is shown in Table 1.
Comparison of different temporal logic schemes
| Linear Temporal Logic (LTL) | Metric Temporal Logic (MTL) | Signal Temporal Logic (STL) | |
|---|---|---|---|
| System | Transition | Transition | Hybrid/Continuous |
| Solution | Automaton | Timed Automaton | Control Signals |
| State | Discrete | Discrete | Continuous |
| Time | Abstract | Real | Real |
| Predicate | Propositions | Propositions | Inequalities |
Conlcusions
The output of this project is a firstly high-quality experiment content pertaining to dynamic system and control concepts as they relate to a cutting-edge robot actuator. A foundation of high-level logic implemented via a finite-state machine and a low-level safety and learning checker allows for safe and meaningful interactions between the student and the actuator to execute experiments. This output supports the deployment of a remote experiment platform, which will increase flexibility for students to complete exercises, and increase access to high-quality experiments to students around the world at different learning institutions to those who are completing degrees remotely.
Based on these developments the next step will be to implement the controllers on the Apptronik P170 SEA in order to test, refine, and deploy the experimental content on this system.
References
[1] Paine, N.A. (2014). High-performance series elastic actuation (Doctoral dissertation).
[2] Raman, V., Donzé, A., Maasoumy, M., Murray, R. M., Sangiovanni-Vincentelli, A., & Seshia, S. A. (2014, December). Model predictive control with signal temporal logic specifications. In 53rd IEEE Conference on Decision and Control (pp. 81-87). IEEE.
[3] da Silva, R. R., Wu, B., & Lin, H. (2016). Formal Design of Robot Integrated Task and Motion Planning. arXiv preprint arXiv:1604.05657.
[4] Raman, V., Donzé, A., Sadigh, D., Murray, R. M., & Seshia, S. A. (2015, April). Reactive synthesis from signal temporal logic specifications. In Proceedings of the 18th International Conference on Hybrid Systems: Computation and Control (pp. 239-248). ACM.
[5] Ghahramani, Z. (2001). An Introduction to Hidden Markov Models and Bayesian Networks. International Journal of Pattern Recognition and Artificial Intelligence. 15(1):9-42.
[6] Hollinger, G. (2007). Partially Observable Markov Decision Processes (POMDPs). [PowerPoint slides]. Retrieved from https://www.cs.cmu.edu/~ggordon/780-fall07/lectures/POMDP_lecture.pdf
Bitbucket Repository
Associated Paper
Attached is a paper that can be further updated and refined in the future to ultimately become a publication.