Abstract

Autonomous vehicles have become a vital part of the modern-day automotive industry that is looking for technologies which can improve and very well ensure the safety and well-being of humanity. In order to make a vehicle fully autonomous, attaining Drive-by-Wire (DbW) control is considered the initial step. In this project the DbW controls for a 2018 Toyota Prius has been developed using the comma.ai hardware, which is used to communicate with the vehicle and can be implemented and tested using a personal computer and joystick. The motivation for this project is to replace the existing system which is costly, bulky and most of the system components are proprietary of some company and furthermore not an open-source. Once the DbW controller is working, then it will make the transition to full autonomy easier.

The main objective of this project is to actually test the by-wire capabilities of the car using a general purpose computer and joystick along with the help of comma.ai hardware which is helping us to communicate with the vehicle’s ECUs by hacking into the CAN bus. The project focuses on the functional safety, design and development of a high level system architecture, component implementation and integration.

Introduction

In a modern day vehicle, with all the advances in electronic control technologies, the vehicle control has been shifting from mechanical systems to electronically controlled subsystems. Electronic control systems are increasingly used by the automotive industry to assist the driver or automate certain maneuvers of the vehicle. This move to partially automate the function of a vehicle has created the opportunity for computer-controlled systems to increase the efficiency of the vehicle's longitudinal and lateral control that ensures safety along with aiding driving comfort. These systems that aid advanced driver assistance also known as ADAS are common in most of the modern vehicles. Further technologies of ADAS that can be seen in several high-end vehicles such as Park-Assist, Adaptive Cruise Control (ACC), Lane Keeping Assist System (LKAS) allow electronic systems to take control of certain functions of the vehicle and use sensor feedback to control action taken by electronic control systems. All these functionalities can be considered to be stepping stones to reaching a fully autonomous vehicle.

Self-driving or autonomous vehicles are considered the future of transportation and artificial intelligence technologies will be required to reach their fruition. All major automakers around the world are investing enormously in improving autonomous vehicle technologies. An autonomous vehicle is fundamentally any vehicle that can move from one point to another without human intervention. One of the key prerequisites for a semi/ fully autonomous vehicle is the Drive-by-Wire (DbW) system. DbW technology allows functions in the vehicle that were formerly performed or controlled by mechanical transfer of forces to be controlled electronically. The technology replaces the traditional mechanical control systems with electronic control systems using electromechanical actuators and human machine interfaces. Once the vehicle’s longitudinal and lateral movement control can be attained using a by wire system, we are getting closer to realizing the final objective of driving the vehicle fully autonomously.

Through this project Eindhoven University of Technology expects to design and develop DbW controls for a 2018 model Toyota Prius with a joystick, a general purpose computer along with comma.ai hardware which includes a universal car interface called Panda and an adapter board called giraffe. In the following section a short description of all the hardware and software that was used in this project has been given.

Background and context

The DbW controller project is a new tailored project that has been introduced to the PDEng Automotive Systems Design team of 2018. The challenge presented to the team is to start from scratch including the functional safety study, design and development of the high level system architecture, planning and implementation, and all of this to be accomplished within a very limited amount of time.

The ASD team of 2018 consisted of seven people with different technical and academic backgrounds. The team has one project manager and two team leaders. The project manager was also assigned the role of a team leader. The remaining five members were divided among the two team leaders. One team focused on the functional safety and the other team focused on the technical design and implementation. Both the teams worked in parallel during the entire project. The team has organized as follows.

NameRoleContact
Functional Safety Team
Momen AbdallatifProject Manager/ Team Leaderm.a.h.abdallatif@tue.nl
Dhruv JaggaDesignerd.jagga@tue.nl
Akshay PakkathDesignera.pakkath@tue.nl
Salih YousifDesigners.e.a.yousif@tue.nl
Technical Design Team
Ashton MenezesTeam Leader/Designera.a.menezes@tue.nl
Achyuthan SundarrajanDesignera.sundarrajan@tue.nl
Varun KhattarDesignerv.khattar@tue.nl

Goal

The main goal of this project is to test the by-wire capabilities of the given 2018 model Toyota Prius. In order for this, a system has to be developed that can perform the DbW operation on the car and to come up with functional safety requirements for the DbW controller by performing a Hazard Analysis and Risk Assessment (HARA) and Failure Mode Effect Analysis (FMEA), developing a high-level system architecture, in which the functional safety requirements are implemented.

System Description

The DbW controller system communicates with the vehicle in a safe way enabling the user to talk to specific vehicle ECUs (LKAS & ACC) which will further facilitate to take control of the vehicle’s lateral and longitudinal movement. The system is installed in a 2018 Toyota Prius with Adaptive Cruise Control and Lane Keeping Assist System. The system acts as a gateway between an application and the low-level vehicle systems.

For the implementation of this project, a joystick, a personal computer, a USB cable, comma.ai hardware and finally the Robot Operating System (ROS) running on Linux and the Python programming language, both of which are installed in the personal computer has been used. The comma.ai hardware enables us to hack into the vehicle’s CAN bus which allows us to read and write CAN messages.  A short description of the functions of all the hardware and software used in this project can be seen below,

Panda

The comma Panda is a universal car interface that enables communication to the car over USB and Wi-Fi.

Giraffe

The comma Giraffe is an adapter board which acts as a fake On Board Diagnostics (OBD) port that help us to write and read CAN messages to and from the vehicle’s CAN buses that are not exposed on the on the main OBD connector.

ROS

The entire communication within our system is managed by ROS which is a robust, general purpose mechanism for creating applications for robotics. For this project Kinetic version of ROS has been used. All the necessary information is present in the form of different channels called ROS topics and processing this information from different channels is done using ROS nodes.

Python

Python is an interpreted, high-level, general-purpose programming language. Python has a design philosophy that emphasizes code readability, notably using significant whitespace.

Joystick

The joystick used is a regular gaming controller with a D-pad or an analogue stick. The controller used in the system is a Logitech gamepad F310 that connects to the PC through USB 2.0.

Computer

The PC is any desktop or laptop computer that runs Ubuntu 16.04. The ROS (version Kinetic) runs on top of it.

USB

USB is an industry standard that establishes specifications for cables, connectors and protocols for connection, communication and power supply between personal computers and their peripheral devices.

CAN

A CAN bus is a robust vehicle bus standard designed to allow micro-controllers and devices to communicate with each other in applications without a host computer.

Implementation

CAN

The CAN network of the Toyota Prius has 5 separate buses, each of which is intended for specific functions (and hence each channel will carry CAN messages related to its function). The primary bus has the highest number of ECUs connected to it, followed by the powertrain CAN and the park-assist CAN buses. The last two are of primary interest to the project because the lateral and longitudinal movement of the car is controlled by communication in those channels. While the OBD port of the vehicle is present in the primary CAN bus (thereby providing access to a lot of information), the autonomous driving functionality is almost always on the park-assist bus. This is so because the ECUs corresponding to the autonomous systems always work on a higher level than the ECUs that work on the car’s basic systems. For example, the lane keep assist ECU or the cruise control ECU operate on a higher level, receive and send messages about vehicle speed, steering angle, etc. from an ECU that works on a lower level and directly controls the engine or brakes or power steering system. Thus, it also means that any damage to the higher-level functionalities will not affect the lower level functions because the latter are more fundamental and hence, more important.

image-20190405140705-1.png

                                                                CAN architecture for 2010 model Toyota Prius                                                                              

It can be seen from the figure that the autonomous driving activity is associated with a CAN bus that is physically isolated from the low-level ECUs that directly control the car’s physical systems. The Drive Support ECU and the PMC ECU act as a sort of gatekeepers that abstract certain messages between the different channels, and only relevant messages can be seen on either bus. This can be partly attributed to hardware constraints of the protocol, where the number of unique message IDs on a channel is limited. While the nomenclature for the ECUs vary between OEMs and variants of the car (the terminology used is specific to the vehicle), here the DLC3 is used to denote the OBD-2 port.

In general, the pins 6 and 14 of the OBD-2 port are used for CAN. The standard set for the port mandates that it is the high-speed CAN, so its data rate will be 500 kbps. Based on the car manufacturer, the OBD port may also feature other communication protocols on other pins, or even another CAN channel. So, to make use of this port for CAN communication, the CAN messages are to be received by a device that reads CAN signals and encode them into meaningful data. Also, in the context of this project, the CAN data should be read and used by a computer, so they should be communicated through a port present in the PC. Hence an OBD to USB converter does the job perfectly. The handling of the data sent to the USB port is taken care of by dedicated software, or by user programs that read the serial input (USB is basically serial communication, and hence can be used by python code running on top of the OS). The device of choice here is a panda bridge from comma.ai, which is capable of bridging CAN and USB communication standards. It has an OBD-2 male port to USB 2.0 female port. It converts standard high-speed CAN bus data to serial USB data that can be read by programs running on top of an OS (USB driver -> OS -> custom software).

The panda bridge device can read the messages on the CAN channel to which it is connected to, however, in order to write messages onto the CAN bus, additional hardware is used. The giraffe adapter helps augment the basic functionality of the panda (which is reading the bus) with blocking the channel (which is similar to unplugging the panda), and passing the signals through panda. It does so with the help of hardware switches, which help configure the giraffe’s operating mode. It has a male and a female OBD-2 port, to accommodate the panda and to connect to the car’s diagnostic port. It also has an Ethernet port that has access to CAN1, CAN2, ignition detect, +12v, and ground. The giraffe is more of a passive device in the context of the project as soon as duplex communication mode is set.

The PC side of the system has the Ubuntu distribution of the Linux operating system running on the hardware, ROS running on Linux, and python code running on top of ROS. Hence the hardware requirement for Ubuntu 16.04 is 2 GHz dual core processor or better, 2 GB system memory, 25 GB of free hard drive space. The hardware requirement for ROS is 2 GB RAM, and a 1.6 GHz processor minimum. Linux has CAN support built-in to the kernel via SocketCAN. So that can be utilised by python files running on top of the OS to work on CAN. The primary python files, in the context of ROS, are called nodes. Nodes in ROS perform a self-contained and specific purpose. There are multiple assisting or auxiliary files, where function definitions, file-access and OS interaction are taken care of. The nodes and files associated with CAN operations are explained below.

ROS

In the figure below, the square boxes represent topics and the circles represent nodes. Information exchange between nodes takes place through topics. A node sends messages under a topic.

image-20190405173528-1.png

                                                                                  ROS Architecture

Writing messages to the CAN bus: There are two basic CAN messages which have to be generated by the joystick: steering and acceleration. The joystick is used to give input to the PC with ROS installed on it. This input is given to nodes Joy_to_acc and Joy_to_steer under the topic /joy. These nodes publish the topics Steer_amount and Acc_amount. For example, the function for calculating the acceleration amount:

class JoyToAcc(object):
    def __init__(self):
# publishing the steering torque
        self.pub_steering = rospy.Publisher('/steering_amount', Int16, queue_size=1)        
# publishing the acceleration amount                                    
        self.pub_acc = rospy.Publisher('/acc_amount', Int16, queue_size=1)    
# subscribing to the input from the joystick
        self.sub = rospy.Subscriber('/joy', Joy, self.joy_cb, queue_size=1)   

    def joy_cb(self, data):
        left_right = data.axes[0] * -1.0
        up_down = data.axes[4] * -1.0
        print data.axes

    # calculating the acceleration amount from the joystick input
        amount_acc = int(up_down * 1023)                     
        if amount_acc < 0:
            amount_acc = 0
        if amount_acc > 1023:
            amount_acc = 1023
        self.pub_acc.publish(amount_acc)

   # calculating the steering torque from the joystick input

        amount_steer = int(left_right * 3840)                
        self.pub_steering.publish(amount_steer)
        print("amount_steer: " + str(amount_steer))
        print("amount_acc: " + str(amount_acc))

The All sender node subscribes to these topics and publishes the messages under the topic Panda board. These are sent by the Panda through the Giraffe to the CAN bus of the car.

class AllPublisher(object):
    def __init__(self):
        self.p = Panda()
        self.p.set_safety_mode(self.p.SAFETY_ALLOUTPUT)
        self.acc_val = 0
        self.steer_val = 0
# subscribing to the acceleration amount
        self.sub_acc = rospy.Subscriber('/acc_amount', Int16, self.acc_cb, queue_size=1)      
# subscribing to the steering torque         
        self.sub_steer = rospy.Subscriber('/steering_amount', Int16, self.steer_cb, queue_size=1)                

    def acc_cb(self, data):
        print("acc_cb: " + str(data))
#  setting the maximum and minimum value of the acceleration
        if data.data > 1023:
            self.acc_val = 1023                             
        elif data.data < -1023:
            self.acc_val = -1023
        self.acc_val = data.data

    def steer_cb(self, data):
        print("steer_cb: " + str(data))
#  setting the maximum and minimum value of the steering torque
        if data.data > 3840:
            self.steer_val = 3840
        elif data.data < -3840:                              
            self.steer_val = -3840
        self.steer_val = data.data

Creation of CAN messages

There is a long hierarchy of functions used to create a CAN message. The highest level functions used to make the steering and acceleration commands use the packer file which uses the class CANPacker e.g. for making the steering command, the following function is used:

def create_steer_command(packer, steer, steer_req, raw_cnt):          

"""Creates a CAN message for the Toyota Steer Command."""

  values = {

# steering request can have 0 or 1 as value
    "STEER_REQUEST": steer_req,      

# the amount of steering torque    
    
"STEER_TORQUE_CMD": steer,              
    "COUNTER": raw_cnt,
    "SET_ME_1": 1,                           
  }
  # creating the steering command using the packer file
  return packer.make_can_msg("STEERING_LKA", 0, values)  

The function make_can_msg uses another function called pack_bytes:

def make_can_msg(self, addr, bus, values, counter=-1):
# calling the pack_bytes function
    addr, msg = self.pack_bytes(addr, values, counter)
    return [addr, 0, msg, bus]

The function pack_bytes appends the variables together using both the defined and built in pack functions:

def pack_bytes(self, addr, values, counter=-1):
    addr, size = self.name_to_address_and_size[addr]

# calling the pack function
    val = self.pack(addr, values, counter)                 
    r = struct.pack(">Q", val)
    return addr, r[:size]

The pack function is defined as:

def pack(self, addr, values, counter):
    values_thing = []
    for name, value in values.iteritems():
      if name not in self.sig_names:
# using the imported ffi package
        self.sig_names[name] = ffi.new("char[]", name)      

      values_thing.append({
        'name': self.sig_names[name],
        'value': value
      })

    values_c = ffi.new("SignalPackValue[]", values_thing)

    return libdbc.canpack_pack(self.packer, addr, len(values_thing), values_c, counter)

The package libdbc is defined as:

import os
import subprocess

from cffi import FFI

can_dir = os.path.dirname(os.path.abspath(__file__))
libdbc_fn = os.path.join(can_dir, "libdbc.so")
subprocess.check_call(["make"], cwd=can_dir)

ffi = FFI()
ffi.cdef()

libdbc = ffi.dlopen(libdbc_fn)

Reading CAN messages from the CAN bus

CAN messages from the CAN bus are read through Panda by a node called can_bridge which subscribes to messages under the topic Send_Can_messages and publishes them under a topic called Can_frame_messages.

class PandaBridgePub(object):
    def __init__(self, rate=1000):
# establishing node as publisher of CAN frame messages
        self.can_pub = rospy.Publisher('can_frame_msgs', Frame, queue_size=10)        
        rospy.loginfo("Setting up publisher to: " +
                      str(self.can_pub.resolved_name))
        self.rate = rate
        rospy.loginfo("Reading from panda board at " + str(self.rate) + " Hz.")
        rospy.loginfo("Connecting to Panda board...")

# using Panda class from the Panda library
        self.panda = Panda()                                                          
        rospy.loginfo("Setting safety mode to SAFETY_ALLOUTPUT")
        self.panda.set_safety_mode(self.panda.SAFETY_ALLOUTPUT)
        self.to_send_msgs = []

# establishing node as subscriber of CAN messages from the Panda
        self.can_sub = rospy.Subscriber('send_can_msg', Frame, self.can_cb,           
                                        queue_size=10)
        rospy.loginfo("Connected.")

    def can_cb(self, data):
        self.to_send_msgs.append(data)

    def run(self):
        rate = rospy.Rate(self.rate)
        while not rospy.is_shutdown():

# Reading gives us up to 256 messages
            can_msg_block = self.panda.can_recv()                              
# print can_msg_block
            # A message looks like:
            # [(420, 55639, bytearray(b'\x00f\x00\x00\x00\x00\x00:'), 0),
            # (428, 55761, bytearray(b'\x7f\xff\x00\x00\x00\x08\x002:'), 0),# ... ]

            if can_msg_block:
                self.process_msg_block(can_msg_block)

            if self.to_send_msgs:
# copying current msgs to send and reset buffer
                msgs = self.to_send_msgs[:]
                self.to_send_msgs = []

# sending all messages
                for msg in msgs:
                    self.send_can_msg(msg)

            rate.sleep()

The CAN frame messages are converted into a readable format for humans i.e. in a dictionary like format with topic string by a node called frame_decoder.

class FrameDecoder(object):
    def __init__(self):
        rp = RosPack()
        pkg_path = rp.get_path('panda_bridge_ros')
        self.can_msg_parser = load_dbc_file(pkg_path + '/config/' + 'honda_civic_touring_2016_can_for_cantools.dbc')

# setting node as a publisher of messages in human friendly format

         self.pub = rospy.Publisher('can_frame_msgs_human_friendly', String,queue_size=10)

# setting node as a subscriber to CAN frame messages                          

        self.frame_sub = rospy.Subscriber('can_frame_msgs', Frame,self._cb, queue_size=10)
        self.msg_id_to_msg_name = {}
        understood_msgs = []
        for msg in self.can_msg_parser.messages:
            this_msg = {}
            this_msg['frame_id'] = msg.frame_id
            this_msg['name'] = msg.name
            this_msg['signals'] = msg.signals
            this_msg['nodes'] = msg.nodes
            understood_msgs.append(this_msg)
            self.msg_id_to_msg_name[msg.frame_id] = msg.name

        rospy.loginfo("To interpret the messages:")
        pp = PrettyPrinter()
        rospy.loginfo(pp.pformat(understood_msgs))

    def _cb(self, data):
        try:
            msg = self.can_msg_parser.decode_message(data.id, data.data)
            msg['frame_id'] = data.id
            msg['message_name'] = self.msg_id_to_msg_name[data.id]
            msg['raw_msg'] = str(data.data)
            human_friendly = str(msg)
# string will look like:
            # {'message_name': 'STEERING_CONTROL', 'CHECKSUM': 5, 'COUNTER': 1,
            # 'STEER_TORQUE': 0, 'frame_id': 228, 'SET_ME_X00': 0,
            # 'raw_msg': '\x00\x00\x00\x00\x15\x00\x00\x00',
            # 'STEER_TORQUE_REQUEST': 0, 'SET_ME_X00_2': 0}

        except KeyError:
            msg = {}
            msg['frame_id'] = data.id
            msg['message_name'] = "UNKNOWN_MESSAGE"
            msg['raw_msg'] = str(data.data)
            human_friendly = str(msg)
        except ValueError:
            msg = {}
            msg['frame_id'] = data.id
            msg['message_name'] = "UNKNOWN_MESSAGE"
            msg['raw_msg'] = str(data.data)
            human_friendly = str(msg)

        self.pub.publish(human_friendly)

The CAN frame messages are also converted to twist type messages, a format readable by ROS, by a node called listener. Twist type messages give information about wheel speed and wheel angle to ROS.

class TwistFromCan(object):
# Constants
    WHEEL_SPEEDS_MSG_ID = 170

    KINEMATICS_MSG_ID=36

    def __init__(self):
        rospy.loginfo("Initializing TwistFromCan")
        rp = RosPack()
        pkg_path = rp.get_path('panda_bridge_ros')

# ensure that the dbc file is in the correct folder
        path_to_dbc = pkg_path + '/opendbc/toyota_prius_2017_pt_generated.dbc'
        self.can_db = cantools.db.load_file(path_to_dbc)
        self.wheel_speed = None
        self.yaw_rate = None

# setting node as a subscriber to can_frame_messages
        self.sub = rospy.Subscriber("can_frame_msgs", Frame,self.callback, queue_size=10)    
   # setting node as a publisher of twist messages

        self.pub = rospy.Publisher("twist_stamped",TwistStamped, queue_size=10)
        rospy.loginfo("Done")

    def callback(self, msg):
        msgID = msg.id
        if (msgID == self.WHEEL_SPEED_MSG_ID):
            data = self.can_db.decode_message(msgID, msg.data)
            speed_RL = data['WHEEL_SPEED_FL']
            speed_RR = data['WHEEL_SPEED_FR']
            self.wheel_speed = (speed_RL + speed_RR) / 2.0

        if (msgID == self.KINEMATICS_MSG_ID):
            data = self.can_db.decode_message(msgID, msg.data)
            self.yaw_rate = data['YAW_RATE']

Modeling of a Supervisor for DbW System

In this section, the plant of DbW system was modeled in Compositional Interchange Format (CIF) 3.0 platform. Overall system automaton was designed using the automatons of subsystems. These automatons consist of states of the plant, a relevant transition can be made in between these states depending upon the trigger of desired events. These subsystems interact with one another to achieve the desired overall system functionality. Later, a supervisory controller was designed for this plant by writing specific requirements coming from the high-level design document and the functional safety document. These requirements put constraints on the system functionality to avoid the conflicting states of the subsystems to be active at the same time and hence ensure system safety. To model the DbW system several plants were identified as follows:

Vehicle State

The vehicle has two states, one is a manual operating state (Manual) in which all the system functionality is controlled by the driver. Another state is system active state (DBW_Active) in which all the system functionality is controlled using the joystick by the operator. The state transition diagram for this plant is shown in Figure 3.

image-20190415102656-1.png

                              Automaton of Vehicle

System State

The identified system is DbW system which has three states. The system is initialized in OFF state (OFF) keeping the vehicle state in manual operating state. Another system state (Checking) allows, making all possible checks required to safely switch on the system. When all these checks are carried out, then only the system can be in the running state (Running). The state transition diagram for this plant is shown in Figure 4.

image-20190415103149-2.png

                              Automaton of DbW System

ON-OFF Button State

Another identified plant is the ON-OFF button. The possible states for this plant are on (ON) and off (OFF). The state transition diagram for the plant is shown in Figure 5.

image-20190415103149-3.png

        Automaton of ON-OFF Button

ROS State

Robot-operating-system (ROS) has three states. It is initialized from the off state (OFF) and can have two more functional states. One of these states is a read-only state (Reading) in which system is only allowed to read the CAN Bus and another is read and write state (Reading_Writing). In this state, the system can read data and write data on the CAN Bus. The ROS is only allowed to be in the read and write state if all the safety requirements are always fulfilled. The state transition diagram for this plant is shown in Figure 6.

1555317194645-683.png

                                             Automaton of Ros

ECUs State

There are two ECUs considered within the scope of the project, the states for the functionality of these ECUs are recognized to be the same which is either the ECU is in working state (OK) or in a state of fault (Not_OK). The state transition diagram for this plant is shown in Figure 7.

image-20190415103312-5.png

                                       Automaton of ECU

Emergency (EM) Button State

The emergency button provides the functionality of shifting entire control over the system from the joystick to the driver, instantaneously. The EM Button can have two states, either pressed (Pressed) or released (Released). The state transition diagram for this plant is shown in Figure 8.

1555317291096-741.png

                             Automaton of EM Button

Communication Channel State

The communication channel includes the hardware from comma.ai (Panda and Giraffe) and the vehicle CAN Bus. If all the connections are intact and messages are received to ROS with proper integrity, then the communication channel is in healthy state (Good_Health) otherwise it is in a bad state of health (Bad_Health). The state transition diagram for this plant is shown in Figure 9.

1555317331714-290.png

       Automaton of Communication Channel

Supervisor

Using the modeling in CIF 3.0, supervisory control is synthesized for the overall Drive-by-Wire functionality which will allow the system to make transitions in safe-states. Without the supervisory control, the subsystems in DbW model can be in any state. Thus, the system model is designed as maximal permissive. Not all of these states are desirable all the times to perform relevant system functions. The transition to these undesirable states can be prevented by creating requirements. These requirements are defined further in this section.

Requirements for the activation of DbW system

  • To initiate the system (initiate_system) it is required that the ON-OFF Button must be in Button_on state.
  • In the activation mode of the system, the ROS can be in Reading state only when the system status is in Checking state, the vehicle is in Manual state and ON-OFF Button is in Button_on state.
  • In the activation mode of the system, the ROS is allowed to be in Reading_Writing state only when the system status is in Running state, the vehicle is in DbW_Active state, ON-OFF Button is in Button_on state, all ECUs are in OK state, EM Button is Released and the communication channel is in Good_Health.

Requirements for the deactivation of DbW system

  • Deactivation of DbW is only possible when system status is in Running state.
  • The system must terminate when ON-OFF Button is in Button_Off state.
  • ROS must go to OFF state when ON-OFF Button is in Button_Off state.

Requirement for the Emergency Stop or Manual Override

  • Deactivation of DbW is allowed if EM Button is Pressed, ECUs are in bad health (Not_OK) or communication channel is in Bad_Health.

With these requirements in practice, the supervisor can trigger the controllable events in the model to ensure the desired functionality. The current supervisor can interact and control the states of several subsystems such as ROS, Vehicle and DbW System. The current requirements are sufficiently residing within the project scope, if required then the supervisor can be upgraded with more stringent requirements. The relationship of the supervisor with relevant subsystems is shown in Figure 10.

image-20190415103957-8.png

                            Relationship between supervisor and modeled subsystems

Simulation

The simulation process is carried out on the modeled plant for the DbW system, with the constraints on the functionality of the plant designed as system requirements. The system functionality is checked first for the modeled plant without any restrictions/requirements. Then it was validated after synthesizing the supervisor with the requirements using the data-based synthesis tool within the CIF tool. GUI and state visualization is used to observe and validate system functionality. See Figure 11.  A video is also provided with this report with the simulation of this exercise.

1555317643701-278.png

                               Simulation window with GUI input and state visualization

HMI

HMI is designed to observe the system state as shown in Figure 10.

image-20190415104111-10.jpeg

                                 HMI of DbW System in CIF 3.0 environment

CIF Simulation Video

Test Plan

Overview

The DbW system being tested is operated by an operator through a joystick, in addition to a safety driver, who is present to take control of the vehicle in case of any failure. The system is used to control the longitudinal and lateral movement of a Toyota Prius 2018 when activated by the operator. The system components under consideration for testing are Giraffe, Panda, ROS PC, USB bus, HMI (including the joystick), Lane Keeping Assist System (LKAS) ECU and Adaptive Cruise Control (ACC) ECUs. Testing of the system individual components is outsourced to different vendors with guarantee on their functionalities separately, the test plan will focus on the integrated system functionalities. 

Test objectives

The objective of these tests is to confirm that the system can correctly perform the functions listed below under normal driving conditions. The functions are:

  1. Steering through the joystick within the allowed latency.
  2. Accelerating through the joystick within the allowed latency.
  3. Steering and accelerating Simultaneous.
  4. Activating brakes through the DbW  system input within the allowed latency.
  5. Transferring control to the driver when pressing the emergency stop button.
  6. Warning in case of CAN bus failure, Giraffe/Panda failure, USB cable failure and joystick failure.
  7. Showing the status of the system (On/Off) to the operator and driver.

Test cases

Test case 1: Steering functionality

Purpose: To verify that the vehicle steers as desired by the operator through the joystick steering input.

Test procedure:

  1. Calibrate the sensors.
  2. Give predetermined values of steering input through the joystick.
  3. Read the measured steering angles from the vehicle steering sensor.

Expected result: The steering sensor should feedback the same steering angle as commanded by the tester.

Test case 2: Vehicle response time to steering command

Purpose: To verify that the latency between giving the steering input and the actual vehicle response is within the determined timeframe.

Test procedure:

  1. Give predetermined values of steering input through the joystick.
  2. Document the time between giving the steering input and the start of actuation.
  3. Document the time when actuation is done.

Expected result: The difference between inputting the steering command and the finish of actuation is within a TBD period.

Test case 3: Acceleration functionality

Purpose: To verify that the vehicle accelerates as desired by the operator through the joystick acceleration input.

Test procedure:

  1. Calibrate the sensors.
  2. Give a predetermined set of values of acceleration input through the joystick.
  3. Read the measured vehicle speed by the sensor.

Expected result: The measured speeds should correspond with the input values

Test 4: Vehicle response time to acceleration command

Purpose: To verify that the latency between giving the acceleration input and the actual vehicle response is within the determined timeframe.

Test procedure:

  1. Give a predetermined set of values of acceleration input through the joystick.
  2. Document the time of the actuation start.
  3. Document the time when actuation is done and vehicle reaches the set speed.

Expected result: The difference between the input of the acceleration command and the vehicle starting to accelerate is within the TBD period.

Test case 5: Steering and acceleration inter-operation test

Purpose: To verify that the vehicle steers and accelerates simultaneously as desired by the operator through the joystick steering and acceleration input.

Test procedure:

  1. Calibrate the sensors.
  2. Give a predetermined set of values of steering input through the joystick while also giving a predetermined set of values of acceleration input simultaneously.
  3. Read the measured vehicle speed by the sensor.
  4. Read the measured steering angle by the vehicle steering sensor.

Expected result: The steering sensor and speed sensor should give readings of the same entered steering angles and speeds.

Test case 6: Brake functionality

Purpose: To verify that the vehicle brakes as desired by the operator through the joystick brake input.

Test procedure:

  1. Give 100% acceleration input through the joystick.
  2. Read the measured vehicle speed by the sensor to confirm that the vehicle speed is 30 km/h.
  3. Press brake button on the joystick until the vehicle is stationary.
  4. Read the speed sensor measurement to confirm that the speed is 0 km/h.

Expected result: The vehicle should come from 30 km/h to a complete stop.

Test case 7: Brakes response time

Purpose: To verify that the vehicle stops completely after receiving the command of maximum braking within TBD time.

Test procedure:

  1. Give 100% acceleration input through the joystick.
  2. Read the measured vehicle speed by the sensor to confirm that the vehicle speed is at 30 km/h.
  3. Press brake button on the joystick until the vehicle is stationary.
  4. Document the time when the button is pressed.
  5. Read the sensor speed feedback.
  6. Document the time when the vehicle starts to decelerate.

Expected result: The vehicle starts decelerating after pressing the brake button within the TBD period.

Test case 8: Emergency button functionality

Purpose: To verify that the vehicle is controlled by the driver once the emergency button is pressed.

Test procedure:

  1. Operator to give an acceleration input through the joystick.
  2. Operator to give a steering input through the joystick.
  3. Read steering and speed sensors feedback to confirm reaching the desired speed and steering angle.
  4. Driver to press emergency button.
  5. Confirm that vehicle is not controllable by joystick anymore.

Expected result: Vehicle control transfers to the driver; operator cannot control vehicle using joystick.

Test case 9: Connection to CAN bus failure

Purpose: To verify that there will be warning in case of loss of information from the vehicle’s CAN bus.

Test procedure:

  1. Confirm vehicle is at rest.
  2. Confirm that the DbW system is active.
  3. Unplug the CAN bus connection – Giraffe or Panda.
  4. Check if there is a warning on the HMI.

Expected result: The HMI shows information loss warning once the bus is disconnected.

Test case 10: Joystick failure

Purpose: To verify that the DbW system warns the operator and driver in case of the failure of the joystick.

Test procedure:

  1. Set the vehicle speed to 0 km/h.
  2. Confirm that the DbW system is active.
  3. Unplug the joystick cable.
  4. Check the HMI for warning.

Expected result: HMI shows warning signal of joystick failure once the joystick cable is unplugged.

Test case 11: System activation and signaling (ON)

Purpose: To verify that the system will be activated and inform the driver and the operator about the status of the system lightbulb.

Test procedure:

  1. Activate the system.
  2. Check the HMI for system status.

Expected result: once the system is activated, the HMI should signal that the system is on.

Test case 12: System deactivation and signaling (Off)

Purpose: To verify that the system will be deactivated and inform the driver and the operator about the status of the system lightbulb_off.

Test procedure:

  1. Activate the system and check it is on
  2. Press emergency button.
  3. Check the status of the system in the HMI.

Expected result: Once the emergency button is pressed, HMI shows the system is off.

Conclusion and Recommendations

Conclusion

The Drive-by-Wire functionality for Toyota Prius has been achieved using comma.ai hardware to read and write CAN messages, a joystick to control the vehicle and a computer running ROS. The CAN messages were encoded and decoded using a dbc-file and functions from openpilot; a github repository of comma.ai. A supervisor has also been implemented to incorporate the functional safety requirements and design the modes of operation for the system.

Recommendations

The goal of this project was to read CAN and joystick information and thereby controlling the car by writing CAN messages to the CAN Bus, while also considering all the safety aspects. Based on the work which was done during this project and the results from the implementation, a list of recommendations are listed below.

  • Here a supervisor is designed to replicate how the system starts and behaves in case of emergency. This supervisor can be integrated with ROS and hardware features like HMI, emergency button and power button.

  • Certain ROS nodes for different application could be clubbed into ROS service. This ROS service could be initiated either by the supervisor or by a higher level controller.

  • CAN messages are published by the ECUs of the steering, engine and braking systems. It is required to know the health status of these ECUs at all times. A solution to this maybe to measure in real-time the frequency at which these messages are put on the CAN bus. This checking can be programmed in one of the ROS nodes by comparing the clock time of every consecutive message of an ECU.

  • The system goes into reading mode when manual override is taken over by actuation of the steering wheel, brakes or the accelerator pedal. Our system has access only to the CAN bus of the vehicle. One of the major challenges here is to identify when a manual input is given to the above actuators. A solution to this problem may be to carry out specific tests like manual steering and read what signals are varying in the CAN bus. By doing this it is possible to identify what information is generated when a manual input is given.

Manual for operating the System

This manual gives the procedure required to setup all the ROS nodes for running the System.

  • While operating ROS it is necessary that the all the information is available to a node before activating this node (python Function).
  • The system is divided into three functional parts, Reading CAN Messages, Reading the Joystick data and Writing CAN Messages.

Note: All the commands are entered in terminal.

Instructions for Reading CAN Messages

InstructionsDescription
Reading CAN Messages
  • Connect the Giraffe between the connector from the car and the front camera of the windscreen.
  • Connect the USB cable from the Giraffe to the Panda Board.
  • Plug in the panda board to the USB port of the laptop.
  • Open terminal in the catkin folder.
  • Run the command catkin_make
  • Initiate ROS by running the command roscore in a Terminal.
  • Type rosrun <package_name> panda_bridge_ros.py
  • Type rostopic echo /frame_messages to validate
  • Run the node Frame_Decoder in terminal using rosrun <package_name> frame_decoder.py.
  • Type rostopic echo /human_friendly to validate
  • Run the node can_listener_toyota in terminal using rosrun <package_name> can_listener_toyota.py
  • Type rostopic echo /Twist to validate

These steps  are required to have access to the CANBUS information in the vehicle. The Giraffe acts like junction box to tap information from this wire.

This compiles all the packages in the SRC folder.

This command activates a  node that creates an object that belongs to the Panda class. It publishes a topic “/frame_messages” using the information received from the Panda.

Make sure that in the code the path of the dbc file is correct. The frame decoder publishes the CAN messages into human readable format in a topic “/human_friendly” by subscribing to the frame messages topic.

Make sure that in the code the path of the dbc file is correct. The can_listener_toyota publishes twist messages by subscribing to “/frame_messages. Validate that the node is working by echoing the messages of the “/Twist” topic.

Instructions for Reading and Publishing Joystick Messages

InstructionsDescriptions
Reading Joystick Messages
  • Connect the joystick to the USB port of the laptop
  • Setup the Joystick node
  • Type rostopic echo /joy to validate
  • Type rosrun <package> joy_to_acc.py
  • Type rostopic echo /acceleartion to validate
  • Type rosrun <package> joy_to_steer.py
  • Type rostopic echo /steer to validate

For setting up the joystick node which publishes to the topic /joy, follow in the instructions given in the ROS Wiki page http://wiki.ros.org/joy/Tutorials/ConfiguringALinuxJoystick.

The joy_to_acc node publishes the /acceleration topic by subscribing to the /joy topic.

The joy_to_steer node publishes the /steer topic by subscribing to the /joy topic.

Publishing CAN Messages
  • Type rosrun <package> set_acc_steer.py

This node creates an object of panda class. This object is used to feed messages into the panda. The integer information from the steering or the acceleration topic is converted  to can messages and published to the panda board.

Tags:
Created by Mps admin on 2019/03/25 17:27
   

Need help?

If you need help with XWiki you can contact:

MPS Group