Software System

The WAM-V utilizes Robot Operating System (ROS), an open-source operating system for the team’s software platform.  Since the team comprises of mainly mechanical engineers, the number of competent students in programming is small. Therefore, ROS allows the team to use drivers and packages that are already developed and add onto it, alleviating the amount of programming needed for the WAM-V.

Color Recognition

Running on OpenCV2 (Open Source Computer Vision Library) environment, the Color Recognition code is written in Python. The main factor in deciding to use Python and OpenCV2 was the experience and proficiency of the team members in said languages. The code itself subscribes to a node in ROS that publishes image data. Then the code identifies color in the image through HSV thresholding. Contours are then formed around the identified pixels to show and track where the identified color is in the video or image.

In initial testing of the code a simulation of the RobotX competition, developed on the platform Gazebo, was used. The simulation, Virtual Maritime RobotX Competition (VMRC), contains not only the WAMV but also sensors, including camera, and obstacles found at the competition. Most importantly is the “Scan the Code” obstacle: the light buoy. VMRC’s light buoy can be programmed to display any color or pattern of colors; which can be then captured by the simulated camera into a compressed image format. This simulated data was ideal for testing because of its ease of programming and availability. In the Figure below is a view of the obstacle from the WAM-V’s perspective, on the left. Next to that image is the code’s color contrasted correction of the image used to determine not only the color detected, but also where in the image it detects it. Finally, on the far right, is the code marking the first image with a simple circle to help identify and track the detected color.

This is one of the first iterations of the code. In later iterations the contours’ color would not only match the identified color but also display the color’s name in text. The camera’s used for this, and other vision software on the WAM-V, were Logitech HD Pro Webcams. These were chosen due to their low price and USB compatibility. A ROS package, usb_cam, creates a node that interfaces with usb cameras and can publish image data to the onboard computer. This image data can then be sent to a second package, image_transport, which turns the raw image data into a compressed format. Compressing the images allows for less bandwidth being used over the wireless communication system and smoother live data at the ground station. The Color Recognition code will be run, manually, at the ground station so as not to take up computing power from the WAM-V’s onboard computer.

Image Recognition

The Image Recognition code was developed in MATLAB. Chosen, again, because it is the language that was most familiar to the team. Another factor for choosing MATLAB over other languages, was the many image and object detection algorithms offered with documentation. The team decided to use R-CNN detector, which allowed to train a neural network with images of the competition shapes supplied by the team itself. The detector returns the location of the objects with a labeled bounding box as seen below.

Robot Localization

The Robot localization code was developed in ROS. A ROS package called robot_loclization, meant to fuse multiple sensors from the sensor suite, 2 IMUs and a GPS, attached to the WAMV. The data is run through a extended Kalman Filter to produce a pose state estimate of the WAM-V’s position, orientation, linear and angular velocity. Once the code is activated its position at that time is used for the origin of the local and map frame for the code.

Waypoint Navigation

The Waypoint Navigation code was developed in MATLAB, and designed to communicate with ROS. The position, orientation and velocity of the WAM-V is published in ROS; the MATLAB code creates its own ROS subscriber to receive this data. The code also creates publishers to the motor inputs and goal inputs in ROS. The goal inputs are simply the x and y position and orientation desired for the WAM-V.

Once the code has its odometry data it calculates the error in position and orientation; as well as the distance, in length and arc, to the goal inputs. From these errors the WAM-V is determined to be in one of 6 states. These states can be set into two main categories: facing correct heading or not. Each of these categories are then broken into sub-categories: outside approach radius, inside approach radius but outside goal radius, and inside goal radius. First the code rotates the WAM-V to face the goal. Then the code is set to drive the WAM-V forward and controlling the path heading toward the goal. Once the WAM-V is within the goal radius and facing the correct heading the code cuts off the thrusters. The radiuses can be adjusted depending on the precision of movement of the WAM-V and accuracy of the sensors. As of right now based on the accuracy of the GPS the goal radius is set to 3 meters. The code controls the speed of the WAM-V by sending messages for the thruster’s voltages to the Arduino controlling the propulsion system.

High Current Box

The high current box of the WAMV contains all components capable of effectively controlling a minimum of 50A.  These components play the pivotal role of controlling the propulsion system. Effectively controlling the increased voltage requirement of the 110 lbs thrusters required replacing the previous iteration of the high current box.  The most major change in the high current box was replacing the previous motor controllers, 4 REV Robotics Spark motor controllers, with motor controllers capable of inputting 50V from the power system. During testing of the motor controllers, it was discovered the motor controller’s internal voltage polarity inverter was inoperable after switching the polarity of a large amount of voltage. In other words, if the WAMV was required to go from full forward to full reverse in a short period of time, the WAMV would be unable to return to full forward. To circumvent this issue, external reverse contactors replaced the motor controller’s internal voltage polarity inverters.  A 5V regulator was needed for a relay board on the reverse contactors; the relay board acts as a kill switch to the thrusters.

The previous motor controllers in the old high current system were controllable using pulse width modulation (PWM) signals. The new motor controllers operate on an analog signal.  One possible option for sending analog signals was using a digital analog converter. This possibility was explored but had too many issues which couldn’t be resolved in a timely manner (see Ineffective Solutions section). The solution that was ultimately chosen was using the Arduino’s PWM pins with a low pass filter to replicate an analog signal.  Low pass filters were designed for PWM frequencies of approximately 500Hz.

Another issue the new motor controller posed was increasing levels of interference in the high current box. This increase in interference caused digital signals to components, such as the relays, to float; resulting in uncontrollable rapid switching of power to the thrusters.  To help remedy this issue, major rewiring was required.

Previously, the master Arduino (controls majority of functions in hardware box except power kill switches) read the voltage of the power supply directly using a voltage divider; with this information the master Arduino can better determine how much power the motor controllers can draw from the power supply preventing overdraw. With the new arrangements, an Arduino Mega was added to the high current system which primarily reads the battery voltage, then sends an analog signal to the master Arduino scaled based on the voltage of the batteries.

To ensure interference does not affect signals from the low current box system, which contains all components responsible for controlling the motor controllers, reverse contactors, and kill switch system, both the low current and high current systems are electrically isolated using optocouplers.  Ten optocouplers were employed; 5 were used for digital signals to the reverse contactors; 4 were used for PWM signals to be converted to the motor controllers; 1 was used for the voltage reading Arduino.

Finally, to prevent overheating of components, a more advanced cooling system was required.  The original casing of the motor controller utilized a heatsink and fan to cool the motor controller. This system wouldn’t work in a sealed box as the air wouldn’t be able to carry the heat out and away from the motor controllers. To solve this problem, water cooling allows the motor controllers to be rapidly cooled. The heat is carried by the water and brought to the external radiator to leave the system.

The high current box has multiple safety systems in place, which include relays, circuit breakers and the regulator disconnecting switch.  These systems are designed to keep human operators safe and prevent damage to the other components in the box.  The relays cut the power that is ultimately going to the thrusters, preventing them from spinning.  This is done via the kill switch system, either by flipping a switch on the RC transmitter or by buttons located on the port and starboard sides of the WAM-V.  The kill switch system sends a signal from these inputs and opens or closes the relay.  The circuit breakers are designed to open in the event of a current spike going to the thrusters. This opening of the circuit prevents current from flowing, protecting the other components in the system. The regulator disconnecting switch is a circuit breaker with a different use. Functionally, it can perform all the previously stated functions of the circuit breaker. In this context, however, the circuit breaker allows for manual disconnection of the voltage regulator modules. When the voltage regulators were directly connected to the main batteries, sparking would occur at the junction between the high current box and the battery connection. Using the circuit breaker allows us to temporarily disconnect the voltage regulator from the main battery and thus preventing sparking.

Low Current Box

The low current box received much less alteration than the high current box. The low current box contains components which requires less than 10A.  As previously stated, the low current box as a whole is responsible for controlling the components as the high current box and ultimately the propulsion system.

The low current box receives signal from the wireless pole and outputs signals to the high current box. All three subsystems are connected using RJ45 Ethernet cables, which have been shown to be much better at maintaining connectivity in maritime conditions compared to the previous ethernet choices. In terms of power, the low current system operates on four 14.4V Lithium Ion batteries, totaling up to 290 Wh of power.

There are 3 main components which make up most of the control electronics of the WAMV: 2 Arduino Megas and an Intel NUC.  One of the Arduino Megas is designated as the Master Arduino; the Master Arduino is responsible for controlling the motor controllers and the reverse contactors. It can switch the WAMV between two modes: manual and autonomous.  Manual mode allows a user with an RC transmitter on shore to control the movements of the WAMV. The other Arduino Mega is designated as the Kill Arduino. (See kill switch system section)

At the moment, the possibility of adding visual data to the adding visual data to the WAMV’s sensor package using cameras is being explored. A cost-efficient option for testing how difficult integrating this new sensor input would be is to use USB connected web cameras. Four such cameras were added to front, back, and sides of the WAMV. Each camera is held in a transparent, waterproof 17.5cm x 11.5 cm (7” x 4.5”) husky box with Styrofoam for cushioning. The camera’s USB cables were connected to the onboard computer through a plate on the side of the low current box with USB ports.

An Intel NUC serves as our onboard computer because of its compact size, compatibility with DC inputs which is how most components in the low current box are powered, and processing power. It functions as a platform for executing ROS packages and autonomous code stored in its memory. The Intel NUC can also broadcast GPS location, odometry data, and camera feeds to the ground station using the onboard wireless router.

Wireless System

The Wireless System encompasses the Fr-Sky Receiver, the 2.5 GHz antenna, 5 GHz antenna, and the 900 MHz antenna.  The Fr-Sky Receiver directly communicates with the manual controller and sends signals to the low current box over 16 channels.  This component is located on the wireless pole, along with the 5 GHz antenna.  The 2.4 GHz and 900 MHz antennas are mounted on the WAM-V platform. The various antenna, the 2.4 GHz, 5 GHz and the 900 MHz, allow for communication between the low current box and the ground station.  The various antenna enables flexibility in networking options. In 2016, all competing teams used a 2.4 GHz frequency for wireless communications which caused interference.  The ability to switch frequencies allows for avoidance of busy frequencies.

Kill switch system

The current kill switch system consists of a Taranis QX7 transmitter, a FrSky R9 receiver, an Arduino, two relay boards, two physical buttons on the WAM-V, and LED strip. Before, what was used was a Digi XBee radio module on an Arduino but changed to using a channel on a R9 receiver to send a kill signal to an Arduino. This change was due to the XBee having difficulty sending a signal over a distance of about 300 meters while the WAM-V is out in open waters. The Taranis QX7 transmitter paired with the FrSky R9 receiver can cover a distance just over 1 km. The operational procedure of the kill switch system is to have the Arduino monitor the state of the WAM-V via the two physical buttons attached to the WAM-V and a remote switch on the Taranis QX7 transmitter. If either of the buttons or remote switch is engaged, the Arduino sends a signal to the relay board which cuts power to the system. In the event the Arduino shall fail, the relay boards are configured to be normally open which kills the WAM-V system. In the event the transmitter loses power, the R9 receiver has a failsafe setting to send the kill signal to the Arduino. An LED strip is used to indicate the state of the WAM-V. This LED strip is also controlled by the kill switch Arduino via the second relay board.

Transmitter

The Taranis QX7 transmitter was chosen for its 16-channel configuration on a 2.4 GHz connection. 16 channels provide more manual control of the WAM-V system and the 2.4 GHz connection provides more channels to avoid signal interference.

Propulsion

The previous propulsion system utilized four 55 lb thrusters in a holonomic drive configuration that had allowed the WAM-V 3 degrees of motion and precise maneuverability. However, the disadvantage of the holonomic configuration lied in the fixed positional angles of the thrusters. This was inefficient for fully moving forward or backwards compared to a normal drive train.  Another issue became apparent when the ocean currents at a sea state level of 2 was able to easily push the WAM-V around during systems testing at Sand Island.

Two solution strategies were developed to mitigate the problems: increasing the power of the propulsion system and developing articulating thruster mounts. The thruster force was changed to 110 lb, mitigating the problem of not being able to hold position in certain orientation, while articulating thruster mounts were developed to provide the WAM-V the capability to mechanically change the propulsion system from a holonomic configuration to a normal drive-train. Assuming the boat is moving at 3 knots and force is applied on the motor shrouds, computational fluid dynamics (CFD) was used and resulted in a max moment of 65 Newton-meters. Thus, the articulation thruster mounts were designed to utilize a worm gear, stepper motor with a torque of 12 Newton-meter, and a 1 to 5 gear reduction to combat the CFD max moment.

Modular Mounts

A Modular Mounting Rack was designed, prototyped, and developed for the battery, hardware, and sensor system. It was created to be mission adaptable for various instrumental setups.

Energy

The power system was improved to be 36 V for it to be able to support the new propulsion system of four 110 lb thrusters. In the old configuration, the power system was 12 V for the 55 lb thrusters. Due to Hawaii’s geographical location, battery shipments are expensive and difficult.  Therefore, the team modified the current battery system by putting them in series, ultimately becoming a 50V power system.

Front Thruster Mounts

The design for the front thrusters mounts had to be changed. The original design had the thrusters positioned on the side of the pontoons, but due to the weight of the thrusters, there was too much torsion. The new design has the thrusters positioned in front of the pontoons, and is made out of aluminum bars, and zinc plated fasteners.

    To ensure that the design would work, some calculations were done to find the maximum moment this is created by thrust when the propulsions are in the forward direction. The total  moment that is created by one thruster is 110lb, so with two thrusters the total moment is 220lb.

When the total moment is distributed equally to each bolt, the moment equals 36.67 lb*ft, but the forces at each bolt differs. To find the forces acting on the individual bolts, the equation

F= M/d [1]

Where M is the moment of the individual bolts, and d is the distance from the center of the arm. The shear forces calculated at each bolts are shown in Table XX, with the maximum shear force at one bolt being 146.68 lb.

Table 2.3.2.1.2.1: Forces Calculated at Individual Bolts In the Forward Direction

Next, the it was calculated to see if the bolts will fail under moment. Since the bolts are rated with tensile stress, it is not calculated with the shear force over the area of the bolt to get shear stress. Another thing is that the plates are held together with compressive force caused by the bolt and nut creating friction between the plates, not with the shear stress of the bolt. The friction coefficient value of  Aluminum is 1.05-1.35, taken from a Friction Table[https://www.engineeringtoolbox.com/friction-coefficients-d_778.html]. As determined by a Fastener torque chart [https://www.engineersedge.com/hardware/torque_vs_tension_bolts_13355.htm], a ¼-20 Bolt has a clamp load of 2020 lb and a torque of 8 lb*ft. The frictional force can be calculated by using the equation

ff= μ N[2]

Where μ is the friction coefficient, and N is the normal force. In this case, the normal force used is the clamp load. Multiplying the clamp load and the aluminum friction coefficient   together, the ff equals 2121 lb, while the tensile strength is 120,000 psi. If the shear forces at each bolt, or just the bolt with the max shear force, is greater than the ff, then there is a greater chance that the bolt will fail. When comparing the max shear force with the frictional force, 2121 lb >>146.68 lb, the bolts will not fail.

Next, calculations were done to find the maximum moment by thrust and weight when the propulsion is going in the reverse direction. If it is assumed that 70% of max thrust is used when going in the reverse direction, then the moment of the thrust is found to be 154 lb*ft, and the moment of the weight is calculated to be 297.4 lb*ft. By adding the thrust moment and the weight moment together, the total moment is 451.702 lb*ft.

    When the total moment is distributed equally to each bolt, the moment equals 75.28 lb*ft, but the forces at each bolt differs. To find the forces acting on the individual bolts, the equation [1] is used. The shear forces calculated at each bolts are shown in Table XX, with the maximum shear force at one bolt being 301.2 lb.

Table 2.3.2.1.2.2: Forces Calculated at Individual Bolts In the Reverse Direction

Next, the it was calculated to see if the bolts will fail under moment. Using the frictional force found for the forward direction, 2121 lb, and comparing that to the maximum shear force, 301.2 lb, the bolts will not fail.

LIDAR

There has been two major changes to the Sensor Fusion and Signal Processing Subsystem.The first being a change in the setup for the LiDAR; second is the inclusion of a smartphone to the sensor suite. The figure below shows the mounting system for the Velodyne LiDAR; including the placement of the new IMU and webcam. Previously the WAM-V had two UM7 IMUs dedicated to fusing in the robot_localization package, in ROS. However, attempts at fusing the IMUs’ and the Adafruit Ultimate GPS raw sensor data have been unsuccessful. Therefore the second IMU was deemed redundant and was determined to be paired with the LiDAR. The IMU provides the LiDAR with its orientation and angular velocity; which is necessary for the LiDAR to perform SLAM. The IMU is mounted as close to the LiDAR as possible, and oriented with their axes in parallel. Any offset in either position or orientation must be corrected in software; therefore this placement attempts to minimize these corrections and make them repeatable when mounting the LiDAR. The webcam added to the setup to provide image data to be fused with the LiDAR point cloud data to reconstruct the WAM-V’s surroundings in 3D. The GPS shown in the figure below came with the Velodyne LiDAR. It provides not only GPS data but also timestamps for the LiDAR.

As mentioned previously, the second change is the inclusion of a smartphone. After failing to fuse raw sensor data in robot_localization it was decided that a new sensor fusion algorithm needed to be pursued. One of the most prevalent algorithms come in the form of smartphone mobile processors; particularly from the company InvenSense. Due to the time constraints, and the unlikely hood of InvenSense partnering with the team by providing the mobile processor, it was decided that the best course of action for accessing the fusion algorithm would be to simply access it from the smartphone itself. This option also came with the benefit of providing another set of IMU and GPS. Shown the figure below, The smartphone’s IMU will provide the InvenSense algorithm with acceleration and heading data; which is fused into an orientation estimate. This new estimate is then fused with the IMU acceleration and the GPS position data in the robot_localization package to provide a new position and orientation estimate. The smartphone is accessed by the WAM-V’s hardware box using an a combination ethernet cable and ethernet to mini-USB adapter.

Team Kanaloa is looking for dedicated students of all backgrounds and skill levels!  No prior knowledge or experience is required; however, dedication, a positive attitude, and a willingness to learn is mandatory.

If you are interested, please fully read our Kanaloa Team Member Expectations document shown below (it’s two pages long!).  Then fill out our new member application form here (note: you must be signed into your hawaii.edu account by default in your browser to access the form!  If you are signed into a personal google account, sign into your hawaii.edu account in a new incognito tab, and open the form in that tab).  After filling out this form, we will contact you to schedule a recruitment meeting.

If you have any questions, please contact us.