Robotic Hand for ASL
August 2018 – December 2018
Summary
My team’s prototype is a robotic hand designed to perform American Sign Language. The basics of our design include 3D-printed fingers and a 3D-printed palm-forearm assembly adhered to a laser-cut acrylic box. The box houses servo motors connected to tendon strings that are threaded through each finger, such that actuating the motors contracts the fingers. Additional elastic strings are also threaded through each finger and attached to the box, so that in the absence of contraction from the servos, the fingers snap back and remain in the open-hand position. Although there is no video for the hand performing the number “one”, it was perfectly able to do this just as well as all the other numbers – we just did not get a recorded video of it before it was disassembled after the demo session ended.
Context
Regardless of what age a person is, learning sign language can reap a number of benefits for them and the people around them. For babies and toddlers, sign language allows them to nonverbally communicate with their parents. At an elementary or high school level, learning American Sign Language (ASL) has the same mental and social advantages as being bilingual. In fact, an increasing number of schools have been trying to offer ASL classes as a secondary language. Several colleges are considering ASL as a potential fulfillment of their foreign language requirements. ASL teachers and advocates even consider it a foreign language due to its unique grammar structure. Lastly, at any age, knowing ASL mitigates the gap between both the hearing and deaf communities. Therefore, society as a whole could benefit from learning ASL.
In order to to aid the learning process and make ASL more accessible, our project aims to build a prototype that will be able to interpret typed (English language) text and produce a physical representation of ASL. The idea is not to replace human interpreters, but supplement them in sign language education classes. This will help make classes be more interactive since one-on-one education may not always be possible where interpreters may not necessarily be plentiful or available. It will also facilitate the retention of ASL since it is a visual language above all. Eventually we would want this project to be able to “translate” most of the English language at a reasonable speed. If the prototype could be able to interpret letters, words, or even common phrases, then its purpose can be extended beyond use in the classroom. It can be applied to locations that are not often accessible to the deaf community or making interpretation services more readily available in cases of emergency.
Currently, what we want to tackle is having a robotic hand be able to interpret a set number of inputs (in our case, the numbers 1-9) and change position accordingly. We chose to have these inputs because these numbers only require the use of one hand. More specifically, they only require movement of the five fingers on the hand, not the palm or wrist. For this reason, our prototype’s main goal is to mimic the contraction and extension of human fingers in a way that will be recognizable as ASL. Specifically, we want each finger segment to be able to bend down 90 degrees from its resting position on its own and be able to return to its resting position after interpretation without intervention. Another design goal is creating infrastructure to keep the hand stable while it is moving. This infrastructure will also hold all the mechanisms needed to move the hand and the keypad for inputs.
Design Description
Feature One - Finger
One key feature of our prototype was an individual finger. In our first implementation, we created one finger with hopes that we could repeat it roughly four and a half times for the rest of the digits on the hand. Initially, we used 5 separate 3D-printed parts to mimic the 3 segments of a finger. The bottom 2 segments would use 2 parts each to allow the strings and hinges to be hidden within the finger. This feature was first made in Fusion with inspiration from other prototypes mentioned in this paper and our own fingers. We then created the hinges to join the segments as well as two sets of holes running through the front and back for the extension and contraction strings. We used nano-cord for the contraction since it was strong enough to pull the finger joints without snapping. We planned to use rubber bands in the back to provide tension to pull the finger back. After assembling the finger, we connected the string to an encoder on a servo motor that was programmed to rotate 180 degrees.
This finger was evaluated on two criteria: could it contract each of its segments to 90 degrees and could it return to its original position without manually being reset? We discovered that the finger could be pulled by the motor but not to the extent that we wanted. However, it could not retract to its original position at all since we could not fit the rubber bands through the back holes. We also had some issues clearing the PLA in the holes we created for the strings in the finger.
We modified our design by revising the Fusion file first. We made both sets of holes twice as large, which made clearing them much easier. We also decided to use a drill to speed up the process. We also ordered elastic string for the extension since it would fit more easily in the hole. We also doubled-up on each string (2 sets of nano-cord in the front and 2 sets of elastic in the back) so that the movement would be stronger.
After we got the finger working individually, we still faced several integration problems. First, we used the same finger design for each digit, but vertically scaled their heights. This made some of their hinges (that connect to the palm) too small to fit in the spaces meant for them. This made them looser than expected, causing them to meet at the palm at an angle rather than perpendicularly. To solve this, we created some additional parts to insert under the fingers to counteract the angle. Another issue was that the fingers would rub against the palm and the friction would delay or stop movement. To solve this, we sanded the edges of both until they would not intercept anymore.
In its final performance, the fingers performed extraordinarily well in contraction. Each of them had the capability of being bent into the palm. The thumb had some issues because of the angle it was put at, but it could still recognizably be bent. However, in its extension, only the middle finger could be returned to its original position without human intervention. The other fingers required manual reset.
Feature Two - Palm & Forearm
Another key feature of our prototype was the palm and forearm. In our first implementation, we used a laser-cut piece of acrylic. At the time being, the purpose of these pieces was to hide the string mechanisms and hold the fingers upright. When we say that acrylic would not serve this purpose well, we 3D-printed the two halves of the palm and forearm. We also made modifications on the placement of the thumb in the palm, moving it move upward to prevent it from falling out.
Some integration problems we had with the palm was that it did not have an easy assembly/disassembly process. For this reason, if we needed to make modifications to the fingers, we would have to remove the hot glue holding the two parts together. To ease this, we would soak it in acetone prior to removing the hot glue. Another issue that we had was that the strings would get tangled within the palm/forearm during their movements. To resolve this, we decided to laser-cut a slat with 10 holes in it for each of the 10 pairs of strings to prevent them from tangling with each other.
In the end, the palm and forearm served their dual function well. They were able to conceal 7 of the 10 pairs of strings. Two of the pairs were for the thumb which would not move properly had the strings been kept in the palm (since it entered at an angle). Another pair of strings were the elastic for the index finger that broke last minute. We decided to thread them outside the palm instead of risking to open the palm again. The palm and forearm were, however, sturdy enough to hold up the hand even whilst it was moving.
Feature Three - Box
The third key feature of our prototype was the box beneath the hand. Our first implementation of the box was in our second checkpoint. We used a cardboard shipping box of small to moderate size, and it essentially served as the housing and structure of our entire hand. The hand was mounted on the top of the box, and the tendons were threaded down into the box. The tendons were all attached to circular gears on multiple axles that were mounted width-wise across the inside of the box, terminating on the outside with small handles. Rotating the handles on the side of the box would in turn rotate the axle and gear that the tendon strings were attached to, providing for contraction of the fingers.
We evaluated the performance of this initial implementation of the box by assessing how feasible it would be to implement in the final design. We knew from the start that our prototype would require unique actuation of each finger. We also knew, and had been advised, that the most practical way to pull and release on the tendon strings would be to use servo motors. Taking all of this into account, using the box to house five axles and gears (one for each individual finger) did not seem feasible. First, each servo takes up a lot of space within the limited real-estate of the box. Second, the forces brought about by the servo motors would complicate the process of creating a secure axle for each one, as well as securely mounting each servo to its axle so that there would be no unwanted sliding or rotation of any of the servos on their axles.
Moving forward, in our final prototype, we modified our approach by doing away with the gear and axle idea. Each servo came with servo encoder horns to which the strings could be directly attached. Thus, we would no longer need gears. Then, by mounting the servos directly to the sides and floor of the box, we would no longer need axles either. This modified design was not only much more feasible, it allowed for better stability while being much less complex. Attaching each servo to the microcontroller as a separate output would allow for discrete control of each servo, and thus, each finger. Laser-cutting the box out of acrylic would give us more structural rigidity and control over the size. Overall, our modifications would allow us to reach our performance target.
We did face a number of obstacles when integrating our new box design into the final prototype. Mounting the hand and forearm was simply enough, only requiring us to include a laser-cut hole on the top of the box through which to thread the strings, as well as hot glue for adhering the hand and forearm assembly to the box. However, mounting the servos to the box proved difficult. There was a lot of force involved in the servos pulling down on the tendon strings, because they were overcoming constant tension from the tension strings counter-attached to each finger. Over time, the servos would detach from the inner surfaces of the box, despite multiple forms of adherent (hot glue, tape, etc). With advice from Professor Guimbretière, we laser cut additional acrylic supports in specific dimensions, such that they could be mounted in-between the servos and the roof of the box. Not only did this keep the servos in place, it helped us solve another force-related box integration issue we encountered; the roof itself flexing during contraction of the fingers. The servos were interspersed evenly across the inside of the box, so the acrylic supports we put in did a good job at keeping the roof stable and in place.
The final performance of the box was very good. The supports we implemented seemed to be the last piece of the puzzle – after they were put in, we did not encounter any more issues with the servos’ mounting-points inside the box, nor any more issues with the box itself losing integrity from the force of contraction. Ultimately, we were able to achieve the stable housing for our mechanisms that we originally envisioned when designing our box.
Future Work
From our work on this prototype, there are several improvements we could make on another implementation.
First, we could improve the Fusion files for our 3D-printed fingers and palm. For the fingers, we would lengthen the hinges so that the segments did not collide as much during movement. Within our prototype we sanded the edges to mitigate this, but we could have also used the “fillet” option to make the edges more smooth to begin with. We also would not have scaled each of the fingers from original finger design. This caused some of our fingers to be too loose within the palm structure. Instead, we would create each finger (or at least each bottom segment) to fit the palm correctly. Lastly, we would include a set of slats within the palm itself to keep the internal strings from becoming tangled. This was an issue with our prototype since tangled strings would cause multiple fingers to move when we would only want a particular one to move.
Second, we would incorporate a pulley system for the thumb. Currently, our thumb does not bend as much as it could because a lot of the work the string does in pulling down is lost in pulling it across an angle. An internal pulley would help to decrease the work lost. This would also keep the strings internal, which we did not have for the thumb in the final prototype.
Finally, we would create a move cohesive structure for the interface. This could include making the box big enough to house the entire breadboard with appropriate holes for the adapter. It would also have enough support to hold up the hand. These supports would ideally have holes to hold the servo motors, so that we would not have to hot glue to the unused breadboards on the bottom. This would also make the structure more stable and more pleasant to look at.