Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Novel Approach for Sensing the Humanoid Hand Finger Position Using Non-contact TMR Sensor

1 School of Mechanical & Manufacturing Engineering, National University of Sciences and Technology Islamabad H-12, 44000, Pakistan ↗
2 U.S.-Pakistan Center for Advanced Studies in Energy, National University of Sciences ↗
Publication year
Digital Object Identifier10.54941/ahfe1001599

Abstract

The robotic hand robustness is necessary for different applications. In the past two decades, the magnetoresistance (MR) sensor technology has developed into a new field of magnetic sensing based of tunnel magnetoresistance technology (TMR) that is expanding rapidly. The hand gesture and position of finger can be found out with the help of different kind of sensors each kind of sensor have its challenges as well as the limitations. Smart sensors like TMR can provide multiple information with very small footprint, low power, and cost. This paper presents a novel technique to sense the robotic or human hand finger position with the help of non-contact magnetic field sensor called TMR sensor. The human hand finger position has been measured at four different positions. The results validate that the four finger positions has been detected with the help of single sensor with precision.

INTRODUCTION

Daily activities such as taking a cup or delivering the information to others, such as gesturing leaving, need the use of hands. As the world’s population ages, neurological illnesses become more common, resulting in a loss of hand function and a lower quality of life (Jiang, Kang, Song, Lo, & Shull, 2021). Gesture recognition is a feature of pattern recognition and recognition chal- lenge in which a movement is labelled as correspond to a specific class. The response of a gesture recognition system could solve problems in a variety of fields, including medical, robotics, sign language, human–computer inter- faces, security, virtual reality, and augmented reality (Nogales & Benalcázar, 2021). MR sensor technology has evolved into a new field of magnetic sensing based on tunnel magnetoresistance technology (TMR) that is quickly incre- asing during the last two decades. Magnetic field alterations are measured

© 2022. Published by AHFE Open Access. All rights reserved. 38

Novel Approach for Sensing the Humanoid Hand Finger Position 39

without any contact with MR technology, which consumes low power and uses low-cost sensors with small integrated circuit (IC) designs. Magnetic field (mf) is recognized to be related to current and to play a vital role in electric power systems (Z. L. Wang, 2017). Magnetic sensors have been used for a variety of applications, including current detection, rotational speed sensing, and motion sensing. Solid state mf sensors have discovered a cost, size, and power consumption benefit over other existing sensor technologies. Magneto-resistive (MR) sensors are one form of sensor that works on the pri- nciple of magneto-resistive effect, in which the resistivity change in a material (resistance change) is measured in the presence of a magnetic field. The disco- very of new types of MR materials, such as anisotropic magneto-resistance (AMR), giant magneto-resistance (GMR), and tunnelling magneto-resistance (TMR) materials, fabrication of advanced electronics, and its application in various aspects have increased the role of MR technology in the field of power grid, energy, and the environment (C. Wang et al., 2018). TMR sensors, which are still relatively new, have improved in terms of sensitivity and small size. These magnetic sensors have recently been found to be suitable for current measuring applications in transmission and distribution systems, according to recent study. They are distinguished by excellent sensitivity, low power consumption, low cost, compact size, and broad band coverage (Jamone, Natale, Metta, & Sandini, 2015).

In literature different techniques are reported regarding static and dynamic hand gesture recognition. Static gesture recognition techniques refer to algo- rithms and approach for recognition of static hand. This can be real human hand, or it can be artificially man-made hand which can be placed statically to recognize the gestures through different learning algorithms. Researcher in (Nguyen, Vo, Huynh, & Meunier, 2014) proposed a vision-based hand gesture recognition technique for static hand gesture recognition. Authors in (Li, Yang, Wu, Xu, & Wang, 2012) proposed a hand gesture detection system based on histograms of oriented gradient (HOG) characteristics for the Kinect sensor. This method involves finding the aspects of hands characte- ristics and selecting hog features that are both geometric moment invariant and suited to the light transform.

Dynamic gestures refer to recognition of gesture for moving hand. Diffe- rent research has been proposed in the literature regarding dynamic gestures recognition using different approaches. Researchers in (Zhang & Tian, 2013) developed a new dynamic temporal pyramid organization approach for an edge enhanced depth motion map framework to model various hand gestures based on their visual effects. The study (Shiravandi, Rahmati, & Mahmoudi, 2013) developed a Bayesian dynamic network-based combina- torial technique for hand gesture recognition. Using the YCBCR color space transformation, study (Lai & Lai, 2014) presented a real-time dynamic hand gesture identification system. A new hand gesture model incorporating star- ting hand posture, middle motion trajectory, and end hand posture was reported in a study published in (Duan, Zhang, & Ma, 2011). The start and end of a hand gesture can be recognized by sensing changes in movement rate. A systematic review for different hand gesture recognition techniques based on different applications is carried in (Yasen & Jusoh, 2019). Recently,

study presented in (Benito Temprano & McPherson, 2021), TMR sensor is deployed on electric guitar for gesture acquisition and disambiguation.

To the best of authors knowledge, based on literature carries, no litera- ture has been found application of TMR sensor in dynamic or static hand gesture recognition. Only one study has been found but it relates to gesture recognition of hand on guitar movement (Benito Temprano & McPherson, 2021). This paper proposes a concept study regarding application of TMR sensor in static hand gestures recognition. The study is carried out on diffe- rent angles of hand, and application and performance of TMR sensor have been evaluated.

Sensors based on magnetic field are used for various purposes such as current sensing, motion and rotational speed sensing, linear and rotary posi- tion sensing, etc. The magnetic sensors are smaller in size and consumes less power as compared to other low field sensing technologies (Yu, Qian, Liu, & Qu, 2018). These sensors are based on the principle of converting magnetic field into equivalent voltage or resistance. This type of sensing is car- ried out in small areas, thus reducing power needs for different applications (Wilson et al., 2007).

Magneto-resistive (MR) sensors are advanced type of magnetic sensors. They are based on magneto-resistive effect that is based on resistivity change of a material due to varying magnetic field. Recently, new types of MR sensors have been developed that have higher sensitivity to the changing magnetic fields. Rapid developments in the field of MR technologies have led to the discovery of new MR based materials such as anisotropic magneto-resistive (AMR), giant magneto-resistive (GMR) and tunnel magneto-resistive (TMR) materials (Gobi, Kannapiran, & Devaraj, 2017).

METHADOLOGY In this study, TMR sensor is used for static hand gesture recognition. TMR sensor is chosen because it has a high sensitivity, strong thermal stability, and a good time response to a frequency-changing uniform magnetic field. Furthermore, TMR sensors are less expensive and use less energy than Hall effect and AMR magnetic field sensors (Malik, Khawaja, Janjua, & Kazim, 2020). The magnetic field three axis values have been taken with the help of TMR sensor. The finger position has been kept fixed and the magnetic field value B for three axis x ,y and z have been stored. The stored data then validated with the help of repeated measurements at the same position. The finger position has been changed to next three positions to collect the data of magnetic field. The method is explained in Figure 1.

The magnet is placed at the front of hand and fingers positions are fixed. TMR sensor measured the x, y and z plan of magnetic field across different fingers position and send the real time measured data to DAQ. This real time measured data is stored in DAQ system and match the finger position by scan- ning the finger position. Data acquisition system (DAQ) is a combination of software and hardware components and play a role of carrier between TMR sensor and graphical user interface. GUI display the movement of object that convey the information regarding corresponding gestures of hand fingers. If

Novel Approach for Sensing the Humanoid Hand Finger Position 41

Figure 1: Flowchart methodology for data collection.
Original PDF Figure 1:

the finer position is matched, the DAQ system gets the real time measured data from the fingers movements and gestures and display a graphical result on the computer screen and repeat the same position for next finer position. If no finer position is matched, the x, y, and z plan measuring process is again repeated till the finger position is matched with the scanned copy. Graphical user interface makes an interface between DAS and TMR sensor. Calibration scope is also used to measure and examine the control of hand gestures and magnetic field of magnet.

EXPERIMENT SETUP The TMR sensor raw data has been collected with the help of National Instru- ment DAQ system. The data later is processed in the PC for finger position identification. The sensor first calibrated with known electric field which is generated through tesla coil and the function generator. The generated signal with tesla coil has been measured with TMR sensor. The error has been remo- ved and the collected data is used to calibrate the TMR sensor. The hardware setup is shown in figure 2.

Figure 2: Hardware setup.
Original PDF Figure 2:
Figure 3: Cartesian plan for under-study static hand fingers.
Original PDF Figure 3:

In the Figure 4 the three positions of the hand have been shown. The data is collected at these three positions. The magnet shape is like small coin with diameter 10 mm and the thickness is 1.5 mm. The magnet north pole is placed upward when it was attached to the middle fingertip.

The relationship for magnetic field strength measurement for cylindrical shape magnetic is shown in Equation 1. The Figure 4 (a) shows the hand in the open state, where the sensor and the magnet position are at 180o with respect to each other. The distance of the magnet and the sensor in this posi- tion is 12.5 cm. In Figure 4 (b) the finger is fully closed while the magnet north pole position changed toward downward. In this position the magnet distance from the sensor became 5 cm.

[Equation or mathematical symbol available in the original paper PDF]

[Equation or mathematical symbol available in the original paper PDF]

Mathematical equation reproduced from the original PDF
Equation reproduced from the original PDF.

[Equation or mathematical symbol available in the original paper PDF] (1)

[Equation or mathematical symbol available in the original paper PDF] D + z q

− z p

2

R2 + z2

R2 + (D + z)2

Where; Br: Remanence field, no co-relation with magnet geometry

Novel Approach for Sensing the Humanoid Hand Finger Position 43

Figure 4: Under-study fingers position w.r.t different angles and gestures.
Original PDF Figure 4:

z: Distance from pole face on symmetrical axis D: Cylinder thickness/height R: Cylinder radius The position 4 is showing in Figure 4 (c) where the middle figure tip is upward bended on its middle joint. The distance of the finger became 9 cm in this position. The last position at 900 is shown in Figure 4 (d) in this position finger is vertical up position. The distance of the magnet center and the sensor in this position is 8.5 cm.

RESULTS

The Figure 5 shows the processed results which are obtained after processing the 1000 samples at each position. The samples were collected for Bx, By and By for three positions. The average value of Bx, By and Bz is plotted in Figure 5. The results clearly show that the shape of each plot is different and can be segregated based on finger position. The results are showing that how TMR sensor sense the magnetic field of each action figure. For example, in Figure 5 (b) a curve line shows the 90-degree tilt in finger, while in Figure 5 (d) the Bz plain goes to zero because 180-degree tilt in hand finger.

CONCLUSION

In this study, a novel technique to sense the static human hand finger gestu- res with the help of non-contact magnetic field sensor called TMR sensor has been proposed. The human hand finger position was measured at four dif- ferent positions i.e., at 180 deg full hand open, 90 deg finger half tilted, 90 deg full finger up, and 180 degree all fingers folded. The results validate that the four finger positions has been detected with the help of single sensor with quite precision. The single sensor will be enough for sensing the position of

Figure 5: Results analysis for four different gestures of hand fingers.
Original PDF Figure 5:

all fingers. So, this concept study can open a new door towards deployment of TMR technology towards gestures recognition of hand fingers.

FUTURE RECOMMENDATIONS The results represent that TMR sensor can be used for angle detection of humanoid hand fingers. Moreover, in future the TMR can be used to con- duct same study for dynamic hand finger gestures recognition. Moreover, a machine learning approaches can also be integrated with current study for more precise results.

ACKNOWLEDGMENT

The authors would like to acknowledge School of Mechanical and Manufa- cturing Engineering (SMME), National University of Sciences and Techno- logy (NUST), Pakistan and European Union (EU)’s Horizon 2020, Research and Innovation Staff Exchange Evaluations (RISE) under grant agreement No 823904 - ENHANCE project (MSCA-RISE 823904) for technical sup- port and funding.

REFERENCES

Benito Temprano, A. L., & McPherson, A. (2021). A TMR Angle Sensor for Gesture Acquisition and Disambiguation on the Electric Guitar. In Audio Mostly 2021 (pp. 256–263). Duan, H.-x., Zhang, Q.-y., & Ma, W. (2011). An approach to dynamic hand gesture modeling and real-time extraction. Paper presented at the 2011 IEEE 3rd International Conference on Communication Software and Networks.

Novel Approach for Sensing the Humanoid Hand Finger Position 45

Gobi, K., Kannapiran, B., & Devaraj, D. (2017). Design of Non-contact transduction based Pressure Sensor using Tunneling magnetoresistive (TMR) principle. Paper presented at the 2017 IEEE International Conference on Intelligent Techniques in Control, Optimization and Signal Processing (INCOS). Jamone, L., Natale, L., Metta, G., & Sandini, G. (2015). Highly sensitive soft

tactile sensors for an anthropomorphic robotic hand. IEEE sensors Journal, 15(8), 4226–4233. Jiang, S., Kang, P., Song, X., Lo, B., & Shull, P. B. (2021). Emerging wearable interfaces and algorithms for hand gesture recognition: A survey. IEEE Reviews in Biomedical Engineering. Lai, H. Y., & Lai, H. J. (2014). Real-time dynamic hand gesture recognition. Paper

presented at the 2014 International Symposium on Computer, Consumer and Control. Li, H., Yang, L., Wu, X., Xu, S., & Wang, Y. (2012). Static hand gesture recognition based on HOG with Kinect. Paper presented at the 2012 4th International Conference on Intelligent Human-Machine Systems and Cybernetics. Malik, S. Z., Khawaja, A. H., Janjua, A. K., & Kazim, M. (2020). A contactless

method for unbalanced loading detection in power distribution lines by magnetic measurements. IEEE Transactions on Instrumentation and Measurement, 69(10), 7472–7483. Nguyen, T.-N., Vo, D.-H., Huynh, H.-H., & Meunier, J. (2014). Geometry-based static hand gesture recognition using support vector machine. Paper presented at the 2014 13th International Conference on Control Automation Robotics & Vision (ICARCV). Nogales, R. E., & Benalcázar, M. E. (2021). Hand gesture recognition using machine

learning and infrared information: a systematic literature review. International Journal of Machine Learning and Cybernetics, 12(10), 2859–2886. Shiravandi, S., Rahmati, M., & Mahmoudi, F. (2013). Hand gestures recognition using dynamic Bayesian networks. Paper presented at the 2013 3rd Joint Confe- rence of AI & Robotics and 5th RoboCup Iran Open International Symposium. Wang, C., He, H., Cui, Z., Cao, Q., Zou, P., & Wang, H. (2018). A novel EMT

system based on TMR sensors for reconstruction of permeability distribution. Measurement Science and Technology, 29(10), 104008. Wang, Z. L. (2017). On Maxwell’s displacement current for energy and sensors: the origin of nanogenerators. Materials Today, 20(2), 74-82. Wilson, S. A., Jourdain, R. P., Zhang, Q., Dorey, R. A., Bowen, C. R., Willander,

M., . . . Quandt, E. (2007). New materials for micro-scale sensors and actuators: An engineering review. Materials Science and Engineering: R: Reports, 56(1-6), 1–129. Yasen, M., & Jusoh, S. (2019). A systematic review on hand gesture recognition techniques, challenges and applications. PeerJ Computer Science, 5, e218. Yu, H., Qian, Z., Liu, H., & Qu, J. (2018). Circular array of magnetic sensors for

current measurement: Analysis for error caused by position of conductor. Sensors, 18(2), 578. Zhang, C., & Tian, Y. (2013). Edge enhanced depth motion map for dynamic hand gesture recognition. Paper presented at the Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops.

Cover of the proceedings volume Industrial Cognitive Ergonomics and Engineering Psychology

Explore the complete digital volume

Industrial Cognitive Ergonomics and Engineering Psychology

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-11-7

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →