Developing a novel haptic device for non-rigid computer graphic objects utilizing unmanned aerial vehicles

— In this paper, we propose a novel haptic device consisting of a Parrot quadcopter AR Drone 2.0 that delivers force-feedback to users when they press on the surface of the drone in the vertical direction. This drone haptic device will free users from any cumbersome devices which were utilized in previous haptics systems and allow them to sense kinesthetic feedback coming from the drone which in turn renders computer graphic objects, for example, a virtual box. Specifically, this system performs damped harmonic oscillation force motion on users’ hands and fingers. The oscillation function is implemented on the drone whenever users nudge the drone down. Overall, we evaluated the system with ten subjects, and results show the effectiveness of using damping oscillation motion to provide force-feedback delivered from the drone.


INTRODUCTION
Haptic devices are instruments in which users can interact with virtual and augmented realities through tactile sensation. The word haptic, from the Greek haptikos, means "pertaining to the sense of touch." Haptic technology could be used in many areas, for example, games and surgery training. One feature of haptics is enabling users to sense computer graphic (CG) objects. A haptic device can convey the stereognosis of virtual objects to users by implementing force-feedback on users' bodies and arms [1].
In our previous research, a finger-mounted haptic device using surface contact has been made [2]. This haptic device consists of a wearable glove which allows users to feel force-feedback observed from a CG object on the thumb and index finger of the left hand. A virtual sphere was utilized as a CG object as Figure 1 shows.

Figure 1 CG Sphere
However, this haptic glove device has two major problems. First, cumbersome electronic devices and wires must be tethered to the users' hands as Figure 2 shows, which makes users feel uncomfortable grabbing it. Second, this glove cannot generate kinesthetic feedback. Specifically, it does not allow users hands to move out of the operating range of the sphere.

Figure 2 Glove Haptic Device
Following that, researchers are still developing new haptic devices capable of delivering tactual perception without any restrictions regarding space and complexities caused by the haptic devices. Unmanned aerial vehicles (UAV) technology is being developed to overcome the constraints in the domain and to eliminate large devices interfacing with users. For example, in other research, an encountered-type haptic display using a drone was developed in which users can sense force-feedback of a virtual creature via a sheet of paper attached to a drone that delivers a rigid feeling of the creature. This system also enables users to feel that they can draw on a virtual wall [3]. However, it has limited force magnitude up to (0.118 N) that comes from the air flow of the attached paper.
The major problem to obtain force-feedback from drones stems from their non-rigid bodies which makes it difficult to receive the accuracy force-feedback. This is a major obstacle to obtaining precise force-feedback from drones, which might be overcome by applying and analyzing damped harmonic oscillation motion. The proposed haptic device will allow users to realize force-feedback that is coming from a drone. This device will free users from any difficulties of wearable devices and enables them to detect a movable virtual object. As a first step, we have implemented damped harmonic motion on the drone whenever users try to push on its surface as Figure 3 shows. They will sense kinesthetic feedback coming from a CG object. In this paper, we describe the proposed method using a flow chart of the system, the damped harmonic motion mathematics relations, explanation of the work, results and a conclusion.

PROPOSED METHOD
In this research, an AR Drone 2.0 is being used to overcome the limitations of fixed and wearable conventional haptic devices. The AR Drone 2.0 is a quad copter UAV which can fly freely in space and perform maneuvers, for example, pitch, roll, and yaw [4].

Figure 3 AR Drone 2.0
We realized that the drone must be safer to enable users to touch it without any worries, because of that we put the handler on its surface as Figure 4 shows.

System Flowchart
The system allows users to sense spring damped oscillation force-feedback according to the steps shown in Figure 5: (c) Once a user pushes the drone down, meaning that the drone will be under 0.6 meters, and will oscillate back and forth mimicking the damped harmonic oscillation.
(d) Once the oscillation stops (velocity = 0), the drone will fly again to the equilibrium state (hover).
(e) To land, a user should use the mouse function again.

Damped Harmonic Oscillation
In real life, when an oscillating motion is applied to an object, the object will experience over time a decrease in amplitude as a result of internal friction and air resistance [5]. Hence, the object has two different forces affecting its motion which is given by is the damping force that acting on the object to impede the oscillation, b is the proportionality constant and v is the velocity of the oscillation.
The spring force follows Hooke's law, where k is the spring constant and x is the displacement from the equilibrium position.
The negative sign in the two functions represents that the forces act in the opposite direction of motion.
According to Newton's second law of motion that stating that the summation of forces acting on an object is given by ∑ = (3) where m is the mass of the object, and a is the instantaneous acceleration.
Therefore, substituting (1) and (2)  The object will take a very long time to achieve its equilibrium position. It will take a very long time to stop oscillation.
(B) Underdamped: 2 < 4 The object Oscillates several times before coming to rest (C) Critical damped: 2 = 4 The object takes the shortest amount of time to come to equilibrium

EXPLANATION
The use of drones with haptics to render virtual objects causes one principal weakness due to the physical status of flying drones. Testing and analyzing Newtons' laws of motion might be the way to reach a solution, specifically starting with the spring force model. Since the spring force considers changes in the distance according to the stretch that a user may indicate from the equilibrium distance to the releasing point, it would be an excellent factor to obtain a reliable force-feedback from the drone. Thus, the spring model force, according to Hooke's law is given by = − (9) K is the spring constant (stiffness) (N/m) X is the displacement from the equilibrium, which is the altitude of the drone subtracted the equilibrium position of the drone for example, of the spring model as Figure 8 shows a ball oscillates in the range between -A and A.

Figure 8 Spring force diagram
However, after testing the harmonic oscillation without damping on the drone, we have tested the damping harmonic oscillation to make users feel the more reliable sensation when they can feel the oscillation with high magnitude force into low magnitude force, and the damped harmonic function that was implemented in the case of a drone. This is given by ( ) = . − ( ) + (10) where X is the displacement of the drone from an equilibrium point (m). A is the amplitude which is the displacement from the natural length of the spring (m). C is the distance from the ground to the equilibrium position of the drone (initial position of the drone) t is the time (s). Then, according to the three kinds of motion related to the damping oscillation, we have chosen underdamped motion that could be shown in Figure 6 and Figure 7 in which the motion of the object would oscillate several times showing users reliable damping oscillation. Hence, since the AR Drone 2.0 could be controlled using the velocity (m/s). Figure 9 plots two different positions so that we have applied the phase velocity function that is given by = − − (11) where d2 and d1 are two different positions from the damping wave (m). t2 and t1 are two different time slots from the same wave representing the d2 and d1 positions (s).

EXPERIMENT AND RESULTS
We evaluated the system by asking ten subjects to answer four different questions after testing the damping oscillation of the drone whenever they try to push the drone down. Q1. Was the response speed of the drone sufficient to feel force-feedback? O2. Did you feel the damping oscillation on your hands and fingers? Q3. Was the drone stable enough in the air? Q4. Did you feel safe when you touched the drone?
Rating scale to answer the questions is (5. Strongly agree, 4. Agree, 3. Neutral, 2. Disagree, 1. Strongly disagree). Table 1 indicates the results of the experiment where the average score and standard deviation were calculated for each question.  Table 1, Q1 and Q2, which they are related to the movement of the drone when acting as a haptic device delivering force-feedback has good scores because we have changed the parameter 'b' of the function (1) and the parameter 'A' of the function (10) many times to obtain the best oscillation. Q4 also has a good result because we enhanced the safety of the drone using handle attached to the surface of the drone allowing users to feel safer more when they grab it. Q3 scored 2.9 which is low value because the AR Drone 2.0 that we use is not entirely stable in the air according to lightning and camera issues. Moreover, the drone is self-controlled in our system which means users can control just takeoff and land of the drone and they cannot control the drone maneuvers. Overall, the results would appear to convey that this novel haptic device represents a powerful tool to render CG objects, but still has limitations with safety and stability issues.

CONCLUSION
In this paper, we developed a new system in which a drone is used as a haptic device to render virtual feedback in the vertical direction by applying the underdamped harmonic oscillation that oscillates several times before coming to rest. However, because of utilizing the AR Drone 2.0 was difficult, according to stability issues, another type of mini-drones that has a protective cage may be used to eliminate interaction hazards. In addition, the Kinect device might be used to allow users to control the drone motions by their hands' position and this would overcome stability problems.