Augmented Reality based Navigation and Recognition System

The current navigation system requires the user to continuously map with the navigation application with the real-world environment. It sometimes leads to an in aut to describe the design of an application that will provide the same reality-based environment as that seen mulation or content to provide an enhanced view to the user during navigation. We will be using Augmented reality to provide users with more information, computer-assisted Google’s existing Maps people can be done from the data stored on the cloud and will be augmented when scanned through the camera of the phone which is the only requirement of the application. It also enables the users to navigate indoors with the help of a radar map and arrow pointing towards the points of interest. This application enables the decisions. The user can also navigate to different points of interestindoor as well as outdoor. It also enables the recognition of places, as well as people and the details related to the same, get augmented.


Introduction
Augmented Reality is a technology that overlays the virtual graphics in the real-world [1] environment. It helps in visualization and improves user experience in applications. The current navigation system has customized road map views which are difficult for the pedestrians [2] as well as the drivers to map with the real world. Augmented Reality, when merged with the navigation system along with various Google APIs [3], provides a better user experience by augmenting the directional details. This helps the user to see the real world along with the navigational details.
In this paper, we focus on the navigation and recognition system with augmented reality integration. It -based augmented reality system for navigation and recognition. The system has the recognition features which recognize objects, scenes, and people and augments their details. These details get augmented with the help of image targets [4]. The details stored in the cloud-related to the database image targets get augmented [5] on their recognition. The augmented details may include the description, ratings, reviews, etc. and can be displayed in the form of texts, images, 3d objects, and videos.
The recognition system is integrated with the navigation system to provide the pedestrians as well as the drivers with the available details [6] on their path which gets augmented on their screen. The navigation system uses Google Maps API, Google Geocoding API and Google Direction API are used to get map-based data, geolocation of the user and geo-coordinates [7][8][9]. The navigation system allows the user to select the source and destination points by searching the desired places for navigation. It provides an augmented route to the destination selected with various details such as the distance between the source and the destination, and the time required to cover the distance.
The system can interact with the user using voice commands and instructions [10] for giving the routing to desired locations. Along with outdoor navigation, the system also provides indoor navigation. The maps for the indoor navigation [11] can be accessed using the QR code and can be used for navigation inside the organizations with augmented details.

Literature Review
In the paper, "A Survey of Augmented Reality", the author R. Azuma explored six classes of potential AR applications. Author explains the characteristics [12] of the AR systems. Author different sites and the issues encountered during the development. The paper summarizes the current efforts to overcome the problem of the registration and sensing errors in building effective Augmented Reality systems.
The author Ronald T. Azuma along with others, discussed in the "Recent Advances in Augmented Reality" [13] about how recent advancements in technology demand AR to be deployable on mobiles and possible to maximum users. Authors discussed the research contribution of Mohring and Wagner on the usage of phones for computer vision in sensing and studied characteristics of web 2.0 and AR 2.0 and compared them. Authors concluded how AR can be deployed with handheld devices, to generate and share information or user content with other users.
presented [14] existing research carried out in augmented reality based navigation systems. It describes the exisiting navigation systems, techniques used and problems faced. The paper concludes that augmented reality is a very powerful mechanism to provide the user with better user experience while navigating and also improves the safety of the user.
In the paper "Augmented reality as a new way of exploring the city: unified platform for data providers" [15] the authors explore Augmented Reality as a part of Location-based services. It describes a platform for the dynamic transformation of the static information obtained for the POIs. The aim was to provide the users real-time [16 user-friendly as much as possible.

Methodology
For building the recognition system we have used the SDK along with android studio which is a web application and for the outdoor navigation we have used BeyondAR SDK along with the android studio which is a native android application:

Scene and Object Recognition
In the scene recognition module, the scene is recognized with the help of image targets. These image targets are which is a collection of targets used to recognize the scene. The live camera feed is analyzed by the image etects the targets. For each target in the target collection, the details regarding the target get augmented. The augmented details can be in the following forms:

Text
The textual details are augmented by converting them in the form of images. The conversion of text to image is done using the Pillow library of python. Image format to display the text is designed and is combined with the text to be augmented.

Image
The image resources to be augmented are passed to the AR.ImageDrawable. The ImageDrawable converts the image into a representable AR object.

Hyperlinks
a particular website into an AR object that will be used for augmentation.

Audio
The details in the audio format can be augmented by passing the corresponding audio file to the AR.sound class. It is initialized on the recognition of the targets in unable .

Video
The video related to the target is augmented with the help of VideoDrawable class. The VideoDrawable class encapsulates the video which can be used to represent the video in the form of an AR object. The VideoDrawables can be represented in the form of transparent pixels.

Navigation
In the navigation module, the start location of the user and selecting it using the Geocoding API. The route is mapped with the help of Google Directions API which provides the map and BeyondAR SDK which recognizes the roads directional details. For the AR-based navigation, the turn by turn navigation route is given by the Directions API on top of Maps SDK by requesting the Geocoding API for source and destination. The response obtained will be the navigation route. The navigation route is displayed with the help of the virtual green spheres which get augmented along the route line with the help of BeyondAR SDK. BeyondAR SDK recognizes the road and helps in augmenting the details. The SDK helps in augmenting the start banner, green spheres along the route of navigation and stop banner on reaching the desired destination along with the time required to reach the destination and the distance to reach there.

BeyondAR GeoObject
A geoAR object is created with an image. This object is used to display the directional details. The world holds the information related to the objects. It helps in augmenting the objects by overlaying it with the camera frames in the application.

BeyondAR Google maps plugin
The google maps plugin provides an option to augment the world object on the maps view of the Map SDK provided by google. The GoogleWorldMapPlugin is used to set the geoAR object on the map view on the navigation path provided by the Directions API at specific geolocations.

Google Maps SDK
Google Maps SDK is used to add maps based data to the application. It handles access to Maps servers, displaying of maps on application screen, responding to the gestures and downloading map data.

Geocoding API
Google Geocoding API is used to get the geolocation of the start and destination place of navigation. It converts the address into geographical coordinates.

Directions API
The Directions API is used to get the navigation route from the source to destination. It provides the geocoordinates along the route of navigation which in coordination with the google maps plugin is used for augmenting the world objects. It also provides the distance to the target and the time required to reach the destination.

System Architecture
The application gets the user's location with the help of GPS satellites. The Google Geocoding API provides the user's location and the Google Maps SDK and Direction API provides the map data required for the navigation de stores the image stores the images, videos and other details needed to be augmented on the targets. As seen in Fig 1., the AR with the augmented reality world components. The AR visualized on the mobile phone screen with a camera providing the realaugmenting the AR components. SDK has been used for augmenting the details on the realworld environment and augment details on them. The AR interface data, GPS data and the compass data are given  The android application uses the geocoding API to get the latitude and longitude values of the searched location. The user can search the source and destination for the navigation. On starting the navigation, the direction spheres get augmented along the path of the navigation.

Result
turns by augmenting the green spheres along the path of navigation. The user can follow the path along the augmented spheres which helps in tracing turns and flyovers accurately thereby avoiding ambiguity while navigating. The distance and time for navigation also gets augmented on the screen which can be seen in the Fig. 4. Local Augmented Reality Navigation allows the user to view different Points of Interest in the area, navigate to them and also get additional details and distance to those places. As seen in Fig. 5., users can navigate to the points of interest by selecting them on which an arrow gets augmented on the screen which directs the user to reach the selected POI by following the path.  The Recognition module recognizes the places and the details regarding it are augmented in the form of image, audio and video which can be seen in Fig. 6. The users can also get virtual buttons augmented to access the websites of the recognized place thereby reducing the effort of browsing the place explicitly. Details in the form of 3D objects as seen in Fig.  7. are also augmented with interactivity features which can be used for more realistic details and advertisement purposes.  Table 1 shows the comparison between the traditional navigation system and the augmented realitybased navigation and recognition system. The traditional navigation system results in ambiguity in turns and flyover because of custom road map view which is overcome by augmented reality-based navigation which uses camera feed and augments spheres to provide exact details of directions. Also, traditional navigation system doesn't recognize places and they need to be explicitly searched, whereas using augmented reality recognition augmented on the phone screen.

Conclusion
In this paper, we presented an Augmented Reality based navigation and recognition system that can be run on clear, user-friendly, and provides multiple features. The user can easily select source and destination and will get a route augmented with spheres for navigation. The user will be able to view the distance between the source and destination and also the time required to reach the , historic places, institutes, etc. by using radar maps and collection of POIs. Augmented details about the nearby locations such as petrol pumps, hotels, mechanic shops, etc. during navigation based on geolocation. The details forms such as text, image, audio, video, and 3D model to provide a better experience to the user. Weather details augmented for traveling s can view POIs present within the range. Information about the selected POI can be viewed and direction to the selected POI is also provided. The experience and provides better user experience and usability.