Recent Changes - Search:

Augmented Reality in SmartSantander

  1. Our Augmented Reality Approach.
    1. The Architecture.
      1. Communication between the components.
    2. Augmented Reality Server Interface.
      1. Register device.
      2. Get Data and submit observations.
      3. Unregister device.
  2. Experience AR in Santander.
    1. SmartSantanderRA App.
      1. Communication with the AR Server.
        1. Register device.
        2. Get POI Information.
    2. NFC/QR tags.

Our Augmented Reality Approach

In this section we explain in detail the tools provided by the SmartSantander platform that assist the developer to build service(s) and/or application(s) that follow an augmented reality approach.In addition, we describe the functionalities provided by the augmented reality server.

The Architecture:

To support all these desired functionalities, the SmartSantander platform provides the Augmented reality Server (AR server) that supports the registration/deresgistration of augmented reality devices into the SmartSantander platform; submission of observations into the SmartSantander platform; and retrieval of sensed data. Moreover, the platform offer over 2500 NFC/QR tags placed all over the city (shops, bus stops, tourist point of interest …) that are accessible to the end users. The following figure shows the interaction between the SmartSantander platform and the Augmented Reality Service:

In this scenario there are several components involved:

  • Smart phone equipped or not with NFC chip, with an Internet connection.
  • NFC and QR tags.
  • Augmented Reality Server.
Communication between the components
  1. Register Device: When the application starts for the first time the user is asked to register his/her information to the system. Furthermore, device information is also sent from the mobile to the Augmented Reality Server so that the device can be registered in the SmartSantander platform.
  2. GetData: Users can get information about specific points of interest in the city either by reading an NFC tag/QR code or by using the augmented reality functionality of the application.
    • When a NFC tag is read, the user is redirected to the link that is contained in the tag. In addition, information about the device is sent to the AR Service Server. This information will be used to have a statistical module showing the number of times a tag was read and a distribution of the types of devices that were used to read the tag. When the user faces the mobile phone camera to a specific point of interest, the GPS coordinates and the compass values of the phone are sent to the AR Service Server and the information about the point of interest the camera is pointing to is matched with the values provided by the device. The information is sent to the mobile phone so that the user can visualize (images and text) describing the place.
    • The additional information is sent from the AR Service Server to the USN in form of an Observation Message.
  3. Delete device: From the SmarSantanderRa App the user can delete his/her account the request is sent to the Augmented Reality Server.

Augmented Reality Server Interface:

The Augmented Reality Server component provides the following operations:

  • RegisterDeviceResponseDTO registerDevice(DeviceDTO device);
  • To get data and submit observation is need to make a HTTP POST request to the Server with a valid url.
  • UnregisterDeviceResponseDTO unregisterDevice(String deviceUUID);
Register device:

The registration message call contains the device information. The device information is represented by the DeviceDTO class. This object contains the following structure:

The DeviceDTO object contains three elements: the deviceInfo of type DeviceInfoDTO, a collection of device capabilities, and a collection of applications. The device capabilities are the physical capabilities that the device has (NFC chip, camera); and the applications are the applications that will make use of the augmented reality; each application has a collection of capabilities that are the application related capabilities that will be used by the application to submit observations to the SmartSantander platform. As result to the device registration call the application or service gets a unique identifier deviceUUID generated by the AR Server when registering the device into the SmartSantander platform. This deviceUUID needs to be stored in the phone since you will need it to interact with the AR Server in further calls.

Get Data and submit observations:

When the application or service ask for POI data needs to send an HTTP Post request to the AR server; the URL used in the request will have and array of parameters that the AR server will use to create observations. The possible parameters includes in the AR server are:

  • Max number of POIs (“maxPoi”): The maximum number of POI that the app showed in the AR views.
  • Max Search Distance (“dist”):
  • InfoType(“type”): The type of info request. The possible values are:
    • TUS: Transport
    • CUL: Culture
    • COM: Commerce
    • TUR: Tourism
    • AGENDA: Agenda
  • Language(“lang”).
  • deviceOS. This parameter is included in the http header of the request.
  • POI id (“id”).
  • Tag Type (“tagType”): NFC or QR
  • Tag id (“tagId”).
  • URL Latitude (“lat”).
  • Longitude (“lon”).
  • deviceUUID (“uuid”).

An example of an URL extract from the SmartSantanderRA: http://fake.server.url/server.php?lat=-3.40005&lon=42.543563&maxPoi=30&lang=EN&type=CUL&dist=10&uuid=bf5eb4cd-97eb-498b-af09-dca3810b68ca

This URL request the server the closest 30 culture POIs to a position (42.543563, -3.40005).

As result a HTTP Response with the available data is returned.

Unregister device:

In order to deregister your device in the SmartSantander platform you will need to call the unregisterDevice method; this method takes the deviceUUID of your device as parameter.

  • UnregisterDeviceResponseDTO unregisterDevice(String deviceUUID);

As result you will receive a boolean; if true the device was successfuly deregistered from the SmartSantander platform; false otherwise.

Experience AR in Santander

In Santander, as in many other cities, there is a variety of information of interest for citizens and tourists, which are rarely used by them. This is due to the high disintegration of the data sources spread and the lack of an easy and clear way to access the data. Information on transport, shopping, beaches, cultural agenda and so on, is available in many different sites and unknown to end users. With the aim of uniting all this relevant information in one application, and displaying the data in an orderly and interactive way, SmartSantanderRA App app was born.

With an AR application users can define their own preferences (language, touristic places to visit, monuments, etc) and have a better experience visiting the city rather than using traditional standalone applications. In this sense, the video stream produced by the smartphone camera will be presented to the users in an AR application, augmenting the live feed of the camera with virtual objects (mainly digital content, video, texts, and photos) of the Point of Interests (POI), based on the current position of the device, in order to create an augmented view.

For each Point Of Interest, it will be provided a description of the place or reference image to be used in the AR view and the type of content to be superimposed. The content itself (3D model, image, videos, audio, etc.) is not part of the Point Of Interest but will link to the content instead. The use of tags will provide information that is not included in the POI itself.

SmartSantanderRA App

The App includes information about 2700 places in the city of Santander divided in different categories: beaches, park and gardens, monuments, Points Of Interest (POI), tourism offices, shops, art galleries, museums, libraries, culture events agenda, shops, public buses, taxis, bikes, parking places, etc.

It allows real time access to traffic and beach cameras, weather reports and forecasts, public buses information and bike-rental services, generating a unique ecosystem for citizens and visitors when walking around the city.

On starting the AR view, the App creates on the smartphone screen, an overlay over the camera with nearby POIs. If a particular POI is selected, further information (title, short description, photo and distance to the POI) is displayed. Apart from that, the App allows for creating the route to that place or playing digital content related to the POI (e.g. videos) if it is available. This App takes advantage of the around 2500 NFC/QR Tags placed around the city. Placing tags on certain POIs such as shops, bus stops or touristic points of interest provides direct access to POIs relevant information.

Apart from that, as the SmartSantander project aims at supporting the experimentation, movement patterns of the users can be recorded in an anonymous way within the USN. It will provide third party developers the possibility to enrich their application and services, based on the user experience within the city:

  • For the municipality, observations sent to the USN by the AR applications will allow the collection of information about how many people visit the different POI, or at least is in the nearby areas, origin from that people, seasonality, etc. Based on this information, they can improve the touristic offer of the city, adapting the cultural events and activities during each season.
  • For the shops, placing tags on certain shops in the city provides new opportunities for shops to build or strengthen customer relationships. The shops can explore the relationship between physical presence and the web. The users can get specific information about the shop, for instance, opening hours, contact, special offers, accessibility in the shop, etc.

The SmartSantanderRA app is available for IOS and Android platforms. More information about the SmartSantanderRA app can be found at SmartSantanderRA.

Communication with the AR Server

The main goal this app achieve to show dynamic inforomation according to the user position. To accomplish this, it is necessary to ask the server each time the user starts the app or the user position changes. At the same time that the user requests information about nearby Points of interest, some information can be collected and sent to the server. In addition, if each smartphone has a unique and anonymous ID, the data collected becomes more valuable and useful to implement new services such as user tracking. For these reasons the application communicates with the server to:

  1. Register the device
  2. Get POI Information
Register device

The AR server implements a Restful web service that allows the application to register devices (nodes) on the platform SmartSantander. The app implements a Restful client to send the device information as defined in the AR Server interface. The data transfer is based on JSON content type. The following figure shows the JSON data send to the server:

The server processes the data and returns an ID (deviceUUID) that the app saved for future communications. The next figure shows an example of registration response:

If the server is not able to register the device on the SmartSantander platform the response message will be empty.

Get POI Information

The SmartSantanderRA app makes two kinds of data requests to the AR Server:

The following figure shows the response:

The following screenshot shows a web page returned by the AR server:

NFC/QR tag

Although The SmartSantander App has a built-in NFC reader (available depending on the device) and a QR reader, any other application that provides this funcionality can be used to read/decode the tags. If the SmartSantanderRA app is used, the observation asociated to the tag reading includes the deviceUUID, otherwise a generic ID will be used. The following picture is an example of NFC / QR tag exposed in a shop.

All the tags have an url encoded. When a device decode it a web browser starts and shows a web page with the info associated to the POI. The communication with the AR server is similar to Get POI Information

Edit - History - Print - Recent Changes - Search
Page last modified on November 30, 2012, at 04:47 AM