Mobile Use – Scenarios for Digital Content and Digital Services – The Augmented Reality
App “Ludwig II” of the Bavarian State Library
Klaus Ceynowa
The Bavarian State Library has developed the smart phone application “Ludwig II-Walking in the Footsteps of a Fairytale King”, which makes use of the possibilities offered by the mobile Internet for an up-to-date presentation of digital cultural assets.
The app represents a location-based multimedia service for using augmented reality technology. A great variety of materials was complemented by features developed specifically for the app, such as audiovisual images, videos of expert interviews, audio recordings of contemporary witnesses. The materials had to be converted into a multitude of short text modules, structured in a more associative fashion, translated into English for accommodating a larger user population, formatted to adapt to the screen layout, and updated to the current situation. Moreover, it is also important that the app can offer a location-independent access to its content. The design is in favour of the location-based access, that is, digital information surrounds the user wherever he goes and is available ubiquitously. The real task is the mutual integration of content and context, so as to create a unique and fascinating use experience for the user to ensure a recurring and long-term user retention to the app.
Keywords﹕ Augmented reality technology; Smart phone application; Ludwig II; Bavarian State Library;
Ubiquitous computing; Location-based services
Klaus Ceynowa: Deputy Director-general, Bavariau State library; E-mail: ceynowa@bsb-muenchen.de
Abstract
Mobile applications offer cultural institutions such as libraries, archives and museums manifold possibilities of presenting their frequently unique contents. With the augmented reality app “Ludwig II -- Walking in the Footsteps of a Fairytale King”
the Bavarian State Library offers an example of what mobile use scenarios for cultural content can look like in the future.
Numerous important cultural institutions have been working on the digitization of their frequently unique cultural objects for many years. They consequently have quickly growing collections of digital content, which, due to the rareness and the value of the underlying analogous objects, puts them in a truly unique position in comparison to the multitude of digital information offers. So far the presentation of these contents was primarily oriented towards use at a stationary Internet workstation, via the website of the institution or special aggregators and portals in the respective field (e.g. WorldCat, Europeana, World Digital Library). Mobile applications offer new and innovative presentation techniques (location-based services, augmented reality, etc.), which have an enormous potential for the areas of teaching and scholarship.
The Bavarian State Library, as one of Germany’s leading institutions in the field of digitisation, has developed some noteworthy services which make use in a paradigmatic fashion of the possibilities offered by the mobile Internet for an up-to-date presentation of digital cultural assets.
1. Digital offers by the Bavarian State Library
The Bavarian State Library (www.bsb-muenchen.de), founded in 1558, is the central state and repository library of the Free State of Bavaria and one of the world’s most important universal libraries. Its collections currently comprise 9.7 million books and 57,500 current 2 journal subscriptions, mostly in electronic form, as well as 1.3 million e-books. With 93,600 manuscripts it ranks among the four largest manuscript libraries of the world, its collection of 20,000 incunabula is the largest worldwide, and around 140,000 printed works of the 16th century represent Germany’s largest library collection in this time segment.
The Munich Digitisation Centre (www.digital-collections.de) was founded at the Bavarian State Library with the support of the German Research Foundation
already in 1997. Today the Munich Digitisation Centre is a national competence centre for innovative digitization technologies and Germany’s leading institution of mass digitization of written cultural material, among other things through the consistent use of scan robotics. Currently the Bavarian State Library already offers almost 700,000 digitised books from its collections for free-of-charge use, among them many unique works. This is the largest digital data collection held by any German library. At the start of 2007 the Bavarian State Library further was the first continental European library to enter into a public-private partnership with Google, initiating the digitization of its complete copyright-free holdings from the 17th to the 19th century, comprising a total of over 1 million volumes.
In spring 2010 the Bavarian State Library started offering its central, net-based information services and its manifold digital offers to its users successively also in the form of mobile applications. The mobile versions of its online catalogue comprising over 10 million titles were followed by two iPad/ iPhone apps: “Famous Books” and
“Oriental Books”. “Famous Books–Treasures of the Bavarian State Library” offers 50 select, digitized outstanding items from the library’s collections in the form of an application for iPads and iPhones, which is available worldwide free of charge via the iTunes app store. On the high-resolution, brilliant colour display of the iPad and the iPhone - which is predestined for presenting digital books in view of its design and usability - now the frequently uniquely illuminated digital colour copies of the genealogies of the Fugger dynasty, the Gutenberg bible, the Song of the Nibe- lungs, the gospel book from the Bamberg cathedral, the Babylonian Talmud, the Theuerdank, the Genji Kokogami and many others can be browsed from the first to the last page. The app implements all typical functions of the iPad/ iPhone such as cover flow, thumbnail preview, zooming, etc. for the comfortable use of the digital copies. The application
“Famous Books” ranked among the most successful free-of-charge apps in the iPad app 3 store and took second place in the ranking in the category “books”. The. “Oriental Treasures of the Bavarian State Library” have been similarly successful. This app presents 20 extremely rare Islamic manuscripts, some of which are held exclusively by the Bavarian State Library.
Image 1 Famous Books - Treasures of the Bavarian State Library
2. The app “Ludwig II - Walking in the Footsteps of a Fairytale King”
On the occasion of the 125th anniversary of the death of Ludwig II in 2011 the Bavarian State Library developed a highly innovative, partly even experimental digital service: the smart phone application “Ludwig II - Walking in the Footsteps of a Fairytale King”. The app was developed in cooperation with the Bavarian Department of State--owned Palaces, Gardens and Lakes with the support of the agency Bokowsky + Laymann–Marketing in Computer- Mediated Environments GmbH. Technically speaking, the app represents a location-based multimedia service for using augmented reality technology. The App is available free of charge via the iTunes app store since 26 September 2011 and has received a major update in October 2012.
In addition to the digitized holdings of the Bavarian State Library, a great variety of materials from the holdings of the Bavarian Department of State-owned Palaces, Gardens and Lakes could be used in the app. This rich content (texts, images, historical photographs, letters, maps, museum objects, etc.) was complemented by features
developed specifically for the app, such as audiovisual images, videos of expert interviews, audio recordings of contemporary witnesses.
This extensive multimedia offer is presented as a location-based service. In the map view (based on Google maps) or in the current camera image of the iPhone the user is shown the app contents relating to locations of Ludwig II in his/
her vicinity. In the Ludwigstrasse in Munich, for example, the camera image of the app indicates the distance between the user’s current location and locations in the vicinity that are related to Ludwig II (for example the Munich Residence or the Cuvillies Theatre). A tap on this display opens an information box with brief information on the respective location, a further tap on the
“more” button in the info box then opens the complete offer of multimedia information items on the location that are available in the app. Since
these information items are in turn linked in a location-and context-dependent fashion with other locations, events and persons in the life of the king, the user is provided with an information space about Ludwig II that is equally broad and thorough. The life of the fairytale king is virtually predestined to become the subject of a location-based multimedia application, not only since Ludwig liked to travel frequently in Bavaria, but also since he made Bavaria in its entirety the theatre for his idea of an “ideal kingdom”
through his realized and planned buildings.
The real-time display of digital information in the smart phone’s camera image is the classical application case of augmented reality technologies. “Augmented reality”
refers to the computer-based augmentation of the visual perception of reality, primarily the enrichment of the camera images of modern mobile devices (smart phones and tablets) with digital additional information or virtual objects by means of insertion Image 2 Start screen of the app
“Ludwig II”
or overlay. In practice in augmented reality applications usually digital information is integrated in the image of reality recorded by means of a smart phone camera. The integrated items can be texts, images, videos, audio files or 3D animations. Augmented reality applications are location-based services and as such depend on the GPS, camera and compass functions of modern smart phones. They consequently represent an important “driving force” for the current transition from the stationary to the mobile Internet. In the HORIZON Report 2011 the relevance of augmented reality is stressed particularly for cultural and academic applications: “The layering of information over 3D space produces a new experience of the world, sometimes referred to as ‘blended reality’, and is fuelling the broader migration of computing from the desktop to the mobile device, bringing with it new expectations regarding access to information and new opportunities for learning.” [1]
3. Features of the app
In detail the app “Ludwig II - Walking in the Footsteps of a Fairytale King” has the following features:
140 locations in Bavaria and Europe relating to Ludwig II
The most important locations which played a role in the life of the fairytale king are navigable comfortably on a map. In the camera view, the location name, the distance to the viewer and short information items are inserted in the camera image of the smart phone in real time. Besides the map view and the camera view, a list view is available as a third viewing option for these points of interest. The list view can be sorted in the order of distance or in alphabetic order. Besides the direct use on the spot, the app is therefore also suitable for comfortable browsing at home, for example to plan a trip “in the footsteps of the fairytale king”.
Detailed description of the 80 most important locations
For the 80 most interesting locations in the life of Ludwig II detailed information texts are available, whose appearance and length are adapted specially to being
[1] The Horizon Report, 2011. http://wp.nmc.org/horizon2011/ (retrieved: 05/05/2011)
used on smart phones. All texts were written specifically for the app by distinguished experts of the Bavarian State Library and of the Bavarian Department of State-owned Palaces, Gardens and Lakes. Thus the app does not offer “infotainment”, but warranted, specialist information for every user with a historical or touristic interest in Ludwig II.
Over 400 illustrated photographs
Extensive photo galleries with historical image material from the holdings of the Bavarian State Library and the Bavarian Department of State-owned Palaces, Gardens and Lakes give the user a vivid impression of king Ludwig II and his time. In the manner typical for iPhones, the images can be browsed comfortably in forward and backward direction by finger gesture. All images are accompanied by meaningful explanations.
Delve into history listening to contemporary witnesses
Spoken quotes of contemporary witnesses, for example by Richard Wagner, Otto von Bismarck or by Ludwig’s Court Chef Theodor Hierneis, convey a vivid impression of how persons from Ludwig’s environment perceived the king and what they reported to posterity about it. Of course the king himself speaks, too!
Audiovisual images covering special topics and locations
The amount of text one wishes to read on the go on the display of a smart phone is rather limited. More comprehensive background information about the “major sights”, for example the king’s castles, was therefore integrated in audiovisual images, which are optimal for use during the journey to the location or upon visiting the location. This feature conveys to the user, among other things, background information about the construction phase of castle Neuschwanstein, about Ludwig’s admiration for the French rulers of absolutism or about the garden of castle Linderhof.
Image 3 Camera view of the app “Ludwig II”
Expert knowledge about Ludwig II in videos
Interviews took place with distinguished experts in the areas of architecture, politics, music, technology as well as the king’s life and myth. In the form of 40 videos of a duration of around 90 seconds each, the interviews are linked in a location- and context-based fashion and convey an immediate understanding of Ludwig and his time. Like all other contents, also the videos are stored in the app itself. An Internet connection is thus not absolutely necessary to use them.
Ludwig’s conservatory in the augmented reality simulation
One augmented reality highlight of the app is the no longer existing conservatory of Ludwig II. It was demolished shortly after the king’s death, but now lives on in the app in the form of an augmented reality simulation. When the user stands in the Emperor’s Court of the Munich Residence and points the iPhone at the front of the building, the conservatory is inserted at the original location in real time in exact register in the camera image as a 3D model. This is one of the first uses worldwide of 3D pattern recognition outside experimental show--cases. The technology used here and in the other augmented reality features is from the company Metaio, the global market leader in the field of augmented reality technologies.
Image 4 Video feature in the app “Ludwig II”
Image 5 The conservatory of Ludwig II as augmented reality simulation
As realistic as the real thing - complete 360degree views of the Throne Hall and the Hall of the Singers
A further augmented reality highlight are the 360--degree panoramic views of the Throne Hall and of the Hall of the Singers of castle Neuschwanstein. Even without looking at the original location, the app conveys a fascinating idea of the neo- -Gothic splendour of the castle. What is special about 360 degrees panoramic views in augmented reality technology is the direct influence of the user on the direction and the viewing angle of the panoramic view: The smart phone’s compass and gyroscope help create the illusion of standing in the middle of the room and exploring it by turning around one’s own axis.
An admission ticket comes to life - digital miniatures of the king’s castles
Visitors of the three castles Linderhof, Herrenchiemsee and Neuschwanstein can use the app to produce a digital miniature from their admission tickets as if by magic:
Upon pointing the iPhone camera at the admission ticket, there appears a model of the visited castle as a 3D simulation. The castle miniature is later stored in the app’s album of gifts and can be opened also without admission ticket.
Image 6 Admittance tickets develop a life
Virtual “gifts” in special locations
The app is intended to encourage its users to visit important locations in the life of king Ludwig II. As a little incentive to do so, there are “gifts” for the users, so---called
“virtual goods”, in some locations. Upon visiting the court kitchen in Hohenschwangau the user obtains a digital original recipe of the royal table, in castle Herrenchiemsee the chime of the astronomical clock of Ludwig II exhibited there as a ring tone. The user who visits the place onLake Starnberg where the king died, is offered an excerpt from the psychiatric opinion by Dr. von Gudden which finally led to the disempowerment of Ludwig. These “gamification” elements are intended to entice the user to make the app his/ her steady companion on trips in the footsteps of the fairytale king. All virtual gifts can be collected in the digital album of the app.
“Your year with the king“ -- Historical events as a push message
A further feature intended to boost the user’s loyalty to the app is the calendar.
Titled “Your year with the king” the calendar contains brief historical notes on 148 days of the year, stating what happened in the life of the king on that day. On 20 October one can read for example that on this day in the year 1870 Ludwig II sent several of his ministers to Versailles, to negotiate Bavaria’s future with Bismarck. The calendar can be opened in the app on the respective days or can alternatively send the information as
a push message.
A film of almost four minutes showing all the features “in action” is available on YouTube. It can be located by inputting the search terms “Ludwig II” and “BayStaatsbibliothek”. The app is available in the German and English languages worldwide in the app store for iPhones. All features - by the exception of the map view - can also be used without Internet access. For this reason the app is a unique opportunity, particularly also for visitors from abroad (possibly high roaming fees!), to delve into the world of Ludwig II. The content of the app is available in a slightly slimmed version also as a channel for the free augmented reality browsers Junaio, Layar and Wikitude for the platforms Android, iOS, Symbian OS, BlackBerry7 and Bada. Just look for the keyword “Ludwig II”.
The use of the app is consequently not limited to iPhones.
4. Lessons Learned
The design of an application such as “Ludwig II”, which combines rich multimedia contents with innovative technology, represents a particular challenge to the partners in the venture. It is not sufficient here when the “content supplier” side - in this case two important, traditional cultural institutions of the Free State of Bavaria - hands over the available contents to the technology partner, who is then responsible for the software
“implementation” on the basis of the contractually agreed features. The real task, which can only be fulfilled in close and continuous cooperation by all parties involved, is the mutual integration of content and context, so as to create a unique and fascinating use experience for the user. The design of the content must adapt universally to the Image 7 Virtual gifts in the
app “Ludwig II”
requirements of the chosen technology, and the other way around the technical features must guarantee a distinctive content experience that would not be possible within the framework of the conventional presentation forms of digital contents. Only then will the unity of content and form be created - in a classical sense - that is desired in every cultural product, or to put it into different words: “content is king, but context is queen”. The experience made in the course of drawing up a concept and implementing the app “Ludwig II” can be reproduced in the form of the following “lessons learned”:
1. Both the Bavarian State Library and the Bavarian Department of State-owned Palaces, Gardens and Lakes have a multitude of texts on Ludwig II, for example from exhibition catalogues, castle and museum guidebooks, scholarly articles, etc. However, none of these texts could be integrated in the app in a one-to-one fashion. Rather, the materials at hand had to be converted into a multitude of short text modules, whose wording in addition had to be designed so that they could be employed in combination with a plurality of links in several “positions” of the multimedia information space, which is structured in a fashion that is more associative than argumentative. It was frequently easier to write the texts again from scratch. For designing mobile applications this means in general: The content is never simply already there, but it must usually at least be optimized for the special usability requirements of the mobile use. Particularly for the content-supplier side this leads to a considerable effort that is frequently not expected at the start of the project. A mere “reformatting” of texts already available, which were usually produced in view of the “sequential” use in the analogous medium “book”, is not going to be successful in mobile use scenarios. Accordingly the motto should be:
“Mobile first – resist recycling!”
2. An app about one of the most well-known persons of German history surely has to be implemented in the German language first. However, in the interest of its worldwide success in the app store an English-language version is also indispensable. In this regard, it must be taken into account that, particularly in the case of a “content-rich”
application, the English translation will cause considerable effort and corresponding costs, which are unremarkable as long as one deals exclusively with experimental showcases or promotional applications. For the app “Ludwig II” a total of 250
pages of text elements, 400 captions, around 6 hours of audiovisual images and contemporary witness statements, as well as 148 calendar entries had to be translated and recorded anew in the English language. This also showed that the English language is by no means always briefer than the German language, so that numerous English text elements had to be formatted anew and required adaptations of the screen layout.
3. The cutting-edge augmented reality features of the app - the 3D pattern recognition of the conservatory, the panoramic view of the Throne Hall and the Hall of the Singers, the “living” admittance tickets - are surely highlights of the app and are the focus of all presentation and promotional activities. However, they are only the icing on the cake, an addition to the multimedia blend of texts, images, videos and audiovisual images, which are in sum decisive for the quality of the offer. On their own the augmented reality features can attract attention for a short time, but cannot ensure a recurring and long-term user retention to the app. The maxim here is: Do not rely on one feature alone. It’s the mix that makes the difference!
4. A location-based app such as Ludwig II should always also offer a location- independent access to its content, since it is rather boring otherwise for all users who are not in the locations of the app (that is for the complete “rest of the world”). In the case of “Ludwig II” all referenced locations can also be accessed via the list view, which makes the app suitable also for use on the user’s sofa at home. Thus the app can also be used - like a traditional travel guide and art guide - for preparing a trip to the castles and locations of the life of Ludwig II.
5. Particularly the gamification elements of the app, which activate a feature only once the user is in the proximity of the location in question, can be tested only at a disproportionately great effort when these features are distributed across great distances (in our case: the whole of Bavaria). Testing will inevitably be limited to a test run at the end of the implementation phase, hoping that the go--live will not yield any nasty surprises.
6. The general rule when implementing a location-based app is: Expect the unexpected!
Apart from the changing light and weather conditions that can affect the function of the augmented reality features, the locations themselves, in which the app is
to function, area generally “incalculable” factor. We had this experience with the augmented reality highlight of the app, the simulation of the conservatory. In a final test shortly before putting the app on offer in the app store, we were greeted by the picture represented in image 8 in the Emperor’s Court of the Munich Residence: The optimal viewing position of the app user, on which the programming of the feature had been based, had been made inaccessible for a long time to come by a newly erected scaffolding and had to be readjusted quickly.
Image 8 Scaffolding in the Emperor’s Court in the Munich Residence
With the multimedia character of its contents the app “Ludwig II – Walking in the footsteps of the fairytale king” shows paradigmatically how the content from libraries, archives and museums can be brought to bear fruit using an interdisciplinary approach. The app can thus also be an example for future offers with a topical focus, in the context of national and international cultural portals such as EUROPEANA and the German Digital Library (Deutsche Digitale Bibliothek, DDB).
Moreover, the app can be understood as an experiment to open up the classic information unit “book” for the multimedia, digital information sphere. The texts of the app themselves, with their highly diverse contextualization and multimedia linking, amount to a total of 250 pages, thus representing a veritable book. In the app the traditional sequentiality of “reading” use, which is characteristic for the medium
book, is completely dissolved in favour of the new order principle of the location- based access: The digital content is presented with reference to the location in which I currently am as user, and with reference to the concrete use interest that I am currently having at this location. The future of the mobile Internet will show whether such scenarios of the digital presentation of cultural contents will prevail - and whether it will in the end possibly even replace the classic travel guide/ art guide and exhibition catalogue.
5. Conclusion
The transition from the stationary to the mobile Internet is more than a technology change regarding the use of digital information. It basically represents the transition to the omnipresent Internet, penetrating all areas of life, the “all-pervasive Internet”, which is fully integrated in our daily lives and social routines. Digital, usually multimedia- based information is no longer something that is retrieved or located by the user at an “Internet workstation”, but surrounds the user wherever he goes and is available ubiquitously. Augmented reality, representing a combination of real-worldly and digital information is doubtlessly the paradigmatic use case of this new digital ecosystem.
This development is almost inevitably accompanied by the loss in importance of the large Internet portals and institutional web sites oriented towards “full service” [2]. There is at least a trend that they will be replaced by apps and app-like web service offers tailored to specific purposes and services, which are adapted to the very concrete and multifarious use scenarios of the digital world regarding their individual focus and design: The Internet will become as individual as the life of its users, being their permanent companion.
The mobile Internet, augmented reality and location based services are - insofar as foreseeable – some of the major trends sketching the outlines of the future Internet.
They form an important part of the technological framework in which the generation,
[2] Beyond the mobile web by yiibu, 2011. http://www.slideshare.net/yiibu/beyond--themobilewebbyyiibu
(retrieved on: 05/05/2011)
processing and distribution of information will take place, in which also libraries have to seek and co-design their future as information providers. In the process the library will be almost completely integrated into the diverse and heterogeneous living and working environments of its users and will be “contextualised”. As a functional unit it will thus inevitably become largely “invisible”. However, at the same time it can gain a new and more far-reaching visibility, in that its offers and services provide the fascinating and immersive use experiences made possible by the new technologies.
