Multi-User 3D Augmented Reality Anwendung für die gemeinsame Interaktion mit virtuellen 3D Objekten

Fakesch, Jens (25. Juli 2016): Multi-User 3D Augmented Reality Anwendung für die gemeinsame Interaktion mit virtuellen 3D Objekten. Masterarbeit, Fakultät für Mathematik, Informatik und Statistik, Ludwig-Maximilians-Universität München.

Abstract

This thesis covers the development of a network supported multi user augmented reality application for mobile devices. A presenter can load 3D models dynamically, display them on an augmented reality 2D tracker and is capable of manipulating certain single objects. These manipulations are already well-defined in advance through the hierarchy of the 3D object. Executable manipulations are translation, rotation, scaling and the change of materials. Any number of spectators can follow the presentation with their own device from a point of view of individual choice. If the data of the model is not present on their own device it will automatically be transferred from the presenter via network. Thoughts that were made in advance are described, followed by the details of implementation and the occured problems as well as chosen solutions. With the prototype a user study was conducted to define guidelines for the choice of different kinds of lighting for certain applications. The choice is between static, dynamic and combined lighting. Additionally the general usability of the app is evaluated in the study.

• Dokumententyp: LMU München: Studienabschlussarbeit
• Keywords: Augmented Reality, Mobile Devices, Multi-User, Interaction Methods, Architectural Visualization
• Fakultät: Mathematik, Informatik und Statistik > Informatik
• Institut oder Departement: Media Informatics
• Themengebiete: 000 Informatik, Informationswissenschaft, allgemeine Werke > 000 Informatik, Wissen, Systeme ; 600 Technik, Medizin, angewandte Wissenschaften > 620 Ingenieurwissenschaften
• Sprache: Englisch

Literaturliste

[1] Aila, Timo and Samuli Laine: Alias-free shadow maps. Rendering techniques, 2004:15th, 2004.

[2] Augment. http://www.augment.com/. Aufgerufen am 22.05.2016.

[3] Autodesk: Fbx. http://www.autodesk.com/products/fbx/overview. Aufgerufen am 02.07.2016.

[4] Azuma, Ronald T, Bruce R Hoff, Howard E Neely III, Ronald Sarfaty, Michael J Daily, Gary Bishop, Vern Chi, GregWelch, Ulrich Neumann, Suya You, et al.: Making augmented reality work outdoors requires hybrid tracking. In Proceedings of the First International Workshop on Augmented Reality, volume 1. Citeseer, 1998.

[5] Borg, Mathias, Martin M Paprocki, and Claus B Madsen: Perceptual evaluation of photorealism in real-time 3d augmented reality. In Computer Graphics Theory and Applications (GRAPP), 2014 International Conference on, pages 1–10. IEEE, 2014.

[6] Calian, Dan A, Kenny Mitchell, Derek Nowrouzezahrai, and Jan Kautz: The shading probe: fast appearance acquisition for mobile ar. In SIGGRAPH Asia 2013 Technical Briefs, page 20. ACM, 2013.

[7] Catmull, Edwin: A subdivision algorithm for computer display of curved surfaces. Technical report, DTIC Document, 1974.

[8] Chan, Eric and Frédo Durand: An efficient hybrid shadow rendering algorithm. Rendering Techniques, 2004:15th, 2004.

[9] Crow, Franklin C: Shadow algorithms for computer graphics. In Acm siggraph computer graphics, volume 11, pages 242–248. ACM, 1977.

[10] Debevec, Paul: Rendering synthetic objects into real scenes: Bridging traditional and imagebased graphics with global illumination and high dynamic range photography. In ACM SIGGRAPH 2008 classes, page 32. ACM, 2008.

[11] Donath, Andreas: Apple kauft Münchner Augmented-Reality-Firma. http://www.golem. de/news/metaio-apple-kauft-muenchner-augmented-reality-firma-1505-114330. html. Aufgerufen am 22.05.2016.

[12] Dorfmuller-Ulhaas, Klaus and Dieter Schmalstieg: Finger tracking for interaction in augmented environments. In Augmented Reality, 2001. Proceedings. IEEE and ACM International Symposium on, pages 55–64. IEEE, 2001. [13] Google: Glass. https://www.google.com/glass/start/. Aufgerufen am 05.06.2016. [14] Google: Project Tango. https://www.google.com/atap/project-tango/. Aufgerufen am 22.05.2016.

[15] Gruber, Lukas, Tobias Langlotz, Pintu Sen, Tobias Hoherer, and Dieter Schmalstieg: Efficient and robust radiance transfer for probeless photorealistic augmented reality. In Virtual Reality (VR), 2014 iEEE, pages 15–20. IEEE, 2014.

[16] Heidmann, Tim: Real shadows, real time. Iris Universe, 18:28–31, 1991.

[17] IDC: Smartphone os market share, 2015 q2. http://www.idc.com/prodserv/smartphone-os-market-share.jsp. Aufgerufen am 05.06.2016.75

[18] Isard, Michael and Andrew Blake: Contour tracking by stochastic propagation of conditional density. In Computer Vision-ECCV’96, pages 343–356. Springer, 1996.

[19] Ishii, Hiroshi and Brygg Ullmer: Tangible bits: towards seamless interfaces between people, bits and atoms. In Proceedings of the ACM SIGCHI Conference on Human factors in computing systems, pages 234–241. ACM, 1997.

[20] Jacobs, Katrien and Céline Loscos: Classification of illumination methods for mixed reality. In Computer Graphics Forum, volume 25, pages 29–51. Wiley Online Library, 2006.

[21] Kähäri, Markus and David J Murphy: Mara: Sensor based augmented reality system for mobile imaging device. In 5th IEEE and ACM International Symposium on Mixed and Augmented Reality, volume 13, 2006.

[22] Kan, Paul and Hannes Kaufmann: Differential irradiance caching for fast high-quality light transport between virtual and real worlds. In Mixed and Augmented Reality (ISMAR), 2013 IEEE International Symposium on, pages 133–141. IEEE, 2013.

[23] Kasahara, Shunichi, Valentin Heun, Austin S Lee, and Hiroshi Ishii: Second surface: multiuser spatial collaboration system based on augmented reality. In SIGGRAPH Asia 2012 Emerging Technologies, page 20. ACM, 2012.

[24] Kato, Hirokazu and Mark Billinghurst: Marker tracking and hmd calibration for a videobased augmented reality conferencing system. In Augmented Reality, 1999.(IWAR’99) Proceedings. 2nd IEEE and ACM International Workshop on, pages 85–94. IEEE, 1999.

[25] Lang, Peter, Alben Kusej, Axel Pinz, and Georg Brasseur: Inertial tracking for mobile augmented reality. In Instrumentation and Measurement Technology Conference, 2002. IMTC/2002. Proceedings of the 19th IEEE, volume 2, pages 1583–1587. IEEE, 2002.

[26] Layar. https://www.layar.com/about/. Aufgerufen am 22.05.2016.

[27] Lowe, David G.: Fitting parameterized three-dimensional models to images. IEEE Transactions on Pattern Analysis & Machine Intelligence, (5):441–450, 1991.

[28] Malik, Shahzad, Chris McDonald, and Gerhard Roth: Hand tracking for interactive patternbased augmented reality. 2002.

[29] Microsoft. https://www.microsoft.com/microsoft-hololens/en-us/why-hololens. Aufgerufen am 14.07.2016.

[30] Microsoft: Hololens. https://www.microsoft.com/microsoft-hololens/en-us. Aufgerufen am 05.06.2016.

[31] Milgram, Paul and Fumio Kishino: A taxonomy of mixed reality visual displays. IEICE TRANSACTIONS on Information and Systems, 77(12):1321–1329, 1994.

[32] Mings, Josh: Excuse me, scrawl. your 3d drawings are crowding my reality. http://www.solidsmack.com/3d-cad-technology/ excuse-me-scrawl-your-3d-drawings-are-crowding-my-reality/. Aufgerufen am 29.05.2016.

[33] Motion, Leap. https://www.leapmotion.com/. Aufgerufen am 14.07.2016.

[34] Newman, Joseph, Martin Wagner, Martin Bauer, Asa MacWilliams, Thomas Pintaric, Dagmar Beyer, Daniel Pustka, Franz Strasser, Dieter Schmalstieg, and Gudrun Klinker: Ubiquitous tracking for augmented reality. In Mixed and Augmented Reality, 2004. ISMAR 2004. Third IEEE and ACM International Symposium on, pages 192–201. IEEE, 2004.

[35] niantic. https://www.nianticlabs.com/img/posts/Encounter0.png. Aufgerufen am 14.07.2016.

[36] Norman, Donald A: The design of everyday things: Revised and expanded edition. Basic books, 2013.

[37] Nowrouzezahrai, Derek, Stefan Geiger, Kenny Mitchell, Robert Sumner, Wojciech Jarosz, and Markus Gross: Light factorization for mixed-frequency shadows in augmented reality. In Mixed and Augmented Reality (ISMAR), 2011 10th IEEE International Symposium on, pages 173–179. IEEE, 2011.

[38] Park, Jun, Suya You, and Ulrich Neumann: Natural feature tracking for extendible robust augmented realities. In Proc. Int. Workshop on Augmented Reality, 1998.

[39] Patow, Gustavo and Xavier Pueyo: A survey of inverse rendering problems. In Computer graphics forum, volume 22, pages 663–687. Wiley Online Library, 2003.

[40] Perez, Sarah: Pokémon go tops twitter’s daily users, sees more engagement than facebook. https://techcrunch.com/2016/07/13/pokemon-go-tops-twitters-daily-users-sees-more-engagement-than-facebook. Aufgerufen am 14.07.2016.

[41] Pinz, Axel, Markus Brandner, Harald Ganster, Albert Kusej, Peter Lang, and Miguel Ribo: Hybrid tracking for augmented reality. ÖGAI Journal, 21(1):17–24, 2002.

[42] Poupyrev, Ivan, Numada Tomokazu, and Suzanne Weghorst: Virtual notepad: handwriting in immersive vr. In Virtual Reality Annual International Symposium, 1998. Proceedings., IEEE 1998, pages 126–132. IEEE, 1998.

[43] Pressigout, Muriel and Eric Marchand: Hybrid tracking algorithms for planar and nonplanar structures subject to illumination changes. In Proceedings of the 5th IEEE and ACM International Symposium on Mixed and Augmented Reality, pages 52–55. IEEE Computer Society, 2006.

[44] PTC: PTC Adds Augmented Reality Leader Vuforia to Portfolio. http://www.ptc.com/ about/history/vuforia. Aufgerufen am 22.05.2016.

[45] Reeves, William T, David H Salesin, and Robert L Cook: Rendering antialiased shadows with depth maps. In ACM Siggraph Computer Graphics, volume 21, pages 283–291. ACM, 1987.

[46] Rohmer, Kai,Wolfgang Buschel, Raimund Dachselt, and Thorsten Grosch: Interactive nearfield illumination for photorealistic augmented reality on mobile devices. In Mixed and Augmented Reality (ISMAR), 2014 IEEE International Symposium on, pages 29–38. IEEE, 2014.

[47] Rolland, Jannick P, Larry Davis, and Yohan Baillot: A survey of tracking technology for virtual environments. Fundamentals of wearable computers and augmented reality, 1:67–112, 2001.

[48] Schmalstieg, Dieter, Anton Fuhrmann, and Gerd Hesina: Bridging multiple user interface dimensions with augmented reality. In Augmented Reality, 2000.(ISAR 2000). Proceedings. IEEE and ACM International Symposium on, pages 20–29. IEEE, 2000.

[49] Shi, Jianbo and Carlo Tomasi: Good features to track. In Computer Vision and Pattern Recognition, 1994. Proceedings CVPR’94., 1994 IEEE Computer Society Conference on, pages 593–600. IEEE, 1994. 77 [50] Stiktu. http://blog.stiktu.com/. Aufgerufen am 22.05.2016.

[51] String. http://string.co/. Aufgerufen am 22.05.2016.

[52] TouchScript. http://touchscript.github.io/. Aufgerufen am 01.07.2016.

[53] Tsang, Michael, George W Fitzmaurice, Gordon Kurtenbach, Azam Khan, and Bill Buxton: Boom chameleon: simultaneous capture of 3d viewpoint, voice and gesture annotations on a spatially-aware display. In Proceedings of the 15th annual ACM symposium on User interface software and technology, pages 111–120. ACM, 2002.

[54] Unity3D: Documentation 4.6 lightmapping in-depth. http://docs.unity3d.com/460/ Documentation/Manual/LightmappingInDepth.html. Aufgerufen am 29.05.2016.

[55] Unity3D: Documentation shadow overview. http://docs.unity3d.com/Manual/ ShadowOverview.html. Aufgerufen am 29.05.2016.

[56] Unreal-Engine: Assetbundles. https://docs.unity3d.com/Manual/ AssetBundlesIntro.html. Aufgerufen am 01.07.2016.

[57] Unreal-Engine: Distance field ambient occlusion. https://docs. unrealengine.com/latest/INT/Engine/Rendering/LightingAndShadows/ DistanceFieldAmbientOcclusion/index.html. Aufgerufen am 29.05.2016.

[58] Unreal-Engine: Ray traced distance field soft shadows. https://docs. unrealengine.com/latest/INT/Engine/Rendering/LightingAndShadows/ RayTracedDistanceFieldShadowing/. Aufgerufen am 29.05.2016.

[59] Unreal-Engine: Shadow casting. https://docs.unrealengine.com/latest/INT/ Engine/Rendering/LightingAndShadows/Shadows/. Aufgerufen am 29.05.2016.

[60] Unreal-Engine: Unreal engine 4 for unity developers. https://docs.unrealengine. com/latest/INT/GettingStarted/FromUnity/. Aufgerufen am 05.06.2016.

[61] Van Krevelen, DWF and R Poelman: A survey of augmented reality technologies, applications and limitations. International Journal of Virtual Reality, 9(2):1, 2010. [62] Vuforia. http://www.vuforia.com/. Aufgerufen am 22.05.2016.

[63] Vuforia: Extended Tracking. https://developer.vuforia.com/library/articles/ Training/Extended-Tracking. Aufgerufen am 22.05.2016.

[64] Vuforia: Getting Started. https://developer.vuforia.com/library/ getting-started. Aufgerufen am 10.06.2016.

[65] Vuforia: How To Play the Penguin App . https://developer.vuforia.com/ library/articles/Training/Vuforia-Object-Scanner-Users-Guide. Aufgerufen am 10.06.2016.

[66] Vuforia: Object Recognition. https://developer.vuforia.com/library/articles/ Training/Object-Recognition. Aufgerufen am 22.05.2016.

[67] Vuforia: Vuforia Object Scanner . https://developer.vuforia.com/library/ articles/Solution/How-To-Play-the-Penguin-App. Aufgerufen am 10.06.2016.

[68] Wikimedia. https://commons.wikimedia.org/wiki/File:Shadow_volume_ illustration.png. Aufgerufen am 29.05.2016.

[69] Wikitude. http://www.wikitude.com/about/. Aufgerufen am 22.05.2016.

[70] Williams, Lance: Casting curved shadows on curved surfaces. In ACM Siggraph Computer Graphics, volume 12, pages 270–274. ACM, 1978.

[71] Woo, Andrew, Pierre Poulin, and Alain Fournier: A survey of shadow algorithms. Computer Graphics and Applications, IEEE, 10(6):13–32, 1990.

[72] Zhang, Xiang, Stephan Fronz, and Nassir Navab: Visual marker detection and decoding in ar systems: A comparative study. In Proceedings of the 1st International Symposium on Mixed and Augmented Reality, page 97. IEEE Computer Society, 2002.

[73] Zhou, Feng, Henry Been Lirn Duh, and Mark Billinghurst: Trends in augmented reality tracking, interaction and display: A review of ten years of ismar. In Proceedings of the 7th IEEE/ACM International Symposium on Mixed and Augmented Reality, pages 193–202. IEEE Computer Society, 2008.