4D sensor perception in relativistic image processing

Müller, Simone und Kranzlmüller, Dieter (2025): 4D sensor perception in relativistic image processing. In: Scientific Reports, Bd. 15, 5862 [Forthcoming]

Abstract

This article introduces the 4D sensor perception in relativistic image processing as a novel way of position and depth estimation. Relativistic image processing extends conventional image processing in computer vision to include the theory of relativity and combines temporal sensor and image data. In consideration of these temporal and relativistic aspects, we process diverse types of information in a novel model of 4D space through 10 different degrees of freedom consisting of 4 translations and 6 rotations. In this way, sensor and image data can be related and processed as a causal tensor field. This enables the temporal prediction of a user’s own position and environmental changes as well as the extraction of depth and sensor maps by related sensors and images. The dynamic influences and cross-sensor dependencies are incorporated into the metric calculation of spatial distances and positions, opening up new perspectives on numerous fields of application in mobility, measurement technology, robotics, and medicine.

• Dokumententyp: Zeitschriftenartikel
• Keywords: 4D Sensor Perception; 4D Informatio;, Relativistic Image Processing; Schlingel Diagram
• Fakultät: Mathematik, Informatik und Statistik ; Physik
• Fakultätsübergreifende Einrichtungen: Centrum für Informations- und Sprachverarbeitung (CIS)
• Themengebiete: 000 Informatik, Informationswissenschaft, allgemeine Werke > 000 Informatik, Wissen, Systeme ; 000 Informatik, Informationswissenschaft, allgemeine Werke > 004 Informatik ; 500 Naturwissenschaften und Mathematik > 500 Naturwissenschaften ; 500 Naturwissenschaften und Mathematik > 510 Mathematik ; 500 Naturwissenschaften und Mathematik > 530 Physik ; 600 Technik, Medizin, angewandte Wissenschaften > 600 Technik
• ISSN: 2045-2322
• Sprache: Deutsch

Literaturliste

1. Trafton, J. G., McCurry, J. M., Zish, K. & Frazier, C. R. The perception of agency: Scale reduction and construct validity. ACM Transactions on Human-Robot Interact. 13, 1–23, DOI: 0.1109/RO-MAN57019.2023.10309544 (2024).

2. Akansie, K. Y. O., Biradar, R. C. & Karthik, R. A multi-sensor-based terrain perception model for locomotion selection of hybrid mobile robots. SN Comput. Sci. 5, DOI: 10.1007/s42979-024-02858-6 (2024).

3. Niloy, M. A. K. et al. Critical design and control issues of indoor autonomous mobile robots: A review. IEEE Access 9, 35338–35370, DOI: 10.1109/ACCESS.2021.3062557 (2021).

4. Loganathan, A. & Ahmad, N. S. A systematic review on recent advances in autonomous mobile robot navigation. Eng. Sci. Technol. an Int. J. 40, 101343, DOI: 10.1016/j.jestch.2023.101343 (2023).

5. Xi, Y. et al. Thp: Tensor-field-driven hierarchical path planning for autonomous scene exploration with depth sensors. Comput. Vis. Media 1–15, DOI: 10.1007/s41095-022-0312-6 (2024).

6. Li, J. et al. Visual perception system design for rock breaking robot based on multi-sensor fusion. Multimed. Tools Appl.83, 24795–24814, DOI: 10.1007/s11042-023-16189-w (2024).

7. Ferrara, A. & Incremona, G. P. Design of an integral suboptimal second-order sliding mode controller for the robust motion control of robot manipulators. vol. 23, 2316–2325, DOI: 10.1109/TCST.2015.2420624 (2015).

8. Siciliano, B. & Khatib, O. Springer handbook of robotics. DOI: 10.1007/978-3-319-32552-1 (Springer Cham, 2016).

9. Faiza Gul, W. R. & Alhady, S. S. N. A comprehensive study for robot navigation techniques. Cogent Eng. 6, 1632046, DOI: 10.1080/23311916.2019.1632046 (2019).

10. Siciliano, B. & Khatib, O. (eds.) Springer Handbook of Robotics. Springer Handbooks (Springer, 2016).

11. Müller, S. & Kranzlmüller, D. Self-organising maps for efficient data reduction and visual optimisation of stereoscopic based disparity maps. DOI: 10.24132/CSRN.2021.3101.3 (International Conference in Central Europe on ComputerGraphics, Visualization and Computer Vision (WSCG), 2022).

12. Müller, S. & Kranzlmüller, D. Dynamic sensor matching based on geomagnetic inertial navigation. DOI: 10.24132/CSRN . 2021.3101.3 (International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision (WSCG), 2022).

13. Höller, B., Mossel, A. & Kaufmann, H. Automatic object annotation in streamed and remotely explored large 3d reconstructions. Comput. Vis. Media 7, 71–86, DOI: 10.1007/s41095-020 (2021).

14. Müller, S. & Kranzlmüller, D. Dynamic sensor matching for parallel point cloud data acquisition. In 29. International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision, DOI: 10.24132/CSRN.2021 . 3101.3 (International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision (WSCG), 2021).

15. Müller, S. & Kranzlmüller, D. 4d-based robot navigation using relativistic image processing. In Proceedings of the AAAI Symposium Series, vol. 4, 367–372, DOI: 10.1609/aaaiss.v4i1.31818 (2024).

16. Yang, Y., Bass, C. & Hatton, R. L. Towards geometric motion planning for high-dimensional systems: Gait-based coordinate optimization and local metrics. In 2024 IEEE International Conference on Robotics and Automation (ICRA), 18494–18500, DOI: 10.1109/ICRA57147.2024.10610063 (2024).

17. Ghosal, S. S., Sun, Y. & Li, Y. How to overcome curse-of-dimensionality for out-of-distribution detection? Proc. AAAI Conf. on Artif. Intell. 38, 19849–19857, DOI: 10.1609/aaai.v38i18.29960 (2024).

18. Condurache, D. A generalization of the bresse properties in higher-order kinematics. In International Symposium on Advances in Robot Kinematics, 96–104, DOI: 10.1007/978-3-031-64057-5_12 (Springer, 2024).

19. Condurache, D. Multidual algebra and higher-order kinematics. In Pisla, D., Corves, B. & Vaida, C. (eds.) New Trends in Mechanism and Machine Science, 48–55, DOI: 10.1007/978-3-030-55061-5 (Springer International Publishing, Cham, 2020).

20. Singh, S., Russell, R. P. & Wensing, P. M. On second-order derivatives of rigid-body dynamics: Theory and implementation. vol. 40, 2233–2253, DOI: 10.1109/TRO.2024.3370002 (IEEE Transactions on Robotics, 2024).

21. Müller, A. Higher derivatives of the kinematic mapping and some applications. vol. 76, 70–85, DOI: 10.1016/j.mechmachtheory.2014.01.007 (Mechanism and Machine Theory, 2014).

22. Condurache, D. Higher-order relative kinematics of rigid body and multibody systems. a novel approach with real and dual lie algebras. vol. 176, 104999, DOI: 10.1016/j.mechmachtheory.2022.104999 (2022).

23. Ramík, D. M., Sabourin, C., Moreno, R. & Madani, K. A machine learning based intelligent vision system for autonomous object detection and recognition. vol. 40, DOI: 10.1007/s10489-013-0461-5 (Kluwer Academic Publishers, USA, 2014).

24. Xu, C. et al. Eqmotion: Equivariant multi-agent motion prediction with invariant interaction reasoning. In 2023 IEEE/CVFConference on Computer Vision and Pattern Recognition (CVPR), 1410–1420, DOI: 10.1109/CVPR52729.2023.00142 (CVPR, 2023).

25. Müller, A. Screw and lie group theory in multibody kinematics: Motion representation and recursive kinematics of tree-topology systems. Multibody Syst. Dyn. 43, 37–70, DOI: 10.1007/s11044-017-9582-7 (2018).

26. Smith, W., Singh, S., Rudy, J. & Guan, Y. Lie theory based optimization for unified state planning of mobile manipulators. arXiv preprint arXiv:2410.15443 DOI: 10.48550/arXiv.2410.15443 (2024).

27. MV, A. Poincaré group spin networks. Int. J. Theor. Phys. 63, 145, DOI: 10.1007/s10773-024-05688-7 (2024).

28. Suri, A. Curvature and stability of quasi-geostrophic motion. J. Geom. Phys. 198, 105109, DOI: https://doi.org/10.1016/j.geomphys.2024.105109 (2024).

29. Bauchau, O. A. & Li, L. Tensorial parameterization of rotation and motion. vol. 6, 031007, DOI: 10.1115/1.4003176 (Journal of Computational and Nonlinear Dynamics, 2011).

30. Ruan, M. & Li, X. Complex dynamical properties and chaos control for a discrete modified leslie-gower prey-predator system with holling ii functional response. DOI: 10.1186/s13662-024-03828-1 (Springer, 2024).

31. Einstein, A. Die grundlage der allgemeinen relativitätstheorie. DOI: 10.1002/andp.19163540702 (Wiley, 1916).

32. Rahaman, F. The special theory of relativity: A mathematical approach. DOI: 10.1007/978-981-19-0497-4 (Springer Singapore, 2022).

33. Boblest, S., Müller, T. & Wunner, G. Spezielle und allgemeine relativitätstheorie. DOI: 10.1007/978-3-662-63352-6 (Springer Spektrum Berlin, Heidelberg, 2022).

34. Feng, G. & Huang, J. An optical perspective on the theory of relativity: Basic concepts and the equivalence principle. Optik 224, 165686, DOI: https://doi.org/10.1016/j.ijleo.2020.165686 (2020).

35. Livadiotis, G. & McComas, D. J. The theory of thermodynamic relativity. Sci. Reports 14, 22641, DOI: 10.1038/s41598-024-72779-0 (2024).

36. Girgis, M. Modified lorentz transformations and space-time splitting according to the inverse relativity model. J. Appl. Math. Phys. 12, 2467–2489, DOI: 10.4236/jamp.2024.127148 (2024).

37. Zhao, J. Lorentz transformation derived from relativity of time. J. Mod. Phys. 13, 851–857, DOI: 10.4236/jmp.2022.136048 (2022).

38. Hawking, S. Singularities and the geometry of spacetime. The Eur. Phys. J. H 39, 413–503, DOI: 10.1007/s10773-024-05688-7 (2014).

39. Chang, D. C. The physical basis of lorentz transformation and minkowski’s four-dimensional space–time. In On the Wave Nature of Matter: A New Approach to Reconciling Quantum Mechanics and Relativity, 179–190, DOI: 10.1007978-3-031-48777-4_ 13 (Springer, 2024).

40. Tsamparlis, M. Solved Problems and Systematic Introduction to Special Relativity (Springer, 2024).

41. Chaichian, M., Kulish, P., Nishijima, K. & Tureanu, A. On a lorentz-invariant interpretation of noncommutative space–time and its implications on noncommutative qft. vol. 604, 98–102, DOI: 10.1016/j.physletb.2004.10.045 (2004).

42. Halliday, D., Resnick, R. & Walker, J. Principles of physics (John Wiley & Sons, 2023).

43. Krig, S. Computer vision metrics: Survey, taxonomy and analysis of computer vision, visual neuroscience, and deep learning. DOI: 10.1007/978-3-319-33762-3 (Springer, 2016).

44. Szeliski, R. Computer vision - algorithms and applications. DOI: https://doi.org/10.1007/978-3-030-34372-9 (Springer, 2022).

45. Murat, I. Comprehensive empirical evaluation of feature extractors in computer vision. PeerJ Comput. Sci. 10, e2415, DOI: 10.7717/peerj-cs.2415 (2024).

46. Hui, M., Wei, Z., Zhu, H., Xia, F. & Zhou, Y. Microdiffusion: Implicit representation-guided diffusion for 3d reconstruction from limited 2d microscopy projections. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 11460–11469, DOI: 10.1016/S1076-5670(06)45002-3 (2024).

47. Xue, W., Lu, F. & Chen, G. Hdmnet: A hierarchical matching network with double attention for large-scale outdoor lidar point cloud registration. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 3393–3403, DOI: 10.48550/arXiv.2310.18874 (2024).

48. Chen, Z. et al. Shc: soft-hard correspondences framework for simplifying point cloud registration. EURASIP J. on Adv. Signal Process. 2024, 13, DOI: 10.1186/s13634-023-01104-0 (2024).

49. Gao, Z. et al. Self-supervised pre-training with symmetric superimposition modeling for scene text recognition. arXiv preprint arXiv:2405.05841 DOI: 10.48550/arXiv.2405.05841 (2024).

50. Han, J. et al. Blind image decomposition. In European Conference on Computer Vision, 218–237, DOI: 10.1007/978-3-031-19797-0_ 13 (Springer, 2022).

51. Jiang, Y. et al. The application and accuracy of feature matching on automated cephalometric superimposition. BMC Med. Imaging 20, 1–7, DOI: 10.1186/s12880-020-00432-z (2020).

52. Kanade, T. Computer vision as a physical science. CMU Comput. Sci. A 25th Anniv. Commem. (1991).

53. Shiba, S., Klose, Y., Aoki, Y. & Gallego, G. Secrets of event-based optical flow, depth and ego-motion estimation by contrast maximization. IEEE Transactions on Pattern Analysis Mach. Intell. DOI: 10.1109/TPAMI.2024.3396116 (2024).

54. Heikkila, J. & Silven, O. A four-step camera calibration procedure with implicit image correction. pp. 1106, DOI: 10.5555/794189.794489 (IEEE Computer Society, 1997).

55. Zou, J., Xu, Z., Wu, S. & Zhou, L. Improved anti-aliasing algorithm based on fast approximate anti-aliasing. In Fifteenth International Conference on Graphics and Image Processing (ICGIP 2023), vol. 13089, 568–579, DOI: 10.1117/12 . 3021120 (SPIE, 2024).

56. Sung, K., Pearce, A. & Wang, C. Spatial-temporal antialiasing. IEEE Transactions on Vis. Comput. Graph. 8, 144–153, DOI: 10.1109/2945.998667 (2002).

57. Yang, L., Liu, S. & Salvi, M. A survey of temporal antialiasing techniques. In Computer graphics forum, vol. 39, 607–621, DOI: 10.1111/cgf.14018 (Wiley Online Library, 2020).