We will demonstrate a Swift-based data storage solution, capable of storing and managing up to 100 exabytes of data. This data storage system is performant, scalable, distributed, and fully encrypted. It features: - Rapid user and key-to-file mapping with minimal and predictable disk seek time. - Robust chunk (blob) storage supporting fixed and variable chunk sizes. - Support for file checksums to ensure data integrity. - Automatic data healing to mitigate of bit rot. - Resilient architecture with no single point of failure, using automatic asynchronous replication. - Rack and data centre-aware replication with automatic performance monitoring and configuration adjustments. - Multi-layered metadata management optimised for data analysis and efficient compression of time-series data. - Automated recovery mechanisms to handle catastrophic hardware failures or complete data center outages. - Geographically-aware data storage to ensure compliance with regional data storage regulations (e.g., storing U.S. customer data exclusively in U.S.-based data centres) - Granular user- and organisation-based data access control.
Data access is provided through REST APIs and customisable callback APIs tailored for client-specific data analysis crawlers
My first real experience with computers was as a teenager when I created a simple application to control my homemade telescope on an Apple ][e. Soon after, I learned to program IBM 370 mainframes, writing FORTRAN programs to calculate FFTs for variable stars, as well as a PL/1-based accounting system. Before finishing high school, I programmed the weather station for Bulgaria’s first satellite and contributed to the development of the control software for the Bulgarian Academy of Sciences’ 2-meter telescope.
During my university studies in Heidelberg, Germany, and subsequent research at the European Molecular Biology Laboratory (EMBL), my work centred around protein folding and computer graphics using Silicon Graphics, SUN, DEC Alpha, Maspar, and NeXT computers. In this period, I developed the first Braille driver for NeXTStep, co-organised two scientific computing conferences, hosted the GNUStep project for two years, and contributed significantly to the MiscKit framework for NeXTStep. Additionally, I was an early adopter and contributor to the WebObjects framework (initially Objective-C-based, later Java-based).
I started developing for NeXTStep around 1990. Alongside engaging extensively in open source projects, I worked on a database for storing infrared astronomical images and developed image-processing software for NMR crystallographic data. Later, I was involved in the initial implementation of Apple’s VoiceOver accessibility technology and presented frequently on Apple’s behalf to the European banking and finance sectors. I also delivered a presentation at Apple’s WWDC focused on scientific applications.
During the last 25 or so years, my team worked on several highly complex systems. Examples include: - An educational portal for the Ministry of Education in Lower Saxony, Germany. - A legal documentation system for the Baden-Württemberg Police Department, Germany. - The Mars Rover image archive for NASA. - A fully robotic astronomical observatory control system, deployed in observatories across the U.S., South Africa, and Europe. - The largest inter-bank document and transaction exchange system in the Netherlands (ongoing) - Observatory control, image processing, and large-scale data storage solutions for space debris monitoring stations (ongoing).