--- title: System architecture description: Introduces front-end, client, back-end, database, object storage, backup, and service split architectures. slug: architecture lang: en category: Architecture And Storage category_order: 7 order: 2 keywords: - Architecture - Docker - SOA - object storage --- # System architecture ## System architecture Architecture overview Front-end architecture Front-end architecture includes the following key components and technologies: Backbone.js was chosen as the main architecture, which provides component-based development, state management, routing control and other functions. On its basis, Gulp, Babel, browserify, Scss, jQuery, underscore, ES6, etc. are introduced to build maintainable and high-performance front-end applications. The WebSocket protocol and background are used for instant messaging of messages. Designed with a unified style, user-friendly and intuitive interface, including search, folder management, file preview and other functions. The website supports full platform preview and uses CSS media query and mobile optimization technology to create a responsive layout to ensure that network disk products have consistent appearance and functionality on different devices. Considering multi-language support, the system currently supports Chinese and English languages, which can be switched at any time on the interface or through the browser language. Front-end performance optimization technologies such as code splitting, lazy loading, resource compression, and CDN acceleration are adopted to improve page loading speed and response time. Implement front-end security best practices, including XSS (cross-site scripting) and CSRF (cross-site request forgery) protection. Ensure data transmission is encrypted and stored securely. Cross-platform and multi-language: The front-end part of the BabelBird system includes: BabelBird iOS version (iPhone, iPad), BabelBird Android version, BabelBird web version, BabelBird WeChat version (H5), BabelBird mac client, BabelBird windows client, and BabelBird Linux (Kirin) client. The native iOS client is developed using Object-C and Swift. The native Android client and Hongmeng are developed using Java (Android studio). Windows, mac, and Kirin clients are developed using Electron and C++, and include four components: client interface, synchronization terminal, transmission management, and mapping disk. Backend architecture Server topology Babel's back-end architecture is mainly responsible for business logic, data processing and interaction with the front-end. It meets the following key objectives: Modular design: The backend adopts a modular design to divide different functions and business logic into independent modules. This helps with code maintainability and scalability. Performance: The Babel backend has good performance and can handle large-scale file uploads, downloads, searches, and collaboration requests. Appropriate load balancing and caching strategies and optimization techniques are adopted to improve response times. Security: The Babel backend focuses on strengthening security, including user authentication, authorization, data encryption, and defensive programming to protect user data from attacks. Scalability: The Babel backend architecture is scalable to accommodate future business needs and the addition of new features. Adopts SOA service architecture and modular component design to support flexible expansion of functions. Data Management: The Babel backend efficiently manages user data, file data, and metadata. This includes database design, backup strategies and data cleansing. Frame selection Babel adopts a loose, service-oriented architecture model. Different services are designed, developed, and deployed independently, thus ensuring the stability and scalability of the system. Eventually, we will use docker to containerize these services for unified management and deployment. For services with simple logic and high-frequency access, we choose Nodejs and the corresponding development framework. For services with complex logic and low consumption, we will choose Struts2+Spring framework. For services with complex logic and high consumption (such as MD5 calculation, file blocking, encryption), we will choose the Rust language for development. data layer Database: Babel chose MySQL as the database management system, and also supports the use of other databases (Gaussian, PostgreSQL, Oracle can be supported on demand) At the same time, Babel adopts ORM (Object Relational Mapping): using the ORM framework (Hibernate) to map database tables to Java objects to simplify data access and persistence. security User Authentication: Use strong user authentication mechanisms, including single sign-on (SSO), OAuth authentication, and multi-factor authentication (MFA). Authorization: Use an authorization framework such as Spring Security to define and enforce access control policies to ensure that users can only access the resources they are authorized to use. Data Encryption: Implement encryption of data transmission and storage to protect the security of user data. business logic File Management: Developed business logic to handle file upload, download, version control, sharing and collaboration functionality. User management: Implement user management, rights management and user organizational structure. Search function: An efficient search engine has been developed to support fast retrieval of files and metadata. Online editing: Responsible for online editing and collaborative editing of documents. File transcoding: Responsible for batch queuing transcoding of files and generating preview files. Object storage: file blocking and disk placement, storage system management. Web services RESTful API: Use RESTful API to support data interaction and collaboration between the front end and other clients. Cache management Caching strategy: Use caching to improve performance, including page caching, data caching, and session caching. Asynchronous task processing Message queue: Use the message queue system RabbitMQ to handle asynchronous tasks, such as file transcoding, file preview generation, etc. By using the Struts2 and Spring framework, combined with the above design considerations, a powerful, high-performance, safe and reliable enterprise network disk back-end system can be built to meet user needs and support future expansion and function additions. Cloud infrastructure selection (public cloud SAAS version BabelBird) The choice of public cloud infrastructure can directly affect the performance, reliability and security of enterprise network disk products. The Babel public cloud version selected Alibaba Cloud's ECS, OSS services and CDN distribution business after many comparisons. Currently (in 2023), the Babel system running in public cloud SAAS mode has been operating online with zero server failures for more than 10 years. Cloud service provider selection The business server SAAS version BabelBird uses Alibaba Cloud's ECS (cloud server) cluster Alibaba Cloud's ECS is an elastic computing service that provides you with virtual machine instances so that you can run applications in the cloud. These are some of the key advantages of choosing ECS: Flexibility: ECS allows you to create, start and stop virtual machine instances as needed. This means you can scale your computing resources up or down based on your traffic needs. Scalability: Alibaba Cloud provides a variety of ECS instances of different specifications to meet the needs of different workloads. You can choose based on the performance requirements of your application. Reliability: Alibaba Cloud's ECS instances are generally highly available, providing multiple data centers and availability zones to ensure business continuity. Security: Alibaba Cloud provides various security features, including network isolation, firewalls, and security groups, to help you protect your virtual machine instances from malicious attacks. The object storage SAAS version of Babel uses Alibaba Cloud's OSS (Object Storage Service) Alibaba Cloud's OSS is a highly scalable, safe and reliable cloud storage service suitable for storing and managing large amounts of files and data. Here are some advantages of choosing OSS: Scalable storage: OSS provides scalable storage capacity that can be dynamically expanded according to your needs. This is very important for file storage in enterprise network disk products. Data backup and recovery: Alibaba Cloud's OSS has data backup and version control functions that can help you protect your data from loss or damage. Security: Data stored in OSS is usually protected by encryption and access control to ensure data security. CDN integration: Integrating with CDN services can improve file distribution speed and user experience. To ensure smooth access around the world, BabelBird also uses CDN distribution A CDN (Content Delivery Network) service caches content to globally distributed edge nodes to reduce loading times and improve user experience. Alibaba Cloud's CDN service can bring the following benefits to enterprise network disk products: Fast distribution: Accelerate the distribution of files, ensuring that users can quickly access and download files, regardless of geographical location. Load balancing: CDN can help share the load of the server, reduce the pressure on the server, and improve the performance of the website. Security: CDN services usually have DDoS attack protection features to protect the website from malicious attacks. Global coverage: Alibaba Cloud's CDN service has global coverage, ensuring that your content is quickly delivered to users around the world. Data center location Currently, BabelBird uses two data centers of Alibaba Cloud: Hangzhou Data Center and Weihai Data Center The configuration of one south and one north can ensure the sustainability of services and disaster recovery needs. Database design Database type BabelBird Enterprise Drive uses various types of databases, and can also support Xinchuang's database products or large databases such as Oracle according to customer needs. The default product database of BabelBird is: The core database is a relational database mysql Auxiliary databases include: Redis database, MongoDB database Database schema Data architecture plays a vital role in enterprise network disk systems, ensuring high availability, performance and consistency of data. The following is the database structure of BabelBird: A. Master-slave database architecture Master The main library is responsible for processing write operations, including users uploading files, modifying file information, etc. The main database provides high availability, data consistency and transaction processing. Slave The slave library is mainly used for reading operations, including file query and search. The slave library can provide load balancing and share the read load of the master library. Data synchronization: Use data replication mechanism to ensure that the data of the slave database and the master database are synchronized. Failover: The slave database can take over write operations when the master database fails, ensuring system continuity. B. Relational database (MySQL) Database design The database is designed to support file and user management, including the definition of tables such as file metadata, user information, and permissions. The relationship between data tables is clear, and foreign keys are used to maintain data consistency. Data redundancy The slave database in the master-slave architecture is used for data redundancy to prevent data loss when the master database fails. The data synchronization mechanism ensures that the data of the slave database is consistent with that of the master database. C. NoSQL database (MongoDB) File metadata storage MongoDB can be used to store metadata of files, such as file name, size, version information, etc. MongoDB with elastic schema works well with changing data schemas. D. Cache database (Redis) Cache read data Redis is used to cache frequently read data to reduce the load on the database. Cached data can include user session information, popular files, access logs, etc. E. Data backup and recovery Backup strategy Regularly back up the data of the master and slave databases to ensure data security. Babel backup data is stored in reliable storage in various geographical locations. disaster recovery We develop detailed recovery plans that include steps and timelines for restoring data from backups. The Babel operations team regularly tests the disaster recovery process to ensure its effectiveness. F. Data consistency and transaction management Transaction support Use database transactions to ensure consistency across a range of operations, such as file uploads, sharing, and permission changes. Transactions should be managed correctly to avoid data corruption and inconsistency. Concurrency control Use database locks and isolation levels to manage concurrent access to avoid data races and conflicts. G. Monitoring and Performance Optimization Performance monitoring Implement database performance monitoring, including query response time, load, and resource utilization. Perform performance adjustments based on monitoring results to ensure database stability and performance. Query optimization Optimize complex queries to improve query performance, including index optimization and query plan analysis. These data architecture elements will help ensure that your enterprise network disk system can achieve high availability, excellent performance, and data security. With proper database design and maintenance, you can provide stable and reliable services that meet user needs and ensure data integrity. Data backup and recovery strategy (public cloud) A. Data backup Regular backup plan: BabelBird has a regular backup plan to ensure continuous backup of data (data backup in the early morning every day). Full backup and incremental backup: Babel adopts a combined strategy of full backup and incremental backup. A full backup backs up all data in the database, while an incremental backup only backs up data that has changed since the last backup to reduce backup time and storage space consumption. Multiple backup locations: The public cloud version of BabelBird has two geographical locations: Hangzhou computer room and Weihai computer room to cope with natural disasters or hardware failures. Cloud storage and remote data centers are common backup storage options. Automated backup: BabelBird's database backup adopts a fully automatic backup mechanism without manual intervention to improve reliability. Backup testing: We regularly test backups to ensure their integrity and availability. Test by restoring the backup data to a test environment and verifying the consistency of the data. B. Disaster recovery Recovery Plan: We create a detailed recovery plan that clearly identifies each step and responsibility. Make sure team members understand how to perform the recovery process. Backup and recovery testing: We regularly test our backup and recovery processes to ensure that data can be restored quickly and efficiently in the event of an emergency. Testing should include simulations of various failure scenarios. Backup monitoring: Monitor the running status of backup tasks, and automatically remind and report errors when problems occur to ensure the normal operation of the backup system. Disaster recovery team: We have a dedicated disaster recovery team, on call 24/7, responsible for taking action in emergencies to ensure the continued operation of the business. Communication plan: We have established an effective communication plan and automatic alert message delivery monitoring to inform relevant stakeholders about disaster events and recovery processes. Documentation and training: Ensure detailed documentation of recovery plans and procedures and provide training to team members on recovery strategies and procedures. The above strategies and efforts are based on ensuring the security and availability of database data and minimizing the risk of data loss and system downtime.