BUILDING A HIGH-PERFORMANCE REAL-TIME CHAT SYSTEM USING MSSQL: A SCALABLE, AI-DRIVEN ALTERNATIVE TO PROPRIETARY CHAT SOLUTIONS
Sipilov I.
Sipilov Ivan — Head of Platform Development, CITADEL HEDGE FUND, FOUNDER OF NANNYSERVICES.CS, NEW YORK, USA
Abstract: this paper explores the design and implementation of a custom-built, real-time chat system using MSSQL as a backend database. Serving over 480,000 active users globally and processing more than 4 million messages daily, the system was developed to be highly performant, cost-efficient, and fully self-hosted, without reliance on external tools such as Firebase. Additionally, the system integrates a GPT-based post-moderation model to automatically detect and flag problematic messages, such as those involving harassment or abuse. The model learns and improves using machine learning based on user moderation flags, enhancing future detection. This article focuses on the technical implementation, scalability, and performance, alongside the ethical considerations of using AI-driven moderation.
Keywords: Real-time chat system, MSSQL, scalable chat platform, AI-driven moderation, GPT-based moderation, custom-built chat solution, machine learning for moderation, SignalR real-time communication, in-memory OLTP, Service Broker, Change Data Capture, horizontal scaling, reactive event store, Firebase alternative, self-hosted chat platform, cost-efficient chat system, privacy in chat systems, global chat infrastructure, harassment detection, message flagging, vendor lock-in, relational database in real-time systems, message throughput, message partitioning, ethical AI moderation, data sovereignty in chat platforms, high-volume messaging system.
References
- RabbitMQ Quorum Queues: RabbitMQ official documentation provides insights into quorum queues, their durability, and how they implement the Raft Consensus Algorithm to ensure reliable message delivery. Quorum Queues Documentation
- MSSQL Service Broker: Microsoft documentation explains how Service Broker allows applications to send and receive asynchronous messages, providing a mechanism for real-time communication. Microsoft Service Broker Documentation
- NET Core SignalR: Microsoft's ASP.NET Core SignalR enables real-time web functionality, allowing server-side code to push content to connected clients instantly. ASP.NET Core SignalR Documentation
- In-Memory OLTP: SQL Server In-Memory OLTP offers memory-optimized tables that improve performance for high-volume transactional workloads. SQL Server In-Memory OLTP Documentation
- GPT-3 for AI Moderation: OpenAI's GPT-3 model has been adapted for text moderation tasks, helping identify harmful and abusive content in messaging platforms. OpenAI GPT-3 Model Overview
- Firebase Real-Time Database: Firebase's NoSQL JSON data structure offers real-time data synchronization for applications but faces challenges with large-scale applications. Firebase Realtime Database Documentation
- Change Data Capture (CDC) in MSSQL: Microsoft’s Change Data Capture feature allows real-time monitoring of database changes, facilitating real-time data-driven applications. CDC Documentation
- SignalR WebSockets: SignalR supports WebSockets for maintaining real-time communication between clients and servers, ensuring low-latency message delivery. SignalR and WebSockets
- AI-Based Post Moderation: AI-based models for text moderation, such as GPT, are becoming increasingly relevant in social platforms, providing scalability and efficiency in detecting inappropriate content. AI Text Moderation Overview
- Cost Comparison Between Firebase and MSSQL: Cost comparisons between Firebase and MSSQL are critical when considering high-scale messaging systems. Firebase's costs can grow exponentially with usage, while MSSQL provides more predictable cost structures for large-scale operations. Firebase Pricing and SQL Server Pricing
Ссылка для цитирования данной статьи
Тип лицензии на данную статью – CC BY 4.0. Это значит, что Вы можете свободно цитировать данную статью на любом носителе и в любом формате при указании авторства. | ||
Sipilov I. BUILDING A HIGH-PERFORMANCE REAL-TIME CHAT SYSTEM USING MSSQL: A SCALABLE, AI-DRIVEN ALTERNATIVE TO PROPRIETARY CHAT SOLUTIONS // Наука, образование и культура - № 4(70), 2024 {см. журнал} |