Could you recommend good resources that give a broad overview of recent advances and interesting projects in this area not focused exclusively on crypto currency? Thanks.

tldr; Radical decentralization of all the things.

I hope this isnt seen as spam, I have recently posted on this subreddit. the subject matter of my posts are looking to provide clarity on different aspects of the app. It would be too much for any single post.

Decentralization can mean a lot of things because it can be open to interpretation. In this article, we would like to talk about what decentralization means to us and how we are approaching our definition of decentralized. We are working on a typical chat app, but our approach is unique and may push the boundaries of what it means for something to be self-hosted.

Our decentralized chat application is built as a Progressive Web Apps (PWA) and leverages modern web technologies to deliver a user experience comparable to native apps. Unlike traditional centralized chat apps, our chat app is designed to operate independently of central servers, ensuring that user data remains private and secure. This article will delve into the various components and architectural decisions that make our app truly decentralized.

The Decentralized Chat App

An overview of how our chat application leverages Progressive Web App (PWA) technology to provide a seamless user experience across different devices and platforms.

https://preview.redd.it/9982rwnzs46d1.png?width=1100&format=png&auto=webp&s=6d192095e61a6ff3229e28b5ae849be628f3c799

Data Storage and Networking

A typical app relies heavily on data storage and networking capabilities. In our decentralized chat application, we’ve implemented advanced solutions to handle these aspects efficiently and securely.

Data Storage

In our app, we use IndexedDB to store data directly in the browser’s storage. While browsers provide multiple options for storing information, ranging from cookies to local storage, IndexedDB offers a more advanced and robust solution.

Capacity: IndexedDB can store significantly larger amounts of data compared to cookies or local storage.
Structure: It allows for complex data structures, including collections of files and blobs.
Performance: IndexedDB operates asynchronously, meaning it won’t block the main thread of your application, leading to better performance for data-heavy applications.

Using IndexedDB, we ensure that user data is stored locally on their devices, providing both privacy and independence from central servers.

Networking

For networking, our app uses PeerJS-server as a connection broker to establish WebRTC connections between peers.

Direct Peer-to-Peer: WebRTC, provided by the browser, enables real-time communication between peers. This allows two users to connect directly and exchange data with minimal latency.
Efficient Routing: Once a connection is established, data is sent via the shortest possible network route, enhancing speed and efficiency.
Multiple Connections: Browsers can handle multiple WebRTC connections simultaneously, though the number of connections can vary depending on the device and network capabilities.

PeerJS-server helps in the initial connection setup by acting as a signaling server. It facilitates the exchange of connection information between peers, enabling them to establish a direct WebRTC connection.

By leveraging IndexedDB for data storage and WebRTC with PeerJS-server for networking, our chat application achieves a high level of decentralization, ensuring user data remains private and connections are efficient and reliable. These components form the backbone of our approach, pushing the boundaries of what it means for an app to be truly self-hosted and decentralized

https://preview.redd.it/q2y7dyq1t46d1.png?width=1100&format=png&auto=webp&s=a65a2623a88fcf583f810389f4229faaf2626d66

Static Distribution and Deployment

Our app is essentially a bundle of static files, which makes it highly portable and easy to distribute. We leverage AWS services such as S3 and CloudFormation to deploy our app, ensuring it is distributed efficiently over a Content Delivery Network (CDN) across AWS servers.

AWS S3 and CloudFormation

AWS S3: We host our app on AWS S3, a scalable storage service that allows us to serve static files reliably. S3 ensures that our app is always available and can handle large numbers of requests without performance degradation.
CloudFormation: By using AWS CloudFormation, we automate the deployment process, managing our infrastructure as code. This makes it easy to replicate and manage our app’s deployment environment.

Hosting on S3 and using CloudFormation means our app benefits from AWS’s global CDN. This ensures that our static files are cached and served from servers closest to our users, reducing latency and improving load times.

Offline Availability and Self-Hosting

We go a step further by providing users with the option to download a zip file of the app directly from within the app. This ensures that users can run the app locally without relying on our servers.

Direct from Index.html: Unlike some other apps, our app is a pure JavaScript implementation, designed to run directly from the index.html file. Users can simply open this file in their browser and start using the app without needing a server.
Docker Option: For users who prefer or require a server setup, we offer a Docker configuration. This allows the app to be run in a containerized environment, providing flexibility and ease of deployment for different use cases.

Unminified Code for Transparency

We believe in transparency and accessibility. Therefore, we run the app as unminified code. This has several advantages:

Ease of Download: Users can download the entire app using simple browser commands like Ctrl/Cmd+S.
Transparency: By providing unminified code, we ensure that users can inspect and understand the code they are running. This fosters trust and allows for easier customization and debugging.

By distributing our app as static files and offering multiple ways to run it, we ensure that users have maximum flexibility and control. Whether using AWS’s robust infrastructure or running the app locally, our approach embodies the principles of decentralization and user empowerment.

Webpack 5 Module Federation

Our app leverages Webpack 5’s Module Federation feature to implement a microfrontend architecture. This approach allows us to modularize our application, making it easier to maintain, develop, and scale. You can explore our open-source implementation here: https://github.com/positive-intentions/frontend-base.

Current Implementation

We have used Webpack 5 Module Federation to build the chat application, which you can find here: https://github.com/positive-intentions/chat. Additionally, we have created a federated module for cryptography that can be imported at runtime, available here: https://github.com/positive-intentions/cryptography. Our current setup is straightforward, allowing us to dynamically load different parts of the application as needed. This modular approach provides flexibility and efficiency in how we develop and deploy our app.

Future Enhancements

We plan to further decompose the application into separate microfrontends, each responsible for a specific aspect of the app. This will not only simplify maintenance but also enhance documentation and development processes for individual components.

Planned Modules:

1. UI Components: A collection of reusable UI components that can be shared across different parts of the application, promoting consistency and reducing duplication.
2. P2P Framework: The core framework for peer-to-peer networking, facilitating real-time communication between users.
3. State Management: This module will manage the application’s state, ensuring efficient data handling and synchronization across different components and peers.
4. Permission Manager: A dedicated module for managing user permissions and access control, enhancing security and user management.

https://preview.redd.it/rfitr0p4t46d1.png?width=1100&format=png&auto=webp&s=e6f3199083403b7aad4b0177859afa0121ee2403

Individual Storybooks and Module Exports

Each module will have its own Storybook and module export, providing a dedicated space for documentation, testing, and showcasing individual components. This approach will:

Enhance Documentation: Each module will be well-documented, making it easier for developers to understand and use them.
Simplify Maintenance: By isolating modules, we can update and maintain them independently, reducing the risk of breaking changes affecting the entire app.
Promote Reusability: Well-defined modules can be reused across different projects, promoting a modular and efficient development approach.

By embracing Webpack 5 Module Federation and breaking down our app into distinct microfrontends, we aim to create a more robust, scalable, and maintainable architecture. This approach not only benefits our development process but also enhances the overall user experience by ensuring that each component is well-crafted and easily accessible.

Redundancy and Reliability

In the past, we have encountered difficulties when deploying our app to AWS due to various technical issues. Given that our app functions as a self-hosted static application, we have started exploring alternative hosting options, such as GitHub Pages. You can view our deployment on GitHub Pages here: https://positive-intentions.github.io/chat.

GitHub Pages Deployment

Our microfrontend architecture allows us to deploy each module independently. This capability has enabled us to host copies of the app on GitHub Pages for each repository. While this approach is unconventional and can lead to redundant copies of the app, we believe it is a valuable exercise in exploring module federation redundancy.

Accessibility: Hosting on GitHub Pages makes the app easily accessible to users and developers.
Simplicity: GitHub Pages offers a straightforward deployment process, reducing the complexity often associated with other hosting solutions.
Cost-Effective: GitHub Pages is free, making it an economical choice for hosting static sites.

Redundancy and Interoperability

We aim to make our app interoperable across different hosting sources. By leveraging Webpack 5 Module Federation, we can ensure that the app works seamlessly whether it is served from AWS or GitHub Pages. This redundancy enhances the app’s reliability and availability.

Module Federation Redundancy: We are investigating ways to make modules interoperable between different sources, ensuring that the app remains functional even if one source becomes unavailable.
Interoperable Deployment: Our goal is to allow the app to fetch and integrate modules from both AWS and GitHub Pages dynamically.

https://preview.redd.it/hdsfylw7t46d1.png?width=1100&format=png&auto=webp&s=2c36f5d0e5ce5dcc204979f15b8b60c81e85cbb4

Future Plans

To further enhance our deployment strategy, we plan to use a infrastructure as code tool, to deploy the app across multiple cloud service providers. This approach will increase the resilience of our deployment, ensuring the app remains online and functional even if one provider experiences downtime.

Multi-Cloud Deployment: We can deploy the app to various cloud service providers, including AWS, Azure, and Google Cloud. This will distribute the app’s load and reduce the risk of a single point of failure.
Scalability: This will enable us to scale the app easily across different providers, ensuring it can handle increased traffic without performance issues.
Cost Management: Static file hosting is relatively a cheap option for hosting a webapp, we can optimize costs and ensure that the app remains affordable to maintain.

By exploring these new hosting options and implementing a robust deployment strategy, we aim to make our decentralized chat app more resilient, scalable, and cost-effective. Our efforts in redundancy and interoperability will ensure that the app continues to function seamlessly, providing a reliable user experience regardless of the hosting source.

Multi-Device Architecture

As our app continues to develop, we are excited to introduce the concept of a decentralized profile that can be shared across multiple devices. This feature is designed for individuals who want to use the same profile on all their devices, providing a seamless and integrated multi-platform chat experience, much like any modern chat application.

Decentralized Profile

A decentralized profile allows users to maintain a consistent identity and settings across different devices. This means that whether you are using a smartphone, tablet, or desktop, your profile can remain synchronized and up-to-date.

Profile Synchronization: Users can access their chat history, contacts, and settings on any device by synchronizing their decentralized profile.
Data Consistency: Changes made on one device are automatically reflected on others, ensuring a consistent experience.

Multi-Platform Chat Experience

Implementing a decentralized profile enables us to offer a robust multi-platform chat experience. Users can switch between devices without losing their chat continuity or settings, enhancing the overall user experience.

Convenience: Users can start a conversation on one device and continue it on another without any interruptions.
Flexibility: The app adapts to various devices, providing a user-friendly interface and experience on each platform.

https://preview.redd.it/nbxfottat46d1.png?width=1100&format=png&auto=webp&s=cd91d4d75a5af32f6baa6bbbfb6354e817bfc7b3

Decentralized File Storage

In addition to profile synchronization, we are exploring the potential of decentralized file storage. This feature would allow users to move large files between devices quickly, easily and securely.

Cross-Device Sharing: Users can share files between their devices effortlessly, whether they are transferring a document from their laptop to their phone or vice versa.
Collaborative Work: Teams can collaborate more effectively by sharing encrypted files directly through the app, regardless of the devices they are using.

By introducing a decentralized profile and exploring decentralized file storage, we aim to enhance our app’s functionality and user experience. These features will provide users with the convenience and flexibility expected from a modern chat application while maintaining the principles of decentralization and security.

Decentralized Peer Discovery

Decentralized peer discovery is a critical component of our app’s architecture. By leveraging innovative technologies such as QR codes, NFC, and BLE, we facilitate seamless connections between peers without relying on a central authority.

QR Codes

QR codes provide a straightforward method for establishing peer connections. Users can generate a QR code that contains their connection details, which other users can scan to initiate a peer-to-peer connection.

Ease of Use: Users can quickly and easily share connection details.
Security: QR codes can be generated dynamically, reducing the risk of interception.

https://preview.redd.it/4l533guet46d1.png?width=793&format=png&auto=webp&s=9e41b16da03a7d834e3c5c88ca0975d64fb24421

NFC (Near Field Communication)

NFC allows devices to establish connections simply by being in close proximity. This technology is particularly useful for quick and secure peer discovery.

Speed: Connections are established almost instantly.
Convenience: Users can connect devices by simply bringing them close together, making it ideal for spontaneous interactions.

BLE (Bluetooth Low Energy)

BLE enables devices to discover and communicate with each other over short distances with minimal power consumption. This makes it a suitable option for maintaining constant peer-to-peer connections.

Energy Efficiency: BLE conserves battery life, making it ideal for mobile devices.
Range: BLE provides a reliable connection over a short range, perfect for personal or localized networking.

Combining Technologies for Enhanced Discovery

By integrating QR codes, NFC, and BLE, we create a robust and versatile peer discovery mechanism. Users can choose the most convenient method for their situation, ensuring that connections are both seamless and secure.

Hybrid Approach: Users can combine different methods for an optimal connection experience. For instance, initial discovery via QR code followed by connection via BLE for sustained communication.
Adaptability: The app adapts to the available technologies on the user’s device, providing the best possible peer discovery experience.

These innovative approaches to peer discovery, coupled with our decentralized microfrontend architecture, form the backbone of our commitment to enhancing decentralization, scalability, and user experience. By continually exploring and implementing cutting-edge technologies, we ensure our app remains at the forefront of decentralized communication solutions

These sections outline the key aspects of our decentralized microfrontend architecture and provide a roadmap for our ongoing and future efforts to enhance decentralization, scalability, and user experience.

Conclusion

Our journey towards building a decentralized chat application has been driven by a desire to push the boundaries of what it means for an app to be truly self-hosted and independent of central servers. By leveraging modern web technologies such as Progressive Web Apps, IndexedDB, WebRTC, and Webpack 5 Module Federation, we have created a robust and scalable architecture that emphasizes privacy, security, and user empowerment.

We have explored innovative hosting solutions like GitHub Pages and plan to implement multi-cloud deployments to enhance resilience and scalability. Our vision for a multi-device architecture, incorporating decentralized profiles and encrypted file storage, aims to provide a seamless, cross-platform user experience. Additionally, our commitment to decentralized peer discovery using QR codes, NFC, and BLE ensures that users can connect easily and securely without relying on centralized authorities.

As we continue to develop and refine our app, we remain dedicated to the principles of decentralization, transparency, and user control. We invite you to join us on this journey, explore our open-source repositories, and contribute to the ongoing evolution of our decentralized chat application. Together, we can redefine what it means to communicate in a decentralized world.

Thank you for taking the time to read about our project. We look forward to your feedback and collaboration as we continue to innovate and improve our decentralized chat application.

Demo: https://chat.positive-intentions.com

I hope this isnt seen as spam. I previously posted the source code of my app which is pretty broad in functionality. In this post i was to discuss about my observations about how the app is managing data. for me it is working unintuitivly well.

Blockchain is a good way to order block of sequential data that can be validated by others. Countless real-world examples show that it scales pretty well.

In my app, I am testing the use of a blockchain for storing "chat app data" selfhosted-only. The app is a work-in-progress proof-of-concept and experimental. It is an investigation into creating a distributed and decentralized app.

Unlike traditional blockchains, the sole purpose of this blockchain is to keep messages between peers in sync. The implementation is have is far from finished, but i have a testable proof-of-concept. The blockchain is entirely in javascript running in a browser.

I have a few observations I would like to make:

• Without the need for mining, it is basically a large array of data. When evaluating the data to be displayed on the UI, it is a "relatively" heavy calculation, but I find that it is more than performant enough to be used in a chat app. I find that the messages and data can scale and the app remains quite performant (I haven't really done much to optimise caching).
• In cases like a group chat, the data can be validated and synced between multiple peers (which may not all be online). (its worth noting: peers may be able to manipulate the database, but it is not a concern for the app where the purpose is only for blockchain as a datastructure)
• Why is this kind of datastructure not used more often? There are other blockchain chat apps, but by putting a messaging system on something like ethereum, it would typically be expensive to users. But in this case, the blockchain is only used for local data storage and validation. I think this is a good use case for blockchain. When working on your device, you don't need to be conservative about things like message size and so we can store files and images in the blockchain.
• With no cryptocurrency, there is no incentive to keep the blockchain alive. The data is easily disposable or persisted as the user prefers. there is no need for a setup process and things like ID's and passphrases can be auto-generated behind the scenes away from the user. (the app is currently very experimental, if your data goes, it goes... but it shouldnt matter because there is no financial value to the blockchain)

I am very interested in the idea of blockchain as a datastructure and I would like to see it used more often. i think this datastructure will play an important role in my app as it will enable the app to move to a single-user-multi-platform architecture.

I would like to hear your thoughts on blockchain as a datastructure. Initially i did it investigating if it works on a basic level to help keep messages in sync, but i find that it is quite performant; especially considering it is only running in a browser. (i expect i can easily improve the performance)

This is my build of the BuckitUp project - the self-hosted Raspberry Pi app that does data storage and communications off-cloud. It is open source and may be usefull for many things from just havig an automated encrypted backup of a valueble files to having separate and decentralised data infrastructure for an entire buisness or community.

This thing is present is exploring the market fit, but anybody having a raspberry pi 4b may try it for free just with a few clicks. It gives you a local server with the wifi hotspot and works entirely offline, but you can also connect them to the same local network - so they sync togather. They are also available for sync over distance through ZeroTire, if you are willing to connect to the internet still.

Share your thoughts - do you find projects like this usefull, and what it may be good for?

I want to share with you a successful solution for secure messaging. My team has been using ADAMANT Messenger for over a year to exchange important information.

ADAMANT leverages blockchain technology to create a decentralized network free from government control, corporations, and even developers. Your communication remains truly anonymous.

ADAMANT's open-source code allows anyone to verify its functionality and security.

Businesses can utilize ADAMANT Business—a private blockchain network built on the underlying ADAMANT protocol—for secure internal communication.

The ADAMANT ecosystem operates on the ADM cryptocurrency.

It has its own wallet and the ability to transfer and exchange coins.

I've read Yggdrasil is a self-healing routing protocol which is intended to be used in decentralized networks. I wounder if a project could be started to pair this with WiFi HaLow? HaLow apparently supports good speed and a range of ~1km. Maybe this could be a good combination to build a solution in the likes of Helium. I've always been interested in an off-grid decentralized messaging solution.

https://yggdrasil-network.github.io/

Budai Radio by Scrambled-Vagabond (itch.io)

This is an internet radio app where every listener is also the DJ. The app streams audio from a shared cloud-based pool of tracks, whose content is dynamically built by users who upload their audio files into the pool. The app accepts .mp3, .WAV and .ogg files.

Please check it out and share it! As the community grows, so to will the variety of audio streaming from the station.