Social media protocols comparison

Comparison of ActivityPub, AT Protocol, and Matrix Protocol (mainly a reminder for myself) “

FeatureActivityPubAT ProtocolMatrix Protocol
Primary Use CaseSocial networking (microblogging, content sharing)Social networking with algorithmic controlReal-time communication (chat, VoIP)
Commands/VerbsCreate, Update, Delete, Follow, Like, Announce (Repost)Create, Update, Delete, Follow, Like, Repost, Algorithmic ChoiceSend, Receive, Join Room, Invite, Leave
Data ModelActivity-Object Model in JSON-LDJSON with user-defined schemasJSON (events model for real-time updates)
Transport ProtocolHTTPS, with JSON-LDHTTPS, with JSONHTTP (REST) and WebSocket, with support for end-to-end encryption (E2EE)
Identity ManagementTied to server domain (e.g., @user@domain.com), uses WebFinger for discoveryPortable DIDs for decentralized identityTied to server domain but portable; user ID format is @user:domain.com
FederationFederated, allowing instances to share content and social connections across domainsFederated with content and algorithm controlFederated, with real-time, synchronized state across servers
InteroperabilityWidely interoperable with other ActivityPub-compliant platforms in the FediverseDesigned for custom app experiences, interoperability is in developmentSupports interoperability with other Matrix clients; bridges to other protocols (e.g., Slack, IRC)
End-to-End EncryptionNot native to protocol but possible with extensionsNot natively specifiedBuilt-in and widely supported, particularly in 1:1 and group chats
ModerationInstance-based moderation policies, customizable filters and blocksUser-level and instance-level moderation, customizable algorithmsRoom-level moderation, with granular permissions for room admins
Popular PlatformsMastodon, PeerTube, Pixelfed, WriteFreelyBluesky Social, upcoming decentralized appsElement (main Matrix client), Synapse (server), bridges for Slack, Discord, Telegram, etc.

Summary of Key Differences

  • ActivityPub is best suited for federated social networking, particularly for applications that prioritize openness and content sharing across platforms in the Fediverse. It uses an Activity-Object model with JSON-LD and supports instance-based identity.
  • AT Protocol focuses on user control over content algorithms and portable identities using DIDs, with a vision for interoperability in custom social applications. It is also designed for federated social networks but with more control over data portability and algorithmic transparency.
  • Matrix Protocol excels in real-time, federated communication, supporting secure, encrypted messaging with granular moderation capabilities. It’s heavily used for chat, VoIP, and collaborative tools, emphasizing interoperability with other platforms through bridges.

Code-First Approach vs. Design-First Approach

Some people aren’t able to think/design a software algorithm without writing the code that implements it; some other can just think/rethink at an algorithm without even writing a single line of code until it is almost totally clear in their mind, and then they code it. These are two extremes of the different approaches to designing/implementing and algorithm. Of course mixed approaches are what happen normally but let’s try to identify the extreme sides in terms of how they work.

Code-First Approach (Bottom-up or Implementation-Driven Approach):

  • This approach involves thinking through the problem by immediately writing code.
  • Developers using this method tend to experiment with implementations as they go.
  • It’s often associated with a more hands-on, trial-and-error style of problem-solving.
  • This approach can be beneficial for smaller problems or when working with familiar concepts.
  • It might be referred to as “thinking with your fingers” or “coding to think.”

Design-First Approach (Top-down or Conceptual Approach):

  • This approach involves thoroughly thinking through and designing the algorithm before writing any code.
  • Developers using this method often use abstract thinking, mental models, or visual aids like flowcharts or pseudocode.
  • It’s associated with a more theoretical or analytical style of problem-solving.
  • This approach is often beneficial for complex problems or when designing large-scale systems.
  • It might be referred to as “whiteboard coding” (even if no actual whiteboard is used) or “algorithmic thinking.”

Both approaches have their merits and can be effective depending on the situation, the complexity of the problem, and the individual developer’s style. Many experienced developers can switch between these approaches as needed.

It’s worth noting that in practice, many developers use a combination of both approaches, iterating between design and implementation as they work through a problem. This hybrid approach allows for both conceptual planning and practical experimentation.

Some related concepts and methodologies that incorporate aspects of these approaches include:

  1. Test-Driven Development (TDD): Writing tests before implementation, which can be seen as a form of design-first thinking.
  2. Rapid Prototyping: Quickly implementing ideas to test them, which aligns more with the code-first approach.
  3. Model-Driven Development: Focusing on creating and analyzing domain models before implementation, which is more aligned with the design-first approach.

Understanding these different approaches can help developers recognize their own problem-solving styles and potentially expand their toolkit by practicing alternative methods when appropriate.

Coding

2024 Reading list (updated as we go)

Neural Networks : Zero to Hero https://karpathy.ai/zero-to-hero.html

state of swe jobs market https://newsletter.pragmaticengineer.com/p/state-of-eng-market-2024

reinforcement learning explained : https://ai.gopubby.com/how-did-alphago-beat-lee-sedol-1a160d76612b

O(1) lfu : http://dhruvbird.com/lfu.pdf

C11 atomics (atomic_int for ex) are still not supported by C++ when including C code : https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2020/p0943r6.html and https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2015/p0063r0.html

Round Robin DNS : https://datatracker.ietf.org/doc/html/rfc8305

Srinivasa Ramanujan : https://www.quantamagazine.org/srinivasa-ramanujan-was-a-genius-math-is-still-catching-up-20241021/

What is legacy code ? According to https://understandlegacycode.com/blog/key-points-of-working-effectively-with-legacy-code/ “Legacy Code is code without tests”

[thoughts] we are transitioning from swe tools to product design tools; llms are blending the boundaries between code, UI/UX, and product ideation.

Are LLMs reasoning ? https://arxiv.org/pdf/2410.05229

Open-Meteo is an open-source weather API https://open-meteo.com/en/docs

Remote work is young and we have not built up methodologies or just even habits or practices : https://intenseminimalism.com/2024/the-myth-of-the-missing-remote-work-culture/

A life spent watching the sky : https://www.majakmikkelsen.com/film

Hard life for rust and linux : a proposal for a rust interface to fs .. https://www.youtube.com/watch?v=WiPp9YEBV0Q&t=67s

Segment anything : a new AI model from Meta AI that can “cut out” any object, in any image, with a single click https://segment-anything.com/

14 years since Go launched : the good and the bad by Rob Pike https://commandcenter.blogspot.com/2024/01/what-we-got-right-what-we-got-wrong.html

Writebook : everything you need to edit and publish your online books

Merchants of complexity : https://world.hey.com/dhh/merchants-of-complexity-4851301b ( on the attraction for complexity read here )

Tired of slack and not owning the data ? https://once.com/campfire#requirements

Something in between a Product Manager and a Software Engineer : Product Engineer i.e. PMs are sometimes not enough technical and SWEs are sometimes not enough product oriented https://refactoring.fm/p/how-to-become-a-product-engineer

myspace reborn https://spacehey.com/

Stephen Wolfram on neural nets : https://writings.stephenwolfram.com/2024/08/whats-really-going-on-in-machine-learning-some-minimal-models/

Some good recommendations https://levelup.gitconnected.com/follow-these-6-patterns-or-i-will-reject-your-pull-request-fc08f908e7fe :

  • Early return and align the happy path left
  • Avoid boolens in methods signature
  • Avoid double negations
  • Use default values to avoid unnecessary else in initializations
  • Avoid functions with side effects


3D Mesh generation with object imageshttps://omages.github.io/

Hetzner de servers auction https://www.hetzner.com/sb

Ransomware victims : https://www.ransomware.live/#/recent

Red and Blue teams in cybersecurity : https://anywhere.epam.com/en/blog/red-team-vs-blue-team

How google is using AI internally https://research.google/blog/ai-in-software-engineering-at-google-progress-and-the-path-ahead/

Protecting artists from gen ai : https://glaze.cs.uchicago.edu/what-is-glaze.html

configuring core dumps in linux/docker https://ddanilov.me/how-to-configure-core-dump-in-docker-container

dolt, a version controlled database mysql compatible https://github.com/dolthub/dolt

MS/DOS 4.01 is open source https://cloudblogs.microsoft.com/opensource/2024/04/25/open-sourcing-ms-dos-4-0/

Contents shortage for AI :

Meta, for instance, trained its new Llama 3 models with about 10 times more data and 100 times more compute than Llama 2. Amid a chip shortage, it used two 24,000 GPU clusters, with each chip running around the price of a luxury car. It employed so much data in its AI work, it considered buying the publishing house Simon & Schuster to find more. 

https://www.bigtechnology.com/p/are-llms-about-to-hit-a-wall

Stop doing cloud if not necessary (I’m saying this since years..) https://grski.pl/self-host

Redis forks (after the licence change) :
– redict : https://redict.io/ Drew DeVault + others?
– valkey : https://valkey.io/ backed by AWS, Google, Oracle, Ericsson, and Snap, with the Linux Foundation; more to come imo.

nginx new fork https://freenginx.org/ (others forks are openresty)

Too much hype about Devin : Debunking Devin: “First AI Software Engineer” Upwork lie exposed! https://www.youtube.com/watch?v=tNmgmwEtoWE

Matt Mullenweg buys Beeper (already owns Texts.com and Element (New Vector)) consolidating his position in Matrix.org based messaging services : https://techcrunch.com/2024/04/09/wordpress-com-owner-automattic-acquires-multi-service-messaging-app-beeper-for-125m/

golang fasthttp (replacement for standard net/http if you need “to handle thousands of small to medium requests per second and needs a consistent low millisecond response time”. “Currently fasthttp is successfully used by VertaMedia in a production serving up to 200K rps from more than 1.5M concurrent keep-alive connections per physical server.” https://github.com/valyala/fasthttp

Back to basics 🙂 Bloom filter https://en.wikipedia.org/wiki/Bloom_filter

1 billion row challenge : https://github.com/gunnarmorling/1brc

golang : alternative to cgo ? https://github.com/ebitengine/purego

Command line benchmark tool : https://github.com/sharkdp/hyperfine

New jpegli, jpeg-xl derived : https://giannirosato.com/blog/post/jpegli/

Edge CDN techniques : Shielding from fastly i.e. use a designated edge cache instead of origin https://docs.fastly.com/en/guides/shielding on a edge cache miss.

Apple car not interesting anymore : https://www.bloomberg.com/news/articles/2024-02-27/apple-cancels-work-on-electric-car-shifts-team-to-generative-ai

golang error handling the Uber way : https://github.com/uber-go/guide/blob/master/style.md#errors

nginx forking : Maxim Dounin annouces https://freenginx.org/en/ on the nginx forum https://forum.nginx.org/read.php?2,299130

Quad 9 free dns 9.9.9.9 : https://www.quad9.net/

UI testing the netflix way : https://netflixtechblog.com/introducing-safetest-a-novel-approach-to-front-end-testing-37f9f88c152d

Check it out : the new super-ide https://zed.dev/

Lex/Yacc today : https://langium.org/

Inside Stripe Engineering Culture, a series a posts : https://newsletter.pragmaticengineer.com/p/stripe

I find truly interesting the point around promoting a write culture (Execs/Directors in tech blog, SWEs on tech blogs/internal technical documents) : https://newsletter.pragmaticengineer.com/i/140970283/writing-culture
I’m a long-time believer that writing clarifies thinking more than talking and writing persists information, makes it searchable, talking does not. “Verba volant, scripta manent” as the Latins use to say. But this idea shifted into “just enough” documentation (which means it is not necessary) in SW engineering latest methodologies so it is interesting that a multi billion company like stripe is going totally against the tide.

Big Internet services backend technology map (always out of date )

  • Airbnb: Ruby on Rails, MySQL, Amazon Web Services (AWS)
  • Uber: Go, Node.js, Java, Python, PostgreSQL, MySQL, Redis, Kafka
  • GitHub: Ruby on Rails, Go, Erlang, MySQL, Redis
  • WhatsApp: Erlang, FreeBSD, Yaws, Ejabberd, Mnesia DB
  • LinkedIn: Java, Kafka, Hadoop, Cassandra, Oracle, Voldemort (LinkedIn’s own distributed database)
  • Netflix: Go, Gluster, Apache Tomcat, Hive, Chukwa, Cassandra, Hadoop, MySQL, Amazon Web Services (AWS)
  • Slack: PHP, Java, MySQL, MongoDB, Kafka
  • Facebook: PHP (HHVM), React, GraphQL, MySQL, Cassandra
  • Twitter: Scala, Java, Ruby, MySQL, Aurora, Redis
  • Spotify: Python, Java, ZeroMQ, PostgreSQL, Cassandra, Google Cloud Platform (GCP)
  • Pinterest: Python (Django), Java, Redis, Cassandra, Kafka
  • Snapchat: Python, Java, PHP, Cassandra, Google Cloud Platform (GCP)
  • Dropbox: Python, Go, Rust, MySQL, Amazon Web Services (AWS)
  • Zoom: C++, Java, MySQL, Kafka, Amazon Web Services (AWS)
  • Shopify: Ruby on Rails, MySQL, Redis, Liquid (templating), Google Cloud Platform (GCP)
  • TikTok (ByteDance): Python, C++, Java, Nginx, MySQL, Kafka
  • Reddit: Python (Django), PostgreSQL, Redis, RabbitMQ, Amazon Web Services (AWS)
  • Instagram: Python (Django), React, PostgreSQL, Redis, Amazon Web Services (AWS)

sunovermonte

The charm of complication

(or the Attraction for Complexity) There is a very common tendency in computer science and it is to complicate solutions. This complication is often referred as incidental/accidental complexity i.e. anything we coders/designers do to make more complex a simple matter. Some times this is called over engineering and stems from the best intentions :

  1. Attraction to Complexity: there’s often a misconception that more complex solutions are inherently better or more sophisticated. This can lead to choosing complicated approaches over simpler, more effective ones.
  2. Technological Enthusiasm: developers might be eager to try out new technologies, patterns, or architectures. While innovation is important, using new tech for its own sake can lead to unnecessary complexity.
  3. Anticipating Future Needs: developers may try to build solutions that are overly flexible to accommodate potential future requirements. This often leads to complex designs that are not needed for the current scope of the project.
  4. Lack of Experience or Misjudgment: less experienced developers might not yet have the insight to choose the simplest effective solution, while even seasoned developers can sometimes overestimate what’s necessary for a project.
  5. Avoiding Refactoring: In an attempt to avoid refactoring in the future, developers might add layers of abstraction or additional features they think might be needed later, resulting in over-engineered solutions.
  6. Miscommunication or Lack of Clear Requirements: without clear requirements or effective communication within a team, developers might make assumptions about what’s needed, leading to solutions that are more complex than necessary.
  7. Premature Optimization: trying to optimize every aspect of a solution from the beginning can lead to complexity. The adage “premature optimization is the root of all evil” highlights the pitfalls of optimizing before it’s clear that performance is an issue.
  8. Unclear Problem Definition: not fully understanding the problem that needs to be solved can result in solutions that are more complicated than needed. A clear problem definition is essential for a simple and effective solution.
  9. Personal Preference or Style: sometimes, the preference for certain coding styles, architectures, or patterns can lead to more complex solutions, even if simpler alternatives would suffice.
  10. Fear of Under-Engineering: there can be a fear of delivering a solution that appears under-engineered or too simplistic, leading to adding unnecessary features or layers of abstraction.