[GoLUG] Mailing list, long term.

[Steve Litt]

[comment]: # (This email is in Markdown)

# TL;DR 

Within 2 years I'm going to make a new, better mailing list
technology that does what mailing lists should have done all along. For
the rest of this document I'll call the new non-email mailing list
software "MindMeld", the working title, and the general concept of
remote group collaboration as "mind meld".


# THIS DOCUMENT'S METADATA

- Authored by Steve Litt, slitt at troubleshooters.com

- First version of the document: 2025/08/11 (yyyy/mm/dd)

- Format: I intended this document to be valid Markdown, but might have
  made mistakes.


# PURPOSE OF MAILING LISTS IN OPEN SOURCE COMMUNICATIONS

In open source projects, clubs and groups, mailing lists serve a very
different purpose than general purpose email or business
communications. In Open Source communications, a mailing list's sole
purpose is to foster cooperative thinking, among many people, to grab
opportunities and iron out problems. This works very well, because the
intelligence of the group far outshines the sum of the intelligences of
its individuals. The preceding sentence completely sums up why I'll be
creating this new technology.

In Open Source communications, the purpose is *not* any of the
following:

- CYA.

- Full conversation archival.

- Compatibility with general purpose email.

- Compatibility with RFCs. RFCs are not infallible.

- Privacy


# NO PRIVACY? REALLY?

Yes really. Worrying about privacy on a mailing list is silly, because
most mailing lists have archives anyway. And once the "mailing list" no
longer depends on email, you don't need to worry about your email
address being seen. As far as your name on the mailing list (jdlspeedy,
basketcase, etc), what can they do with it? They can't push any spam to
you without pushing it to the MindMeld server, and they can't do that
without a password, and they can be blacklisted at the server level.


# ALTERNATIVES

When the slightest deficiency in mailing lists is encountered,
everybody and their dog has a lame suggestion...

## FORUMS

Yeah sure, I want to log into each forum every day and go searching for
new stuff. If a mind meld is like a nuclear reactor, forums are like
lowering all the damping rods all the way into the reactor. "Oh, but
there are all sorts of ways to get it via email!" Nuf said!

## GOOGLE GROUPS

You have to play by their rules, which are constructed for their profit,
not for your benefit. Rules that change constantly. And if you need to
move the group elsewhere, your archives are toast.


## FACEBOOK GROUPS

Oh yeah, that'll work out well. Facebook's a swamphole of purposely
churned arguments. And a lot of people haven't signed up for Facebook
because they're smart enough to have avoided that mess.

## LINKEDIN GROUPS

Once again, their house, their rules, lost archives when you move. Plus
the fact that LinkedIN is rapidly devolving into a combination of right
and left wing propaganda, plus some of the most syrupy politically
correct pablum ever seen. And once again, move and your archives are
toast.

## SLACK

Oh yeah sure, make people learn a massive, do everything technology
just to achieve a mind meld. Yeah, that'll attract a lot of users, LOL.

## DISCOURSE AND DISCORD

Now which is which and does what? Their house, their rules, toasted
archives.

## ALTERNATIVES: A SUMMARY

If you wonder why, after all these decades, email based mind melds
still not only exist but thrive, it's because the alternatives are so
ineffective in creating mind melds. Also, most are owned by
corporations, and if it's free, then you're the product. At least
anybody who is very proficient can set up their own mailman server.


# WHY NOT STICK WITH EMAIL?

Mind melding software based on email should have never happened. Mind
melding has very little in common with email. However, back in the 20th
century, you couldn't always rely on a tcp/ip connection between both
ends, and sometimes had to store and forward through intermediate
servers, so mind melding software of necessity had to be bolted to UUCP
or email to achieve that store and forward. But by 2010, reliable end
to end realtime connections were possible, so the need to affiliate
mind melding software became unnecessary.

Meanwhile, spam became such a problem, and in response
yahoo/hotmail/google created difficult to admin stuff like SPF, DMARC,
DCOM, and others. These measures play hell with mailing lists, making
mailing lists more and more impossible every year that goes by.

Achieving a mind meld has nothing to do with email. You don't need
privacy (which email based mailing lists don't give you anyway). You
don't need to jump through all of email's hoops. You don't need every
hop to tack on yet another header. You don't need to become a slave to
email RFCs, nor do you have to endure criticism when you break email
RFCs, because except for that incredibly stupid RFC stating the default
should be respond to user (damping rods in the mind meld reactor),
email RFCs are orthogonal to achieving a mind meld.

In short, if you remove all the email baggage, client-server software
implementing a mind meld becomes incredibly simple.


# MINDMELD DESIGN CONSIDERATIONS

MindMeld will achieve mind meld without email and all its baggage,
including email RFCs. MindMeld will be completely orthogonal to email.
It will be a simple client/server architecture with very few
dependencies, easily installed by a person with normal intelligence.
Installation without a distro package will be possible and in fact
encouraged. There will be no "bug tracker", because that's where bugs
go to die. Instead, bugs will be discussed on MindMeld itself, with IRC
as a backup should MindMeld go down.

## LICENSE

This will be an ultra simple client/server installable by anyone with
half a brain. I do *not* want corporation X making some sort of walled
garden community for it like Github did with git (and then Github
changed the rules). Although I normally favor the Expat license or
GPLV2, in this situation I want to prevent abuse by private equity
investors and venture capitalists, so I'll go either GPLV3 or maybe go
all the way and make it GNU Afero.


## SERVER

The top priority of the server is simplicity and very few dependencies.
Because I'm not an especially good programmer, I anticipate writing it
in Go, because from the few hours I've studied Go, it's a reasonably
easy language and it has what you need to make a fairly effective,
safe, safely threaded server. Rust is a little too difficult for this
task, although it might get written in Rust, for ultimate safety, later.

Version 0.1 will be missing quite a few features, so that I can
actually create the thing. Later versions can incorporate necessary
features, as they begin to appear necessary, based on usage.

The server will handle three or four data pieces:

- The data store, a tree of all messages for this server.

- A server config file in MS .ini format. That format sucks, but for a
  simple application, it's the best of many bad config alternatives. It
  will include a message size max and various file locations.

- A whitelist, consisting of usernames and *non-valuable* passwords.

- A blacklist, consisting of spammers and scoundrels.

I plan the datastore's directory structure to look something like this:

`servername/subgroup/yyyy/mm/dd/sendername`

Each message consists of one file. The file naming convention will be
something like the following:

`sendername_servername_subgroup_yyyy_mm_dd_hh_mm_ss_mm`

The first mm is month, the second is minute, the third is millisecond.
The time is time of arrival at the server, no time zones or UTC
accounted for. Twice a year it screws up. I'll accept that for version
0.1, just so the thing gets built at all.

The structure of a message file will look something like the following:

```
Date: yyyy/mm/dd at hh:mm:ss
From: Person who sent the email
Message_ID: servername_subgroup_yyyy_mm_dd_hh_mm_ss_mm_sendername
In_Reply_to: Empty or the message id being replied to
Subject: Whatever
List_Id: servername_subgroup

This is the body of the email. It will have an upper limit on number of
bytes. Also, including attachments is a must, but I haven't figured out
how to do that yet.

This is a second paragraph of the body.

```

All the server really does in version 1 is:

- Store all messages from everybody

- Add comments when sent by a member

- Membership is defined by a unique name (jdlspeedy, basketcase) 
  plus a non-valuable password. That's right, security
  by obscurity. TLS can be added in later versions.

- Take care of quoting marks (`>`, `>>`, `>>>`, etc.). This is a 
  job for the server because mark my words, people will write clients
  that do quoting stupidly, and even users will find stupid ways to
  quote confusingly.

- Encourage interleave posting and discourage top posting. Top posting 
  in a mind meld is like putting contaminents in your nuclear reactor:
  It causes brain-busting misunderstandings.

One of the things the server *doesn't* do is word wrap or maximum words
per line. The definition of a paragraph will be a blank line
delineation, same as Plain TeX or Markdown, and the server will store
an entire paragraph as a single line, except in the case of code (I'll
need to figure out how to impleent that). This "maximum words per line
that email people just love to yell at each other about made sense when
using vi as your email composer, but makes no sense today. It's the
client's job to word-wrap.


## CLIENT

For version 0.1, ease and simplicity are the goal. Later versions will
make it more put together, enabling things like custom folders, sorts
and selections, and UI conveniences. Because I actually need to get
this done, I'll do at least version 0.1 in Python, because it's easy.

### VERSION 0.1

Responsibilities are:

- Maintain a copy of the server/subgroup messages on the client hard
  disk, with the same directory structure and naming conventions as on
  the server.

- Keep a list of all the messages you've sent, whether accepted or
  rejected, on the local hard disk.

- Format and word wrap downloaded messages.

- Retrieve without logging in. No privacy difference between this and
  public viewable archives.

- Enable you to log in, with your name (basketcase, jdlspeedy,
  whatever) plus a non-valuable password. This combo stays in RAM as
  long as the client is running.

- Reply to email and send. Requires login.

- Compose and send. Requires login.

- Upon request, sort by thread, sender, date, subject. In version 0.1,
  these views might have no user interface except viewing.

- Download messages using the rsync executable, so the rsync executable
  is a dependency.


### LATER VERSIONS

- Be multi-server, multi-subgroup, for the convenience of getting
  everything from a simplified immitation of .fetchmailrc. This can be
  done by a very simplistic subset of fetchmail's functionality. I'll
  write it. It won't be hard.

- Implement a very simplified procmail immitation so that each user can
  make his/her own folders and subfolders. This might end up being
  include/exclude files for rsync.

- Make the various views have a convenient user interface. Perhaps make
  it multi-threaded so that people don't get stuck when an unseen
  window is blocking.

- Much Later: Implement TLS as an option for those who want it. Lack of
  TLS won't stop interested individuals from downloading the data with
  rsync, but will prevent password sniffing when posting.


# BENEFITS

- Easily Installable and maintainable by anyone with half a brain.

- Installable and maintainable with or without distro packaging.

- No baggage from 20th century email or RFC rules.

- Knowledge sharing: Nothing more, nothing less.

- Backup is as easy as rsync, and anybody can back it up.

- What email mailing lists were supposed to be before they failed
  miserably.


# THANK YOU FOR LISTENING

Sincerely, 

Steve Litt, slitt at troubleshooters.Com