"Run Your Own Mail Server" by M.W.Lucas

Tranditionally any computer was able to send and receive email, without much effort. And with the advent of the Internet, computers became connected and email started to work globally.
"Without much effort" is sort of a joke. Until the early 90s, any computer capable of using e-mail was by definition a large machine, administered by professionals. There were no "personal" computers with network connections, with very rare exceptions. There were BBNs that people dialed into, but that wasn't really e-mail, more of a chat or forum site. E-mail was heavily used since the early 70s: until the mid 90s, IBM's internal VNET had more e-mail than the Internet. I started using e-mail in 1982. To hruodr's comment that 1-day delays were normal: No, even in the 80s delays of 10 minutes or at most an hour were expected for sites that were permanently connected; only uucp sites and hobbyists on BBNs had longer delays.

Even in those days, there was a clear distinction between a computer that transmitted and stored e-mail (typically a mainframe or minicomputer), and the device that the end user needed to view or write e-mails (until the 90s that was a terminal, like a VT100 or 3277; later it was a personal computer like a TRS-80 or IBM PC). The important part was that the mail remained on the server. In the 80s, I either drove to the office or lab to use e-mail, and later I had a Hazeltine or VT100 at home, with a modem.

That is when shops like Yahoo etc. jumped in and offered a delegated email service, based on web pages.
Beginning in the early 90s, the Internet (and by that I mean the TCP-IP based heir to the ARPAnet) reached consumers, who started browsing. Since using telnet to dial into a shell machine to read the e-mail there was considered too difficult, lots of web-based e-mail services sprang up; I think Hotmail and AOL may have been even earlier than Yahoo.

And then, in the 90s, individuals suddenly had full-fledged computers at home, with network connections. A whole bunch of hobbyists (including me) decided it would be a good idea to have their own e-mail hosts, and run MTAs, and store e-mail. In the early days of Internet protocols, that was easy enough, but it was immediately open to all manners of abuse.

It is important to understand that web pages (HTTP/HTML) and email (SMTP) have nothing at all to do with each other, ...
No, it's much more complicated than that. To begin with, HTML is an information rendering language. I can transport an e-mail (by any protocol capable of transporting mail, be it SMTP, RSCS, or IMAP) independent of what information the e-mail contains. It can be clear text, it can be HTML encoded, it can be a binary file like a JPG. So yes, HTML and mail are natural partners.

The other side is that the human end user can use a variety of protocols to read and write e-mail. They can use SMTP if they are running their own MTA. They can keep the the e-mail on a server host, and use IMAP and POP to look at it. They can access it via text-based encodings (for example, when mutt or elm use ANSI rendering commands to put a message on their screen). And if the mail server happens to present the mail as web pages, then accessing those via HTTP transporting HTML is also a natural fit.

The important thing here is to always remember the distinction between MTA, MUA, and whatever mechanism the MUA uses to communicate with the human.

But bringing both together is the cause for about 100% of all scams, malware attacks, ransomware and other cyber criminal actions. It is in fact the absolute worst one can do.
Nonsense; e-mail based malware existed before HTTP and HTML were even a thing. Remember the Christmas Tree Exec, or Robert Morris' Internet Worm? E-mail is a good mechanism for transporting information, whether the information is good or bad. The flow of bad information is not caused by web-based protocols.

Furthermore, the original demand or usecase (people wanting to switch off their computers at night) is not even true anymore. Nowadays we are expected to own a smartphone that runs day and night, and to be reachable 24/7. There is no longer a technical problem in receiving your own mail at any time.
Smart phones do not even store mail (except for caching), they are just the visible surface of the MUA. Try the following experiment (done it a few times): Throw your smart phone into the swimming pool. Go to the store and get a new one (Apple or Android). Put the SIM card into the new phone after drying it, power up, and within minutes all your mail is back again.

But, the web based mail shops like Yahoo etc., being not only pernicious, but also superfluous now, nevertheless want to stay in place and continue to get fat on income from feeding their users with unwanted advertisements.
As explained above, we still need mail hosts (those machines that store mail), and MTAs. Very few people run their own. As we have discussed in this thread, it is possible to run your own, but it has become quite tedious. Whether it is worth it is a judgement call; IMHO, it is not worth it.

In the golden times of email, around 2000, when you met somebody at a conference and wanted to exchange further ideas in private, you would just exchange email addresses. It was easy to communicate, back then. Nowadays you don't get somebody's email address anymore, because it is considered dangerous to give it away (what good is email then, anymore?)
Nonsense. Several times this week, in interactions with various companies, the following happened: I call their 800 number, or bring up the chat bot on their web page. I ask my question, and they connect me to the correct person. I explain my problem to the person, they tell me their e-mail address and ask me to give a detailed description via e-mail, or send pictures of the broken part via e-mail. Then we have e-mail based discussions.

I'll ignore the rest of your paranoid anti-business rant. Angry much?
 
"Without much effort" is sort of a joke. Until the early 90s, any computer capable of using e-mail was by definition a large machine, administered by professionals.
When I say computer in these forums, I usually mean computer running Berkeley unix.

E-mail was heavily used since the early 70s: until the mid 90s, IBM's internal VNET had more e-mail than the Internet.
Yeah, and sendmail was used to convert between these.

I started using e-mail in 1982. To hruodr's comment that 1-day delays were normal: No, even in the 80s delays of 10 minutes or at most an hour were expected for sites that were permanently connected; only uucp sites and hobbyists on BBNs had longer delays.
And sendmail.cf could also be made to convert from/to these.
Been there, done it.

No, it's much more complicated than that. To begin with, HTML is an information rendering language. I can transport an e-mail (by any protocol capable of transporting mail, be it SMTP, RSCS, or IMAP) independent of what information the e-mail contains.
That is what I am saying. It has nothing to do with each other.

It can be clear text, it can be HTML encoded, it can be a binary file like a JPG. So yes, HTML and mail are natural partners.
Natural partners to abuse, yes.

Nonsense; e-mail based malware existed before HTTP and HTML were even a thing. Remember the Christmas Tree Exec, or Robert Morris' Internet Worm? E-mail is a good mechanism for transporting information, whether the information is good or bad. The flow of bad information is not caused by web-based protocols.
The show me how you click-through onto a rogue URL in a text-based mail reader.

Smart phones do not even store mail (except for caching), they are just the visible surface of the MUA.
Sure they do not. They do not do anything useful, because they are intended only as slave-collars for the disempowered masses.

But they have enough compute power and availability so they could, if a proper protocol were devised.

Try the following experiment (done it a few times): Throw your smart phone into the swimming pool. Go to the store and get a new one (Apple or Android). Put the SIM card into the new phone after drying it, power up, and within minutes all your mail is back again.
Irrelevant. You missed the point.

As explained above, we still need mail hosts (those machines that store mail), and MTAs. Very few people run their own. As we have discussed in this thread, it is possible to run your own, but it has become quite tedious. Whether it is worth it is a judgement call; IMHO, it is not worth it.
The question is about cause and effect: who is making things tedious, and what agenda is driving the actors?
If we don't ask such questions and only take things as practical constraints, we could as well argue for slavery as a natural state of being. But I thought we (as mankind) were over this.

Nonsense. Several times this week, in interactions with various companies, the following happened: I call their 800 number, or bring up the chat bot on their web page. I ask my question, and they connect me to the correct person.
I did not ever get that far anymore, for many years already.
 
Google making it sound like SPF and DKIM mandatory but so far only PTR seems to be; that’s why I was interested in your set-up.

I do have SPF, and I also check for hard SPF failures. Other than that it's a bare mail daemon.

I don't really consider IPv4 PTR record as a feature, because I've always had that whenever being given an address. With IPv6 I wasn't so lucky because one of my VPS providers doesn't support that, though this should be generally supported.

Fun fact: back in the day many sysadmins considered A/PTR mismatch to be so bad, that there used to be dedicated software that would reject connections outright in such cases.
 
Not having at least 3 static open IP(s) with Pass-Through routing from an ISP that offers IPV6 support, not knowing how BGP and DNS work, plus not being able to run own Secondary DNS would be the main reasons for NOT TO run any TCP/IP services open to the outside world of Internet's mayham :( Providing own Internet Presence Services today, as a one man show, is like trying to maintain and fix own Tesla EV or any other modern vechicle - things will break enventaully which may lead to a bad crash :)
 
Your own internet presence starts with an AS, your own uplinks and core infra like routing, firewalling and DNS. You can easily outsource much of that to a VPS hoster. It's quite doable to run a hosting business from VPS'es alone. Knowing how things work is obviously helpful. Mail is finicky so you need to know what you're doing, even with parts outsourced.
 
Not having at least 3 static open IP(s) with Pass-Through routing from an ISP that offers IPV6 support, not knowing how BGP and DNS work, plus not being able to run own Secondary DNS would be the main reasons for NOT TO run any TCP/IP services open to the outside world of Internet's mayham :( Providing own Internet Presence Services today, as a one man show, is like trying to maintain and fix own Tesla EV or any other modern vechicle - things will break enventaully which may lead to a bad crash :)

That would mean to give up the Internet to the greedy ones - to those who came after 2000 with the sole abition to have their future already made for them, to exploit everything that we had built before, and to make big profit while investing negligible effort and monopolizing everything to them.
As a longterm unemployed you may well know how whatever BGP and AS and routing and all such gimmicks do technically work, but with zero money you cannot get such.
 
That would mean to give up the Internet to the greedy ones - to those who came after 2000 with the sole abition to have their future already made for them, to exploit everything that we had built before, and to make big profit while investing negligible effort and monopolizing everything to them.
As a longterm unemployed you may well know how whatever BGP and AS and routing and all such gimmicks do technically work, but with zero money you cannot get such.
Yes Sir,
The days of Internet for all and to use it as you wish, without being monitored, guided and told what you can or cannot do, for a fee or for free, are gone ! Big Corps are running Internet now. And if you know better than them how to do it, they'll either buy you out or just destroy you if you refuse to cooperate.
 
Yes Sir,
The days of Internet for all and to use it as you wish, without being monitored, guided and told what you can or cannot do, for a fee or for free, are gone ! Big Corps are running Internet now. And if you know better than them how to do it, they'll either buy you out or just destroy you if you refuse to cooperate.
Aye, rhat's drastically put, but yes, that is clearly the direction of development I perceive.

My question is: why do we just accept it that way? Without even complaining, without proper analysis and critique? The western civilisation is built upon successful struggles against those in power, i.e. the french revolution defeating feudalism and creating the foundations of democracy (egalite, liberte, fraternite), or Karl Marx resolving the issues of industrialization (labour rights). And now we seem to abandon both of these, in favour of governemnt-megacorp alliances enslaving the people.

The majority of people can bring the excuse tha they just do not know how all this works. Even the governemts could bring that excuse (then they would have to be asked why they enforce a development which they not even understand - but that wouldn't be tjhe first time). But we here cannot.

It is not so terribly difficult to operate an internet facing server. It is not easy either, not everybody can do it - but probably one in a hundred people could do it, and that is well sufficient for independent structures.
 
Yes Sir,
The days of Internet for all and to use it as you wish, without being monitored, guided and told what you can or cannot do, for a fee or for free, are gone ! Big Corps are running Internet now. And if you know better than them how to do it, they'll either buy you out or just destroy you if you refuse to cooperate.
That's mostly true for the IPv4 world. IPv6 liberates a huge part of this for us. I don't really *NEED* an AS anymore, just an ISP that has a sane process for RDNS and isn't afraid to actually allocate a /48 to my uplink. Nowadays I can get this from one consumer ISP where I live. I'm pushing hard at $work to implement IPv6 where it fits.
 
Now if only my ISP would join the future present and offer IPv6 to personal customers...
Same here, they already say for 3 years, it is "High on the priority list". Nobody believes that anymore, though they do offer 8 gbit up/down for normal price, so that is nice.
 
Same here, they already say for 3 years, it is "High on the priority list". Nobody believes that anymore, though they do offer 8 gbit up/down for normal price, so that is nice.
What's a normal price of an 8Gb/s connection? And what would you use that data transfer rate for?
 
What's a normal price of an 8Gb/s connection? And what would you use that data transfer rate for?
60 euro per month ish, some discounts if you have mobile sim from them too etc. I use 1 gbit for now, but other two options are 2/2gbit and 8/8gbit. I do transfer large amounts of data for work, pondering about getting 2 gbit (most infra at home is 2.5 gbit) but I don't feel like upgrading all switches and buying 10 gbit NICs yet.
 
Now if only my ISP would join the future present and offer IPv6 to personal customers...
Some ISP(s), at least in US, use IPV6 extensively for their internal network operations with WAN TCP/IPV6 routing and services such as DNS. You can do your own LAN over IPV6. There are many semi-commercial LAN/WAN routers not very expensive and free pfSense routing software.
But, it's up to your ISP or IP Gateway provider to offer IPV4 or IPV6 routing to and from your WAN facing interface.
 
So if you have the choice and don't want to waste your precious time with idiots - don't run a mailserver. Not because it's *technically* very hard, complicated or tedious - the RFCs are pretty clear and simple - but because you have to deal with idiots that aren't able/willing to follow even the most simple rules...
I beg to differ. Wietse Venema, the well known author of Postfix, held several talks in the 2010s about Postfix and its development. In one of this talks he made a joke about SMTP being an easy protocol on the surface with about 10 commands or so, but when doing the deep dive you are in for an amazing brain fuck and stuff gets really complicated.

Anyway regarding mail servers - I am running them since the mid 90s. I started with Smail and Taylor UUCP back then, later migrated to Exim because it was similar to Smail (I never liked how Qmail managed its configuration, too many ways to get lost quickly IMHO, I am a big fan of having one big master configuration file), but had a big development momentum and Cyrus IMAP, later switched over to Courier IMAP. Then I ditched Exim for Postfix years ago, because of security and replaced Courier with Dovecot and never looked back. Postfix and Dovecot is still the standard stack I do use everywhere.

So by my own experience setting up an own mail server and running it still is easy. What makes it way more complicated compared to the past is if you want to get it into a usable and stable state due to all the annoyances which pester the modern world.

Keeping this up and staying on top of the game got more complicated over the years, and probably still will continue to do so.

What I do mean with it is this:

1. you've got to secure your MTA in a way it will not become a spam relay or will be abused by 3rd parties, which still is the easiest task
2. since SPAM is everywhere, you want to have a good enough spam filtering stack on your system in order to keep your inbox clean. This is not exactly science, more then experience what works and what does not. Back in the past I used Spamassassin, but made the switch to Rspamd years ago, which works great and is much quicker than Spamassassin.
3. if you need that your emails are being processed by the big ones like Google Mail, Outlook and so on you have to implement their mail requirements, which nowadays is mostly setting up DKIM, SPF and DMARC. If you don't do this, then they will refuse to accept your emails. This is especially fun given the fact Google was one of the biggest spam senders in the past.

For spam filtering you've got always to keep on top, since the spam industry keeps innovating their stuff. This is a never ending battle. The same goes for ensuring that Google and so on will accept your emails, because now and then they introduce a new requirement you've got to implement then you've never probably heared about before.

Additionally you might want to monitor your MTA's IP actively for appearances on big RBLs regularly.
 
My journey started with Fetchmail over dialup to UW-IMAP. Moved to Sendmail when I got broadband. At some point I ditched Sendmail for Postfix and UW-IMAP for Dovecot. It's a really nice system. Very little maintenance, and Just Works most of the time.
 
That's mostly true for the IPv4 world. IPv6 liberates a huge part of this for us. I don't really *NEED* an AS anymore, just an ISP that has a sane process for RDNS and isn't afraid to actually allocate a /48 to my uplink. Nowadays I can get this from one consumer ISP where I live. I'm pushing hard at $work to implement IPv6 where it fits.
Does that help against censorship, suppression, exploit and harassment? If so, how?

See, if I switch to IPv6, I cannot use google at all, because every five minutes they force me to solve a very difficult captcha. Since my IP address is constant, they could learn at the first time that I am a human - which shows that they do it not to find that out, but for the sole purpose of harassment.

My ISP allows me only to receive mail from commercial spam distributors (like google, outlook, etc.), they censor mails from private people. So if somebody wants to mail me, they first have to become customers of these spam distributors where they are forced to watch lots of advertisements.
Can IPv6 change that strategy?
 
See, if I switch to IPv6, I cannot use google at all, because every five minutes they force me to solve a very difficult captcha. Since my IP address is constant, they could learn at the first time that I am a human - which shows that they do it not to find that out, but for the sole purpose of harassment.

You could be a nasty proxy
 
Back
Top