In Simple Terms: How do I Merge During [freebsd-update -r]

[SilverC3ll]

Hello everyone,

The FreeBSD Handbook, chapter 26, concerning my question is presently still using jargon beyond my level of learning; searching the forum, the Internet, YouTube, etc. did not yield an explanation for my rather basic question. I hope I do not disturb the quality of the forums by placing my beginner's question here.

I'm attempting to use the freebsd-update command for the first time. I am upgrading from FreeBSD 13.2 to FreeBSD 14.0.
I do: freebsd-update -r 14.0-RELEASE upgrade
I get:

Attempting to automatically merge chages in files... done.

The following file could not be merged automatically: /etc/Master.passwd
Press Enter to edit this file in vi and resolve the conflicts manually...

As a UNIX-neophyte, I do not know what data the Master.passwd file contains, but it looks like it contains encrypted passwords. This makes me uneasy about sharing a screenshot of the file in question, though I would feel very helped if I could share my actual case. Please let me know if it is safe for me to share a screenshot of this file.

I noticed the freebsd-update binary does not let me escape the updating process without having first merged the data in the file. This poses a problem, because anyone who first encounters this phenomenon simply does not know what to do. Ironically I did manage to escape the installation process by pressing print-screen; I'm using VMWare Workstation, if that matters.

Can it be explained, in simple terms, how one merges data in a file? Kindly forgive that I may not understand some of the jargon you will use; I will try to look up the terms not understood or otherwise kindly allow me to inquire deeper until I have finally attained understanding.

Gratefully yours,​

​

SilverC3ll​

 

[SirDice]

I do not know what data the Master.passwd file contains, but it looks like it contains encrypted passwords.

Indeed it does.

This makes me uneasy about sharing a screenshot of the file in question,

Well, why make a screenshot when you can just copy/paste the information? That's preferred over a picture anyway. You can replace the passwords with a <placeholder> or something similar, because we all understand the reservation of not putting your password out there, even if it's encrypted.

 

[Perceval]

Hi ! Yep, just like SirDice said, you can "clean" the data before posting if any doubt ?
For everything else, maybe this thread can help :

14.0-RELEASE Upgrade question: Merge conflict markers remain

What am I supposed to do on this screen? It's showing my /etc/group file compared to the new version, correct? If I add my change to the new version it still complains. > 14.0-RELEASE daemon:*:1: kmem:*:2: sys:*:3: tty:*:4: operator:*:5:root mail:*:6: bin:*:7: news:*:8: man:*:9: games:*:13...

forums.freebsd.org

Edit/Warning : vi is inside ?

 

[SilverC3ll]

Thank you for the reactions so far and not deeming my question too unworthy; it was quite a journey just to get a copy of master.passwd on my Windows host. I'm using VMWare Player that by all appearances does not support the shared clipboard feature. Trying to mount a shared folder was not successful either; mounting USB drives with a respective filesystem is presently still beyond me. In the end I managed to ssh into the VMWare guest and also used scp to actually copy the file, having a copy of the needed file ready on a user's home directory (now removed). I used this approach because of permission issues.
I have however noticed that this is not the actual file shown to me by freebsd-update, which contained additional data (data for potential merging). I'm not sure if that file is still existent as the update process was interrupted.
Will it help if I do:
freebsd-update -r 14-0-RELEASE upgrade | tee /usr/home/username/update..txt
Will update.txt contain the content of master.passwd, when the update process makes me open the file for editing in vi?

Likely there are more elegant solutions.
Thank you for sticking with me so far; I am indeed enjoying my learning process.

Edit/Warning : vi is inside ?

Every newbie fears vi ;-).

Kindly yours,

SilverC3ll​

 

[SirDice]

I'm using VMWare Player that by all appearances does not support the shared clipboard feature

Use PuTTY or some other Windows SSH client to SSH into the VM. You can do the upgrade "remotely". Then you can simply copy/paste the information when it shows up. Just the master.passwd itself isn't much help, you need to show the exact merge that's shown on screen. We can then tell you which lines to keep and which to remove. Also checkout the thread Perceval posted above. Similar situation, similar solution.

 

[SilverC3ll]

Thank you for being helpful everyone. It took some time to figure out that PuTTy copies text just by highlighting it (ctrl+c of course terminates a current process which caused a bit of stagnation).
I have changed my passwords to dummy ones, so kindly allow me to paste the master.passwd file here. I will try to understand what means what. Corrections are much appreciated.

The following file could not be merged automatically: /etc/master.passwd
Press Enter to edit this file in vi and resolve the conflicts
manually...

<<<<<<< current version
# $FreeBSD$
#
root:$6$4I2Wu3VbYsU0k5cN$yFB0SjMjO/LJY7AskdHiyl9Ncw5k0e8gz9wrp3u9NMNuxwKU8KDD.1wIClr32ArD1YA1tdBo73qsfAjKGj5hg1:0:0::0:0:Charlie &:/root:/bin/csh
=======
root::0:0::0:0:Charlie &:/root:/bin/sh
>>>>>>> 14.0-RELEASE
toor:*:0:0::0:0:Bourne-again Superuser:/root:
daemon:*:1:1::0:0:Owner of many system processes:/root:/usr/sbin/nologin
operator:*:2:5::0:0:System &:/:/usr/sbin/nologin
bin:*:3:7::0:0:Binaries Commands and Source:/:/usr/sbin/nologin
tty:*:4:65533::0:0:Tty Sandbox:/:/usr/sbin/nologin
kmem:*:5:65533::0:0:KMem Sandbox:/:/usr/sbin/nologin
games:*:7:13::0:0:Games pseudo-user:/:/usr/sbin/nologin
news:*:8:8::0:0:News Subsystem:/:/usr/sbin/nologin
man:*:9:9::0:0:Mister Man Pages:/usr/share/man:/usr/sbin/nologin
sshd:*:22:22::0:0:Secure Shell Daemon:/var/empty:/usr/sbin/nologin
smmsp:*:25:25::0:0:Sendmail Submission User:/var/spool/clientmqueue:/usr/sbin/nologin
mailnull:*:26:26::0:0:Sendmail Default User:/var/spool/mqueue:/usr/sbin/nologin
bind:*:53:53::0:0:Bind Sandbox:/:/usr/sbin/nologin
unbound:*:59:59::0:0:Unbound DNS Resolver:/var/unbound:/usr/sbin/nologin
proxy:*:62:62::0:0:Packet Filter pseudo-user:/nonexistent:/usr/sbin/nologin
_pflogd:*:64:64::0:0:pflogd privsep user:/var/empty:/usr/sbin/nologin
_dhcp:*:65:65::0:0:dhcp programs:/var/empty:/usr/sbin/nologin
uucp:*:66:66::0:0:UUCP pseudo-user:/var/spool/uucppublic:/usr/local/libexec/uucp/uucico
pop:*:68:6::0:0:Post Office Owner:/nonexistent:/usr/sbin/nologin
auditdistd:*:78:77::0:0:Auditdistd unprivileged user:/var/empty:/usr/sbin/nologin
www:*:80:80::0:0:World Wide Web Owner:/nonexistent:/usr/sbin/nologin
ntpd:*:123:123::0:0:NTP Daemon:/var/db/ntp:/usr/sbin/nologin
_ypldap:*:160:160::0:0:YP LDAP unprivileged user:/var/empty:/usr/sbin/nologin
hast:*:845:845::0:0:HAST unprivileged user:/var/empty:/usr/sbin/nologin
tests:*:977:977::0:0:Unprivileged user for tests:/nonexistent:/usr/sbin/nologin
nobody:*:65534:65534::0:0:Unprivileged user:/nonexistent:/usr/sbin/nologin
Logan:$6$6NPRU88bARLchVsn$9DxD0aKYXuyTZ2Oqk6gVdGBDdunjLlWXYyMiY/vgRtleAJCayDDQQD4.D1k/mHZD6xUhXDwSc4ZuLN7YSRncV.:1001:0::0:0:User &:/home/Logan:/bin/sh

I understand the data between <<<<<<< current version and ======= is a part of my present configuration.
What comes after >>>>>>> 14.0-RELEASE is information added by the freebsd-update -r session.
If I understand correctly, what is in the section for the current version, is data set apart for my inspection and to be merged by removing the current and new-version indicators (<<<<<, =====, >>>>>>).

Highlighted in bold in the quote, I'm not sure what root::0:0::0:0:Charlie &:/root:/bin/sh is doing as it is placed between the current and 14.0-RELEASE sections of the document.

I'm very curious about what
root:$6$4I2Wu3VbYsU0k5cN$yFB0SjMjO/LJY7AskdHiyl9Ncw5k0e8gz9wrp3u9NMNuxwKU8KDD.1wIClr32ArD1YA1tdBo73qsfAjKGj5hg1:0:0::0:0:Charlie &:/root:/bin/csh and root::0:0::0:0:Charlie &:/root:/bin/sh actually signify and how or why the update process has made changes in them (if at all).

Everything is just speculation here from my part and would love your feedback.
If I'm not mistaken, after merging the data in the file I am to reboot, and then execute: freebsd-update install. I that correct?
Thank you for helping me learn about this fascinating operating system.

I will do my best to study the thread you have referred me to and hope you don't mind the presence of mine, as I of course have a unique use-case.

Yours sincerely,

silverC3ll​

 

[SirDice]

<<<<<<< current version                            <---- REMOVE
# $FreeBSD$                                        <---- REMOVE
#                                                  <---- REMOVE
root:$6$<long hash>:0:0::0:0:Charlie &:/root:/bin/csh   <---- KEEP!
=======                                            <---- REMOVE
root::0:0::0:0:Charlie &:/root:/bin/sh             <---- REMOVE
>>>>>>> 14.0-RELEASE                               <---- REMOVE
toor:*:0:0::0:0:Bourne-again Superuser:/root:
daemon:*:1:1::0:0:Owner of many system processes:/root:/usr/sbin/nologin
operator:*:2:5::0:0:System &:/:/usr/sbin/nologin
bin:*:3:7::0:0:Binaries Commands and Source:/:/usr/sbin/nologin

If I'm not mistaken, after merging the data in the file I am to reboot, and then execute: freebsd-update install. I that correct?

The freebsd-update -r <version> upgrade command doesn't change anything yet. It's just preparation. You install those changes with freebsd-update install. Make sure to follow the instructions! You have to run freebsd-update install a total of THREE times! And pay attention when it says to reinstall your installed packages, do not skip this step or everything will be broken[*] after the third freebsd-update install.

[*] Broken but fixable if you know what you're doing. Best not to skip the step. It'll save you a headache.

 

[SilverC3ll]

<<<<<<< current version                            <---- REMOVE
# $FreeBSD$                                        <---- REMOVE
#                                                  <---- REMOVE
root:$6$<long hash>:0:0::0:0:Charlie &:/root:/bin/csh   <---- KEEP!
=======                                            <---- REMOVE
root::0:0::0:0:Charlie &:/root:/bin/sh             <---- REMOVE
>>>>>>> 14.0-RELEASE                               <---- REMOVE
toor:*:0:0::0:0:Bourne-again Superuser:/root:
daemon:*:1:1::0:0:Owner of many system processes:/root:/usr/sbin/nologin
operator:*:2:5::0:0:System &:/:/usr/sbin/nologin
bin:*:3:7::0:0:Binaries Commands and Source:/:/usr/sbin/nologin

The freebsd-update -r <version> upgrade command doesn't change anything yet. It's just preparation. You install those changes with freebsd-update install. Make sure to follow the instructions! You have to run freebsd-update install a total of THREE times! And pay attention when it says to reinstall your installed packages, do not skip this step or everything will be broken[*] after the third freebsd-update install.

[*] Broken but fixable if you know what you're doing. Best not to skip the step. It'll save you a headache.

Thank you SirDice,

Are you, or anyone else, willing to explain exactly what these lines of data represent:

1. # $FreeBSD$
2. root:$6$<long hash>:0:0::0:0:Charlie &:/root:/bin/csh
3. root::0:0::0:0:Charlie &:/root:/bin/sh

If I understand what they mean, I might begin to understand a principle and apply it to a next upgrade session. I'm curious why they need to be removed, and what the FreeBSD upgrade session has changed there, and why.
'

And pay attention when it says to reinstall your installed packages

Does this simply imply packages installed via pkg install packageName -y? Is there a command that will automatically reinstall every package; is pkg-static > upgrade or pkg-static upgrade -f useful here?

My gratitude once more; maybe in a year or two it will be my turn to assist others.

Sincerely,

SilverC3ll​

 

[SirDice]

1) SVC tag. It used to get translated by CVS and subversion. Useless now because git doesn't update them any more. Just remove it, it serves no purpose anymore.
2) That's your current root account, including the encrypted password.
3) 14.0 changed root's shell from /bin/csh to /bin/sh. That's why it's suggesting to merge it. You can change your root's shell to /bin/sh if you want. Or not, and keep /bin/csh.

Does this simply imply packages installed via pkg install packageName -y? Is there a command that will automatically reinstall every package; is pkg-static > upgrade or pkg-static upgrade -f useful here?

pkg-upgrade(8) will automagically detect you're doing a major version upgrade.

 

[SilverC3ll]

1) SVC tag. It used to get translated by CVS and subversion. Useless now because git doesn't update them any more. Just remove it, it serves no purpose anymore.
2) That's your current root account, including the encrypted password.
3) 14.0 changed root's shell from /bin/csh to /bin/sh. That's why it's suggesting to merge it. You can change your root's shell to /bin/sh if you want. Or not, and keep /bin/csh.


pkg-upgrade(8) will automagically detect you're doing a major version upgrade.

Thank you, that was very helpful.
It looks like you need in-depth understanding indeed before you can properly update your system. I'm sure the release notes would have clarified everything to erudite eyes, though. For instance, I could not have guessed changing root::0:0::0:0:Charlie &:/root:/bin/sh to root:$6$<long hash>:0:0::0:0:Charlie &:/root:/bin/csh would pertain to changing the root shell. I'll read the 14.0 release notes and see if I can correlate that data with the data presented in the merge session.

Kindly yours,

SilverC3ll​

 

[SilverC3ll]

I hope that I have progressed through the update procedure successfully:

Logan@kungfu:~ $ freebsd-version
14.0-RELEASE-p4
Logan@kungfu:~ $ uname -a
FreeBSD kungfu.io 14.0-RELEASE-p3 FreeBSD 14.0-RELEASE-p3 #0: Mon Dec 11 04:56:01 UTC 2023 root@amd64-builder.daemonology.net:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64

Of course, as a beginner I will need to look more than once at the apparent inconsistent data produced via freebsd-version and uname -a.
If I understand correctly, freebsd-version will yield results concerning the userland, referring to all the components of the operating system that aren't part of the kernel, such as system libraries, utilities, and applications.

uname -a however provides information about the FreeBSD kernel. This includes the kernel version, machine hardware name, processor type, hardware platform, and operating system.

I'm still a bit confused about the data I am looking at. When uname -a shows 14.0-RELEASE-p3 FreeBSD 14.0-RELEASE-p3 how does the release version (14.0) actually distinguish itself from the package version (p3)? Does this mean it is the kernel of release 14.0 with a kernel patch of number 3? In that sense I would be able to understand there are two different types of patches: patches for the userland, and patches for the kernel.
Have I understood this correctly?
And, with the data shown, have I performed my upgrade successfully?

Thank you for patience and wisdom.

Sincerely,

SilverC3ll​

 

[richardtoohey2]

Look at freebsd-version r, u and k flags. The kernel patch level only gets changed if there’s a kernel change. So yes, for FreeBSD 14 the kernel is at p3 and userland has had an additional patch so it’s at p4.

 

[SilverC3ll]

Thank you Richard,

May I kindly ask a final question: when one is presented with a file merge, are the release notes the sole source of information one should go to, to cross-check the data in the release notes and the data presented in your merge-files, so that one can identify what is what?

Thank you for this educational exchange.

Sincerely,

SilverC3ll​

 

[Cath O'Deray]

… are the release notes the sole source of information one should go to, to cross-check the data in the release notes and the data presented in your merge-files, so that one can identify what is what? …

No, I should not expect release notes to predict merge conflicts.

(Some conflicts are fairly predictable, however these predictions are not release note material.)

Looking ahead​

pkgbase is the future.

PkgBase - FreeBSD Wiki

freebsd-update is an axe candidate.

FreeBSD 15.0 planning

Retrospective: https://forums.freebsd.org/threads/freebsd-14-0-planning.82875/ Looking ahead: {link removed ? link provider grahamperrin is dead}

forums.freebsd.org

 

[SilverC3ll]

No, I should not expect release notes to predict merge conflicts.

(Some conflicts are fairly predictable, however these predictions are not release note material.)

Looking ahead​

pkgbase is the future.

PkgBase - FreeBSD Wiki

freebsd-update is an axe candidate.

FreeBSD 15.0 planning

Retrospective: https://forums.freebsd.org/threads/freebsd-14-0-planning.82875/ Looking ahead: {link removed ? link provider grahamperrin is dead}

forums.freebsd.org

Thank you so much grahamperrin; it looks like PkgBase might simplify the process of upgrading the system a bit.
Do you have advice on how I can properly understand the data presented to me in a merge-file?

Kindly,

SilverC3ll​

 

[Cath O'Deray]

There's a wealth of frighteningly complex documentation.

Git is not relevant, but I'll quote the one short paragraph below from <https://git-scm.com/docs/git-merge#_how_conflicts_are_presented>:

The area where a pair of conflicting changes happened is marked with markers <<<<<<<, =======, and >>>>>>>. The part before the ======= is typically your side, and the part afterwards is typically their side.

diff3 is mentioned at the Git page above.

You need not read the related manual page diff3(1), but essentially:

compare three files line by line

 

[SilverC3ll]

There's a wealth of frighteningly complex documentation.

Git is not relevant, but I'll quote the one short paragraph below from <https://git-scm.com/docs/git-merge#_how_conflicts_are_presented>:



diff3 is mentioned at the Git page above.

You need not read the related manual page diff3(1), but essentially:

It looks like you need quite a bit of understanding of FreeBSD architecture ere you can understand the lines of data presented to you in a file merge; manuals, as you mentioned, are not necessarily all that accessible. Especially to the newbie. I have really come to see FreeBSD is ultimately for the person wanting to attain professional status in understanding and operating a UNIX operating system. This is luckily my ambition, but I find there aren't really any coherent or truly sequential stepping stones. My humble feedback as a beginner is that those who create the documentation might have difficulty with empathically placing themselves in the shoes of the beginner, assuming some jargon as naturally understood because the author is already so much at home in certain nomenclature and concepts. Allow me to give this example from the FreeBSD Handbook:

All compositors using Wayland will need a runtime directory defined in the environment, which can be achieved with the following command in the bourne shell:

% export XDG_RUNTIME_DIR=/var/run/user/`id -u`

First of all, the term compositor was never explained. The term runtime directory is also not explained, nor what a bourne shell is or how to use it. The beginner will also not notice that the percentage sign at the beginning of the command line implies a user should be active as root, and this (I think) was also not specifically explained.
I always feel that if an operating system is to reach a greater amount of people, the introductory documentation should be, what I call, self-sustainable or a stand-alone read (as much as possible). What I mean with this, is that the documentation assumes as little computing knowledge as possible, and introduces sheer computation from the perspective of the respective operating system; in this way no epistemic gaps are left in the explanations given and any user can start using a computer system with your operating system with greater ease. I have recently befriended someone from the mailing lists (the FreeBSD community is remarkably kind!), and expressed my desire (and even my own ambition) to see a book titled: A FreeBSD Introduction to Computing. For instance, on reading Michael W. Lucas's Absolute FreeBSD, the respectable author demonstrates he creates several partitions for the installation: / (root), /tmp, /var, and /usr.
I'm not someone with a complaintive attitude so please do not misunderstand me, but as a beginner you don't understand why this approach is at all used. Why create these partitions at all? What is their function? Why is it ordered like this? I recognise a correlation with the directories in the FreeBSD operating system; but does that then mean these directories now exist as partitions; what for? Why are storage devices designated as ada[n] and not just "hdd" or "ssd" "usb" etc? A beginner asks all these questions that he needs answered before an introductory communication can succeed in its purpose, not leaving epistemic gaps. Of course, one should not be lazy and be willing to do a lot of research, but at the same time I think I do feel I want to humbly and sincerely offer this feedback. If an introductory documentation cannot be used as a stand-alone reference, and is not a "self-sustainable" source of information, it would be good if, at the very beginning, it provides a sequential list of sources of prerequisite information to be understood prior to reading the present documentation. For example, if I'd want to write a book about calculus, it would be my duty to inform my reader s/he first needs a mastery of basic arithmetic, algebra, geometry, trigonometry, pre-calculus, functions and graphs. To communicate successfully, I would refer to specific books or other sources of information about these prerequisite topics that are most compatible and conductive to the information presented in the present book/documentation on calculus. So it must be with operating systems.

Circling back to the subject of file-merge, in my mind it is important that the operating system provides data as to what the relevant lines actually are, what changes are implied, and why. My inquiries so far have not shown me that this topic is effectively or easily answered, especially as there doesn't seem to be a central platform whose data one can correlate with the data present in a file-merge, which I first thought would be the release notes. Of course, I can imagine changes are made to the system via an update, and configuration files might need to be edited of binaries that have nothing to do with the base operating system and hence FreeBSD as a platform cannot account for them in terms of providing information. How would one deal with this?
Of course, I am enjoying the learning process and am curious where I will stand in a year or two from now .

Kindly yours,

SilverC3ll​

 

[Cath O'Deray]

SilverC3ll thank you. Truly, very valuable observations.

… this (I think) was also not specifically explained. …

There's some explanation in the preface, under Examples.

The first example uses the word floppies and targets the A: drive.

---

For additional light amusement (but still, not to detract from the need to improve documentation), here's another phrase from the preface:

Keys are shown in bold

No, they're not as demonstrated by the keys that appear below.