Solved How do I use the command “cut” to select the 2nd field in a string variable?

F

fixit9660

I'm using FreeBSD version: 11.3-RELEASE-p14, Shell version: tcsh 6.20.00 (Astron) 2016-11-24 (x86_64-amd-FreeBSD) options wide, nls, dl,al, kan, sm, rh, color, filec and trying to use cut to harvest fields in a string variable, with no success.
Code: 
#! /bin/sh
.
# Variables
filelength=0
start=0
reachability=""
.
filelength=$(sed -n '$=' /$workdir/ntpstatus.txt)

start=3
tr -s ' ' < /$workdir/ntpstatus.txt > /$workdir/ntpstatus2.txt
while [ "$start" -le "$filelength" ]
        do
                peer_line=echo sed -n $start,1p /$workdir/ntpstatus2.txt
                echo $peer_line
                reachability=$(cut -d " " -f 2 $peer_line)
                echo $reachability
                start=$(( start + 1 ))
        done
If I comment out the reachability= line, it happily prints out the 3 whole lines from the file ntpstatus2.txt, so string variable peer_line is reading a line from the file ntpstatus2.txt fine. But I can't for the life of me get the string variable reachability to equal field 2 of the string variable peer_line. I've tried adjusting stdbuf to -o0 and -oL to no avail. I've tried just about every combination of cut(1) parameters I can find on the web. Sample of peer_line content: 1 40822 94fa yes yes none candidate sys_peer 15. I've been at this for nearly two days now and I'm losing the will. Could someone please point me in the right direction?
 
It doesn't work for me, just prints a blank line at echo $reachability. I've tried several values for -f to no avail. I don't get any error messages.
 
Ahh sorry, I missed you're alternative answer. The second version, reachability=$(cut -d " " -f 2 "$peer_line"), gave me an error message "cut: : No such file or directory".

I added a line prior to just try "echo "$peer_line"| cut -d " " -f2" and that just produces a blank line too, so the error is with "cut" somehow.

peer_line contains " 1 40822 94fa yes yes none candidate sys_peer 15". It originates as the output from "ntpq -c associations".

I've attached the whole script if useful.
 
Thinking about the error message "cut: : No such file or directory", does it means that "cut" doesn't work on string variables, only files?
In which case could I amend the earlier file read something like this?
echo sed -n $start,1p /$workdir/ntpstatus2.txt | cut -d " " -f2
 
Looks complicated to me. Not sure what you need, but awk is your friend:

Code: 
awk '!/===/ && !/condition/ {print $2}' ntpstatus.txt
 
Personally I would not set $workdir as you do then place "/" in front of it in shell variables. One mistake can mean placing stuff in "/" root.
It's not needed, you've already placed it in the variable.
I'd also do ntpstat2="${workdir}/ntpstatus2" and use $ntpstat2 for each access. Just a personal preference, really.

I would change:
Code: 
 peer_line=echo sed -n $start,1p ntpstatus2.txt
 to
 peer_line=`sed -n $start,1p ntpstatus2.txt`

And
Code: 
reachability=$(cut -d " " -f 2 "$peer_line")
 to
reachability=`echo "$peer_line"| cut -d " " -f 2`
 
usually you use something like 'cat | cut' instead of 'echo' to extract parts of a string from a file, or just use awk...

what exactly are you trying to accomplish with this script? I suspect there's an easier way.
 
oOiOo said:
Looks complicated to me. Not sure what you need, but awk is your friend:

Code: 
awk '!/===/ && !/condition/ {print $2}' ntpstatus.txt
Click to expand...
Thanks for the suggestion, I appreciate it, and I will try awk if cut doesn't work. I need to write code in single step, easy-to-understand (for me), pieces because when I come back to them later I can understand them. I find awk not very intuitive. I write very little code.
 
fixit9660 said:
I need to write code in single step, easy-to-understand (for me), pieces because when I come back to them later I can understand them. I find awk not very intuitive. I write very little code.
Click to expand...

That's why you always comment your code. Write a short paragraph on *WHAT* the script does at the top, *HOW* it works and its options/arguments. Then write short descriptions on what part of a script does (e.g. # the following block extracts status information from ntpd ) and if you have some complex logic you can/should also comment that line e.g. why you chose this way...
 
mark_j said:
Personally I would not set $workdir as you do then place "/" in front of it in shell variables. One mistake can mean placing stuff in "/" root.
It's not needed, you've already placed it in the variable.
I'd also do ntpstat2="${workdir}/ntpstatus2" and use $ntpstat2 for each access. Just a personal preference, really.

I would change:
Code: 
 peer_line=echo sed -n $start,1p ntpstatus2.txt
to
peer_line=`sed -n $start,1p ntpstatus2.txt`

And
Code: 
reachability=$(cut -d " " -f 2 "$peer_line")
to
reachability=`echo "$peer_line"| cut -d " " -f 2`
Click to expand...
Suggestions on directory variables noted and adopted, thank you for them. As I said before, I'm not a code writer, and my knowledge and discipline writing scripts shows that up very obviously. All advice therefore gratefully received.
Changing
Code: 
 peer_line=echo sed -n $start,1p ntpstatus2.txt
to
peer_line=`sed -n $start,1p ntpstatus2.txt`
made no apparent difference, however changing
Code: 
reachability=$(cut -d " " -f 2 "$peer_line")
to
reachability=`echo "$peer_line"| cut -d " " -f 2`
made exactly the change I wanted, it works!!! Thank you!!!
 
sko said:
That's why you always comment your code. Write a short paragraph on *WHAT* the script does at the top, *HOW* it works and its options/arguments. Then write short descriptions on what part of a script does (e.g. # the following block extracts status information from ntpd ) and if you have some complex logic you can/should also comment that line e.g. why you chose this way...
Click to expand...
I have to apologise for the lack of comments in this script. I usually do put it copious comments as I need them the most if/when I revisit it. I write so little that I virtually have to re-learn everything again when I come to write another one.
I was prompted to write this script after my NAS system clock stopped synchronising to web-based NTP and drifted, several times. It took me several months to realise what had happened and I couldn't find any monitoring tools to generate an alarm, which surprises me as there seems to be one for every other server failure condition.
When it's finished, it will monitor as many of the ntp daemon status conditions as I can manage and email me accordingly. I'll also get it to send entries to the appropriate log, (which ever that may be). The basic functionality was just to be able to read the ntpq statuses output, and pick out the individual report values to be able to compare them to working/non-working values, which I can now do thanks to the above replies.
Thank you for your valuable help.
 
fixit9660 said:
Suggestions on directory variables noted and adopted, thank you for them. As I said before, I'm not a code writer, and my knowledge and discipline writing scripts shows that up very obviously. All advice therefore gratefully received.
Changing
Code: 
 peer_line=echo sed -n $start,1p ntpstatus2.txt
to
peer_line=`sed -n $start,1p ntpstatus2.txt`
made no apparent difference, however changing
Code: 
reachability=$(cut -d " " -f 2 "$peer_line")
to
reachability=`echo "$peer_line"| cut -d " " -f 2`
made exactly the change I wanted, it works!!! Thank you!!!
Click to expand...
I'm happy to help.

When writing scripts, try to break it into stages or sections. Then stop, run it and make sure it works. This might mean putting dummy data into peer_line, for example, to see if cut works. That is, take a line of output from ntpq and placing it in peer_line (peer_line="blah blah") then run your cut or tr or whatever on it.

Also, often it's helpful to put set -x just before you suspect a problem so that the script outputs its debugging information.
As a general rule you should be very careful running any script as root especially if using redirection/pipes and double especially using rm -f :eek:
Can your script be run as a non-root user? If so, it's safer.

Keep plodding, you're doing well. :)
 
...it's helpful to put set -x just before you suspect a problem...
Click to expand...
I write scripts by either copying from my other working scripts, or as you say, making them up in small sections and testing them. I also try to create their own working directory for all the work to happen in. On this occasion I pretty much forgot to do all of it ?.

set? That is a new, and really useful command that I've never come across!!! Thank you. I shall play with it for a while, I'll doubtless need it again.
I agree with your comment about running as root; I thought I'd tried the command "
Code: 
ntpq -c associations
" as a user and it ran but failed to produce any values. However I've just tried it again and it works as expected, so back to running as it User. Thank you for the prompt.

Thanks again for your help, and patience, very much appreciated!!
 
fixit9660 said:
I was prompted to write this script after my NAS system clock stopped synchronising to web-based NTP and drifted, several times. It took me several months to realise what had happened and I couldn't find any monitoring tools to generate an alarm, which surprises me as there seems to be one for every other server failure condition.
When it's finished, it will monitor as many of the ntp daemon status conditions as I can manage and email me accordingly. I'll also get it to send entries to the appropriate log, (which ever that may be). The basic functionality was just to be able to read the ntpq statuses output, and pick out the individual report values to be able to compare them to working/non-working values, which I can now do thanks to the above replies.
Click to expand...

So you are trying to check if ntpd fails to peer?

Instead of all this monitoring, logging etc that needs constant observation and manual intervention, why not just use ntpd -gq or sysutils/rdate in a cronjob to ensure the time is forcefully set at a given interval? If that fails, cron will send you an email with the returned error. I often use rdate on systems that don't have a hardware clock to set the system time at startup.

OTOH: ntpd is usually one of those "config and forget" services, as it 'just works'™. I'd rather check *why* it doesn't work on your host instead of monitoring some parts of it which still won't tell you why it fails. (wild guess: ageing CMOS battery on a cheapo-NAS-board with a bad or no hardware clock)


But as a general recommendation for scripts: the most useful and universal scripts are always simple filters. Don't use hard-coded paths and avoid using files to receive simple data like a status. This way you can simply pipe output of one command into your scritp and it returns either a status code or some other (text) output that could be piped into the next command (e.g. to send the output via mail or to a logfile). Pipes are a powerful tool - write scripts that can make use of pipes.

I don't want to discourage you from writing scripts - they are essential duct-tape for many things, but don't get carried away and try to solve (or worse: cover up) every small problem with a bunch of scripts, especially if the right way would be to fix the root cause of the problem. You'll end up with a host that consists of layers and layers of hacks and workarounds that tend to break in very ugly and hard to debug ways and are a nightmare to maintain. A lot of "virtual appliances" (read: some Linux installation cobbled together with tons of scripts which run as root and everything is stuffed into a single folder...) are built this way and they usually blow up on larger updates...
 
So you are trying to check if ntpd fails to peer?
Click to expand...
Yes, that's correct.
I did try to determine why it failed and the only entry in all the messages file was as follows:
Nov 18 16:30:15 freenas ntpd[1117]: restrict: ignoring line 10, address/host '0.freebsd.pool.ntp.org' unusable.
Nov 18 16:30:15 freenas ntpd[1117]: restrict: ignoring line 11, address/host '1.freebsd.pool.ntp.org' unusable.
Nov 18 16:30:15 freenas ntpd[1117]: restrict: ignoring line 12, address/host '2.freebsd.pool.ntp.org' unusable.

Not a whole heap of help really, so I thought I'd start by i) letting myself know immediately it happened, ii)recording some of the values in disk files prior to the failure to give me something to work on when it does fail again.
Hence the script.
avoid using files to receive simple data like a status
Click to expand...
If the only information available is in a running script, then it's not available if the script/server stops; I'm afraid I have to disagree with you about using files to receive status data.
The NAS is behind a UPS so the HP motherboard CMOS battery is irrelevant, but thanks for the reminder though, I will check the battery.
I've always relied on NTP, so far without issue, and as you say, it's "usually one of those "config and forget" services, as it 'just works'", so I'll dig a little further before I consider rdate, or similar. Thanks for the info though, I wasn't aware of its availability.
 
shell fun (set positional params to split a string at whitespace (IFS))
Code: 
#!/bin/sh
A=$(ls -l /dev/null)
set - $A
echo third: $3
echo fourth: $4
echo first: $1

Code: 
$sh e.sh
third: root
fourth: wheel
first: crw-rw-rw-
 
Just FTR: FreeNAS is not supported here, as they do some really weird stuff to the underlying FreeBSD installation (which might even include another version of/build options for ntpd which causes your problem!):
forums.freebsd.org

GhostBSD, pfSense, TrueNAS, and all other FreeBSD Derivatives

Questions about 'derivative FreeBSDs', like GhostBSD DesktopBSD TrueNAS XigmaNAS OPNsense pfSense PacBSD BSD Router Project NomadBSD helloSystem should be asked on the forums and/or mailing lists for these specific products. See below for links. If you still think your questions should be...
forums.freebsd.org forums.freebsd.org

Show your ntp.conf, respectively the lines that ntpd is complaining about. They should look something like:
Code: 
pool 0.freebsd.pool.ntp.org iburst

check if your host can resolve the hostnames (N.freebsd.pool.ntp.org).
As said: ntpd on FreeBSD is working out-of-the-box without touching its config. Otherwise ask at the FreeNAS forums, because they might have replaced it with another variant...
 
Code: 
root@freenas:/mnt/FreeNAS_1/home/andrew # cat /etc/ntp.conf         
server 0.freebsd.pool.ntp.org iburst maxpoll 10 minpoll 6
server 1.freebsd.pool.ntp.org iburst maxpoll 10 minpoll 6
server 2.freebsd.pool.ntp.org iburst maxpoll 10 minpoll 6
restrict default ignore
restrict -6 default ignore
restrict 127.0.0.1
restrict -6 ::1
restrict 127.127.1.0
restrict 0.freebsd.pool.ntp.org nomodify notrap nopeer noquery
restrict 1.freebsd.pool.ntp.org nomodify notrap nopeer noquery
restrict 2.freebsd.pool.ntp.org nomodify notrap nopeer noquery

...which is odd because in the entries of the messages log above it mentions lines 10, 11, and 12 when there are only 11 lines in the file...

Code: 
root@freenas:/mnt/FreeNAS_1/home/andrew # ping 0.freebsd.pool.ntp.org
PING 0.freebsd.pool.ntp.org (176.58.109.199): 56 data bytes
64 bytes from 176.58.109.199: icmp_seq=0 ttl=53 time=9.190 ms

Name Resolution is fine. It must be because the NAS emails me about several things event month.

Code: 
May  8 09:25:56 freenas ntpd[70420]: ntpd 4.2.8p12-a (1): Starting
May  8 09:25:56 freenas ntpd[70420]: Command line: /usr/sbin/ntpd -g -c /etc/ntp.conf -p /var/run/ntpd.pid -f /var/db/ntpd.drift
May  8 09:25:56 freenas ntpd[70420]: ----------------------------------------------------
May  8 09:25:56 freenas ntpd[70420]: ntp-4 is maintained by Network Time Foundation,
May  8 09:25:56 freenas ntpd[70420]: Inc. (NTF), a non-profit 501(c)(3) public-benefit
May  8 09:25:56 freenas ntpd[70420]: corporation.  Support and training for ntp-4 are
May  8 09:25:56 freenas ntpd[70420]: available at https://www.nwtime.org/support
May  8 09:25:56 freenas ntpd[70420]: ----------------------------------------------------
 
Just a side note: I see you use tcsh while your script runs with /bin/sh. If you try something on the command line, it may give a different result in the script. Tiny differences only :cool: .
Running Linux you don't have those differences, /bin/sh is usually a link to /bin/bash then. Just keep it in mind.
 
Tieks said:
Just a side note: I see you use tcsh while your script runs with /bin/sh. If you try something on the command line, it may give a different result in the script. Tiny differences only :cool: .
Running Linux you don't have those differences, /bin/sh is usually a link to /bin/bash then. Just keep it in mind.
Click to expand...
still behaves differently. when run as sh bash does not support some?/all? of the non posix bash stuff (iirc)
 
Tieks said:
Just a side note: I see you use tcsh while your script runs with /bin/sh. If you try something on the command line, it may give a different result in the script. Tiny differences only :cool: .
Running Linux you don't have those differences, /bin/sh is usually a link to /bin/bash then. Just keep it in mind.
Click to expand...
?!?!?!?! That never even crossed my mind. o_O Well spotted, and thank you. I'll make a point of being more specific in my research.
 
You must log in or register to reply here.
Back
Top