linux unix

network ip calculation with ifconfig

Most *nix are different. I’ll start with a plain Linux output

ifconfig eth0
eth0: flags=4163  mtu 1500
        inet  netmask  broadcast

to get the network ip, I just bitwise-and the inet and the netmask. To do it with the shell, I convert the ip to an integer and use the & (AND) operator

IP=$(ifconfig eth0|grep inet|awk '{print $2}')
NM=$(ifconfig eth0|grep inet|awk '{print $4}')

I get my IP= and NM= out of the ifconfig output

IPDEC=0;IFS=. ;for f in $IP;do ((IPDEC*=256));((IPDEC+=$f));done
NMDEC=0;IFS=. ;for f in $NM;do ((NMDEC*=256));((NMDEC+=$f));done

By converting the IP-base-256 address, I get IPDEC=1572395042 and NMDEC=4294967040 in decimal


That’s simple. My network IP is 1572395008

Let’s print it


Thanks for reading me that far. Ok let blogger Mathieu Trudel-Lapierre tell you : If you’re still using ifconfig, you’re living in the past

ip addr

ip shows your ip, and ipcalc do the calculation

ipcalc -n "$(ip -o -4  -br address show eth0 |awk '{print $3}')"

on parsing arguments in shell

While most programming languages are accepting arguments as an array of strings, shell doesn’t


int main(int argc, char **argv) {
int i;
for (i=1; argc>i; i++)

$ make arglist
cc arglist.c -o arglist
$ ./arglist one two three four

To do the same in shell, it requires some dynamic evaluation, for instance with eval

while [ $i -le $# ]
eval ARGV[$i]=\$$i
echo "\$$i=$(eval echo \"\${ARGV[$i]}\")"

$ ./ one two three four

To further send the arguments to another script or function, it is important to take consideration of white spaces and file expansion (e.g.: *.*), this is achieved with double-quotes and the at-sign

f "${ARGV[@]}"

I recommend against using eval whenever possible. While less awesome, I would prefer something more verbose and without eval

[ -n "$1" ] && echo "\$1=$1"
[ -n "$2" ] && echo "\$2=$2"
[ -n "$3" ] && echo "\$3=$3"
[ -n "$4" ] && echo "\$4=$4"
[ -n "$5" ] && echo "\$5=$5"

$ ./ one two three four

Using eval is difficult and dangerous. The innocent may messed up with the quotes resulting in random effects. It is also a common source of code injection

eval x=$1

$ ./ 1
$ ./ "1; echo uh-oh"

Ref: Eval Injection


on logical and physical working directories

many ignore the difference meaning of .. (dot dot) as an argument when used with ls or cd

this leads to buggy coding and wrong parsing of arguments in scripts

let’s start with an example

$ mkdir $HOME/test $HOME/test/physical $HOME/test/foo 
$ cd $HOME/test/foo
$ ln -s ../physical logical
$ cd logical
$ ls -l ..
total 8
drwxr-xr-x. 2 oracle dba 18:01 foo
drwxr-xr-x. 2 oracle dba 18:01 physical
$ cd ..; ls -l
total 0
lrwxrwxrwx. 1 oracle dba 18:01 logical -> ../physical

Wait… how could cd ..; ls and ls .. have a different output?

Most programs except cd use the physical path in arguments.

If you are in the physical directory $HOME/test/physical and you issue

program argument

it will behave the same as if you were in the logical path. This is somehow consistent, but confusing

Let’s try

$ cd $HOME/test/physical
$ ls -l ..
total 8
drwxr-xr-x. 2 oracle dba 4096 May 21 18:01 foo
drwxr-xr-x. 2 oracle dba 4096 May 21 18:01 physical
$ cd $HOME/test/foo/logical
$ ls -l ..
total 8
drwxr-xr-x. 2 oracle dba 4096 May 21 18:01 foo
drwxr-xr-x. 2 oracle dba 4096 May 21 18:01 physical

BINGO! I got the same output

Got it? The arguments are parsed using the physical path.

This does not relate to absolute or relative path. While most bug occurs with relative path, a relative path is neither logical nor physical. It is relative. Whether it relates to a physical or it relates to a logical path is the scope of this post.

Okay, we get now that most programs use the “filename” as if you were in the physical path.

Path Logical Physical
/home/user01/test/physical /home/user01/test/physical /home/user01/test/physical
/home/user01/test/foo/logical /home/user01/test/foo/logical /home/user01/test/physical

In most case, it makes no difference (which render the bugs less evident to trap). For instance browsing paths in dbca doesn’t do it right, but it is just a side note.

If you use /physicalpath/file or /logicalpath/file or ./file, it really doesn’t matter. It’s relevant with symbolic links on directories and relative path to parents.

So for instance if you want to change to the directory of the first argument it would be wrong to do

cd $(dirname $1)

because cd does use logical path and your program should NOT (to make it symlink-independent).

a not-properly documented (missing for instance on aix 7.2 cd manpage) way is to use the -P option

In Linux there is also a -e option which gives you a non-zero error code on non-existent current working directory (if you are in a path that does not exits, the cd -P won’t work but return 0 by default), but for now, just stick to -P

Let’s see

$ pwd
$ cd -P ..
$ pwd

wait, you change to .. and went two step back? this is not the default behavior of cd. The default behavior is -L

$ pwd
$ cd -L ..
$ pwd

hmm… is that not easier? No way! this is just fine for cd (where you navigate to parent regarding to the logical working directory). But it is not the way the arguments are interpreted.

Apart from cd , there is another command that deals with symlink path hassle : pwd. Again, it is not really well documented (missing in Solaris 10 pwd manpage), but it has always been there.

$ pwd
$ pwd -L
$ pwd -P

next time you use cp, ls, cat with a .. and symlinks, remember this post !

Last note, one may like to try the long option. Don’t!

$ man pwd | grep -- -P
-P, --physical
$ cd -P .
$ cd --physical .
-bash: cd: --: invalid option
cd: usage: cd [-L|-P] [dir]

linux unix

on input and output file descriptors

Let’s start with some basics. The basics works as well on Unix, Linux and Windows. Later techniques only work on linux/unix

$ ls -l hosts          
-rw-r--r--. 1 root root 211 Oct  5  2015 hosts
$ ls -l xxx  
ls: cannot access xxx: No such file or directory
$ read x

Outpout and error are displayed on screen and input is read from your keyboard

The output is kwown as file-descriptor-1 or stdout. Sometimes, depending on your OS, it may be exposed as /dev/fd/1 or /dev/stdout. But not all *nix have this.
The error is kwown as file-descriptor-2 or stderr.
The input is known as file-descriptor-0 or stdin.

Instead of keyboard and screen, it could be a file or any other devices, e.g. /dev/null or just a simple file.

$ ls -l hosts 1>file1
$ ls -l xxx 2>file2          
$ read x 0<file3

0 and 1 are optional here.

If is also possible to redirect stdout and vice versa

$ ls -l hosts 1>&2         
-rw-r--r--. 1 root root 211 Oct  5  2015 hosts
$ ls -l xxx 2>&1 
ls: cannot access xxx: No such file or directory

It is possible to close the file descriptor.

sleep 1 1>&- 2>&- 0<&-

Well, sleep has no output and no input and no error, so the effect is not impressive.

If you write to a closed file descriptor, you get an error. Ok, if you close both stdout and stderr, the error will be silent. But there will still be an error.

$ (echo foo) 1>&-     
bash: echo: write error: Bad file descriptor
$ echo $?        
$ (echo bar 1>&2) 2>&-      
$ echo $?             

if you want to redirect stdin to stdout and stdout to stdin, you better use a new file descriptor

$ (ls -l hosts xxx 1>&2 2>&3) 3>&1            
xxx not found
-rw-rw-r-- 1 root system 2133 Jun 22 2017 hosts

An old trick is to use additional file descriptor to find a return code of command before the pipe.

$ ((((ls hosts; echo $? >&3) | tr '[a-z]' '[A-Z]' >&4) 3>&1) | (read rc; exit $rc)) 4>&1       
$ echo $?
$ ((((ls xxx; echo $? >&3) | tr '[a-z]' '[A-Z]' >&4) 3>&1) | (read rc; exit $rc)) 4>&1        
ls: cannot access xxx: No such file or directory
$ echo $?

As I know the trick for so long that I could not credit the author, only found some 21st century posts

If you want to redirect all your outputs to a logfile, you can use exec

exec 1>>/tmp/mylog
exec 2>>/tmp/mylog
cd /etc 
ls -l hosts          
ls -l xxx

If you want to be able to still use your stdout / stderr, again, open new descriptors

exec 3>&1
exec 4>&2
exec 1>>/tmp/mylog
exec 2>>/tmp/mylog
cd /etc 
ls -l hosts          
ls -l xxx
echo INFO >&3
echo ERROR >&4
exec 3>&-
exec 4>&-

Bash has also one shortcut

ls xxx host &>log

& redirect both 1 and 2 in one step. Doesn't work on ksh.


Standard date format

Let’s start with Powershell

Get-Date -format "o"

The “O” or “o” standard format specifier represents a custom date and time format string using a pattern that preserves time zone information and emits a result string that complies with ISO 8601

Now Linux

date "+%Y%m%dT%H:%M:%S.%3N%z"


to_char(current_timestamp, 'YYYYMMDD"T"HH24:MI:SS.FF3TZH:TZM') 

for my XML fans

extractvalue(xmlelement(t, current_timestamp),'/*') 

Now in AIX

 perl -e '
      use strict;
      use POSIX "strftime";
      use Time::Piece;
      use Time::HiRes "gettimeofday";
      my $s=Time::Piece->new;
      my $t=$s->tzoffset;
      printf "%s.%03d%+03d:%02d\n",

Could not have figured out without google πŸ˜‰
The GNU date could also be installed in AIX, but I am not root

A more generic unix version would be the UTC date

date -u "+%Y%m%dT%H:%M:%SZ" 
linux sybase unix

Unix ODBC Sybase

very similar to Unix ODBC Oracle

instead of tnsnames, the connections are defined in $SYBASE/interfaces.

the odbc.ini must exists as well in $SYBASE.

if you test with unixODBC-devel, keep in mind to use /usr/bin/isql and not $SYBASE_OCS/bin/isql

Driver = Sybase16
DSN = syb

Description = Adaptive Server Enterprise
Driver = /u01/app/sybase/product/16.0/DataAccess64/ODBC/lib/

DSN = syb

master tcp ether 15000
query tcp ether 15000

/usr/bin/isql -v syb user01 passw01
| Connected! |
| |
| sql-statement |
| help [tablename] |
| quit |
| |

linux unix

Unix ODBC Oracle

To connect via ODBC, check

This article is related to Unix/Linux. Often you have a fat client written in C, while java uses JDBC instead of ODBC.

Okay, it’s pretty easy, if you have an oracle client, you probably already have in your LD_LIBRARY_PATH. In this case you can connect using ODBC.

What you need is an odbc.ini where you defined your connections
Driver = OracleODBC18
DSN = DB01
ServerName = DB01

and an odbcinst.ini where you define your driver
Description = Oracle ODBC driver for Oracle 18
Driver = /u01/app/oracle/product/18.1.0/client_64/lib/

the name / location and options may depend on your software / driver vendor.

ODBC uses TNSNAMES, so it really easy, you just the odbc entry ServerName = DB01 that matches
in tnsnames.ora.

ODBC full client allows you to use all connection features like LDAP and SSL.

To test it, I installed unixODBC-devel
sudo yum install unixODBC-devel
export ODBCSYSINI=/home/user1/odbc
vi odbc.ini odbcinst.ini

the machine-wide ODBCSYSINI is /etc. You can chose to define the ODBCINI user-wide (/home) and the ODBCSYSINI machine-wide (/etc). I wouldn’t use machine-wide passwords. But configuring the drivers only once may be an option. If you are root and you have not too many drivers/versions/bitcode.

Now try to connect :
isql DB01 scott tiger
SQL> select * from scott.emp;
| 7369 | SMITH | CLERK | 7902 | 1980-12-17 00:00:00| 800 | | 20 |
| 7499 | ALLEN | SALESMAN | 7698 | 1981-02-20 00:00:00| 1600 | 300 | 30 |
| 7521 | WARD | SALESMAN | 7698 | 1981-02-22 00:00:00| 1250 | 500 | 30 |
| 7566 | JONES | MANAGER | 7839 | 1981-04-02 00:00:00| 2975 | | 20 |
| 7654 | MARTIN | SALESMAN | 7698 | 1981-09-28 00:00:00| 1250 | 1400 | 30 |
| 7698 | BLAKE | MANAGER | 7839 | 1981-05-01 00:00:00| 2850 | | 30 |
| 7782 | CLARK | MANAGER | 7839 | 1981-06-09 00:00:00| 2450 | | 10 |
| 7788 | SCOTT | ANALYST | 7566 | 1987-04-19 00:00:00| 3000 | | 20 |
| 7839 | KING | PRESIDENT| | 1981-11-17 00:00:00| 5000 | | 10 |
| 7844 | TURNER | SALESMAN | 7698 | 1981-09-08 00:00:00| 1500 | 0 | 30 |
| 7876 | ADAMS | CLERK | 7788 | 1987-05-23 00:00:00| 1100 | | 20 |
| 7900 | JAMES | CLERK | 7698 | 1981-12-03 00:00:00| 950 | | 30 |
| 7902 | FORD | ANALYST | 7566 | 1981-12-03 00:00:00| 3000 | | 20 |
| 7934 | MILLER | CLERK | 7782 | 1982-01-23 00:00:00| 1300 | | 10 |
SQLRowCount returns -1
14 rows fetched

linux security unix

disallow pseudo terminal in ssh

Some Oracle documentation wants you to setup ssh with no password and no passphrase.

Configuring ssh

This is not really something your security admin will like.

ssh-keygen -t dsa
First, using DSA, which is deprecated and disabled by default in OpenSSH 7.0, is a pretty dump instruction
OpenSSH 7.0 and greater similarly disable the ssh-dss (DSA) public key algorithm. It too is weak and we recommend against its use.
The two recommended key types are rsa and ecdsa. You should not use dsa

Second, ssh-key without passphrase is a huge security hole. If one get access to your key, for instance on a disk, a tape backup, etc, she’ll get access as oracle to all your database nodes. Best practice to use a pass phrase. Depending on your setup, it is sufficient to get ssh keys at installation/upgrade time only.

Third, providing interactive ssh-login as Oracle is against best practice for tracability. You better use SUDO or another elevation mechanism.

Let’s try:

First, use a recommended algoryhtm and key-length.
ssh-keygen -t rsa -b 4096
ssh-keygen -t ecdsa -b 521

Then, use a passphrase

Enter passphrase (empty for no passphrase): ***
Enter same passphrase again: ***

Then, when creating you authorized key, disable unwanted features, like pseudo terminal



ecdsa-sha2-nistp521 AAAABBBB/cccc== oracle@srv001


no-agent-forwarding,no-port-forwarding,no-pty,no-user-rc,no-x11-forwarding ecdsa-sha2-nistp521 AAAABBBB/cccc== oracle@srv001

Also, you could deactivate some features on the client config


This could also be done one the server sshd_config, but if you are not the sysadmin, don’t mess up with it.

Because you have a passphrase, you need to use an agent before starting your installation. Because pseudo-terminal (no-pty) is disabled, you cannot get a prompt. Because x11 is disabled (no-x11-forwarding), you cannot start an xterm

$ ssh srv002
Permission denied
$ eval $(ssh-agent)
Agent pid 12345
$ ps -fp 12345
oracle 123451 0 ssh-agent
$ ssh-add ~/.ssh/id_ecdsa
Enter passphrase for ~/.ssh/id_ecdsa:
Identity added: ~/.ssh/id_ecdsa (~/.ssh/id_ecdsa)
$ ssh -t srv002
PTY allocation request failed on channel 0
$ ssh -Y srv002 aixterm
X11 forwarding request failed on channel 0
1363-008 X server named was not found.
$ ssh srv002 date
Fri Jul 13 12:50:22 CEST 2018

Those are basic steps to make your ssh less unsecure.


Don’t call it test

There are quite a few names to avoid in your scripts. Even if there are not reserved-words, keep away !

I’ll start with test

cd $HOME/bin
vi test
echo hello world
chmod +x test
hello world

The problem is that it may break your other scripts

$ ssh localhost test 1 = 2 && echo WHAT???
hello world

And it may break sooner or later, depending on your OS / version / default shell / default path / others.

There are quite a few filenames you should not use, like test, date, time, hostname, mail, view, touch, sort and make. The command type lists some of those as reserved word, shell builtin, tracked alias, shell keyword. But again it is not consistent over Unix flavors.

$ uname -sr; type date
SunOS 5.10
date is /usr/bin/date
$ uname -sr; type date
Linux 2.6
date is a tracked alias for /bin/date

Your sysadmin may also alias things for colors and safety in the common profile: for instance vi, ls, rm. But if it annoys you, then use \vi instead of vi.

linux security unix

Untrusted X11 forwarding

I wrote a while ago about my security concerns regarding

xhost +
xterm -display mypc:0

Way back then, I suggested ssh tunnel. SSH is pretty easy to set up, by enabling the X11Forwarding option.

In OpenSSH 3.8 release note, 2004, there was a new default .

ssh(1) now uses untrusted cookies for X11-Forwarding

In the man ssh_config page, it’s still documented as being the default

ForwardX11Trusted The default is ‘no’

But it actually isn’t on most *ix derivates, e.g. RedHat /etc/ssh/ssh_config

# If this option is set to yes then
# remote X11 clients will have full access
# to the original X11 display. As virtually
# no X11 client supports the untrusted
# mode correctly we set this to yes.
ForwardX11Trusted yes

Who is we?

Okay, let’s go back.

If you use the unsafest method, xhost + and xterm -display pc:0, then you grant everybody the right to manipulate X.

If you use trusted ssh, which is the _undocumented_ default in Linux, then you grant this right only to anyone with access to your authority, most probably located in the file $HOME/.Xauthority. So root and yourself, at least.

If you trust neither yourself nor root, you could restrict access to your resource, preventing one hacker from switching your mouse buttons or doing a screenshot. But this is probably going to prevent most of your applications from working. Also, it probably won’t work at all if you use putty, reflection and (virtually any?) other client tools.

If you want to force Trusted mode, use -Y or -o ForwardX11Trusted=yes.

If you want to force Untrusted mode, use -X and -o ForwardX11Trusted=no.

If you use only -X, it may transparently defaults to the more convenient but less secure -Y. Sometimes. At least on Linux OpenSSH. But if you use different Unix / SSH flavours, the -X may ends with an error message like connection to “localhost:10.0” refused by server. In that case, simply use -Y. Actually, always use -Y if you want Trusted.

linux security unix

run sudo, ssh, password, su in simulated interactive mode

Some commands do not like non-interactive mode

$ passwd < oldpassword
> newpassword
> newpassword
Changing password for user lsc.
Current password for passwd: Authentication token manipulation error
$ echo oraclepassword | su - oracle
standard in must be a tty
$ echo sudopassword | sudo su - oracle
[sudo] password for lsc:
sudo: sorry, you must have a tty to run sudo

But ok, if you really want to run those in a script, you have plenty of clean (no hack there) ways of doing this.

For instance, let’s use a screen called BAR.

$ xterm -e "screen -S BAR" &
[1] 31732

Now we have an interactive terminal. It could be redirected to a frame buffer device if no x server is started.

Not really a human device, but an interactive terminal.

Now let’s send stuff

$ CR="$(echo '\r')"
$ screen -S BAR -X stuff "sudo su - oracle$CR"
$ screen -S BAR -X stuff "sudopassword$CR"
$ screen -S BAR -X stuff "id > /tmp/xxx$CR"
$ screen -S BAR -X stuff "exit$CR"
$ screen -S BAR -X stuff "exit$CR"
[1] + Done xterm -e "screen -S BAR" &
$ cat /tmp/xxx
uid=100(oracle) gid=100(dba) groups=100(dba)

Usual disclaimer: it is a bad security practice to hardcode your passwords in scripts. Use this only if you really understand security. Read man openssl about how to use openssl to encrypt your password. Ask your security friends before trying

linux unix

How to *really* send a script to the background

Let’s check this small script

echo foo.1:`date` | tee $HOME/tmp/foo.txt
sleep 3
echo foo.2:`date` | tee -a $HOME/tmp/foo.txt

$ $HOME/tmp/
foo.1:Thu Nov 27 17:34:53 CET 2014
foo.2:Thu Nov 27 17:34:56 CET 2014

Very obvious, I write to the console, wait three seconds, then write to the console.

Ok, let’s take another script that would call this script in the background using &

echo bar.1:`date`
$HOME/tmp/ &
echo bar.2:`date`

$ $HOME/tmp/
bar.1:Thu Nov 27 17:36:32 CET 2014
bar.2:Thu Nov 27 17:36:32 CET 2014
foo.1:Thu Nov 27 17:36:32 CET 2014
foo.2:Thu Nov 27 17:36:35 CET 2014

bar is printing the date, calling foo in the background, then printing the date, then it returns to you, and foo is still running.

BUT this is only in a relative background …

Let’s try this

$ time $HOME/tmp/ > /dev/null

real 0m0.01s
user 0m0.00s
sys 0m0.00s

So it takes no time to run bar you believe ?

Let’s try, for instance, over ssh (or cron or whatever)

$ time ssh localhost $HOME/tmp/ > /dev/null
real 0m3.81s
user 0m0.01s
sys 0m0.01s

running bar suddenly waits 3 seconds for foo to finish.

To be sure the script is sent to the farest background, you need to close the file descriptors, stdin, stdout, stderr

I rewrote it as

echo bar.1:`date`
$HOME/tmp/ <&- >&- 2>&- &
echo bar.2:`date`

$ time ssh localhost $HOME/tmp/ >/dev/null
real 0m0.44s
user 0m0.00s
sys 0m0.00s

Now the script baz is immediately finished and does not wait for foo to complete

linux unix

fun with cron

Today I find out that my scheduler was too busy to execute all jobs in my crontab !?

* * * * * (while :;do ssh :; done)
59 23 19 06 * touch /tmp/bang

my while loop is going to produce so much hangs on the cron deamon that it may not be able to read the crontab once a minute. If it reads it at 23:58 and at 00:00, the 23:59 won’t be run.

This is actually the first time I see this behaviour. And -believe me- it’s annoying!

linux unix

Testing for (non-)empty string in shell

One way to test for (non-)empty string is to use test and -z (-n)

$ x=foo
$ test -z "$x"
$ echo $?

This is mostly seen with an if and [ -z … ] syntax

$ y=bar
$ if [ -n "$y" ];
then echo non-empty;

Instead of a variable, it could be the output of a script.


if [ -n "$(grep ORA- alertDB01.log)" ]
echo there is an error in the alert log
echo "fine :)"

This will work for years until one day you get :

ksh: no space

Why that? This is the way the shell works. Your shell (here ksh on AIX) starts having errors as soon as your subshell (here the grep) is exhausting the space.

$ wc -l alertDB01.log
2 alertDB01.log
$ if [ -n "$(grep ORA- alertDB01.log)" ];
then echo non-empty;
else echo "fine :)";
$ wc -l alertDB01.log
75025 alertDB01.log
$ if [ -n "$(grep ORA- alertDB01.log)" ];
then echo non-empty;
else echo "fine :)";
ksh: no space

You got a memory error, how the shell will react is random (core dump, errors, continue, crashes). It will just bug and you do not want this.

There is more than one to circumvent this. For instance you could use the return code of grep

$ if grep ORA- alertDB01.log >/dev/null;
then echo non-empty;
else echo "fine :)";

Different shells (Bash / Bourne) and different OSs (Linux / AIX / HPUX) may react differently. If AIX crashed with a 50’000 lines, it may scale up to millions of lines in recent Linux’s – but still use trucks of memory

linux security unix

hot to bypass requiretty in sudo

You can execute it a command without password from the commande line

$ sudo -l
User lsc may run the following commands on this host:
(root) NOPASSWD: /usr/local/bin/helloworld
$ sudo /usr/local/bin/helloworld
Hello World!

Now you try to run it via cron and you get

sudo: sorry, you must have a tty to run sudo

The message is clear, you need a terminal.

Either you edit your sudoers files to disable requiretty, or you just get yourself a terminal.

Maybe you tried to assign a pseudo terminal with ssh -t, but you may get an error if ssh has no local tty

Pseudo-terminal will not be allocated because stdin is not a terminal.

Don’t despair, read man ssh

Multiple -t options force tty allocation, even if ssh has no local tty.

Let’s try

* * * * * ssh -t -t sudo /usr/local/bin/helloworld >> /tmp/txt

This should work, providing you configured ssh keys πŸ™‚

linux unix

use cron to schedule a job only once

I wrote about not using DAY OF MONTH and DAY OF WEEK simultanously in how to cron

The correct method is to use
15 14 15 05 * /tmp/run-my-job

But… I wrote this five years ago. Hmmm ! Not that correct then since it would run every year πŸ˜‰

Ok, periodically I check for jobs are scheduled to run a specific date only
$ crontab -l|awk '$1!~/#/&&$3*$4'
15 14 15 05 * /tmp/run-my-job

I have 9 more days to remove this before it runs for the fifth time πŸ™‚

sqlplus unix

return code and sqlplus

Calling a shell script from within sqlplus is buggy…

I have reported bug 3798918 in (back in 2004) and bug 13349119 in because some metalink guru closed 3798918 as not reproducible.

As written in return code, host does not return the correct code

SQL> host exit 7
SQL> def _RC

If you never use _RC, you may believe you are safe. But watch this :

SQL> get list
1 #!/bin/sh
2 if /bin/false
3 then
4 echo this is wrong
5* fi
SQL> host ./
this is wrong

The return code not being set is not only affecting the _RC variable, but it is also affecting all subshells !

Note this is not reproducable with SQLPLUS /NOLOG

SQL> host false
SQL> def _rc
SQL> conn x/x
ORA-01017: invalid username/password; logon denied
SQL> host false
SQL> def _rc

After my (failed or successfull) tentative to connect as x/x, it is reproducible again

linux Uncategorized unix

How to quit crontab -e without overwritting cron

Imagine my crontab

* * * * * /usr/bin/date > /tmp/foo

I am writing the date to /tmp/foo every minute
$ cat /tmp/foo
Thu Jul 5 08:45:01 CEST 2012

Now I want to view my crontab in my EDITOR (vi).

$ crontab -e

I do not quit yet.

In the meantime, my colleague modify the crontab.

* * * * * /usr/bin/date > /tmp/bar

Later, I quit vi with :q!

O Surprise, the crontab is * * * * * /usr/bin/date > /tmp/foo again

According to the doc :

When you finish creating entries and exit the file, the crontab command
copies it into the /var/spool/cron/crontabs directory

Even if you did not make change, you overwrite the content of your crontab !

If you want to exit your crontab editor really without overwritting the crontab, you need to kill yourself.


[2] + Stopped (SIGTSTP) crontab -e
$ kill %2
[2] + Stopped (SIGTTOU) crontab -e

Thanks to Colin comment, I realized I could not kill with kill, let’s kill with -9

$ kill -9 %2
[2] + Killed crontab -e

security unix windows

xhost+ security hole part 2

Five years ago I wrote xhost+ is a huge security hole, I turned out red this morning when my neighbour sent me a smiley via X.

Do I really want everyone to have full access to my screen? No, I don’t. And I don’t do xhost+.

So why did it happen to me ???

I am using X-Window Attachmate aka Reflection X. And in this tool, according to the doc, the default X policy is unrestricted. This is in my opinion a huge flaw in the security design. Make sure you always change this to something more secure.

In Reflection X Manager Settings, Category Security, choose for instance User-based security and Prompt. Configuring X Cookies is probably more cumbersome.

Then when you or someone else will start an XTERM on your desktop, you will get a nice dialog box :

[Reflection X]
Client could not successfully authenticate itself to Reflection X server. Would you like Reflection X to connect to this client as an UNTRUSTED client ? Client originated from (RX1303)

Ok, I have to click one more button, but at least I can deny access to my screen πŸ™‚

news unix

[alert] AIX Posix Timezone issue

Maybe you did get or you will get an issue with the date command in AIX.

expected behavior, Linux

$ TZ=NZST-12NZDT,M10.1.0/2,M3.3.0/3 date
Sat Mar 17 00:14:54 NZDT 2012
$ TZ=Pacific/Auckland date
Sat Mar 17 00:14:58 NZDT 2012

unexpected behavior, AIX

$ TZ=Pacific/Auckland date
Sat Mar 17 00:15:50 GMT+13:00 2012
$ TZ=NZST-12NZDT,M10.1.0/2,M3.3.0/3 date
Fri Mar 16 23:15:52 NZST 2012

The consequence : date, and all other unix commands like ls, who, ps that display the date in human readable format, and all programs that use ctime are affected

$ TZ=NZST-12NZDT,M10.1.0/2,M3.3.0/3 perl -e 'use POSIX;print ctime(time)'
Fri Mar 16 23:19:51 2012

Reference and link to the fixes :

dba unix

delete unused shared memory segments from an Oracle instance

Once upon a time, a dba issues some kill -9 to clean up dying database processes. Or the database instance crashes. This will left some shared memory segments. Note 68281.1 describe how to remove them on a server with multiple databases.

First, list the ipc process

$ ipcs
IPC status from /dev/mem as of Mon Nov 14 11:28:58 CET 2011
Message Queues:
q 0 0x4107001c -Rrw-rw---- root printq

Shared Memory:
m 0 0x7800006f --rw-rw-rw- itmuser1 itmusers
m 1 0x78000070 --rw-rw-rw- itmuser1 itmusers
m 5242882 0x41d2ba80 --rw-r----- oracle dba
m 99614723 0xb0d4d164 --rw-rw---- oracle dba
m 12582917 0xb84cbc28 --rw-rw---- oracle dba
m 79691782 0x1058873f --rw------- oracle dba
m 638582792 0x78000382 --rw-rw-rw- root system
m 218103817 0x780003b7 --rw-rw-rw- root system
s 1 0x6202c477 --ra-r--r-- root system
s 6291461 0x0102c2d8 --ra------- root system
s 6 0xa100004b --ra-ra-ra- root system

Get a list of the running databases

$ ps -ef | grep pmon | grep -v grep
oracle 483334 1 1 Aug 16 - 6:46 ora_pmon_db03
oracle 1253476 1 0 Oct 31 - 2:00 ora_pmon_db01
oracle 2298042 1 0 Sep 05 - 11:07 ora_pmon_db02

Then, for each database, get the ipc information
$ export ORACLE_SID=db01
$ sqlplus / as sysdba
SQL> oradebug setmypid
Statement processed.
SQL> oradebug ipc
Information written to trace file.
SQL> oradebug tracefile_name
$ awk '/Shared Memory:/{getline;getline;print}' /u01/app/oracle/admin/db01/udump/db01_ora_2625574.trc
5242882 0x41d2ba80
$ export ORACLE_SID=db02
$ sqlplus / as sysdba
SQL> oradebug setmypid
Statement processed.
SQL> oradebug ipc
Information written to trace file.
SQL> oradebug tracefile_name
$ awk '/Shared Memory:/{getline;getline;print}' /u01/app/oracle/diag/rdbms/db02a/db02/trace/db02_ora_2441408.trc
99614723 0xb0d4d164
$ export ORACLE_SID=db03
$ sqlplus / as sysdba
SQL> oradebug setmypid
Statement processed.
SQL> oradebug ipc
Information written to trace file.
SQL> oradebug tracefile_name
$ awk '/Shared Memory:/{getline;getline;print}' /u01/app/oracle/diag/rdbms/db03b/db03/trace/db03_ora_2617416.trc
12582917 0xb84cbc28

Compare it with the first list, and if you are absolutely sure to do what you are doing, remove the oracle segments that are not assigned to any database instance with ipcrm. If possible try first to figure out where they come from and do a shutdown abort of the not-correctly-stopped database.

Ok, with ipcrm

$ ipcrm -m 79691782

I removed the segment that apparently does not relate to any running instance

This could help you if you are really forced to remove some shared memory segments and you cannot afford shutting down other databases.

linux unix

Check if it a program is already running in Unix

There is more than one way to do it, the safe is probably to check if /home/lsc/OH_YES_I_AM_RUNNING exists and believe it. This is called the file.PID method and is widely used (Apache used to use it since a long long time). It needs file. It needs cleanup if you reboot your server in the middle of something (and surely you do not want to delete old pid files yourself)

Ok, often you see this :

ps -ef | grep program

There you list all processes and check the lines that contain program. So some does a vi program or anything worse (emacs?), you will get more rows than needed.

Maybe it is fine to run program with different arguments, this must be decided.

Well, take a simple test case : and :
while :
date > /dev/null

let’s try to use ps

$ nohup ./ &
$ nohup ./ &
$ jobs
[2] + Running nohup ./ &
[1] - Running nohup ./ &
$ ps -ef | egrep 'x[12]'
u22 9240796 6226164 30 14:56:52 pts/2 0:00 /bin/ksh ./
u22 20840608 6226164 31 14:56:48 pts/2 0:01 /bin/ksh ./

So fine so good, I see I have one instance of each program.

Let’s try to see if the results are consistent over time :

$ n=9999;while :
ps -ef |
egrep 'x[12].sh'>f
if [ $(wc -l $n"

Fri Oct 28 15:01:01 CEST 2011
u22 9240796 6226164 32 14:56:52 pts/2 0:14 /bin/ksh ./
u22 20840608 6226164 28 14:56:48 pts/2 0:14 /bin/ksh ./
==> 2

Fri Oct 28 15:01:08 CEST 2011
u22 9240796 6226164 50 14:56:52 pts/2 0:14 /bin/ksh ./
==> 1

Fri Oct 28 15:01:09 CEST 2011
u22 9240796 6226164 52 14:56:52 pts/2 0:14 /bin/ksh ./
u22 20840608 6226164 53 14:56:48 pts/2 0:15 /bin/ksh ./
==> 2

Fri Oct 28 15:01:17 CEST 2011
u22 9240796 6226164 40 14:56:52 pts/2 0:15 /bin/ksh ./
u22 10944520 9240796 0 15:01:17 pts/2 0:00 /bin/ksh ./
u22 20840608 6226164 31 14:56:48 pts/2 0:16 /bin/ksh ./
==> 3

the fact that a subshell (pid 10944520 ) of x2 appear is not a problem for me. I have much more of a problem at 15:01:08 where x1 disappeared !

Conclusion : you cannot trust ps

linux unix

shell and list of files

How do you loop thru a list of files?

For instance you want to archive than delete all pdf documents in the current directory :

Bad practice :

tar cvf f.tar *.pdf
rm *.pdf

There are multiple issue with the command above

1) new files could come during the tar, so the rm will delete files that have not been archived

filelist=$(ls *.pdf)
tar cvf f.tar $filelist
rm $filelist

2) if there is no file, tar and rm will return an error

filelist=$(ls|grep '\.pdf')
if [ -n "$filelist" ]
tar cvf f.tar $filelist
rm $filelist

3) this will not work for long list (above 100k documents)

filelist=/tmp/filelist.$(date "+%Y%m%d%H%M%S").$$.$RANDOM
ls|grep '\.pdf' > $filelist
if [ -s "$filelist" ]
tar cvfL f.tar $filelist
for f in $(

As you see, this require special handling. tar for instance use the -L option to accept a list of files, rm could delete files one by one (or in bunches with xargs -L).

This 100'000 limit (the limit may vary for your shell/os) is something that often gets forgotten.

Typical error that could occur are

ksh: no space
bash: Arg list too long


pstree in AIX

For those who do not want to download some linuxlike freeware on your aix box, use ps -T πŸ™‚

ps -fT 2412672
oracle 2412672 1 0 Sep 05 - 0:00 /u01/app/oracle/product/OAS
oracle 630956 2412672 0 Sep 05 - 6:11 \--/u01/app/oracle/prod
oracle 1347672 630956 0 Sep 05 - 15:32 |\--/u01/app/oracle/
oracle 1437836 630956 0 Sep 05 - 1:02 |\--/u01/app/oracle/
oracle 880820 1437836 0 Sep 05 - 0:32 | |\--/u01/app/ora
oracle 1036532 1437836 0 Sep 05 - 0:00 | |\--/u01/app/ora
oracle 1134796 1437836 0 Sep 05 - 0:01 | |\--/u01/app/ora
oracle 1343712 1437836 0 Sep 05 - 0:33 | |\--/u01/app/ora
oracle 1368166 1437836 0 Sep 05 - 1:11 | |\--/u01/app/ora
oracle 1384684 1437836 0 Sep 05 - 0:33 | |\--/u01/app/ora
oracle 1392862 1437836 0 Sep 05 - 0:32 | |\--/u01/app/ora
oracle 1396898 1437836 0 Sep 05 - 0:33 | |\--/u01/app/ora
oracle 1482978 1437836 0 Sep 05 - 0:32 | |\--/u01/app/ora
oracle 1527890 1437836 0 Sep 05 - 0:00 | |\--/u01/app/ora
oracle 1781798 1437836 0 Sep 05 - 0:32 | |\--/u01/app/ora
oracle 2195474 1437836 0 Sep 26 - 0:13 | \--/u01/app/ora
oracle 1626296 630956 0 Sep 05 - 13:49 \--/u01/app/oracle/

dba linux unix

Generate network graph from command line

I recently wrote on gnuplot, today I tried another command line utility to generate graphs, graphviz, version 2.24.0 on AIX5L.

Pretty straightforward syntax :
echo "digraph Emp {"
sqlplus -s -L scott/tiger << EOF set pages 0 lin 120 hea off feed off select ename ||'->'||
(select ename from emp where empno=e.mgr) || ';'
from emp e where mgr is not null;
echo "}"
)| neato -Tpng | uuencode Emp.png | mailx

(or neato -Tpng -o Emp.png to save locally, or -Tps|lp, etc…)

linux sqlplus unix

send graph per mail from sqlplus

How to send a graph with a single command from your database to your mail in Unix?

I tried this (gnuplot is available for Solaris, AIX and most Unix derivates) :

echo '
set hea off pages 0 feed off
prom set title "salaries of EMP"
prom unset key
prom unset xtics
prom unset xlabel
prom set term png
prom plot "-" with circle
select row_number() over (order by sal),sal from emp;
prom e
prom quit
' |
sqlplus -s scott/tiger |
gnuplot |
uuencode emp.png |

The picture is sent as attachment.

If you want to embed your png in an HTML mail, use sendmail

/usr/sbin/sendmail -t <Below a graph...

Content-Type: image/png
Content-Disposition: inline; filename="png.png"
Content-Transfer-Encoding: base64


Sendmail syntax and more tips on : Sending Emails With Sendmail – Part 3


vi large files

Once upon a time a colleague asked me if there is a better editor than vi installed on my db server. Well, I was not really about arguing the benefit of ed (less memory usage, no useless error message). But one advantage of ed was (I believed) the ability to read large files

$ vi alert_DB01.log
Out of memory saving lines for undo - try using ed
"alert_DB01.log" 612637 lines, 20458891 characters
Thu Aug 25 16:08:20 2011
LNS: Standby redo logfile selected for thread 1 sequence 6898 for destination LOG_ARCHIVE_DEST_2

of course most of readers are using some clicky fancy tools with colors and/or unlimited undo.

Well, for those who use vi on AIX, check this

$ vi -y 9999999 alert_DB01.log
Tue Oct 26 13:59:12 2010
Starting ORACLE instance (normal)
sskgpgetexecname failed to get name
Picked latch-free SCN scheme 3
Autotune of undo retention is turned on.
ILAT =27
"alert_DB01.log" 612662 lines, 20459538 characters

Oh yes !!!

linux tuning unix

scp tuning

I twitted yesterday :

copying 1TB over ssh sucks. How do you fastcopy in Unix without installing Software and without root privilege?

I got plenty of expert answers. I have not gone to far in recompile ssh and I did not try plain ftp.

Ok, let’s try first to transfer 10 files of 100M from srv001 to srv002 with scp :

time scp 100M* srv002:
100M1 100% 95MB 4.5MB/s 00:21
100M10 100% 95MB 6.4MB/s 00:15
100M2 100% 95MB 6.0MB/s 00:16
100M3 100% 95MB 4.2MB/s 00:23
100M4 100% 95MB 3.4MB/s 00:28
100M5 100% 95MB 4.2MB/s 00:23
100M6 100% 95MB 6.4MB/s 00:15
100M7 100% 95MB 6.8MB/s 00:14
100M8 100% 95MB 6.8MB/s 00:14
100M9 100% 95MB 6.4MB/s 00:15

real 3m4.50s
user 0m27.07s
sys 0m21.56s

more than 3 minutes for 1G.

I got hints about the buffer size, about SFTP, about the cipher algorythm, and about parallelizing. I did not install new software and I have a pretty old openssh client (3.8). Thanks to all my contributors tmuth, Ik_zelf, TanelPoder, fritshoogland, jcnars, aejes, surachart, and the ones the will answer after the writting of this blog post…

Ok, let’s try a faster algorythm, with sftp (instead of scp), a higher buffer and in parallel
$ cat batch.ksh
echo "progress\nput 100M1" | sftp -B 260000 -o Ciphers=arcfour -R 512 srv002&
echo "progress\nput 100M2" | sftp -B 260000 -o Ciphers=arcfour -R 512 srv002&
echo "progress\nput 100M3" | sftp -B 260000 -o Ciphers=arcfour -R 512 srv002&
echo "progress\nput 100M4" | sftp -B 260000 -o Ciphers=arcfour -R 512 srv002&
echo "progress\nput 100M5" | sftp -B 260000 -o Ciphers=arcfour -R 512 srv002&
echo "progress\nput 100M6" | sftp -B 260000 -o Ciphers=arcfour -R 512 srv002&
echo "progress\nput 100M7" | sftp -B 260000 -o Ciphers=arcfour -R 512 srv002&
echo "progress\nput 100M8" | sftp -B 260000 -o Ciphers=arcfour -R 512 srv002&
echo "progress\nput 100M9" | sftp -B 260000 -o Ciphers=arcfour -R 512 srv002&
echo "progress\nput 100M10" | sftp -B 260000 -o Ciphers=arcfour -R 512 srv002&
$ time batch.ksh
real 0m19.07s
user 0m12.08s
sys 0m5.86s

This is a 1000% speed enhancement πŸ™‚

linux sqlplus unix windows

What is the current setting of NLS_LANG in sqlplus?

I just learnt a neat trick from Oracle Support.

How do you see the current value of NLS_LANG in SQLPLUS ?

HOST is not the right answer.


SQL> host echo $NLS_LANG



The correct setting is revealed by @.[%NLS_LANG%]

SP2-0310: unable to open file ".[AMERICAN_AMERICA.WE8ISO8859P1]"


SP2-0310: unable to open file ".[AMERICAN_AMERICA.WE8ISO8859P1]"

It could well be that both return the same answer, but not necessarly, as shown above.

The unix discrepancy is related to the subshell created by HOST. The subshell may read some .profile and overwrite the value of NLS_LANG

In Windows, the NLS_LANG setting may be set by sqlplus according to some registry entries

linux unix

Time offset in Unix

What is the time offset of the current date in Unix?

perl -e '
if($gd<$ld){$d+=24}; if($gd>$ld){$d-=24}
print ($d."\n")'

Am I in summer (DST)?
perl -e 'if((localtime)[8]){print"yes"}else{print "no"}'

linux unix

to cvs or to subversion

First surprise, after migration, the size of my subversion folder is double the size of my cvs folder. With a bunch of 2Gb disks shared amoung dozens of unix persons, and regular reminders the current usage reached 100%, you will feel the pain of having each developers doublesizing its home directory…
The reason is a .svn/test-base directory containing a duplicate of your local copy.
The benefit there is that it reduces network usage, for instance when diffing with the working version.

Second surprise, not sure if it is a generic issue or related to the one above, a full checkout take ages πŸ™ .

svn and cvs command line interfaces almost have the same options, a bit like vi and vim -vi is my favorite- but the “tagging” massively differs. The is no cvs tag command in subversion, you use a different syntax.
cvs :

$ cd /home/lsc/CVS/proj1/source/pkg
$ cvs tag MYTAG helloworld.pkb helloworld.pks

now enjoy the power of subversion! [updated as I found the –parents option]

$ cd /home/lsc/SVN/proj1/source/pkg
$ svn copy --parents helloworld.pkb -m "my first svn tag"
$ svn copy --parents helloworld.pks -m "my first svn tag"

I always loved the cvs rename command
$ cvs rename pkg1.pkb pkg2.pkb
cvs [rename aborted]: Remote server does not support rename

Joke apart, in CVS I used to physically logon to the server and manually move the pkg1.pkb,v to pkg2.pkb,v
Renaming works in subversion :

$ svn move pkg1.pkb pkg2.pkb
A pkg2.pkb
D pkg1.pkb

Update: One more annoyance, you cannot checkout a single file πŸ™
subversion faq