Laurent Schneider

Good news, go there http://www.oracle.com/technology/software/products/database and there
Quick install guide

11gR2, news, unix release

Something I like in ksh is to change from /my/old/directory/path to /my/new/directory/path by typing cd old new. This does not work in bash

So I had to find a workaround

$ ksh
$ cd /app/oracle/product/11.1.0.6/database/rdbms/admin
$ cd 6 7
$ pwd 
/app/oracle/product/11.1.0.7/database/rdbms/admin
$ bash
$ cd 7 6
bash: cd: 7: No such file or directory
$ c() { cd ${PWD/$1/$2}; }
$ c 7 6
$ pwd
/app/oracle/product/11.1.0.6/database/rdbms/admin

linux, unix bash

This is one of the thing I hate to see, recursively changing everything to 777

If you want to give read access to all, then 644 is enough for files and 755 for directories. If you want to give execution permission too, you could give 755 to executable files.

Also sometimes you have files with special permission like suid or guid bit (to run as the owner instead of to run as the run user), so it would be best to use relative (go+r) to make the file readable to group and others.

Therefore I prefer relative change. Still there is one thing I do not want, is making every file executable…

Ok, here it is

chmod -R a+rX .

note the big X we change recursively all files and dirs to be readable, and we set the executable flag ONLY IF the file is executable for the owner !

linux, unix chmod

Ftp is seen as an old-time unsecure protocol. Many shops nowadays have switched or are switching to sftp. I will try to point out some differences :

Compatibility: none. the protocol is completly different. Multiple graphical clients however do support both mode. But the basic “ftp” client will not work with sftp.

Ascii mode: only in ftp. In sftp, it is always binary so there will be no conversion. Also no blocksize, recordlength or primary/secondary space for your OS/390 connections.

Interactive mode: similar. you enter your username and password, do cd, put and get. But to quit, by will not work in sftp Use quit or exit instead

Batch mode: different. Most probably you will end up setting a private/public key infrastructure for your ssh connection and use scp (secure copy). If you are using a ssh client like putty, it is possible to do something like pscp -l user -pw password server:file .

Security: sftp is secure, ftp is not.

Speed: ftp is fast, sftp is slow !

Oh NOOOOOOO!!!!! What’s the point is bringing something new if it is slower !!!

Ok, let’s try to download a 100m file:

$ time (echo "open dbsrv01
user oracle secret
bin
get 100m"|ftp -n )

real    0m24.673s
user    0m0.030s
sys     0m0.016s
$ time scp -q oracle@dbsrv01:100m .

real    1m46.978s
user    0m0.108s
sys     0m0.202s

it is about 4x slower! Is there anything we could do about it?

Well, maybe :

$ time scp -q -o Compression=yes oracle@dbsrv01:100m .

real    0m18.634s
user    0m0.748s
sys     0m0.452s

ssh/scp/sftp have a compression mode. If you are transferring your large files across a slow network, this may be an interesting option to consider !

OpenSSH homepage : http://www.openssh.org

linux, security, unix ssh

It is Friday, I wanted to give my readers some stuff for the week-end

Imagine that piece of code :

countlines() {
  c=$(sqlplus -s /nolog <<EOF | grep -v Connected
connect scott/tiger
set feed off head off
select count(*) from $1;
EOF
)
  echo there are $c lines in $1
}

countlines EMP
countlines DEPT

I can run this

time ./script1
there are 14 lines in EMP
there are 4 lines in DEPT

real    0m0.46s
user    0m0.06s
sys     0m0.09s

Sounds like a regular shell script. How could we optimize the countlines function? Well, we could create the connection only once and use coprocess pipe (with |& that is pipe ampersand)

sqlplus -s /nolog |&

print -p "connect scott/tiger"

read -p line
if [ $line != Connected. ]
then
  exit 1
fi

print -p "set feed off head off"

countlines() {
  print -p "select count(*) from $1;"
  read -p c
  echo "there is $c lines in $1"
}

countlines EMP
countlines DEPT

print -p disconnect

A two-ways pipe is opened with sqlplus. There is only one connect, and one disconnect.

Let’s check the performance :

$ time ./script2
there is 14 lines in EMP
there is 4 lines in DEPT

real    0m0.23s
user    0m0.00s
sys     0m0.01s

About twice as fast! Note the “Connected” output may not exist in recent version of sqlplus in silent mode. If you have a script that generates hundreds of connections, or which create a connection every 5 seconds or so, think about it

Enjoy your week-end

dba, unix coprosess pipe

In my previous post hide-password-from-ps-output-sql-loader I mentioned a way to pass the password to the loader thru a parameter file. As correctly suggested by Brian Tkatch, the password could be passed as standard input

sqlldr control=x.ctl silent=header,feedback <<EOF
scott/tiger
EOF
Username:

The Username: prompt is displayed  

How do we get rid of this ?

sqlldr control=x.ctl silent=header,feedback <<EOF | grep -v "^Username:"
scott/tiger
EOF

There is no output. But what’s the error code

echo $?
1

The return code is 1

This is not the error code from sqlldr, but the error code from grep !

Ok, here is the trick, a bit cryptic if you are not familiar with file descriptors

( ( (sqlldr control=x <<EOF;echo $? >&3) |grep -v "^Username:" >&4 ) 3>&1 |(read x;exit $x) )4>&1
scott/tiger
EOF
echo $?
0

The return code is 0

dba, linux, unix return code

I have been inspired by Chen to talk about bash…

I have been using ksh for many years, and I mean ksh88 not ksh93. The main reason is, I want my script to run the same way in any Unix flavor.

ksh93 has never been too much popular. I used it a few time to sleep half a second

echo sleep 0.5| /usr/dt/bin/dtksh

ksh has a lot of nice features. I just used one of them in my script :

$ typeset -u name
$ read name?"Enter your name : "
Enter your name : Laurent
$ echo $name
LAURENT

Way easier to force a variable to be uppercase rather than using echo|tr etc

Bash has some nice features too, but unfortunately every OS release come with a different bash version, which is the same pain as perl when you want to write a script that last for a decade or two.

Ok, just4fun

$ mkdir -p {a..z}/{1..9}
... create directories a/1 a/2 ... z/8 z /9
$ [[ text =~ t..t ]] 
... check if text matches regular expression t..t
$ echo ${text/pattern/string}
... replace pattern by string

The first two commands require bash3, the last is just fine with bash2.

Have fun shell-scripting

linux, unix shell

Let’s start with an example :

$ cat AI
#!/usr/bin/bash
while :
do
  echo "What's your name ?"
  read a
  if [ ! $a ]
  then
    break
  fi
  echo "Your name is :"
  echo $a
  echo
done
echo "Bye"

$ ./AI
What's your name ?
Jo
Your name is :
Jo

What's your name ?
Jack
Your name is :
Jack

What's your name ?
-e
Your name is :

What's your name ?

Bye

This artificial intelligence is not very intelligent, it cannot recognize me if I am called “-e” (it is Friday, have a look at Little Bobby Tables ).

Most unix tools consider
-- signals the end of options and disables further option processing

But not echo

$ touch -e
touch: invalid option -- e
Try `touch --help' for more information.
$ touch -- -e
$ ls -l -e
ls: invalid option -- e
Try `ls --help' for more information.
$ ls -l -- -e
-rw-r--r-- 1 lsc dba 0 Oct 31 15:44 -e
$ rm -e
rm: invalid option -- e
Try `rm ./-e' to remove the file `-e'.
Try `rm --help' for more information.
$ rm -- -e
$ echo -e

$ echo -- -e
-- -e

So, what’s the solution? well, probably not using “echo”, for example printf

$ (echo “What’s your name ?”
read a
echo “Your name is :”
printf “%s\n” “$a”)
What’s your name ?
-e
Your name is :
-e

linux, unix

updatedb and locate are parts of the findutils package, which exists since a long time on most unix / linux flavors.

Something I never get worked is to search for files on network drives with Cygwin, I always got :

find: /cygdrive/y/Favorites/Links/del changed during execution of find (old inode number -474324984, new inode number -44545478
4, filesystem type is system) [ref 1114]
find: /cygdrive/y/Favorites/Links/del changed during execution of find (old inode number -513303800, new inode number -47432498
4, filesystem type is system) [ref 1114]

which is because the file system is mounted and unmounted on demand and get new inodes.

So I wrote a hack in /usr/bin/updatedb

# $find $NETPATHS $FINDOPTIONS …
# lschnei2
(
for d in c: y:
do
cd $d
/cygdrive/c/OS/system32/attrib /s
done
) |
sed ’s,………..\(.\):,/cygdrive/\1,;y,\\,/,’ |
tr ‘\r’ ‘\0′ | tr -d ‘\n’

Then I can search efficiently

$ time updatedb

real 0m0.261s
user 0m0.265s
sys 0m0.015s

cygwin, linux, unix

RTFM is not the best answer …

man crontab
SunOS 5.10          Last change: 10 Nov 2005

User Commands                                          crontab(1)
     A crontab file consists of lines of  six  fields  each.  The
     fields  are  separated by spaces or tabs. The first five are
     integer patterns that specify the following:

     minute (0-59),
     hour (0-23),
     day of the month (1-31),
     month of the year (1-12),
     day of the week (0-6 with 0=Sunday).

so far so good. But read this

     Each of these patterns can be either  an  asterisk  (meaning
     all legal values) or a list of elements separated by commas.
     An element is either a number or two numbers separated by  a
     minus sign (meaning an inclusive range). Time specified here
     is interpreted in the timezone of the cron(1M) daemon, which
     is  set system-wide in /etc/default/init. Entries do not use
     the invoking user's timezone. The specification of days  can
     be  made  by  two  fields  (day  of the month and day of the
     week). Both are adhered to if specified as a  list  of  ele-
     ments. See .

See .

Let’s imagine you want to run a job today, Thursday May 15, 2008 at 2:15pm
You write:
15 14 15 05 4 /tmp/run-my-job

The job will be run Today at 2:15, next year Friday May 15, 2009 at 2:15pm and next week Thursday May 22, 2008 at 2:15pm…

The correct method is to use
15 14 15 05 * /tmp/run-my-job

Specifying both day of week and day of month is not what I expected

dba, linux, unix