One Hundred Hellos - Blog

iperf setup and results - 19 May 2007

Link to iperf home page http://dast.nlanr.net/Projects/Iperf/
Simple:
Server side: ./iperf -s
Client side: ./iperf -c

Explicit:
./iperf -s -P 0 -i 1 -p 5001 -l 8K -f m -t 10

./iperf -c

aria has a linksys EG1032
lascala has an onboard NVidia nForce Gbe
cantor has a Trendnet TEG-PCITXR (RTL-8169)
piaget has a D-Link DGE-530T

Results (lascala was busy.. redo, and add openfiler)
cantor-lascala: 418 Mbits/sec
lascala-lascala: 594 Mbits/sec
cantor-cantor: 2.25 Gbits/sec
aria-aria: 637 Mbits/sec
lascala-aria: 251 Mbits/sec
aria-cantor: 453 Mbits/sec
cantor-aria: 538 Mbits/sec
Link to iperf home page http://dast.nlanr.net/Projects/Iperf/
Simple:
Server side: ./iperf -s
Client side: ./iperf -c

Explicit:
./iperf -s -P 0 -i 1 -p 5001 -l 8K -f m -t 10

./iperf -c

aria has a linksys EG1032
lascala has an onboard NVidia nForce Gbe
cantor has a Trendnet TEG-PCITXR (RTL-8169)
piaget has a D-Link DGE-530T

Results (lascala was busy.. redo, and add openfiler)
cantor-lascala: 418 Mbits/sec
lascala-lascala: 594 Mbits/sec
cantor-cantor: 2.25 Gbits/sec
aria-aria: 637 Mbits/sec
lascala-aria: 251 Mbits/sec
aria-cantor: 453 Mbits/sec
cantor-aria: 538 Mbits/sec

transcode re-encode mp3 with lame - 13 May 2007

Doin this to shrink collection for Sylvie'e iPod.

yum install lame
yum install id3lib

# make parallel directory structure
find -type d -exec mkdir ../mp3_128/{} \;
# do actual transcoding
find -type f -name \*mp3 -exec
lame -b 128 --mp3input {} ../mp3_128/{} \;
# and now id3cp -1 -2 (necessaru caus lame does not bring over id3 tags
find -type f -name \*mp3 -exec id3cp -1 -2 {} ../mp3_128/{} \;

-=-= dual Core 2 = 4 cores ::
To accelerate on dual core 2 machines, manually split into 4(5) source directories, all of which encode to same mp3_128 target dfirectory.
Doin this to shrink collection for Sylvie'e iPod.

yum install lame
yum install id3lib

# make parallel directory structure
find -type d -exec mkdir ../mp3_128/{} \;
# do actual transcoding
find -type f -name \*mp3 -exec
lame -b 128 --mp3input {} ../mp3_128/{} \;
# and now id3cp -1 -2 (necessaru caus lame does not bring over id3 tags
find -type f -name \*mp3 -exec id3cp -1 -2 {} ../mp3_128/{} \;

-=-= dual Core 2 = 4 cores ::
To accelerate on dual core 2 machines, manually split into 4(5) source directories, all of which encode to same mp3_128 target dfirectory.

WWYDIYWNA - 21 January 2007

Considering options for rapid developpment.
Lifted considerations:
-common environment, simplified installtion recipes
-distribution
-source for java sdk, , can use java 1.5, from sun/ibm, 1.6 when it gets FLOSSED.
-svn to replace cvs
-planing: trac for details vs solodesk.
Considering options for rapid developpment.
Lifted considerations:
-common environment, simplified installtion recipes
-distribution
-source for java sdk, , can use java 1.5, from sun/ibm, 1.6 when it gets FLOSSED.
-svn to replace cvs
-planing: trac for details vs solodesk.

Trac on cantor - 21 January 2007

Installing trac on cantor
Decided to up2date instead of yum
http://trac.edgewall.org/wiki/TracOnRhel4
Change the directories from instructions:
Subversion repos
mkdir -p /var/lib/subversion/repos
svnadmin create --fs-type fsfs /var/lib/subversion/repos
Trac projects.
mkdir -p /var/lib/trac
trac
-admin /var/lib/trac/solotrac initenv
permissions, root.apache didn't cut it...
chown -R apache:apache /var/lib/trac
chown -R apache:apache /var/lib/subversion/repos

and put a logo into /usr/share/trac/htdocs/sologlobe.png by changing /var/lib/trac/solotrac/conf/trac.ini

for i in CONFIG_VIEW MILESTONE_ADMIN MILESTONE_CREATE MILESTONE_DELETE MILESTONE_MODIFY REPORT_ADMIN REPORT_CREATE REPORT_DELETE REPORT_MODIFY REPORT_VIEW ROADMAP_ADMIN ROADMAP_VIEW SEARCH_VIEW TICKET_ADMIN TICKET_APPEND TICKET_CHGPROP TICKET_CREATE TICKET_MODIFY TICKET_VIEW TIMELINE_VIEW TRAC_ADMIN WIKI_ADMIN WIKI_CREATE WIKI_DELETE WIKI_MODIFY WIKI_VIEW; do trac-admin solotrac

Then we svn,
And define milestones for hibernate and clustering unit/integration/load testing...
Installing trac on cantor
Decided to up2date instead of yum
http://trac.edgewall.org/wiki/TracOnRhel4
Change the directories from instructions:
Subversion repos
mkdir -p /var/lib/subversion/repos
svnadmin create --fs-type fsfs /var/lib/subversion/repos
Trac projects.
mkdir -p /var/lib/trac
trac
-admin /var/lib/trac/solotrac initenv
permissions, root.apache didn't cut it...
chown -R apache:apache /var/lib/trac
chown -R apache:apache /var/lib/subversion/repos

and put a logo into /usr/share/trac/htdocs/sologlobe.png by changing /var/lib/trac/solotrac/conf/trac.ini

for i in CONFIG_VIEW MILESTONE_ADMIN MILESTONE_CREATE MILESTONE_DELETE MILESTONE_MODIFY REPORT_ADMIN REPORT_CREATE REPORT_DELETE REPORT_MODIFY REPORT_VIEW ROADMAP_ADMIN ROADMAP_VIEW SEARCH_VIEW TICKET_ADMIN TICKET_APPEND TICKET_CHGPROP TICKET_CREATE TICKET_MODIFY TICKET_VIEW TIMELINE_VIEW TRAC_ADMIN WIKI_ADMIN WIKI_CREATE WIKI_DELETE WIKI_MODIFY WIKI_VIEW; do trac-admin solotrac

Then we svn,
And define milestones for hibernate and clustering unit/integration/load testing...

D-Link DGE-530T 10/100/1000 Gigabit Ethernet - 28 October 2006

How to install D-Link DGE-530T 10/100/1000 Gigabit Ethernet driver
on CentOS-4.4

/Linux driver/install-8_23.tar.bz2 from CD in D-Link box,
note: there is also a half-height bracket in the box.

make sure the link
ln -s ... /usr/src/linux is ok, especially if a new kernel is installed
When I built the driver :
ls -ltrd /usr/src/linux
lrwxrwxrwx 1 root root 37 Oct 27 21:26 /usr/src/linux -> /usr/src/kernels/2.6.9-42.0.3.EL-i686

/etc/modeprobe.conf - replace sky2 driver with installed sk98lin
...
alias eth0 forcedeth
alias eth1 sk98lin

Quoted from:
http://www.phoronix.com/lch/?k=entry&l=53&t=D-Link%20DGE-530T%2010/100/1000
I installed the sk98lin driver in the supplied CDROM.

I followed the installation instructions in the README file in /Linux driver/install-8_23.tar.bz2. I did not follow the procedure in /Linux driver/sk98lin.txt - I suspect that is for an older driver version.

I loaded the driver module and it worked first time.
How to install D-Link DGE-530T 10/100/1000 Gigabit Ethernet driver
on CentOS-4.4

/Linux driver/install-8_23.tar.bz2 from CD in D-Link box,
note: there is also a half-height bracket in the box.

make sure the link
ln -s ... /usr/src/linux is ok, especially if a new kernel is installed
When I built the driver :
ls -ltrd /usr/src/linux
lrwxrwxrwx 1 root root 37 Oct 27 21:26 /usr/src/linux -> /usr/src/kernels/2.6.9-42.0.3.EL-i686

/etc/modeprobe.conf - replace sky2 driver with installed sk98lin
...
alias eth0 forcedeth
alias eth1 sk98lin

Quoted from:
http://www.phoronix.com/lch/?k=entry&l=53&t=D-Link%20DGE-530T%2010/100/1000
I installed the sk98lin driver in the supplied CDROM.

I followed the installation instructions in the README file in /Linux driver/install-8_23.tar.bz2. I did not follow the procedure in /Linux driver/sk98lin.txt - I suspect that is for an older driver version.

I loaded the driver module and it worked first time.

CentOS-4.4 Samba config - 28 October 2006

Use System Settings/Server Settings/Samba Gui editor
Firewall / system-config-securitylevel / other ports
137:udp, 138:udp, 139:tcp, 445:tcp
Use System Settings/Server Settings/Samba Gui editor
Firewall / system-config-securitylevel / other ports
137:udp, 138:udp, 139:tcp, 445:tcp

Growing a raid5 array - 06 October 2006

Grow a raid5 array from 3 disks to 4.

mdadm - lvm(2) - evm - extfs

A- Growing a raid5 directly
get mdadm 2.5.2 or above
-found http://www.cse.unsw.edu.au/~neilb/source/mdadm/RPM/mdadm-2.5.3-1.i386.rpm
confir kernel >2.6.17

first create 4 devices in vmware. /dev/sd[bdce]1
mdadm --create --verbose /dev/md0 -l5 -n3 /dev/sdb1 /dev/sdc1 /dev/sdd1
# how big does the backup file need to be?
mdadm --grow /dev/md0 -n4 --backup-file=/tmp/raidbackup
mdadm -a /dev/md0 /dev/sde1

Then redo this with a filesystem on /dev/md0, or better ext3fs over lvm over /dev/md0

B- Alternatively using multiple raid5 pv's we can simply grow each pv independantly:
* pvmove /dev/md3 # Move all data off of /dev/md3
* vgreduce vg /dev/md3 # Remove /dev/md3 from the volume group
* pvremove /dev/md3 # Remove the LVM signature from /dev/md3
* mdadm --stop /dev/md3 # Stop the array
* mdadm --zero-superblock /dev/md3 # Remove the md signature from the disk
* mdadm --create /dev/md3 --level=5 --raid-devices=4 /dev/hda3 /dev/hdc3 /dev/hde3 /dev/hdg3 # Create the new array
* pvcreate /dev/md3 # Prepare /dev/md3 for LVM use
* vgextend vg /dev/md3 # Add /dev/md3 into the array

---- found this enabling note---
http://www.gagme.com/greg/linux/raid-lvm.php
For those now reading this, you can actually expand a raid 5 array with mdadm for software based raid in linux. At least as of kernel 2.6.17 and mdadm 2.5.2 it is possible. One easy way to stest this is to create 4 files with dd, I created 4 100MB files named raidfile[1-4]. Next, use losetup to set them up as loop back devices, I used loop[0-3]. To create the initial array do:
mdadm --create --verbose /dev/md0 -l5 -n3 /dev/loop0 /dev/loop1 /dev/loop2
The array should now be initializing, when it is finished, increase the number of raid devices in the array by issuing the command:
mdadm --grow /dev/md0 -n4 --backup-file=/tmp/raidbackup
This part requires a backup file in case power is lost during the reordering of data, this backup file has to be used to assemble the raid array in that case.
Next, to add the other drive to the array use:
mdadm -a /dev/md0 /dev/loop4
the new drive will then be added to the array with no data lose hopefully. During my testing it has worked flawlessly so far, but as for using it on a production box its probably not recommended unless you have backups. You do keep backups correct?
--------------------------------------------------
Grow a raid5 array from 3 disks to 4.

mdadm - lvm(2) - evm - extfs

A- Growing a raid5 directly
get mdadm 2.5.2 or above
-found http://www.cse.unsw.edu.au/~neilb/source/mdadm/RPM/mdadm-2.5.3-1.i386.rpm
confir kernel >2.6.17

first create 4 devices in vmware. /dev/sd[bdce]1
mdadm --create --verbose /dev/md0 -l5 -n3 /dev/sdb1 /dev/sdc1 /dev/sdd1
# how big does the backup file need to be?
mdadm --grow /dev/md0 -n4 --backup-file=/tmp/raidbackup
mdadm -a /dev/md0 /dev/sde1

Then redo this with a filesystem on /dev/md0, or better ext3fs over lvm over /dev/md0

B- Alternatively using multiple raid5 pv's we can simply grow each pv independantly:
* pvmove /dev/md3 # Move all data off of /dev/md3
* vgreduce vg /dev/md3 # Remove /dev/md3 from the volume group
* pvremove /dev/md3 # Remove the LVM signature from /dev/md3
* mdadm --stop /dev/md3 # Stop the array
* mdadm --zero-superblock /dev/md3 # Remove the md signature from the disk
* mdadm --create /dev/md3 --level=5 --raid-devices=4 /dev/hda3 /dev/hdc3 /dev/hde3 /dev/hdg3 # Create the new array
* pvcreate /dev/md3 # Prepare /dev/md3 for LVM use
* vgextend vg /dev/md3 # Add /dev/md3 into the array

---- found this enabling note---
http://www.gagme.com/greg/linux/raid-lvm.php
For those now reading this, you can actually expand a raid 5 array with mdadm for software based raid in linux. At least as of kernel 2.6.17 and mdadm 2.5.2 it is possible. One easy way to stest this is to create 4 files with dd, I created 4 100MB files named raidfile[1-4]. Next, use losetup to set them up as loop back devices, I used loop[0-3]. To create the initial array do:
mdadm --create --verbose /dev/md0 -l5 -n3 /dev/loop0 /dev/loop1 /dev/loop2
The array should now be initializing, when it is finished, increase the number of raid devices in the array by issuing the command:
mdadm --grow /dev/md0 -n4 --backup-file=/tmp/raidbackup
This part requires a backup file in case power is lost during the reordering of data, this backup file has to be used to assemble the raid array in that case.
Next, to add the other drive to the array use:
mdadm -a /dev/md0 /dev/loop4
the new drive will then be added to the array with no data lose hopefully. During my testing it has worked flawlessly so far, but as for using it on a production box its probably not recommended unless you have backups. You do keep backups correct?
--------------------------------------------------

galo on drupal and gallery2 - 27 August 2006

New implementation choice for galo:
vmware running ubuntu server with drupal and gallery2.
Here is the logo image:



New implementation choice for galo:
vmware running ubuntu server with drupal and gallery2.
Here is the logo image:



NSLU2 dl-slug - 01 August 2006

After a part swap from linksys, which turned out to be another Power supply failure...

Restoring the slug:
-confirm working
configure : ip 192.168.5.200
Server Name dl-slug
Workgroup : SOLO
Lang Support: Western Europe/Latin-1

Before updating to unslung firmware
test telnet into redboot as in
http://www.nslu2-linux.org/wiki/HowTo/TelnetIntoRedBoot
I used the following (Third manual method):
while ! ping -W 1 -c 1 192.168.0.1 2>&1 >/dev/null; do true; done && telnet 192.168.0.1 9000

From the readme : Unslung-6.8-beta-README.txt
I think It will be better to upgrade to linksys V23R63 first.
Having to do with the unslng disk being in USB port1 or port2.
And I intend putting the drive in port1.
But if I unsling a brand new drive, Then I will do as sugessted and unsling to port2.

HOLD EVERYTHING, will redo this as per readme with a brand new drive.....

Checksums for reference...

-update to latest firmware : NSLU2_V23R63.bin
423f65fb2121bbf9855d060dce29b3af NSLU2_V23R63.zip
007fa7a32c20e29411922b41370962c8 NSLU2_V23R63.bin
- or unslung
b0c1c53c8d707477ac8f14e23f344713 Unslung-6.8-beta-firmware.zip
After a part swap from linksys, which turned out to be another Power supply failure...

Restoring the slug:
-confirm working
configure : ip 192.168.5.200
Server Name dl-slug
Workgroup : SOLO
Lang Support: Western Europe/Latin-1

Before updating to unslung firmware
test telnet into redboot as in
http://www.nslu2-linux.org/wiki/HowTo/TelnetIntoRedBoot
I used the following (Third manual method):
while ! ping -W 1 -c 1 192.168.0.1 2>&1 >/dev/null; do true; done && telnet 192.168.0.1 9000

From the readme : Unslung-6.8-beta-README.txt
I think It will be better to upgrade to linksys V23R63 first.
Having to do with the unslng disk being in USB port1 or port2.
And I intend putting the drive in port1.
But if I unsling a brand new drive, Then I will do as sugessted and unsling to port2.

HOLD EVERYTHING, will redo this as per readme with a brand new drive.....

Checksums for reference...

-update to latest firmware : NSLU2_V23R63.bin
423f65fb2121bbf9855d060dce29b3af NSLU2_V23R63.zip
007fa7a32c20e29411922b41370962c8 NSLU2_V23R63.bin
- or unslung
b0c1c53c8d707477ac8f14e23f344713 Unslung-6.8-beta-firmware.zip

RPC options - 17 July 2006

Results:
Basic transport is workable now.
For initial tests php-o-lait will be simpler, (requires php 4.3 (.1)+ (not boole)

JSON-RPC
Server side -
JSON.php wrapped by me
php-o-lait includes JSON.php , and rpcproxy classes
json-rpc-java-1.0

Client side
php-o-lait rpcproxy classes
Hand written code in HttpPostTest
including cookie tracking.

Impedence mismatch: Types!
If php is in the chain, whoe dow we adapt the class hinting scheme from java client:
remove it from the source by hand ?
Implement Objects in php ?

Other transports:
REST, XMLRPC, SOAP
JAX-RPC -> supplanted by JAX-WS 2.0 (which supplants JAX-RPC), JAXB 2.0, and SAAJ 1.3.

What about http://xins.sourceforge.net/
handles all three: REST, XMLRPC, SOAP

for xmlrpc :
implementation list : http://www.xmlrpc.com/directory/1568/implementations
top candidates for java:
http://ws.apache.org/ or http://ws.apache.org/xmlrpc/index.html
http://xmlrpc.sourceforge.net/ (redstone , was Marquée)
top candidates for php: not yet reviewed
There is already a module in php :php-xmlrpc (on euler,abel,rack2)
article describing this extension :
http://www.devshed.com/c/a/PHP/Using-XMLRPC-with-PHP/
this is the xlmrpc-epi (epinions extension)
http://keithdevens.com/software/xmlrpc
http://scripts.incutio.com/xmlrpc/
http://phpxmlrpc.sourceforge.net/ (has some json stuff too, had trouble making it work ?)
but maybe that was on boole)
Results:
Basic transport is workable now.
For initial tests php-o-lait will be simpler, (requires php 4.3 (.1)+ (not boole)

JSON-RPC
Server side -
JSON.php wrapped by me
php-o-lait includes JSON.php , and rpcproxy classes
json-rpc-java-1.0

Client side
php-o-lait rpcproxy classes
Hand written code in HttpPostTest
including cookie tracking.

Impedence mismatch: Types!
If php is in the chain, whoe dow we adapt the class hinting scheme from java client:
remove it from the source by hand ?
Implement Objects in php ?

Other transports:
REST, XMLRPC, SOAP
JAX-RPC -> supplanted by JAX-WS 2.0 (which supplants JAX-RPC), JAXB 2.0, and SAAJ 1.3.

What about http://xins.sourceforge.net/
handles all three: REST, XMLRPC, SOAP

for xmlrpc :
implementation list : http://www.xmlrpc.com/directory/1568/implementations
top candidates for java:
http://ws.apache.org/ or http://ws.apache.org/xmlrpc/index.html
http://xmlrpc.sourceforge.net/ (redstone , was Marquée)
top candidates for php: not yet reviewed
There is already a module in php :php-xmlrpc (on euler,abel,rack2)
article describing this extension :
http://www.devshed.com/c/a/PHP/Using-XMLRPC-with-PHP/
this is the xlmrpc-epi (epinions extension)
http://keithdevens.com/software/xmlrpc
http://scripts.incutio.com/xmlrpc/
http://phpxmlrpc.sourceforge.net/ (has some json stuff too, had trouble making it work ?)
but maybe that was on boole)