PCI Express over fiber / cable is an architectural oppurtunity

It is now few years that my mind has been fascinated about the opportunities that the PCI Express specification provides at the system architecture level. My recent tests with a simple, one lane PCI Express extension kit using a 50 meter fiber confirmed that my vision is quickly coming reality.

First, two very simple rules about the COTS systems:

1. All industrial COTS cards that got a CPU on them get insufficient to turn a commercial Operating System (OS) within a three years time span
2. Industrial I/O cards can live much longer: ten year is not an exception!

Today, the bus has dematerialized and it has become a software protocol that we transfer encapsulated in a transmission protocol on a serialized media, which can be cable, fiber or wireless. So why not finally make a modular system where the slow and fast I/O problems are each separated in to their own entities and the ever changing CPU/OS in its own? Presented with a block diagram, we would get something like this:

The high end solution (fast disks, PCI-Express detector cards, and so on) would look equally modular:

There are some industrial grade 1U height rackable servers around. The following link is not the only one that I found, but the product comes with PCI Express x8 riser card:
http://www.stealthcomputer.com/industrial_rackmounts_sr1501datasheet.htm.
An example of a bus extension system could be the following product:
http://www.amplicon.co.uk/IPC/product/PCI-Rackmount-MAGMA-2906.cfm.

Where’s the beef ?

Wouldn’t the proposed system be more expensive? Sure, for a single purchase. But seeing the annoying fact that the CPUs and Operating Systems are aging much more rapidly than the I/O systems the Total Cost of Ownership (TCO) would be equal or less in the timespan of ten to fifteen years. Only the unit containing the CPU and the operating system needs to be changed while the PCI/PCIe extension unit will not be touched. The immediate saving in money is not substantial but what a relief for the maintenance and software upgrades!

 

Laptop as VGA console – second generation got it right!

Epiphan KVM2USB is a USB device that connects between your (portable) Linux or Windows PC and a headless VGA/XVGA enabled computer, turning the PC as Keyboard, Video Mouse (KVM) of the headless system. The following two pictures illustrates a typical usage:

Epiphan KVM2USB - second generation

 

Quick testing Adnaco S2 remote PCI / PCI Express fiber kit

Hardware contents in Adnaco S2 PCI/PCIe fiber kit

Recently I have been testing Adnaco S2 remote PCI/PCIe (PCI Express) over fiber kit.

Installation

Adnaco has done careful job and there is practically no hick-ups in the installation. At least there is no drivers to install and from the remote host’s point of view the entire operation is really transparent (Trenton MCXT PICMG1.3, motherboard BPX6719). I experienced some problems initially, after the installation when the test system refused to fire up. The problem was obviously related to the fact the remote PCI/PCIe extension chassis was not turned on. Here is a point of improvement, the remote chassis should turn automatically on, when the remote host fires up.

Test Arrangement

To test the performances, I moved the LSI SAS3041 RAID controller of the remote host into the Adnaco S2 extension chassis and replaced its slot with Adnaco’s H1 PCIe/Fiber interface. Between the two chassis I put 50 meters of multi-mode cable. The SFP transceivers are SR (for Short Range) type. As you can see from the below picture, I did not moved the actual SAS disks into the other chassis, there is no scientific reason to do so.. The LSI SAS3041 card is connected with three 15 krpm SAS disks. The embedded LSI controller is not used for RAID operations but the three disks create a software RAID-0 array as /dev/md0 device in Red Hat Linux 5.2 64-bit system. The below picture illustrates the test arrangement.

Performance

We are interested especially in the write performance of the RAID-0 array. We are using basically two tests, each writing big chunks of data of 4 GB into the RAID-0 array. We are not interested of the short cycle read-write operations and that is why I have no measurement data other than block write. The first tests consist of a test program written in C. It creates a memory block and using write() system call moves the 4 GB memory block into the RAID-0 array using Linux /dev/md0 mounted EXT2 file system:

• Local performance: 264 MB/s
• Same test with the LSI controller on remote Adnaco R1BP1: 160 MB/s

That’s great, since it is but 38 percent drop in performance and not 75 percent that could have followed from the fact that we have to switch from four PCI Express lanes into a single lane in the fiber. This means that the performance bottleneck in this particular application is rather in the SAS disk array, the LSI SAS controller and Linux file system and Linux RAID system.

Another test allows us to limit take out the Linux file system performance from the test. It is called “dd test“, from the GNU/Linux dd(1) utility. The test allows to write directly from the memory into the raw device, served by the Linux RAID system. The commands within the (test) script are:

Fill Random

dd if=/dev/random of=/dev/md0 bs=1G count=4

Fill Zero

dd if=/dev/zero of=/dev/md0 bs=1G count=4

Please note that the random filling is used between each /dev/zero test to make it less probable that the LSI device and/or SAS disk can cache the zero stream.

• Local performance: 324 MB/s
• Adnaco R1BP1: 175 MB/s

This is truly a great result from Adnaco! Let me remind you that theoretical maximum speed of one lane PCI Express (first generation) is 2.5 Gbit/s. With 10-bit payload, that makes theoretical transfer speed of 250 MB/s. With Adnaco we get up to 70 percent bandwidth occupation.

Conclusion

I am very satisfied with the above results. I have learned that the oversimplification in performance calculations is bad. You cannot just divide the local performance with the number of lanes when you move from eight lane PCI Express operation to one lane operation. You need to try what actually will happen in your specific problem. There is always so much software and firmware overhead that you have to make a test before you can tell. Adnaco S2 sorts out from the test as a stable product. I would like to see more slots, both PCI and PCI Express on the remote motherboard. The proposed enclosure should not be a “mini-tower” but a 19 inch, rackable, 4U height unit.

 

Simple Flash-like photo-album with CSS and JavaScript

I needed to make a single page “house-for-sale” announcement on the web. I did not want to take the use of the commercial reap-estate sites where all that is automatized but make a simple, but elegant page with modern web standards. My web design information was dating from the days where elements were actually table cells. I decided that there was a need for a modern, CSS  Layer based design. I wanted also to take the use of layered techniques to propose a small photo-album that would look a little bit like it would have been created with Flash. Below I will explain how the photo-album was created. Should you want to take a look at the final output already, check the French and English versions of my sales leaflet.

 

Nokia N95 Firmware Upgrade 31.0.014 – then 31.0.017

In my ongoing saga on my excellent, unbranded Nokia N95 smartphone : Now the PC Suite’s Nokia Software Updater (NSU) announced that firmware version 31.0.014 is available. I have been thinking to switch to N96 but after the announcement of N97 I guess that it is really not worth of it. Therefore I thought that since I need to keep the Good Old Faithful alive still for awhile, I would give the latest upgrade a go.

 

Installation of lm_sensors 3.x package for Intel Xeon temperature monitoring on RedHat 5

This modification is required because RedHat 5.1 (x86_64) has lm_sensors 2.x package(s) installed with no Xeon temperature monitoring device driver. You can verify this by running the sensors-detect progam and answer (YES – default) to all questions. The process will create a file /etc/sysconfig/lm_sensors.conf that determines which I2C device drivers are needed at startup:

 

Multi-Ethernet RHEL4 or RHEL5 installation stucked – how to get out

You should always use /usr/sbin/system-config-network to modify the entire configurtion of all interfaces at once. But it sometimes fails, if you forget to define something: for example sometimes the outcome will be that the /etc/sysconfig/network contains hostname localhost.localdomain.. In any case, the system does not boot anymore, but remains stucked on cups printer services startup, or something similar happens.

 

Sigma 70-300m APO zoom – problems with Canon EOS 450D

I am the model...

I recently purchased a new Canon EOS 450D (a.k.a. Digital Rebel XSi) as a digital alternative to my faithful companion, Canon EOS 10. I did my homework around my existing lenses but missed the point that several people have already experienced problems with my faithful 300 mm zoom lens, Sigma 70-300mm 1:4-5.6 APO Macro (ca.1996!), when used with digital EOS series. Apparently there is an upgrade available for certain Sigma lenses but I wrote to Sigma only after having purchased the 450D. Of course, my Sigma lens was not upgradeable.

 

Microphone jack does not work on Dell Inspiron 1525

I just bought identical Dell Inspiron 1525 laptops with Windows Vista 32-bit, EN-US. After the initial, Dell FAI (Fully Automated Installation) the front panel microphone jack of one of the computers did not work. After verifying that the headset microphone worked on another computer, I checked the recording level settings from the Control Panel -> Hardware and Sound -> Sound -> Manage audio devices -> Recording Tab -> (You can stop here, talk into the microphone and actually see the volume meter moving) -> Microphone/Line In -> Properties -> Levels Tab (how user friendly).

 

Iomega StorCenter 1TB (2×500GB) Setup

Iomega StorCenter 1TB
Netgear DG834PN

Surfing on the web before buying Iomega StorCenter 1TB was not very encouraging. There are quite a few people who consider the device noisy, overheating and difficult to connect. Because I have had good hardware experience in my work with Iomega’s 160 GB USB disk, I dediced to give them try.

It was clear before buying the product that the quality of Iomega software is not so great and their minimalist documentation suggest to customers a Mac-like, easy User Experience. This makes an impossible combination and I spent quite a while to prepare Windows Shares on the box. You can actually get it working somehow quite quickly but as soon as you try to make it work as you have planned, the