How to set IP over Infiniband.

With Infiniband be could gain more speed for data transfer between 2 nodes. There are a lot of ways to use Infiniband. In this particular case we will use it for TCP/IP connection. It’s not the fastest method for data transfer but it’s just a start. We could user iSCSI over Infiniband as well. We will work with QLE7342 Card on Debian Jessie.

  • First we will configure some kernel options on grub. We just need to modify this line in /etc/default/grub
    • GRUB_CMDLINE_LINUX_DEFAULT=”quiet”
  • for
    • GRUB_CMDLINE_LINUX_DEFAULT=”quiet options ib_qib singleport=1 krcvqs=2″
  • (Here you could find the configuration for your processor)

 

  • We update grub’s configuration:
    • update-grub
  • And reboot the server.

 

  • Once again on our server, we have to install the Infiniband tools and some dependencies:
    • apt-get install -y build-essential infiniband-diags opensm
  • Next we need to add the modules to /etc/modules:

# Protocol modules
# Common modules
rdma_ucm
ib_umad
ib_uverbs

# IP

Continue Reading

How to Mount an iScsi device.

These steps work with Debian Jessie and a Fibrechannel QLogic ISP2432 card.

  • Then we need to update:
    • apt-get update
  • And install the multpath package:
    • apt-get install multipath-tools
  • Once we have Multipath installed, we need to rescan looking for any device:
    •   for host in $(ls -1d /sys/class/fc_host/*); do echo “1” > ${host}/issue_lip; done
    •   for host in $(ls -1d /sys/class/scsi_host/*); do echo “- – -” > ${host}/scan ; done
  • After that we could check the devices:
    • multipath -ll
  • That command list all the devices detected.
  • To set the storage for autodetect and automont we have to find the ID with:
    • /lib/udev/scsi_id -g -u -d /dev/sdb
  • This is a simple example of the file /etc/multipath.conf that
  • Continue Reading