Because of the large number of requests for a simpler HCA driver, I
decided to clean up my "science project" driver for the Mellanox HCA
and release it. The driver is extremely immature but does enough for
IPoIB to work. I created a branch of the subversion repository in
https://openib.org/svn/gen1/users/roland/mthca, and this new "mthca"
driver code is in the directory
https://openib.org/svn/gen1/users/roland/mthca/src/linux-kernel/infiniband/hw/mthca
This driver currently cannnot be used as a full replacement for
Mellanox's driver, as it is quite incomplete and does not support many
critical features (more details below).
However, this driver was written from scratch as a fully native Linux
2.6 driver (in fact, because CLEAR_BITMAP was for some reason renamed
to bitmap_zero, you will need to build against Linux 2.6.6 or newer).
I believe that the driver can serve at least as a good proof of
concept and should clarify the OpenIB driver architecture. The stack
with the mthca low-level driver looks like:
ib_core etc. (ib_mad, ib_client_query etc. will be gradually
merged into a single ib_core module):
The core IB midlayer.
ib_mthca:
A standard Linux PCI driver for the Mellanox HCA, which plugs
into ib_core.
ib_ipoib, ib_sdp, ib_srp, etc.:
Upper-level protocol drivers that use the ib_core API to
provide useful protocols over IB.
The mthca driver currently does _far_ less than Mellanox's mature HCA
driver. The following list of limitations is probably incomplete but
should give some idea of the level of completeness:
No userspace support: ib_mthca provides _no_ facilities for
userspace access to the HCA, beyond the generic MAD services
provided by the ib_useraccess module.
Firmware commands are polled: ib_mthca does not implement
event-driven firmware commands (which limits it to one
outstanding command at a time).
Only UD transport: ib_mthca does not handle transports other than
UD correctly, so the only functional ULP is IPoIB.
No FMR support: ib_mthca does not support Mellanox-style "fast
memory regions" as used by SDP and SRP.
No SRQ support: ib_mthca does not support shared receieve queues.
Only handles successful completions: the code to process
completions with an error status is not written yet.
Limited support for asynchronous events: only port change
asynchronous events will be dispatched.
Only physical memory registration: the register virtual memory
region verb is not implemented.
UD address vectors are in host memory: putting address vectors in
HCA memory increases performance; ib_mthca does not implement
this yet.
Comments, questions and discussion are very welcome. I'm happy to get
even the most nitpicking suggestions on the code.
- Roland
--
To unsubscribe send an email with subject unsubscribe to
openib-general@xxxxxxxxxxx
Please contact moderator@xxxxxxxxxx for questions.