What is FAbRIC

Derek Chiou (PI, UT Austin/Microsoft)

System admins: Xiaoyu Ma Alex Hsu (UT Austin), David Carver (UT Austin) and Laura Timm (UT Austin)

...

In addition to NSF support,  Intel (formerly Altera), Nallatech, Xilinx and Alpha-data have all committed FPGA/board donations.  Altera Intel has committed their entire suite of CAD tools, IBM has donated POWER8 servers, Intel has provided funds for the CAD tool servers, Microsoft has provided Catapult servers and funds for operating them, Nvidia has donated their GPUs, Bluespec has committed their Bluespec compiler, and Impulse Accelerated Technologies has commited their ImpulseC C-to-gates compiler. We are in active discussions with other companies who are interested in contributing their technologies to FAbRIC.

This material is based on work supported by the National Science Foundation under Grant No. 1205721 and generous donations and technical support from Alpha-data, AlteraIntel, Bluespec, IBM, ImpulseC, Intel, Microsoft, Nallatech, Nvidia and Xilinx.

In any publications that use FAbRIC, please include the following acknowledgment: "This material is based on work supported by the National Science Foundation under Grant No. 1205721."

User Access

If you would like to use the FAbRIC system, please first request and obtain a TACC account at https://portal.tacc.utexas.edu/account-request.  Make sure you get a confirmation email from TACC and that you can login to the user portal. Keep in mind you will need to login to your TACC account at least once to activate your account.

Forward that confirmation email to account-request@openfabric.org and the system admin of the platform you'd like to use, along with a brief reason why you want the account. Please provide your full legal name, your citizenship, and your current residential address, and your university name.  You should send the email from your university email address.

In addition, include the following statement in your email.

• "All of the code that I will pass through FAbRIC CAD tools (such as Verilog files, Bluespec files, etc.) and the files needed to process that code (such as Makefiles) is either already open source (GPL version 2 or above, BSD, or MIT licenses) or I have the right to make it open source and are hereby making all of the code that I pass through FAbRIC CAD tools open source by one of those licenses.  I will provide access to my source code to the CAD tool vendors and the FAbRIC administrators immediately.  The simplest way to do that is to provide a repository account to the FAbRIC administrators.  By default, the CAD tool vendors and/or the FAbRIC administrators agree not to publish the code publicly for at least 12 months.
   
  I acknowledge that the tools, servers, and FPGAs are potentially subject to export controls under U.S. and other applicable Government laws and regulations.  I will comply with these laws and regulations and agree to obtain all required Government authorizations.
  
  In any publications that use FAbRIC, I agree to include the following acknowledgment: [[[This material is based on work supported by the National Science Foundation under Grant No. 1205721.]]]"

If you want access to Microsoft Catapult, add the following to your email.

• "I acknowledge that my access to and use of Microsoft ‘s Project Catapult Academic Shell and Driver and related hardware provided by Microsoft is governed by, and subject to, the terms and conditions of the Microsoft Research License Agreement for the Microsoft Project Catapult Academic Shell and Driver. By accessing or using Microsoft Project Catapult materials, I represent and warrant that I have read the agreement, and I agree to be bound by it."

If you would like access to Intel FPGAs and tools (IBM Power8+CAPI, Microsoft Catapult), please also get a myAltera account  (https://www.altera.com/mal-all/mal-signin.html) and forward the confirmation email to account-request@openfabric.org as well.

You will receive an email once your fabric FAbRIC account is approved and becomes active. 

...

• Nallatech 385 A7 Stratix V Altera -based FPGA adapter
• Alpha-data 7V3 Virtex7 Xilinx-based FPGA adapter
• NVIDIA Tesla K40m GPGPU card

...

A quick user start guide: https://docs.google.com/document/d/1EkWfSXGTOqAwsiI2pzNj99ovmzB9d-ad-C6YrOaFvDo/edit#heading=h.qf46aqhtt3ic

Primary system admins: Xiaoyu Ma (xma@taccAlex Hsu (alex.hsu@tacc.utexas.edu) and David Carver (dcarver@tacc.utexas.edu)

Microsoft Catapult cluster

The Microsoft Catapult system consists of 432 two-socket Intel Xeon-based nodes, each with 64 GB of memory and an Altera Stratix V D5 FPGA with 8 GB of local DDR3 memory. FPGAs communicate to their host CPUs via a PCIe Gen3 x8 connection, providing 8GB/s guaranteed-not-to-exceed bandwidth, and each FPGA can read and write data stored on its host node using this connection. The FPGAs are connected to one another via a dedicated network using high-speed serial links. This network forms a two dimensional 6x8 torus within a pod of 48 servers, and provides low latency communication between neighboring FPGAs. This design supports the use of multiple FPGAs to solve a single problem, while adding resilience to server and FPGA failures.

Per Node:

• Two Xeon E5Two Intel® Xeon® E5-2450, 2.1GHz, 8-core, 20MB Cache, 95W
• 64GB RAM
• Four 2TB 7.2k 3G SATA 3.5"; Two 480GB 6G Micron SATA SSD 2.5"
• Intel 82599 10GbE Mezz Card
• Altera Stratix V FPGA Card
• Operating System: Windows Server 2012

The system is being brought up by TACC and Microsoft. Will go live soon. 

...

The gateway node: Dell R720 server, 64GB memory, 16 cores (Intel Xeon Intel® Xeon®  CPU E5-2670 @ 2.60GHz)

The compute node is a Convey MX system, with 128GB of RAM, 100GB of 64b granularity bandwidth, and 4 user "application" FPGAs. 

...

Note: The Convey MX system is being phased out due to old hardware and low usage.

Primary system admins: Xiaoyu Ma (xma@taccAlex Hsu (alex.hsu@tacc.utexas.edu) and David Carver (dcarver@tacc.utexas.edu) 

History News 

News (November 10, 2015): The second FAbRIC platform is IBM POWER8+CAPI.  We have nine of those systems that are being brought up right now.  We will be opening up the systems to a limited number of beta testers in the next month or two, and plan to open up to the general research community in the first quarter of next year. Thanks to IBM for donating the POWER8 servers, Altera/Nallatech for donating their FPGA boards, Nvidia for donating their GPU boards, and Xilinx/Alpha Data for donating their FPGA boards.

News (November 12, 2015): The third system type is the Microsoft Catapult platform.  384 data center servers, each equipped with an Altera Stratix V D5 FPGA are in TACC being brought up right now.  We will be opening up the systems to alpha users in the next month or two, and plan to open up to beta testers early next year.  Thanks to Microsoft and Altera for providing the systems, FPGAs, and tools.  Here is a blog post.  http://blogs.msdn.com/b/msr_er/archive/2015/11/12/project-catapult-servers-available-to-academic-researchers.aspx