KU Community Cluster
The KU Center for Research Computing (CRC) supports a shared Community Cluster for multidisciplinary research, housed in the NIH funded Advanced Computing Facility (ACF) data center, located in Nichols Hall, with an environmental and electrical infrastructure designed to accommodate current and future computing requirements. The KU Community Cluster supports the research of hundreds of faculty and researchers, postdoctoral associates, and graduate students. The KU Community Cluster is comprised of computing and data-storage resources well suited for distributed computing, shared memory parallel computations, and high-throughput serial computing. The cluster provides over 200 TeraFlops of peak performance with over 380 nodes, 9400 CPU cores, and over 50 TB of memory. Approximately 500 TB of network-accessible, high-performance disk is provided by a Panasas storage system. The Community cluster also provides over 100 NVIDIA GPGPUs and 20 Intel Xeon Phi coprocessors for ofﬂoading and accelerating computations and for software development. As is the case with most of the processors in the KU community cluster, these GPU and PHI accelerators are only available to all users for six-hour jobs when not in use by the owner groups who purchased them.
Advanced Computing Facility (ACF) Data Center
The ACF has environmental and electrical infrastructure designed to accommodate current and future computing requirements. The ACF is located in Nichols Hall, and was built in 2013 with a $4.7 million grant from the National Center for Research Resources at NIH, led by Steve Warren, who was the Vice Chancellor of Research (VCR) at the Ofﬁce of Research. The facility is secure, with access only to authorized personnel. The ACF provides capacity for 32 hardware racks of high-density computing equipment using in-row cooling, hot-aisle containment, and a modular uninterruptable power supply (UPS) capable of delivering 15kW to each rack. An on-site 1500 kW backup generator is installed which provides the assurance of 24/7 uptime capability.
The KU network is connected to the internet via 100 Gbps edge routers. This 100 Gbps network was extended to the KU Community Cluster, Research File Storage, and other locations on campus by a NSF CC* Networking Infrastructure Science DMZ grant, allowing friction-free movement of scientiﬁc data.
KU provides several storage solutions as part of KU’s commitment to providing a comprehensive data management plan for research data.
Immediate High Performance Cluster Storage
A high performance disk storage solution is directly connected to the cluster allowing high-throughput and high IOPs jobs to run without any bottlenecks. There is approximately 500TB of storage available which is distributed amongst all users of the cluster.
Intermediate Term Storage
KU Community Cluster users, as well as the KU research community, have access to the Research File Storage (ResFS) for intermediate (post-publication) data storage. Research File Storage (ResFS) provides easily accessible ﬁle sharing services to KU research projects, research groups and service labs. Any tenure-track faculty member, designated research center, department or principal investigator on a research grant can purchase ResFS storage. Stored data is accessible within the KU network or remotely via the KU Anywhere virtual private network (VPN). ResFS is protected by a RAID system. Data that is inadvertently deleted or otherwise lost can be recovered through self-service snapshots, which provide access to previous versions of ﬁles and directories for up to 30 days. ResFS data is mirrored in the University of Kansas Medical Center datacenter for disaster recovery purposes.
Long-Term Storage (Archiving)
The KU CRC and KU Information Technology have partnered to provide Research Archive Storage (RAS) for long-term storage of research data. RAS provides storage for data that has long-term value, but is not accessed for extended periods of time (months to years). Any tenure-track faculty, designated research center, department or principal investigator on a research grant can purchase RAS storage, which uses low cost, high density tape storage. The system will store data to two copies of tape once a month. The second copy of tape will be taken offsite for disaster recovery purposes.
The KU CRC is directed by Dr. Hoang Tran. Dr. Tran is assisted by two FTE staff, Mr. Riley Epperson and Mr. Bradley Fleming. CRC staff have a combined 30+ years of experience in managing large-scale computing clusters and networks.
When publishing, please include the following acknowledgment to the KU Center for Research Computing.
This work was supported by the HPC facilities operated by the Center for Research Computing at the University of Kansas.