📢 New Feature: Track your HPC cluster usage directly from the HPC User Portal. Cluster Usage Poster   •   Research Outputs: Please update your publications, theses, patents, and other research outputs carried out using Param Shakti and the GPU Cluster. Click here to submit/update.
Notice

PARAM Shakti

A state-of-the-art high-performance computing system based on a heterogeneous and hybrid architecture, combining Intel Xeon Skylake processors with NVIDIA Tesla V100 GPUs for computational efficiency across scientific applications.

Overview

PARAM Shakti is a heterogeneous, hybrid supercomputer based on Intel Xeon Skylake processors and NVIDIA Tesla V100 GPUs, designed and implemented by the HPC Technologies team at the Centre for Development of Advanced Computing (C-DAC). It comprises 2 master nodes, 8 login nodes, 10 service/management nodes, and 442 CPU+GPU compute nodes, with a total peak computing capacity of 1.66 PFLOPS.

Its compute nodes are connected via a Mellanox (EDR) InfiniBand interconnect network, and the system uses the Lustre parallel file system for high-throughput storage.

Hybrid CPU + GPU
Architecture
442
Compute Nodes
1.66 PFLOPS
Peak Performance
Mellanox EDR InfiniBand
Interconnect
Lustre
Parallel File System
C-DAC
Designed & Implemented By

Node Configuration

Full Hardware Specifications
Popular

Master Nodes

Master Nodes supervise and coordinate the cluster. They manage hardware health, workloads, and track utilization across all components.

  • 2 × Intel Xeon SKL G-6148
  • Cores = 40, 2.4 GHz
  • Total Cores = 80 cores
  • Memory = 384 GB
  • Total Memory = 768 GB
  • HDD = 900 GB

Login Nodes

Login Nodes act as user entry points. They support tasks like file transfers, editing scripts, and job submissions, with time and memory limits.

  • 2 × Intel Xeon SKL G-6148
  • Cores = 40, 2.4 GHz
  • Total Cores = 320 cores
  • Memory = 384 GB
  • Total Memory = 3,072 GB
  • HDD = 900 GB

Service Nodes

Service Nodes handle job scheduling and cluster services. They maintain reliability and ensure smooth day-to-day operation of PARAM Shakti.

  • 2 × Intel Xeon SKL G-6148
  • Cores = 40, 2.4 GHz
  • Total Cores = 240 cores
  • Memory = 384 GB
  • Total Memory = 2,304 GB
  • HDD = 900 GB

CPU Compute Nodes

CPU Compute Nodes are the workhorses of PARAM Shakti. They execute both interactive and batch jobs with local SSDs for fast scratch storage.

  • 2 × Intel Xeon SKL G-6148
  • Cores = 40, 2.4 GHz
  • Total Cores = 15,360 cores
  • Memory = 192 GB
  • Total Memory = 73,728 GB
  • SSD = 480 GB

High Memory Nodes

High Memory Nodes provide extended RAM per node, enabling simulations and jobs with very large memory requirements beyond standard compute nodes.

  • 2 × Intel Xeon SKL G-6148
  • Cores = 40, 2.4 GHz
  • Total Cores = 1,440 cores
  • Memory = 768 GB
  • Total Memory = 27,648 GB
  • SSD = 480 GB

GPU Compute Nodes

GPU Compute Nodes combine CPUs with NVIDIA V100 GPUs. CUDA and OpenCL optimized applications achieve huge speedups for AI and HPC workloads.

  • 2 × Intel Xeon SKL G-6148
  • Cores = 40, 2.4 GHz
  • Total Cores = 880 cores
  • Memory = 192 GB
  • Total Memory = 4,224 GB
  • 2 × NVIDIA V100 (16 GB each)

Storage

PARAM Shakti uses the Lustre parallel file system. It offers scalable and reliable storage with high throughput, ideal for large scientific workloads.

  • Primary Storage = 2.1 PiB
  • Archival Storage = 500 TiB
  • Throughput = 50 GB/s

Operating System

The cluster runs on Linux (CentOS 7.6), a stable operating system widely used in HPC. It provides compatibility and reliable performance for users.

  • OS = Linux
  • Distribution = CentOS 7.6
Get Started

Ready to Run Your Research on PARAM Shakti?

Apply for an HPC account and access India's premier supercomputing facility at IIT Kharagpur.

HPC Account Quick Start Guide

Network

System Architecture

NSM/HPC users reach the cluster through a firewall into the login/compute tier, which fans out across four dedicated networks — Secondary (1G), 10G, IPMI, and Infiniband — to the switch tier, and onward to the service, master, and storage nodes.

PARAM Shakti network architecture — login/compute/GPU nodes connect through Secondary (1G), 10G, IPMI, and Infiniband networks to the service, master, and storage nodes. NSM HPC Users Firewall Login Nodes (8 no.) Compute Nodes (384 nos.) ⋮ Compute with High Memory (36 nos.) ⋮ GPU Nodes with NVIDIA V100 (22 nos.) ⋮ Secondary Communication IPMI 10 G (2 nos.) L2 - Infiniband EDR 100 Gbps (18 nos) ⋮ L1 - Infiniband EDR 100 Gbps (28 nos.) ⋮ Service/Management Nodes (8 nos.) ⋮ Master Nodes (2 nos.) Storage Primary Storage (1.5 PiB) Archival Storage (500 TiB) Secondary (1G) 10 G IPMI Infiniband

Param-Seva

Software Stack
HPC Programming Tools Performance Monitoring HPCC IMB/OSU IOR HPCG C-DAC Tools IDE CAPC
Visualization Tools Ferret GrADS ParaView VisIt/ VMD
Application Libraries NetCDF/ HDF/ etc. Math Libraries Python Libraries GNU Scientific Library CHReME
Development Tools Intel Cluster Studio GNU CUDA Toolkit/ OpenACC
Communication Libraries Intel MPI MVAPICH2 Open MPI PGAS C-Chakshu
Middleware Applications and Management Cluster Monitoring/ Help Desk Ganglia C-DAC Tools Nagios XDMoD osTicket
Resource Management/ Scheduling/ Accounting SLURM SLURM Accounting HPC Tasks Automation Scripts
Provisioning OpenHPC (xCAT)
File System NFS Local FS (XFS) Lustre GPFS
Operating Systems Drivers OFED CUDA Network & Storage Drivers Cluster Checker Scripts
Operating System Linux (CentOS 7.x)

Need Help or Have Questions?

Our HPC support team is here to assist. Open a ticket or reach us by email.

Open a Ticket