Wulver Filesystems

The Wulver environment is quite a bit like Lochness, but there are some key differences, especially in filesystems and SLURM partitions and priorities.

Wulver Filesystems are deployed with more attention to PI ownership / group efforts:

1. The $HOME directory is not intended for primary storage and has a 50GB quota. The main location for storing files is the group project directory which has 2TB of storage per PI group. To run the simulations, compilations, etc., users need to use a project directory which has 2TB of storage per PI group. Students can store their files under their corresponding PI’s UCID in the /project directory. For example, if PI’s UCID is doctorx, then students need to use the /project/doctorx/ directory.
2. Users can also store temporary files under the /scratch directory, likewise under a PI-group directory. For example, PI’s UCID is doctorx, so students need to use the /scratch/doctorx/ directory. Please note that the files under /scratch will be periodically deleted. To store files for longer than computations, please use the /project directory. Files under /scratch are not backed up. For best performance simulations should be performed in the /scratch directory. Once the simulation is complete, the results should be copied into the $HOME or /project directory. Files are deleted from /scratch after they are 30 days old. Users will get notified prior deletion so that they can review the files and move them to /project if required.

Filesystem	Purpose	Characterics	Backup policy	Total Size	Default Quota	Deletion Policy	Cost per TB
/home	Non-research user files, such as profile, history, etc., files not intended for sharing. Not for actual research files.	Kalray Pixstor / GPFS (expensive)	Daily	~1 PB	50GB per user (locked)	One year after owner leaves NJIT	Not possible to increase size, use /project or /research instead
/project	Active research by groups. Deployed as /project/$PI_UCID/$LOGIN/	Kalray Pixstor / GPFS (expensive)	Daily	~1 PB	2TB per group	TBD	$200 per TB for a duration of five years, minimum storage allocation of 5TB is required
/scratch	Temporary space for intermediate results, downloads, checkpoints, and such. MOVE YOUR RESULTS & IMPORTANT FILES TO /project or /research	Nvme (very expensive)	NEVER	~ 150 TB	10TB per group	Files deleted after 30 days or sooner if 80% full	No charge, but files are automatically deleted
/local	Very high speed temporary storage, can be accessed while running jobs on the node via $TEMP_DIR	Node-local SSD or NVME (very expensive)	NEVER	800 GB per node shared by all users	NONE	Files deleted after job completion	No charge, but files are automatically deleted
/research	Long term archive. Users can buy as much as they need. The users need to buy this space. Existing purchases/quotas will be kept over from Lochness.	NFS (inexpensive)	Daily	8 TB	Whatever the PI purchases.	TBD	$100 per TB per five years