For the first time in login node

load conda and create a virtual environment

## Sam's recommended practice to install packages
## this is a combination of conda and venv

module load conda/2023-01-10-unstable 
conda activate base

## create venvs directory under ~ for now, later we'll use /grand/project/TFXcan or something like this
mkdir -p venvs/polaris/2023-01-10
python3 -m venv venvs/polaris/2023-01-10 --system-site-packages
source venvs/polaris/2023-01-10/bin/activate
python3 -m pip install --upgrade pip setuptools wheel

# now free to pip install packages as needed into an isolated env
# python3 -m pip install wandb numpy # etc

<aside> 👉🏽 Most of the work should be done on a compute node

</aside>

how to request a compute node

login to the login node and run

qsub -I -A AIHPC4EDU -l walltime=1:00:00 -l filesystems=home:grand -q debug

you can also try to get more hours using the preemtable queue

qsub -I -A AIHPC4EDU -l walltime=08:00:00 -l filesystems=home:grand -q preemptable

If it takes a while to get the compute node, you may want to check the number of free nodes in polaris

pbsnodes -avjS | grep free | wc -l

or check the web-based visualization of load in https://status.alcf.anl.gov/static/polarisDisp.html

polaris system automatically logs you into the compute node

activate the environments in the compute node

activate the base conda (created by the sysadmin) and activate your virtualenv where you installed your packages. you need to do this every time you start a compute node or login to the login node