Arkansas High Performace Computing Center [hpcwiki]

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
===== Condo Queues =====

The Condo program allows individual users to purchase computing hardware with their own funds and use AHPCC staff to provide installation, power, cooling, networking and maintenance services.  The condo hardware owner has priority access to the condo nodes and has no job walltime restriction.  However, the condo nodes are also available for public use for up to 6 hour walltime jobs.  The 6 hour limit may be extended upon request if there are no condo owner jobs waiting in the queue.

All condo nodes are accessible via two queues: **''qcondo_public''** and **''qcondo''**.  The qcondo-public queue is unrestricted and available for all users.  The ''qcondo'' queue is restricted and available for use by condo node owners and their research groups **only**.


==== Trestles ====

To get a listing of all condo nodes on Trestles and their properties:

<code>
tres-l1:pwolinsk:$ for i in `node-queue-map |grep qcondo_public |cut -d" " -f3-`; do echo -n "$i "; pbsnodes $i |grep properties; done
tres0733      properties = mib,douglas,m768gb
tres0734      properties = mib,douglas,v4,m256gb
tres0735      properties = mib,douglas,v4,m256gb
tres0736      properties = mib,douglas,v4,m256gb
tres0833      properties = mib,abi,m3072gb
tres0834      properties = mib,laurent,v4,m64gb
tres0835      properties = mib,laurent,v4,m64gb
tres0836      properties = mib,laurent,v4,m64gb
tres-l1:pwolinsk:$ 
</code>


The properties can be specified following the ppn= directive, i.e. '' qsub -l nodes=1:ppn=1:<property> ''. To submit a job to ''qcondo_public'' queue on trestles to use 1 node with property m64gb (64 GB or RAM):

<code>tres-l1:pwolinsk:$ qsub -q qcondo_public -l walltime=6:00:00 -l nodes=1:ppn=32:m64gb hostname.pbs
283336.torque
tres-l1:pwolinsk:$ 
</code>

==== Razor ====

To get a listing of all condo nodes on Razor and their properties:

<code>
razor-l1:pwolinsk:$ for i in `node-queue-map |grep qcondo |cut -d" " -f3-`; do echo -n "$i "; pbsnodes $i |grep properties; done
compute0805      properties = mib,ib,gpu,k40gpu,m64gb
compute1153      properties = qib,ib,laurent,m24gb
compute1154      properties = qib,ib,laurent,m24gb
compute1155      properties = qib,ib,laurent,m24gb
...
compute3131      properties = mib,ib,pmillett,m64gb
compute3132      properties = mib,ib,pmillett,m64gb
compute3133      properties = mib,ib,v4,gpu,k80gpu,pmillettgpu,m128gb
compute3134      properties = mib,ib,v4,gpu,k80gpu,pmillettgpu,m128gb
compute3135      properties = mib,ib,v4,gpu,k80gpu,pmillettgpu,m128gb
...
compute3231      properties = mib,ib,nair,m64gb
compute3232      properties = mib,ib,nair,m64gb
razor-l1:pwolinsk:$ 
</code>

==== qcondo_public - Unrestricted Access ====
The ''**qcondo_public**'' queue can be used by all users.  To get a listing of the available nodes in that queue on Trestles:

<code>
tres-l1:pwolinsk:$ node-queue-map |grep qcondo_public
qcondo_public 8 tres0733 tres0734 tres0735 tres0736 tres0833 tres0834 tres0835 tres0836 
</code>

In a regular (non-condo) queue all nodes are identical; have same cpus, memory, etc. - have the same properites.  The ''**qcondo_public**'' queue is different.  Individual nodes where purchased by different users and have different properties.  To get a listing of properties for an individual node:

<code>
tres-l1:pwolinsk:$ pbsnodes tres0733 |grep properties
     properties = mib,douglas,m768gb

</code>

Below is an example of submitting a job to ''**qcondo_public**'' queue on Razor to use 2 node with property k80gpu (node with Nvidia K80 GPU):

<code>
razor-l1:pwolinsk:$ qsub -q qcondo_public -l nodes=2:ppn=24:k80gpu hostname.pbs 
1774072.sched
</code>

==== qcondo - Restricted Access ====

The ''**qcondo**'' queue should only be used by condo node owners and their research groups.  To submit jobs to this queue the owner property must be specified during job submission.  Below is an example of submitting a job to one of the nodes in the qcondo queue which belongs to the "douglas" condo group:

<code>tres-l1:pwolinsk:$ qsub -q qcondo -l walltime=200:00:00 -l nodes=1:ppn=32:douglas hostname.pbs
283336.torque
tres-l1:pwolinsk:$ 
</code>

**NOTE:** the owner property must be present when submitting to the ''qcondo'' queue.

==== Property Code Summary ====
<csv>
Property code, Meaning	
ib, Infiniband equipped node
mib, Mellanox Ininiband card
qib, QLogic Infiniband card
abi, condo owner code: Arkansas Bioinformatics Institute
douglas, condo owner code: Douglas group
itzaneta, condo owner code: Tzanetakis group
laurent, condo owner code:  Bellaiche group
nair, condo owner code: Nair group
pmillet, condo owner code: Millet group
xqian, condo owner code: Qian group:w
v4, Xeon Broadwell cpu
m64gb, node memory size 64GB
m128gb, node memory size 128GB
m256gb, node memory size 256GB
m512gb, node memory size 512GB
m768GB, node memory size 768GB
3072GB, node memory size 3TB
gpu, node equipped with a GPU card
k40gpu, node equipped with Nvidia K40 GPU(s)
k80gpu, node equipped with Nvidia K80 GPU(s)
</csv>
Arkansas High Performace Computing Center [hpcwiki]

User Tools

Site Tools

Page Tools
