**This is an old revision of the document!**
Ollama LLM
Ollama is an open source inference engine for running LLM models. https://ollama.com. To run a local LLM, you need two ingredients: the model itself, and the inference engine, which is a piece of software that can run the model. Conceptually, the inference engine processes the input (a text prompt), feeds it through the neural network of the model, and retrieves the response.
AHPCC is making this tool available to our users in a limited fashion so as not to impact the broad array of research tasks making their way thru the queues. As such, Ollama is only available on a single, specific node and should be accessed as follows:
srun -p pcon06 -N1 -n1 -q comp -t 01:00:00 -C 4titanv --pty /bin/bash
Once you are logged in to your session, you need to determine which LLM model you wish to use for the session. To see the available models:
$ ollama list NAME ID SIZE MODIFIED all-minilm:latest 1b226e2802db 45 MB 3 weeks ago bakllava:latest 3dd68bd4447c 4.7 GB 3 weeks ago codellama:latest 8fdf8f752f6e 3.8 GB 3 weeks ago deepseek-coder:latest 3ddd2d3fc8d2 776 MB 3 weeks ago deepseek-llm:latest 9aab369a853b 4.0 GB 3 weeks ago dolphin-mistral:latest ecbf896611f5 4.1 GB 3 weeks ago llama2:latest 78e26419b446 3.8 GB 3 weeks ago llama2-chinese:latest cee11d703eee 3.8 GB 3 weeks ago llama2-uncensored:latest 44040b922233 3.8 GB 3 weeks ago llama3:latest a6990ed6be41 4.7 GB 26 hours ago ....
More information about the various models can be seen here: https://ollama.com/library
There are two ways to utilize Ollama. Interactively, and as a single command query.
To use a session interactively…
ollama run <model>
When you are done with the session, exit with the command…
/bye>
For querying with a single command from outside the Ollama shell…
ollama run <model> <prompt query>
Here is a brief example of running a simple query with result…
ollama run llama3 How many states are in the USA? There are 50 states in the United States of America (USA).
You can do some interesting things with the tool. For example, you can specify a specific output format…
ollama run llama3 Please list in JSON format the names, state capitols, and populations of all 50 US states
Here is the list of 50 US states with their names, state capitals, and populations in JSON format:
```
{
"US States": [
{
"name": "Alabama",
"capital": "Montgomery",
"population": 4983317
},
{
"name": "Alaska",
"capital": "Juneau",
"population": 739795
},
{
"name": "Arizona",
"capital": "Phoenix",
"population": 7294493
},
....
If you want to save output to a file for later use, you use common redirects like so…
ollama run llama3 Please list all 50 US states >> states.txt
To summarize a short text file…
ollama run llama3 "$(cat nas.f90)" Please provide a brief summary of the file This is a Fortran code file, specifically a main program and subroutine for solving a partial differential equation (PDE) using a finite difference method. The code appears to be part of a larger numerical simulation package. The main program (`main`) defines several arrays and initializes some variables, then calls the `timestamp` subroutine to print the current date and time. The `timestamp` subroutine prints the current date and time in the format "DD MMM YYYY HH:MM:SS.AM/PM", where DD is the day of the month, MMM is the month name, YYYY is the year, HH is the hour, MM is the minute, SS is the second, and AM/PM indicates whether it's morning or afternoon. The rest of the code appears to be related to solving a PDE using a finite difference method. It defines several arrays (`f`, `x`, `y`) and performs calculations involving these arrays, possibly updating them based on some initial conditions (`d`, `e`) and boundary values. The code also seems to perform a "back sweep" solution at the end. Overall, this code is likely part of a larger numerical simulation package for solving partial differential equations using finite difference methods.
Input size for Ollama is somewhat limited (perhaps 1000 lines?), so it may be necessary to pare down input using other Linux tools. For example, a protein database file with 2200 lines won't fit…but paring the data down to the first 1000 lines does…
ollama run llama3 "$(head -n 1000 3nir.pdb)" Please provide a brief summary of the file This appears to be a PDB (Protein Data Bank) file, which contains structural information about a protein or protein complex. The file is formatted in a specific way and includes information such as: * Atom coordinates (3D positions) for each atom in the protein * Atom names and types (e.g. N, CA, C, O) * Chain identifiers (e.g. A, B) indicating which chain a particular residue belongs to
As we are all still learning how to efficiently and properly utilize LLM's in our daily life, you may be better served looking on the Ollama GitHub site under “Issues”, or sign up for their Discord server should you have encounter specific problems or if you should have unique use cases you'd like to explore…
