Arkansas High Performace Computing Center [hpcwiki]

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
=== Ollama LLM ===

Ollama is an open source inference engine for running LLM models. [[https://ollama.com]].
To run a local LLM, you need two ingredients: the model itself, and the inference engine, which is a piece of software that can run the model. Conceptually, the inference engine processes the input (a text prompt), feeds it through the neural network of the model, and retrieves the response. Ollama is different than Chat-GPT and Gemini as it runs locally without accessing the internet for model data. The models are stored locally on Pinnacle.

AHPCC is making this tool available to our users in a limited fashion so as not to impact the broad array of research tasks making their way thru the queues. As such, Ollama is available and can be accessed as shown:

For a query to a model, you can do so via a login node as follows:

<code>
ollama-ahpcc run <model> <query>
</code>

You need to determine which LLM model you wish to use for the query. More information about the various models can be seen here: https://ollama.com/library
To see the available models:

<code>
$ ollama-ahpcc list

NAME                           	ID          	SIZE  	MODIFIED     
all-minilm:latest              	1b226e2802db	45 MB 	3 weeks ago 	
bakllava:latest                	3dd68bd4447c	4.7 GB	3 weeks ago 	
codellama:latest               	8fdf8f752f6e	3.8 GB	3 weeks ago 	
deepseek-coder:latest          	3ddd2d3fc8d2	776 MB	3 weeks ago 	
deepseek-llm:latest            	9aab369a853b	4.0 GB	3 weeks ago 	
dolphin-mistral:latest         	ecbf896611f5	4.1 GB	3 weeks ago 	 	
llama2:latest                  	78e26419b446	3.8 GB	3 weeks ago 	
llama2-chinese:latest          	cee11d703eee	3.8 GB	3 weeks ago 	
llama2-uncensored:latest       	44040b922233	3.8 GB	3 weeks ago 	
llama3:latest                  	a6990ed6be41	4.7 GB	26 hours ago	
....
</code>

For querying with a single command from Ollama...

<code>
ollama-ahpcc run <model> <prompt query>
</code>

Here is a brief example of running a simple query with result...

<code>
ollama-ahpcc run llama3 How many states are in the USA?

There are 50 states in the United States of America (USA).
</code>

You can do some interesting things with the tool. For example, you can specify a specific output format...

<code>
ollama-ahpcc run llama3 Please list in JSON format the names, state capitols, and populations of all 50 US states

Here is the list of 50 US states with their names, state capitals, and populations in JSON format:

```
{
  "US States": [
    {
      "name": "Alabama",
      "capital": "Montgomery",
      "population": 4983317
    },
    {
      "name": "Alaska",
      "capital": "Juneau",
      "population": 739795
    },
    {
      "name": "Arizona",
      "capital": "Phoenix",
      "population": 7294493
    },
    ....
</code>

If you want to save output to a file for later use, you use common redirects like so...

<code>
ollama-ahpcc run llama3 Please list all 50 US states >> states.txt
</code>

To summarize a short text file...

<code>
ollama-ahpcc run llama3 "$(cat nas.f90)" Please provide a brief summary of the file

This is a Fortran code file, specifically a main program and subroutine for solving a partial differential 
equation (PDE) using a finite difference method. The code appears to be part of a larger numerical simulation 
package.

The main program (`main`) defines several arrays and initializes some variables, then calls the `timestamp` 
subroutine to print the current date and time.

The `timestamp` subroutine prints the current date and time in the format "DD MMM YYYY HH:MM:SS.AM/PM", where DD 
is the day of the month, MMM is the month name, YYYY is the year, HH is the hour, MM is the minute, SS is the 
second, and AM/PM indicates whether it's morning or afternoon.

The rest of the code appears to be related to solving a PDE using a finite difference method. It defines several 
arrays (`f`, `x`, `y`) and performs calculations involving these arrays, possibly updating them based on some 
initial conditions (`d`, `e`) and boundary values. The code also seems to perform a "back sweep" solution at the 
end.

Overall, this code is likely part of a larger numerical simulation package for solving partial differential 
equations using finite difference methods.
</code>

Input size for Ollama is somewhat limited (perhaps 1000 lines?), so it may be necessary to pare down input using other Linux tools. For example, a protein database file with 2200 lines won't fit...but paring the data down to the first 1000 lines does...

<code>
ollama-ahpcc run llama3 "$(head -n 1000 3nir.pdb)" Please provide a brief summary of the file

This appears to be a PDB (Protein Data Bank) file, which contains structural information about a protein or 
protein complex. The file is formatted in a specific way and includes information such as:

* Atom coordinates (3D positions) for each atom in the protein
* Atom names and types (e.g. N, CA, C, O)
* Chain identifiers (e.g. A, B) indicating which chain a particular residue belongs to
</code>

It should be noted that for analyzing images and other various files, it requires a multi-modal model like llava or bakllava and results seems to be a bit mixed.

As we are all still learning how to efficiently and properly utilize LLM's in our daily life, you may be better served looking on the Ollama GitHub site under "Issues", or sign up for their Discord server should you have encounter specific problems or if you should have unique use cases you'd like to explore...

https://github.com/ollama/ollama/issues

https://discord.com/invite/ollama
Arkansas High Performace Computing Center [hpcwiki]

User Tools

Site Tools

Page Tools
