This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision | ||
ollama_llm [2024/04/25 17:02] pwolinsk |
ollama_llm [2024/04/27 02:59] (current) jpummil |
||
---|---|---|---|
Line 4: | Line 4: | ||
To run a local LLM, you need two ingredients: | To run a local LLM, you need two ingredients: | ||
- | AHPCC is making this tool available to our users in a limited fashion so as not to impact the broad array of research tasks making their way thru the queues. As such, Ollama is only available | + | AHPCC is making this tool available to our users in a limited fashion so as not to impact the broad array of research tasks making their way thru the queues. As such, Ollama is available |
- | For a single | + | For a query to a model, you can do so via a login node as follows: |
< | < | ||
Line 12: | Line 12: | ||
</ | </ | ||
- | For an extended session with the inference engine (a string of successive queries)... | + | You need to determine which LLM model you wish to use for the query. More information about the various models can be seen here: https:// |
+ | To see the available models: | ||
< | < | ||
- | srun -p pcon06 -N1 -n1 -q comp -t 01:00:00 -C 4titanv --pty /bin/bash | + | $ ollama-ahpcc list |
- | </ | + | |
- | Once you are logged in to your session, you need to determine which LLM model you wish to use for the session. More information about the various models can be seen here: https:// | ||
- | To see the available models: | ||
- | |||
- | < | ||
- | $ ollama list | ||
NAME | NAME | ||
all-minilm: | all-minilm: | ||
Line 37: | Line 32: | ||
</ | </ | ||
- | + | For querying with a single command from Ollama... | |
- | There are two ways to utilize Ollama. Interactively, | + | |
- | + | ||
- | To use a session interactively... | + | |
- | + | ||
- | < | + | |
- | ollama run < | + | |
- | </ | + | |
- | + | ||
- | When you are done with the session, exit with the command... | + | |
- | + | ||
- | < | + | |
- | /bye | + | |
- | </ | + | |
- | + | ||
- | For querying with a single command from outside the Ollama | + | |
< | < | ||
- | ollama run < | + | ollama-ahpcc |
</ | </ | ||
Line 61: | Line 41: | ||
< | < | ||
- | ollama run llama3 How many states are in the USA? | + | ollama-ahpcc |
There are 50 states in the United States of America (USA). | There are 50 states in the United States of America (USA). | ||
Line 69: | Line 49: | ||
< | < | ||
- | ollama run llama3 Please list in JSON format the names, state capitols, and populations of all 50 US states | + | ollama-ahpcc |
Here is the list of 50 US states with their names, state capitals, and populations in JSON format: | Here is the list of 50 US states with their names, state capitals, and populations in JSON format: | ||
Line 97: | Line 77: | ||
< | < | ||
- | ollama run llama3 Please list all 50 US states >> states.txt | + | ollama-ahpcc |
</ | </ | ||
Line 103: | Line 83: | ||
< | < | ||
- | ollama run llama3 "$(cat nas.f90)" | + | ollama-ahpcc |
This is a Fortran code file, specifically a main program and subroutine for solving a partial differential | This is a Fortran code file, specifically a main program and subroutine for solving a partial differential | ||
Line 128: | Line 108: | ||
< | < | ||
- | ollama run llama3 " | + | ollama-ahpcc |
This appears to be a PDB (Protein Data Bank) file, which contains structural information about a protein or | This appears to be a PDB (Protein Data Bank) file, which contains structural information about a protein or |