Llama 3 - Meta's Open Source Large Language Model

What is Llama 3

Llama 3 is an open-source large language model developed and released by Meta (formerly Facebook). It’s released under a commercially usable license, enabling local execution and fine-tuning.

Model Variations

Benchmarks

MMLU (Knowledge):
- GPT-4: 86.4%
- Llama 3 70B: 82.0%
- Llama 3 8B: 68.4%

HumanEval (Coding):
- GPT-4: 67.0%
- Llama 3 70B: 62.5%
- Llama 3 8B: 45.8%

Local Execution

Ollama

# Install Ollama
curl -fsSL https://ollama.ai/install.sh | sh

# Run Llama 3 8B
ollama run llama3:8b

# Interactive mode
>>> What is the capital of France?
The capital of France is Paris.

Python (transformers)

from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "meta-llama/Meta-Llama-3-8B-Instruct"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Explain quantum computing simply."}
]

input_ids = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt"
).to(model.device)

outputs = model.generate(
    input_ids,
    max_new_tokens=256,
    do_sample=True,
    temperature=0.7
)

response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

llama.cpp (C++ Implementation)

# Build llama.cpp
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
make

# Download GGUF model
# Get quantized version from Hugging Face

# Run
./main -m models/llama-3-8b-instruct.Q4_K_M.gguf \
  -p "What is machine learning?" \
  -n 256

Using as API Service

vLLM

from vllm import LLM, SamplingParams

llm = LLM(model="meta-llama/Meta-Llama-3-8B-Instruct")
sampling_params = SamplingParams(temperature=0.7, max_tokens=256)

prompts = [
    "Explain the theory of relativity",
    "Write a Python function to sort a list"
]

outputs = llm.generate(prompts, sampling_params)

for output in outputs:
    print(output.outputs[0].text)

OpenAI-Compatible API (Ollama)

# Start server
ollama serve

# OpenAI-compatible endpoint
curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama3:8b",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

// Use from JavaScript/TypeScript
import OpenAI from 'openai';

const openai = new OpenAI({
  baseURL: 'http://localhost:11434/v1',
  apiKey: 'ollama'  // Ollama requires no authentication
});

const response = await openai.chat.completions.create({
  model: 'llama3:8b',
  messages: [{ role: 'user', content: 'Hello!' }]
});

Fine-Tuning

Using LoRA

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import LoraConfig, get_peft_model

model = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Meta-Llama-3-8B",
    torch_dtype=torch.bfloat16
)

lora_config = LoraConfig(
    r=16,
    lora_alpha=32,
    target_modules=["q_proj", "v_proj"],
    lora_dropout=0.05
)

model = get_peft_model(model, lora_config)
# Run training...

Use Cases

RAG Application

from langchain_community.llms import Ollama
from langchain_community.vectorstores import Chroma
from langchain.chains import RetrievalQA

llm = Ollama(model="llama3:8b")
vectorstore = Chroma(...)

qa_chain = RetrievalQA.from_chain_type(
    llm=llm,
    chain_type="stuff",
    retriever=vectorstore.as_retriever()
)

result = qa_chain.invoke("What is our return policy?")

Code Generation

prompt = """
Write a Python function that:
1. Takes a list of numbers
2. Filters out negative numbers
3. Returns the sum of remaining numbers

Include docstring and type hints.
"""

response = llm.generate(prompt)

Hardware Requirements

Summary

Llama 3 is a powerful open-source large language model available for commercial use. It enables flexible usage including local execution, fine-tuning, and API service deployment. Ideal for privacy-focused applications and use cases requiring customization.