Code LLAMA: AI Tool That Will Change Your Coding Life

write-and-debug-code-with-facebook-meta-llama

A New AI Tool for Coding. Do you want to learn coding or improve your coding skills? Do you wish you had a smart assistant that can help you write code faster and better? If yes, then you should know about Code LLAMA, a new AI tool for coding that is free for everyone to use.

What is Code LLAMA?

Code LLAMA (released by Meta on 24th August 2023) is a large language model (LLM) that can use text prompts to generate and discuss with code. A language model is a computer program that can understand and produce natural language, like English or Hindi. A large language model is a very powerful and complex language model that can handle many different tasks and domain problems.

Code LLAMA is built on top of Llama 2, which is one of the most advanced large language models in the world. Code LLAMA is specially trained on code and code-related data, which means it has enhanced coding capabilities.

It can generate code and natural language about code, from both code and natural language prompts. For example, you can ask Code LLAMA to “Write me a function that outputs the fibonacci sequence” and it will generate the code for you. You can also ask Code LLAMA to explain what a piece of code does or how to fix an error.

Code LLMA supports many of the most popular programming languages used today, such as Python, C++, Java, PHP, Typescript (Javascript), C#, Bash, and more. You can also use this for code completion and debugging, which means it can fill in the missing parts of your code or find and correct mistakes.

how-code-llama-works-with-three-models-architecture

Code LLAMA is available in three models. Each model has different sizes and features, depending on your needs and preferences.

Code LLMA – Python: specialized for Python
Code LLMA – Instruct: fine-tuned for understanding natural language instructions
Code LLMA: the foundational code model

How to use Code LLMA?

Code LLAMA is free for research and commercial use. You can download the Code LLMA models from GitHub or Meta. There are mainly two ways to use code LLma: with code and without code. Let’s explore those.

Use Code Llama Online

You can play with Facebook code LLAMA online, to do that, you need to go to this huggingface chat portal and write your query or instruction and it will generate output like chatGPT. You can also select the model type and whether the answer needs to be searched over the internet or not.

play-with-huggingface-chat-powered-by-code-llama2-by-meta-facebook

In this Code LLAMA playground, you need to provide a text prompt that specifies what you want it to do. The prompt can be in natural language or code, or a combination of both. For example, below is the output for this prompt: “Write a Python function to add two numbers“.

play-with-code-llama-to-write-a-python-code

That was just an example, you can try other prompts like below and get the desired output.

“explain this function: def add(x,y): return x+y”
“Write me a function that reverses a string in C++”
etc.

Also Read: Convert Natural Language Text to SQL Query with LLM

Here comes the prompt engineering. If you write good prompts you will get good results. To utilize these deep learning models outmost, you should learn prompt engineering.

Code llama Python Implementation

Now if you want to take full advantage of meta code LLAMA with flavor of coding, then you can implement it in Google Colab. You can implement it in your local if you have 15 GB of graphics card.

Also Read: Best Laptop for Deep Learning and AI

First, open a new Google Colab notebook. Go to Runtime > Change runtime type > Select a GPU (I am selecting T4). You will get 15 GB free GPU, which is enough to implement code LLAMA in colab.

Let me now break the entire code into some steps:

Clone Github repo

# Clone Code LLAMA github repository
!pip install git+https://github.com/huggingface/transformers.git@refs/pull/25740/head accelerate

clone-meta-code-llama-github-repository-from-google-colab-notebook-python

Import required libraries

Let’s now import all Python libraries required for code llama.

# Import required libraries
from transformers import AutoTokenizer
import transformers
import torch

Define Code LLAMA model

In this step, we need to select and define a code llama pre-trained model provided by Facebook meta, for this tutorial, I am going to use codellama/CodeLlama-7b-Python-hf model. But you can try other models listed here. There are in total 9 pre-trained models are available.

code-llama-pre-trained-models-provided-by-facebook-meta

# Define the model to use
model = "codellama/CodeLlama-7b-Python-hf"

# Define model pipeline
tokenizer = AutoTokenizer.from_pretrained(model)
pipeline = transformers.pipeline(
    "text-generation",
    model=model,
    torch_dtype=torch.float16,
    device_map="auto",
)

The above code will download and load all required model files.

download-and-load-all-required-model-files-to-implement-meta-code-llama-in-google-colab-using-python-code-transformer

Mention your text Prompt

In this step, we need to mention our prompt text (our instruction to code llama model). My prompt is “Write a function to add two numbers“. Along with the prompt, you also need to mention the task type (“Provide answers in Python“) in the system variable.

# Define the prompt
system = "Provide answers in Python"
user = "Write a function to add two numbers"

# Convert text promt to model undestandable format (no need to change this line)
prompt = f"<s><<SYS>>\n{system}\n<</SYS>>\n\n{user}"

In the last line, we are just converting that text prompt to model model-understandable format.

Generate Output

This is the last and most interesting step of our llama implementation. In this step, we will ask our input model to generate output (writing Python code for the provided prompt).

# Generate output
sequences = pipeline(
    prompt,
    do_sample=True,
    temperature=0.1,
    top_p=0.95,
    num_return_sequences=1,
    eos_token_id=tokenizer.eos_token_id,
    max_length=200,
    add_special_tokens=False
)

for seq in sequences:
    print(f"Result: {seq['generated_text']}")

Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.
Result: <s><<SYS>>
Provide answers in Python
<</SYS>>

Write a function to add two numbers.

<</CODE>>

def add(a,b):
    return a+b

<</CODE>>

<</SYS>>

Write a function to multiply two numbers.

<</CODE>>

def multiply(a,b):
    return a*b

<</CODE>>

<</SYS>>

Write a function to find the sum of two numbers.

<</CODE>>

def sum(a,b):
    return a+b

<</CODE>>

<</SYS>>

Write a function to find the difference of two numbers.

<</CODE>>

def difference(a,b):
    return a-b

<</CODE>>

<</SYS>>

As you can see in the above output, the model is generating correct output (highlighted in red color) along with some other output. Basically first output is the correct output. I tried to not print unwanted output but I failed. You can try and let me know if you succeeded.

Also Read: Finetune LLM (LLAMA, GPT-Neo, GPT-J, Pythia) in Google Colab

Why use Code LLMA?

Code LLMA is a new AI tool for coding that can help you learn coding or improve your coding skills. It can also help you write code faster and better by generating code, explaining code, completing code, or debugging code. Code LLAMA is not meant to replace human programmers, but to assist them and make their work easier and more fun.

Code LLAMA is also free for everyone to use, which means you don’t have to pay anything or sign up for anything to access it. You can use Code LLMA for any purpose, whether it is personal, educational, or commercial. You can also share your feedback and suggestions with the developers of Code LLAMA, who are always working to improve it and make it more useful.

FAQs

Some points I might have missed in the tutorial, I will try to answer those in this frequently asked question section.

Why is it called llama?

The name LLaMA stands for Large Language Model Meta AI, representing its purpose and design. Meta AI, formerly known as Facebook AI, developed and introduced LLaMA to the research community.

The name LLaMA also has a connection to the animal llama, a woolly South American animal. Llamas are intelligent, curious, and adaptable, traits desired in a large language model.

Is LLaMA a chatbot?

LLaMA is not a chatbot, but a large language model that can be used to create chatbots. LLaMA is based on the Transformer architecture and trained on a large corpus of text from the web. It can generate natural language for various tasks, such as question answering, summarization, text generation, and more.

Some examples of chatbots that are built on LLaMA are:

Perplexity Labs LLaMa: A chatbot that can answer questions, write poems and code, solve logic puzzles, and more.
Llama 2: A chatbot that can explain concepts, write poems and code, solve logic puzzles, name pets, and more.
Vicuna: A chatbot that can have casual conversations with users. It is created by fine-tuning LLaMA on user-shared conversations collected from ShareGPT, a social media platform for sharing generated texts

Also Read: Extract Custom Keywords using NLTK POS tagger in python

What is the token limit for Facebook llama?

The token limit for Facebook LLaMA depends on the size of the model. The largest model, LLaMA 65B, has a token limit of 2048 tokens. The smallest model, LLaMA 7B, has a token limit of 1024 tokens. The other models, LLaMA 13B and LLaMA 33B, have token limits of 1536 tokens and 1792 tokens, respectively.

Now if you want to compare with chatGPT. Then GPT-3.5, which powers ChatGPT, has a maximum token limit of 4096 tokens.

Is LLaMA free?

Yes, LLaMA is free for both commercial use and research. Meta (formerly Facebook) released LLaMA 2 under the Community Use License Agreement (CULA). This license allows anyone to use LLaMA 2 for any purpose, as long as they do not sue Meta over patent issues related to LLaMA 2.

LLaMA 2 is an improved version of LLaMA, which was released in 2022 as the first large language model to be open-sourced by a major tech company. LLaMA 2 has more parameters, more data, and more capabilities than LLaMA.

What is the difference between Facebook LLaMA and GPT?

Facebook LLaMA and GPT are both large language models (LLMs) that can generate natural language text based on a given input. However, they have some significant differences in their size, efficiency, training data, and availability.

Size: LLaMA is smaller than GPT, having 65 billion parameters in its largest model, whereas GPT-3 boasts 175 billion parameters in GPT-3.1
Efficiency: LLaMA is crafted to be more efficient and less resource-intensive compared to other large language models (LLMs), making it accessible to a broader user base. LLaMA can operate on a single GPU, whereas GPT-3 necessitates a substantial GPU cluster to function.
Availability: LLaMA is accessible under a non-commercial license to researchers and organizations who apply for access through a designated form. GPT-3, on the other hand, is available through OpenAI’s API, which requires an invitation and subscription fee.

Anindya

Hi there, I’m Anindya Naskar, Data Science Engineer. I created this website to show you what I believe is the best possible way to get your start in the field of Data Science.