PandasAI: Conversational Data Analysis for Software Engineers

sinaptik-ai/pandas-ai

Think of pandas-ai as a bridge between your data and natural language. It's not just about a simple data query; it's about enabling a conversational experience. Here's how it can be a game-changer for you

Rapid Data Exploration and Prototyping
Need to quickly understand a new dataset? Instead of writing a bunch of pandas code, you can just ask questions. For example, "What's the average salary?" or "Show me the distribution of ages." This is great for fast-tracking exploratory data analysis (EDA) and building prototypes.

Creating Intelligent Applications
You can integrate pandas-ai into your applications to give non-technical users the ability to analyze data without writing code. Imagine a dashboard where a user can type a question like "Which store had the highest sales last quarter?" and get an instant, accurate answer. This "conversational data analysis" is a huge value-add.

Automating Reports and Visualizations
You can prompt pandas-ai to generate complex reports and visualizations with a single command. It can create plots and charts based on your natural language requests, saving you a ton of time on manual visualization coding.

Secure and Safe Execution
pandas-ai can be run in a Docker sandbox, which is a huge benefit for security. This isolates the code execution, protecting your system from any malicious or unexpected code that the LLM might generate. It ensures you can safely let the AI write and run code on your data.

Installing pandas-ai is pretty straightforward. It's a Python library, so you'll use pip.

First, you need to install the library.

pip install pandasai

You'll also need an LLM to power the conversational aspect. pandas-ai supports many different LLMs, including OpenAI, Hugging Face, and others. For this example, let's use the pandasai-openai provider, so you'll need to install that as well.

pip install pandasai-openai

To use an LLM, you'll need an API key. This example uses OpenAI, so you'll need an OpenAI API key. You can store it securely in an environment variable or directly in your code (though using an environment variable is highly recommended for production).

import pandas as pd
from pandasai import SmartDataframe
from pandasai_openai.openai import OpenAI

# It's best practice to set your API key as an environment variable
# If you don't, you can set it directly like this:
# from dotenv import load_dotenv
# load_dotenv()
# os.environ["OPENAI_API_KEY"] = "YOUR_API_KEY"

# Or if you're just testing:
# llm = OpenAI(api_token="YOUR_API_KEY")

llm = OpenAI()

Now, let's see it in action. We'll use a sample CSV file.

# Create a dummy CSV file for demonstration
data = {
    'country': ['United States', 'United Kingdom', 'France', 'Germany', 'Italy', 'Spain', 'Canada', 'Australia'],
    'gdp': [21.43, 2.83, 2.71, 3.86, 2.07, 1.40, 1.71, 1.39],
    'population': [328, 67, 65, 83, 60, 47, 37, 25]
}
df = pd.DataFrame(data)
df.to_csv('countries.csv', index=False)

# Load the CSV file into a SmartDataframe
sdf = SmartDataframe("countries.csv", config={"llm": llm})

# Now, you can chat with your data!
response = sdf.chat("What is the total population of all countries combined?")
print(response)

# Output: 712.0
# The result is automatically calculated by pandas-ai.

# Let's try something more complex
response = sdf.chat("Which country has the highest GDP per capita?")
print(response)

# Output: United States
# pandas-ai understands that "GDP per capita" is a new column to calculate and then finds the max.