The Data Analysis Evolution

Data analysis is being transformed by LLMs. You can now ask questions in plain English and get charts, insights, and reports.

Tools

  • PandasAI: Natural language queries on DataFrames.
  • LangChain Pandas Agent: LLM generates and executes Pandas code.
  • ChatGPT Code Interpreter: Upload CSV, get analysis instantly.
  • Julius AI: AI data analyst with visualization.

Workflow Pattern

1. Load data into a DataFrame.

2. Ask questions in natural language.

3. LLM generates Python code to answer.

4. Execute code and return results + visualization.

Example Questions

  • "What's the average revenue by region last quarter?"
  • "Show me the correlation between marketing spend and conversions."
  • "Find outliers in the customer age distribution."

Best Practices

  • Always verify generated code before execution.
  • Provide column descriptions as context.
  • Use sandboxed execution environments.
  • Combine with traditional EDA for thorough analysis.
  • Set column types explicitly to avoid misinterpretation.

Limitations

  • LLMs can hallucinate incorrect analysis.
  • Complex statistical tests may be unreliable.
  • Large datasets need sampling before LLM processing.
  • Always cross-validate important findings.

Security and Sandboxing

Never run LLM-generated code directly on production data without sandboxing. Use Docker containers or subprocess isolation with strict resource limits. Set memory caps (e.g., 2GB) and timeouts (30 seconds) to prevent runaway queries. For sensitive data, mask PII columns before passing DataFrames to the LLM.

Scaling to Large Datasets

LLMs cannot process millions of rows directly. Strategies for large datasets:

  • Sampling: Send a representative 1000-row sample for the LLM to understand the schema, then apply the generated code to the full dataset.
  • Chunked processing: Split data into chunks, run analysis per chunk, then aggregate results.
  • DuckDB + PandasAI: Use DuckDB as an in-memory SQL engine that can query Pandas DataFrames directly—much faster than row-by-row processing.

Common Pitfalls

  • Hallucinated columns: LLMs sometimes invent column names. Always validate against df.columns before executing.
  • Type mismatches: Dates stored as strings cause silent failures. Set dtypes explicitly: df[‘date’] = pd.to_datetime(df[‘date’]).
  • Aggregation errors: LLMs may use .mean() when you need .median() for skewed data. Specify the statistical method in your prompt.

Let the Model Write Queries, Not Numbers

The reliable pattern for AI-assisted analysis is to have the model produce code that produces numbers, and never to have it produce the numbers directly. A model asked to compute a total will approximate; a model asked to write a pandas expression and have the runtime evaluate it will produce a real result you can verify. Keep the execution step in your own process, print the intermediate DataFrame, and read it. The extra ten seconds of inspection is what turns a plausible narrative into an analysis.

Describing the Data Is Half the Work

Most bad generated analyses come from a vague schema. Before asking a question, give the model the column names, dtypes, a few sample rows, and the null counts. With that, it stops inventing columns and stops implying joins that do not exist. It is worth writing a small helper that emits this profile automatically, because you will want it on every subsequent question and hand-writing it is where errors creep in.

Guardrails for Generated Code

Generated pandas code runs with your credentials and your filesystem. Before executing it, restrict what it can do: no network calls, a read-only data directory, a wall-clock timeout, and a hard cap on memory. Print the code and skim it before running it. This is cheap insurance against a model that decides to download something, write a file, or build a cartesian join that exhausts memory.

Trust the DataFrame, Not the Sentence

Every claim in the final write-up should be traceable to a line of output you saw. If a sentence has a number in it that you cannot point to in a printed DataFrame, delete the sentence or go and compute the number.