# Tutorial

This is a quick tutorial of some basic commands and usage patterns of LETSQL.
We'll cover selecting data, filtering, grouping, aggregating, and ordering.

## Prerequisite

Please start by following the [installation instructions](installation.qmd), and then install [pandas](https://pandas.pydata.org/docs/getting_started/install.html).


## Loading the dataset
LETSQL can work with several file types, but at its core is an Ibis backend so it connects to existing databases and
interacts with the data there.

We'll use the iris dataset, which is a classic dataset used in machine learning and statistics.
It contains measurements of four features (sepal length, sepal width, petal length, petal width) for 150 flowers from three species
(setosa, versicolor, virginica).

```{python}
#| eval: true
#| code-fold: false
#| code-summary: loading the iris dataset
import pandas as pd

import letsql as ls

con = ls.connect()

# Load the dataset
url = "https://raw.githubusercontent.com/mwaskom/seaborn-data/master/iris.csv"
df = pd.read_csv(url)

# Register it
iris = con.register(df, "iris")
```

You can now see the example dataset copied over to the database

```{python}
con.list_tables()
```

There's one table, called `iris`. We can interact with it through the object that was returned when registering the table.
Or we can use the table method to get it.

```{python}
iris = con.table("iris")
```

LETSQL is an Ibis Backend so it is lazily evaluated, so instead of seeing the data, we see the schema of the table, instead. To peek at the data, we can call `head` and then `execute` to get the first few rows of the table.

```{python}
iris.head().execute()
```

<!-- FIXME Add example reading from local file -->

### Interactive mode

For the rest of this intro, we'll turn on interactive mode, which partially executes queries to give users a preview of the results. There is a small difference in the way the output is formatted, but otherwise this is the same as calling `to_pandas` on the table expression with a limit of 10 result rows returned.

```{python}
import ibis

ibis.options.interactive = True
iris.head()
```

## Common operations

LETSQL has a collection of useful table methods to manipulate and query the data in a table (or tables).

### select

Selecting data involves choosing specific columns from the table. To select a column you can use the name of the column as a string:

```{python}
#| code-summary: selecting columns

# Select the 'sepal_length' and 'sepal_width' columns
iris.select("sepal_length", "sepal_width")
```

### selectors

Typing out ALL of the column names _except_ one is a little annoying. Instead of doing that again, we can use a `selector` to quickly select or deselect groups of columns.

```{python}
#| code-summary: using selectors

import ibis.selectors as s

iris.select(
    ~s.matches("sepal_length")
    # match every column except `sepal_length`
)
```

You can also use a `selector` alongside a column name.

```{python}
iris.select("species", s.numeric())
```

### filter

Filtering data involves selecting rows that meet a condition or a set of conditions.
We can filter so we only have flowers with `sepal_length` greater than 5.0

```{python}
#| code-summary: showcasing filtering

# Filter the dataset for flowers with sepal length greater than 5.0
iris.filter(iris.sepal_length > 5.0)
```

Additionally, or filter for Setosa flowers with sepal length greater than 5.0:

```{python}
#| code-summary: showcasing filtering multiple filters

# Filter the dataset for flowers with sepal length greater than 5.0 that are setosa
iris.filter((iris.sepal_length > 5.0) & (iris.species == "setosa"))
```

### order_by

Ordering data involves sorting the rows of the DataFrame based on the values in one or more columns. This can be achieved
with `order_by` because it arranges the values of one or more columns in ascending or descending order.

```{python}
#| code-summary: showcasing order_by
# Sort the dataset by 'sepal_length' in ascending order
iris.order_by(iris.sepal_length)
```

You can sort in descending order using the `desc` method of a column:

```{python}
#| code-summary: showcasing order_by desc
# Sort the dataset by 'sepal_length' in descending order
iris.order_by(iris.sepal_length.desc())
```

### aggregates

LETSQL has the same aggregate functions as Ibis to help summarize data.

`mean`, `max`, `min`, `count`, `sum` (the list goes on).

To aggregate an entire column, call the corresponding method on that column.

```{python}
#| code-summary: showcasing aggregates
iris.sepal_length.mean()
```

You can compute multiple aggregates at once using the `aggregate` method:

```{python}
#| code-summary: showcasing multiple aggregates
iris.aggregate([iris.sepal_length.mean(), iris.petal_width.max()])
```

But `aggregate` _really_ shines when it's paired with `group_by`.

### group_by

Grouping data involves splitting the data into groups based on some criteria. For creating the groupings
of rows that have the same value for one or more columns use `group_by`.

To compute summary statistics for each group pair it with `aggregate` to get a result.

```{python}
#| code-summary: showcasing group_by

# Group the dataset by the 'species' column
iris.group_by('species').aggregate()
```

We grouped by the `species` column and handed it an "empty" aggregate command. The result of that is a column of the unique values in the `species` column.

Now, if we add an aggregation function to that, we start to really open things up.

```{python}
#| code-summary: showcasing group_by with aggregates
iris.group_by("species").aggregate(iris.sepal_length.mean())
```

## Conclusion

In this tutorial, we covered the basics of selecting, filtering, grouping, aggregating, and ordering data using the Iris dataset in LETSQL.
Explore further by applying these techniques to your datasets and combining them to perform more complex data manipulations.