# AGENTS.md

Apache Spark - A unified analytics engine for large-scale data processing

## Setup

```
# TODO: the install command. No lockfile was found, so this could not be inferred.
```

## Commands

```
ruff check .   # lint
```

## Layout

- `sql/`      5039 source files
- `python/`   1307 source files
- `core/`     1179 source files
- `mllib/`    610 source files
- `examples/` 523 source files
- `common/`   439 source files

## Conventions

- Tests live alongside the code they cover, following `.github/workflows/build_and_test.yml`.
- CI defines what passing means. See `.github/workflows/benchmark.yml`, and keep it green.
- TODO: add the two or three conventions a newcomer always gets wrong here.

## Gotchas

- Generated output is committed (`build/`). Search will return the compiled copy, so edit the source, not that.
- `python/pyspark/sql/functions/builtin.py` is 1135KB. It will not fit comfortably in context, so read it in parts.
- No lockfile is committed, so an install here may not match what CI produced.

---

Drafted by agentreadme.com from what is in this repository. Everything marked TODO
needs a human. Check it in as AGENTS.md at the root.