df-research/dataframe/docs/StardustDocs/topics/schemas/schemas.md

The Kotlin DataFrame library provides typed data access via generation of extension properties for the type DataFrame<T> (as well as for DataRow<T>), where T is a marker class representing the DataSchema of the DataFrame.

A schema of a DataFrame is a mapping from column names to column types.
This data schema can be expressed as a Kotlin class or interface.
If the DataFrame is hierarchical — contains a column group or a column of dataframes — the data schema reflects this structure, with a separate class representing the schema of each column group or nested DataFrame.

For example, consider a simple hierarchical DataFrame from .

This DataFrame consists of two columns:

• name, which is a String column
• info, which is a column group containing two nested value columns:
  • age of type Int
  • height of type Double

name	info
	age	height
Alice	23	175.5
Bob	27	160.2

The data schema corresponding to this DataFrame can be represented as:

// Data schema of the "info" column group
@DataSchema
data class Info(
    val age: Int,
    val height: Float
)

// Data schema of the entire DataFrame
@DataSchema
data class Person(
    val info: Info,
    val name: String
)

Extension properties for DataFrame<Person>
are generated based on this schema and allow accessing columns or using them in operations:

// Assuming `df` has type `DataFrame<Person>`

// Get "age" column from "info" group
df.info.age

// Select "name" and "height" columns
df.select { name and info.height }

// Filter rows by "age"
df.filter { age >= 18 }

See for more information.

Schema Retrieving

Defining a data schema manually can be difficult, especially for dataframes with many columns or deeply nested structures, and may lead to mistakes in column names or types. Kotlin DataFrame provides several methods for generating data schemas.

• generate..() methods are extensions for DataFrame (or for its schema) that generate a code string representing its DataSchema.

• Kotlin DataFrame Compiler Plugin cannot automatically infer a data schema from external sources such as files or URLs. However, it can infer the schema if you construct the DataFrame manually — that is, by explicitly declaring the columns using the API. It will also automatically update the schema during operations that modify the structure of the DataFrame.

For best results when working with the Compiler Plugin, it's recommended to generate the initial schema using one of the generate..() methods. Once generated, the Compiler Plugin will automatically keep the schema up to date after any operations that change the structure of the DataFrame.

Plugins

The current Gradle plugin is under consideration for deprecation and may be officially marked as deprecated in future releases.

The KSP plugin is not compatible with KSP2 and may not work properly with Kotlin 2.1 or newer.

At the moment, data schema generation is handled via dedicated methods instead of relying on the plugins.
{style="warning"}

• The Gradle plugin allows generating a data schema automatically by specifying a source file path in the Gradle build script.

• The KSP plugin allows generating a data schema automatically using Kotlin Symbol Processing by specifying a source file path in your code file.

Extension Properties Generation

Once you have a data schema, you can generate extension properties.

The easiest and most convenient way is to use the Kotlin DataFrame Compiler Plugin, which generates extension properties on the fly for declared data schemas and automatically keeps them up to date after operations that modify the structure of the DataFrame.

Extension properties generation was deprecated from the Gradle plugin in favor of the Compiler Plugin.
{style="warning"}

• When using Kotlin DataFrame inside Kotlin Notebook, the schema and extension properties are generated automatically after each cell execution for all DataFrame variables declared in that cell. See extension properties example in Kotlin Notebook.

Compiler Plugin is coming to Kotlin Notebook soon.

• If you're not using the Compiler Plugin, you can still generate extension properties for a DataFrame manually by calling one of the generate..() methods with the extensionProperties = true argument.