{ "cells": [ { "metadata": {}, "cell_type": "markdown", "source": "# Quickstart" }, { "metadata": {}, "cell_type": "markdown", "source": "To start working with Kotlin DataFrame in a notebook, run the cell with the next code:" }, { "cell_type": "code", "metadata": { "collapsed": true, "ExecuteTime": { "end_time": "2025-05-01T13:42:31.795685Z", "start_time": "2025-05-01T13:42:24.748992Z" } }, "source": [ "%useLatestDescriptors\n", "%use dataframe" ], "outputs": [], "execution_count": 1 }, { "metadata": {}, "cell_type": "markdown", "source": "This will load all necessary DataFrame dependencies (of the latest stable version) and all imports, as well as DataFrame rendering. Learn more here" }, { "metadata": {}, "cell_type": "markdown", "source": "## Read DataFrame" }, { "metadata": {}, "cell_type": "markdown", "source": "Kotlin DataFrame supports all popular data formats, including CSV, JSON and Excel, as well as reading from various databases. Read a CSV with the \"Jetbrains Repositories\" dataset into `df` variable:" }, { "metadata": { "ExecuteTime": { "end_time": "2025-05-01T13:42:32.441575Z", "start_time": "2025-05-01T13:42:31.823137Z" } }, "cell_type": "code", "source": [ "val df = DataFrame.readCsv(\n", " \"https://raw.githubusercontent.com/Kotlin/dataframe/master/data/jetbrains_repositories.csv\"\n", ")" ], "outputs": [], "execution_count": 2 }, { "metadata": {}, "cell_type": "markdown", "source": "## Display And Explore" }, { "metadata": {}, "cell_type": "markdown", "source": "To display your dataframe as a cell output, place it in the last line of the cell:" }, { "metadata": { "ExecuteTime": { "end_time": "2025-05-01T13:42:32.538828Z", "start_time": "2025-05-01T13:42:32.465804Z" } }, "cell_type": "code", "source": "df", "outputs": [ { "data": { "text/html": [ " \n", " \n", " \n", " \n", " \n", " \n", "
full_namehtml_urlstargazers_counttopicswatchers
JetBrains/JPShttps://github.com/JetBrains/JPS23[]23
JetBrains/YouTrackSharphttps://github.com/JetBrains/YouTrack...115[jetbrains, jetbrains-youtrack, youtr...115
JetBrains/colorSchemeToolhttps://github.com/JetBrains/colorSch...290[]290
JetBrains/ideavimhttps://github.com/JetBrains/ideavim6120[ideavim, intellij, intellij-platform...6120
JetBrains/youtrack-vcs-hookshttps://github.com/JetBrains/youtrack...5[]5
JetBrains/youtrack-rest-ruby-libraryhttps://github.com/JetBrains/youtrack...8[]8
JetBrains/emacs4ijhttps://github.com/JetBrains/emacs4ij47[]47
JetBrains/codereview4intellijhttps://github.com/JetBrains/coderevi...11[]11
JetBrains/teamcity-nuget-supporthttps://github.com/JetBrains/teamcity...41[nuget, nuget-feed, teamcity, teamcit...41
JetBrains/Grammar-Kithttps://github.com/JetBrains/Grammar-Kit534[]534
JetBrains/intellij-starteam-pluginhttps://github.com/JetBrains/intellij...6[]6
JetBrains/la-clojurehttps://github.com/JetBrains/la-clojure218[]218
JetBrains/MPShttps://github.com/JetBrains/MPS1241[domain-specific-language, dsl]1241
JetBrains/intellij-communityhttps://github.com/JetBrains/intellij...12926[code-editor, ide, intellij, intellij...12926
JetBrains/TeamCity.ServiceMessageshttps://github.com/JetBrains/TeamCity...39[c-sharp, teamcity, teamcity-service-...39
JetBrains/youtrack-rest-python-libraryhttps://github.com/JetBrains/youtrack...118[]118
JetBrains/intellij-scalahttps://github.com/JetBrains/intellij...1066[intellij-idea, intellij-plugin, scala]1066
JetBrains/teamcity-messageshttps://github.com/JetBrains/teamcity...125[]125
JetBrains/teamcity-cpphttps://github.com/JetBrains/teamcity...27[]27
JetBrains/kotlinhttps://github.com/JetBrains/kotlin39402[compiler, gradle-plugin, intellij-pl...39402
\n", " \n", " \n", " " ], "application/kotlindataframe+json": "{\"$version\":\"2.1.1\",\"metadata\":{\"columns\":[\"full_name\",\"html_url\",\"stargazers_count\",\"topics\",\"watchers\"],\"types\":[{\"kind\":\"ValueColumn\",\"type\":\"kotlin.String\"},{\"kind\":\"ValueColumn\",\"type\":\"java.net.URL\"},{\"kind\":\"ValueColumn\",\"type\":\"kotlin.Int\"},{\"kind\":\"ValueColumn\",\"type\":\"kotlin.String\"},{\"kind\":\"ValueColumn\",\"type\":\"kotlin.Int\"}],\"nrow\":562,\"ncol\":5},\"kotlin_dataframe\":[{\"full_name\":\"JetBrains/JPS\",\"html_url\":\"https://github.com/JetBrains/JPS\",\"stargazers_count\":23,\"topics\":\"[]\",\"watchers\":23},{\"full_name\":\"JetBrains/YouTrackSharp\",\"html_url\":\"https://github.com/JetBrains/YouTrackSharp\",\"stargazers_count\":115,\"topics\":\"[jetbrains, jetbrains-youtrack, youtrack, youtrack-api]\",\"watchers\":115},{\"full_name\":\"JetBrains/colorSchemeTool\",\"html_url\":\"https://github.com/JetBrains/colorSchemeTool\",\"stargazers_count\":290,\"topics\":\"[]\",\"watchers\":290},{\"full_name\":\"JetBrains/ideavim\",\"html_url\":\"https://github.com/JetBrains/ideavim\",\"stargazers_count\":6120,\"topics\":\"[ideavim, intellij, intellij-platform, jb-official, kotlin, vim, vim-emulator]\",\"watchers\":6120},{\"full_name\":\"JetBrains/youtrack-vcs-hooks\",\"html_url\":\"https://github.com/JetBrains/youtrack-vcs-hooks\",\"stargazers_count\":5,\"topics\":\"[]\",\"watchers\":5},{\"full_name\":\"JetBrains/youtrack-rest-ruby-library\",\"html_url\":\"https://github.com/JetBrains/youtrack-rest-ruby-library\",\"stargazers_count\":8,\"topics\":\"[]\",\"watchers\":8},{\"full_name\":\"JetBrains/emacs4ij\",\"html_url\":\"https://github.com/JetBrains/emacs4ij\",\"stargazers_count\":47,\"topics\":\"[]\",\"watchers\":47},{\"full_name\":\"JetBrains/codereview4intellij\",\"html_url\":\"https://github.com/JetBrains/codereview4intellij\",\"stargazers_count\":11,\"topics\":\"[]\",\"watchers\":11},{\"full_name\":\"JetBrains/teamcity-nuget-support\",\"html_url\":\"https://github.com/JetBrains/teamcity-nuget-support\",\"stargazers_count\":41,\"topics\":\"[nuget, nuget-feed, teamcity, teamcity-plugin]\",\"watchers\":41},{\"full_name\":\"JetBrains/Grammar-Kit\",\"html_url\":\"https://github.com/JetBrains/Grammar-Kit\",\"stargazers_count\":534,\"topics\":\"[]\",\"watchers\":534},{\"full_name\":\"JetBrains/intellij-starteam-plugin\",\"html_url\":\"https://github.com/JetBrains/intellij-starteam-plugin\",\"stargazers_count\":6,\"topics\":\"[]\",\"watchers\":6},{\"full_name\":\"JetBrains/la-clojure\",\"html_url\":\"https://github.com/JetBrains/la-clojure\",\"stargazers_count\":218,\"topics\":\"[]\",\"watchers\":218},{\"full_name\":\"JetBrains/MPS\",\"html_url\":\"https://github.com/JetBrains/MPS\",\"stargazers_count\":1241,\"topics\":\"[domain-specific-language, dsl]\",\"watchers\":1241},{\"full_name\":\"JetBrains/intellij-community\",\"html_url\":\"https://github.com/JetBrains/intellij-community\",\"stargazers_count\":12926,\"topics\":\"[code-editor, ide, intellij, intellij-community, intellij-platform]\",\"watchers\":12926},{\"full_name\":\"JetBrains/TeamCity.ServiceMessages\",\"html_url\":\"https://github.com/JetBrains/TeamCity.ServiceMessages\",\"stargazers_count\":39,\"topics\":\"[c-sharp, teamcity, teamcity-service-messages]\",\"watchers\":39},{\"full_name\":\"JetBrains/youtrack-rest-python-library\",\"html_url\":\"https://github.com/JetBrains/youtrack-rest-python-library\",\"stargazers_count\":118,\"topics\":\"[]\",\"watchers\":118},{\"full_name\":\"JetBrains/intellij-scala\",\"html_url\":\"https://github.com/JetBrains/intellij-scala\",\"stargazers_count\":1066,\"topics\":\"[intellij-idea, intellij-plugin, scala]\",\"watchers\":1066},{\"full_name\":\"JetBrains/teamcity-messages\",\"html_url\":\"https://github.com/JetBrains/teamcity-messages\",\"stargazers_count\":125,\"topics\":\"[]\",\"watchers\":125},{\"full_name\":\"JetBrains/teamcity-cpp\",\"html_url\":\"https://github.com/JetBrains/teamcity-cpp\",\"stargazers_count\":27,\"topics\":\"[]\",\"watchers\":27},{\"full_name\":\"JetBrains/kotlin\",\"html_url\":\"https://github.com/JetBrains/kotlin\",\"stargazers_count\":39402,\"topics\":\"[compiler, gradle-plugin, intellij-plugin, kotlin, kotlin-library, maven-plugin, programming-language]\",\"watchers\":39402}]}" }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "execution_count": 3 }, { "metadata": {}, "cell_type": "markdown", "source": "Kotlin Notebook has special interactive outputs for `DataFrame`. Learn more about them here." }, { "metadata": {}, "cell_type": "markdown", "source": "Use `.describe()` method to get dataset summaries — column types, number of nulls and simple statistics." }, { "metadata": { "ExecuteTime": { "end_time": "2025-05-01T13:42:32.765356Z", "start_time": "2025-05-01T13:42:32.666531Z" } }, "cell_type": "code", "source": "df.describe()", "outputs": [ { "data": { "text/html": [ " \n", " \n", " \n", " \n", " \n", " \n", "
nametypecountuniquenullstopfreqmeanstdminmedianmax
full_nameString5625620JetBrains/JPS1nullnullJetBrains/Android-Tuts-SamplesJetBrains/lightbeamJetBrains/ztools
html_urlURL5625620https://github.com/JetBrains/JPS1nullnullnullnullnull
stargazers_countInt56216501100244,7597861862,8019820839402
topicsString5621450[]401nullnull[2d, graphics, java, skia][][youtrack, youtrack-workflow]
watchersInt56216501100244,7597861862,8019820839402
\n", " \n", " \n", " " ], "application/kotlindataframe+json": "{\"$version\":\"2.1.1\",\"metadata\":{\"columns\":[\"name\",\"type\",\"count\",\"unique\",\"nulls\",\"top\",\"freq\",\"mean\",\"std\",\"min\",\"median\",\"max\"],\"types\":[{\"kind\":\"ValueColumn\",\"type\":\"kotlin.String\"},{\"kind\":\"ValueColumn\",\"type\":\"kotlin.String\"},{\"kind\":\"ValueColumn\",\"type\":\"kotlin.Int\"},{\"kind\":\"ValueColumn\",\"type\":\"kotlin.Int\"},{\"kind\":\"ValueColumn\",\"type\":\"kotlin.Int\"},{\"kind\":\"ValueColumn\",\"type\":\"java.io.Serializable\"},{\"kind\":\"ValueColumn\",\"type\":\"kotlin.Int\"},{\"kind\":\"ValueColumn\",\"type\":\"kotlin.Double?\"},{\"kind\":\"ValueColumn\",\"type\":\"kotlin.Double?\"},{\"kind\":\"ValueColumn\",\"type\":\"kotlin.Comparable<*>?\"},{\"kind\":\"ValueColumn\",\"type\":\"kotlin.Comparable<*>?\"},{\"kind\":\"ValueColumn\",\"type\":\"kotlin.Comparable<*>?\"}],\"nrow\":5,\"ncol\":12},\"kotlin_dataframe\":[{\"name\":\"full_name\",\"type\":\"String\",\"count\":562,\"unique\":562,\"nulls\":0,\"top\":\"JetBrains/JPS\",\"freq\":1,\"mean\":null,\"std\":null,\"min\":\"JetBrains/Android-Tuts-Samples\",\"median\":\"JetBrains/lightbeam\",\"max\":\"JetBrains/ztools\"},{\"name\":\"html_url\",\"type\":\"URL\",\"count\":562,\"unique\":562,\"nulls\":0,\"top\":\"https://github.com/JetBrains/JPS\",\"freq\":1,\"mean\":null,\"std\":null,\"min\":null,\"median\":null,\"max\":null},{\"name\":\"stargazers_count\",\"type\":\"Int\",\"count\":562,\"unique\":165,\"nulls\":0,\"top\":\"1\",\"freq\":100,\"mean\":244.75978647686833,\"std\":1862.8019819171673,\"min\":\"0\",\"median\":\"8\",\"max\":\"39402\"},{\"name\":\"topics\",\"type\":\"String\",\"count\":562,\"unique\":145,\"nulls\":0,\"top\":\"[]\",\"freq\":401,\"mean\":null,\"std\":null,\"min\":\"[2d, graphics, java, skia]\",\"median\":\"[]\",\"max\":\"[youtrack, youtrack-workflow]\"},{\"name\":\"watchers\",\"type\":\"Int\",\"count\":562,\"unique\":165,\"nulls\":0,\"top\":\"1\",\"freq\":100,\"mean\":244.75978647686833,\"std\":1862.8019819171673,\"min\":\"0\",\"median\":\"8\",\"max\":\"39402\"}]}" }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "execution_count": 4 }, { "metadata": {}, "cell_type": "markdown", "source": "## Select Columns" }, { "metadata": {}, "cell_type": "markdown", "source": [ "Kotlin DataFrame features a typesafe Columns Selection DSL, enabling flexible and safe selection of any combination of columns.\n", "Column selectors are widely used across operations — one of the simplest examples is `.select { }`, which returns a new DataFrame with only the columns chosen in Columns Selection expression.\n", "\n", "After executing the cell where a `DataFrame` variable is declared, an extension with properties for its columns is automatically generated.\n", "These properties can then be used in the Columns Selection DSL expression for typesafe and convenient column access.\n", "\n", "Select some columns:" ] }, { "metadata": { "ExecuteTime": { "end_time": "2025-05-01T13:42:32.986689Z", "start_time": "2025-05-01T13:42:32.810436Z" } }, "cell_type": "code", "source": [ "// Select \"full_name\", \"stargazers_count\" and \"topics\" columns\n", "val dfSelected = df.select { full_name and stargazers_count and topics }\n", "dfSelected" ], "outputs": [ { "data": { "text/html": [ " \n", " \n", " \n", " \n", " \n", " \n", "
full_namestargazers_counttopics
JetBrains/JPS23[]
JetBrains/YouTrackSharp115[jetbrains, jetbrains-youtrack, youtr...
JetBrains/colorSchemeTool290[]
JetBrains/ideavim6120[ideavim, intellij, intellij-platform...
JetBrains/youtrack-vcs-hooks5[]
JetBrains/youtrack-rest-ruby-library8[]
JetBrains/emacs4ij47[]
JetBrains/codereview4intellij11[]
JetBrains/teamcity-nuget-support41[nuget, nuget-feed, teamcity, teamcit...
JetBrains/Grammar-Kit534[]
JetBrains/intellij-starteam-plugin6[]
JetBrains/la-clojure218[]
JetBrains/MPS1241[domain-specific-language, dsl]
JetBrains/intellij-community12926[code-editor, ide, intellij, intellij...
JetBrains/TeamCity.ServiceMessages39[c-sharp, teamcity, teamcity-service-...
JetBrains/youtrack-rest-python-library118[]
JetBrains/intellij-scala1066[intellij-idea, intellij-plugin, scala]
JetBrains/teamcity-messages125[]
JetBrains/teamcity-cpp27[]
JetBrains/kotlin39402[compiler, gradle-plugin, intellij-pl...
\n", " \n", " \n", " " ], "application/kotlindataframe+json": "{\"$version\":\"2.1.1\",\"metadata\":{\"columns\":[\"full_name\",\"stargazers_count\",\"topics\"],\"types\":[{\"kind\":\"ValueColumn\",\"type\":\"kotlin.String\"},{\"kind\":\"ValueColumn\",\"type\":\"kotlin.Int\"},{\"kind\":\"ValueColumn\",\"type\":\"kotlin.String\"}],\"nrow\":562,\"ncol\":3},\"kotlin_dataframe\":[{\"full_name\":\"JetBrains/JPS\",\"stargazers_count\":23,\"topics\":\"[]\"},{\"full_name\":\"JetBrains/YouTrackSharp\",\"stargazers_count\":115,\"topics\":\"[jetbrains, jetbrains-youtrack, youtrack, youtrack-api]\"},{\"full_name\":\"JetBrains/colorSchemeTool\",\"stargazers_count\":290,\"topics\":\"[]\"},{\"full_name\":\"JetBrains/ideavim\",\"stargazers_count\":6120,\"topics\":\"[ideavim, intellij, intellij-platform, jb-official, kotlin, vim, vim-emulator]\"},{\"full_name\":\"JetBrains/youtrack-vcs-hooks\",\"stargazers_count\":5,\"topics\":\"[]\"},{\"full_name\":\"JetBrains/youtrack-rest-ruby-library\",\"stargazers_count\":8,\"topics\":\"[]\"},{\"full_name\":\"JetBrains/emacs4ij\",\"stargazers_count\":47,\"topics\":\"[]\"},{\"full_name\":\"JetBrains/codereview4intellij\",\"stargazers_count\":11,\"topics\":\"[]\"},{\"full_name\":\"JetBrains/teamcity-nuget-support\",\"stargazers_count\":41,\"topics\":\"[nuget, nuget-feed, teamcity, teamcity-plugin]\"},{\"full_name\":\"JetBrains/Grammar-Kit\",\"stargazers_count\":534,\"topics\":\"[]\"},{\"full_name\":\"JetBrains/intellij-starteam-plugin\",\"stargazers_count\":6,\"topics\":\"[]\"},{\"full_name\":\"JetBrains/la-clojure\",\"stargazers_count\":218,\"topics\":\"[]\"},{\"full_name\":\"JetBrains/MPS\",\"stargazers_count\":1241,\"topics\":\"[domain-specific-language, dsl]\"},{\"full_name\":\"JetBrains/intellij-community\",\"stargazers_count\":12926,\"topics\":\"[code-editor, ide, intellij, intellij-community, intellij-platform]\"},{\"full_name\":\"JetBrains/TeamCity.ServiceMessages\",\"stargazers_count\":39,\"topics\":\"[c-sharp, teamcity, teamcity-service-messages]\"},{\"full_name\":\"JetBrains/youtrack-rest-python-library\",\"stargazers_count\":118,\"topics\":\"[]\"},{\"full_name\":\"JetBrains/intellij-scala\",\"stargazers_count\":1066,\"topics\":\"[intellij-idea, intellij-plugin, scala]\"},{\"full_name\":\"JetBrains/teamcity-messages\",\"stargazers_count\":125,\"topics\":\"[]\"},{\"full_name\":\"JetBrains/teamcity-cpp\",\"stargazers_count\":27,\"topics\":\"[]\"},{\"full_name\":\"JetBrains/kotlin\",\"stargazers_count\":39402,\"topics\":\"[compiler, gradle-plugin, intellij-plugin, kotlin, kotlin-library, maven-plugin, programming-language]\"}]}" }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "execution_count": 5 }, { "metadata": {}, "cell_type": "markdown", "source": "## Raw Filtering" }, { "metadata": {}, "cell_type": "markdown", "source": [ "Some operations use `RowExpression`, i.e., expression that applies for all `DataFrame` rows. For example `.filter { }` that returns a new `DataFrame` with rows that satisfy a condition given by row expression.\n", "\n", "Inside a row expression, you can access the values of the current row by column names through auto-generated properties.\n", "Similar to the Columns Selection DSL, but in this case the properties represent actual values, not column references.\n", "\n", "Filter rows by \"stargazers_count\" value:" ] }, { "metadata": { "ExecuteTime": { "end_time": "2025-05-01T13:42:33.082510Z", "start_time": "2025-05-01T13:42:33.031897Z" } }, "cell_type": "code", "source": [ "// Keep only rows where \"stargazers_count\" value is more than 1000\n", "val dfFiltered = dfSelected.filter { stargazers_count >= 1000 }\n", "dfFiltered" ], "outputs": [ { "data": { "text/html": [ " \n", " \n", " \n", " \n", " \n", " \n", "
full_namestargazers_counttopics
JetBrains/ideavim6120[ideavim, intellij, intellij-platform...
JetBrains/MPS1241[domain-specific-language, dsl]
JetBrains/intellij-community12926[code-editor, ide, intellij, intellij...
JetBrains/intellij-scala1066[intellij-idea, intellij-plugin, scala]
JetBrains/kotlin39402[compiler, gradle-plugin, intellij-pl...
JetBrains/intellij-plugins1737[]
JetBrains/Exposed5688[dao, kotlin, orm, sql]
JetBrains/kotlin-web-site1074[kotlin]
JetBrains/idea-gitignore1181[gitignore, ignore-files, intellij, i...
JetBrains/swot1072[]
JetBrains/phpstorm-stubs1110[]
JetBrains/gradle-intellij-plugin1058[gradle, gradle-intellij-plugin, grad...
JetBrains/svg-sprite-loader1815[sprite, svg, svg-sprite, svg-stack, ...
JetBrains/resharper-unity1017[hacktoberfest, jetbrains, plugin, re...
JetBrains/kotlin-native7101[c, compiler, kotlin, llvm, objective-c]
JetBrains/create-react-kotlin-app2424[create-react-app, jetbrains-ui, kotl...
JetBrains/ring-ui2836[components, jetbrains-ui, react]
JetBrains/kotlinconf-app2628[]
JetBrains/JetBrainsMono6059[coding-font, font, ligatures, monosp...
JetBrains/intellij-platform-plugin-te...1133[intellij, intellij-idea, intellij-id...
\n", " \n", " \n", " " ], "application/kotlindataframe+json": "{\"$version\":\"2.1.1\",\"metadata\":{\"columns\":[\"full_name\",\"stargazers_count\",\"topics\"],\"types\":[{\"kind\":\"ValueColumn\",\"type\":\"kotlin.String\"},{\"kind\":\"ValueColumn\",\"type\":\"kotlin.Int\"},{\"kind\":\"ValueColumn\",\"type\":\"kotlin.String\"}],\"nrow\":24,\"ncol\":3},\"kotlin_dataframe\":[{\"full_name\":\"JetBrains/ideavim\",\"stargazers_count\":6120,\"topics\":\"[ideavim, intellij, intellij-platform, jb-official, kotlin, vim, vim-emulator]\"},{\"full_name\":\"JetBrains/MPS\",\"stargazers_count\":1241,\"topics\":\"[domain-specific-language, dsl]\"},{\"full_name\":\"JetBrains/intellij-community\",\"stargazers_count\":12926,\"topics\":\"[code-editor, ide, intellij, intellij-community, intellij-platform]\"},{\"full_name\":\"JetBrains/intellij-scala\",\"stargazers_count\":1066,\"topics\":\"[intellij-idea, intellij-plugin, scala]\"},{\"full_name\":\"JetBrains/kotlin\",\"stargazers_count\":39402,\"topics\":\"[compiler, gradle-plugin, intellij-plugin, kotlin, kotlin-library, maven-plugin, programming-language]\"},{\"full_name\":\"JetBrains/intellij-plugins\",\"stargazers_count\":1737,\"topics\":\"[]\"},{\"full_name\":\"JetBrains/Exposed\",\"stargazers_count\":5688,\"topics\":\"[dao, kotlin, orm, sql]\"},{\"full_name\":\"JetBrains/kotlin-web-site\",\"stargazers_count\":1074,\"topics\":\"[kotlin]\"},{\"full_name\":\"JetBrains/idea-gitignore\",\"stargazers_count\":1181,\"topics\":\"[gitignore, ignore-files, intellij, intellij-plugin, java]\"},{\"full_name\":\"JetBrains/swot\",\"stargazers_count\":1072,\"topics\":\"[]\"},{\"full_name\":\"JetBrains/phpstorm-stubs\",\"stargazers_count\":1110,\"topics\":\"[]\"},{\"full_name\":\"JetBrains/gradle-intellij-plugin\",\"stargazers_count\":1058,\"topics\":\"[gradle, gradle-intellij-plugin, gradle-kotlin-dsl, groovy, intellij, intellij-ides, intellij-platform, intellij-plugin, intellij-sdk, jetbrains-plugin, kotlin, plugin-verifier, publishing-dsl, setup-dsl, teamcity, travis-configuration]\"},{\"full_name\":\"JetBrains/svg-sprite-loader\",\"stargazers_count\":1815,\"topics\":\"[sprite, svg, svg-sprite, svg-stack, webpack, webpack-loader, webpack-plugin, webpack2, webpack3]\"},{\"full_name\":\"JetBrains/resharper-unity\",\"stargazers_count\":1017,\"topics\":\"[hacktoberfest, jetbrains, plugin, resharper, resharper-plugin, rider, unity, unity-editor]\"},{\"full_name\":\"JetBrains/kotlin-native\",\"stargazers_count\":7101,\"topics\":\"[c, compiler, kotlin, llvm, objective-c]\"},{\"full_name\":\"JetBrains/create-react-kotlin-app\",\"stargazers_count\":2424,\"topics\":\"[create-react-app, jetbrains-ui, kotlin, react, webpack]\"},{\"full_name\":\"JetBrains/ring-ui\",\"stargazers_count\":2836,\"topics\":\"[components, jetbrains-ui, react]\"},{\"full_name\":\"JetBrains/kotlinconf-app\",\"stargazers_count\":2628,\"topics\":\"[]\"},{\"full_name\":\"JetBrains/JetBrainsMono\",\"stargazers_count\":6059,\"topics\":\"[coding-font, font, ligatures, monospaced-font, programming-font, programming-ligatures]\"},{\"full_name\":\"JetBrains/intellij-platform-plugin-template\",\"stargazers_count\":1133,\"topics\":\"[intellij, intellij-idea, intellij-idea-plugin, intellij-platform, intellij-plugin, intellij-plugins, jetbrains-plugin]\"}]}" }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "execution_count": 6 }, { "metadata": {}, "cell_type": "markdown", "source": "## Columns Rename" }, { "metadata": {}, "cell_type": "markdown", "source": [ "Columns can be renamed using the `.rename { }` operation, which also uses the Columns Selection DSL to select a column to rename.\n", "The `rename` operation does not perform the renaming immediately; instead, it creates an intermediate object that must be finalized into a new `DataFrame` by calling the `.into()` function with the new column name.\n", "\n", "Rename \"full_name\" and \"stargazers_count\" columns:" ] }, { "metadata": { "ExecuteTime": { "end_time": "2025-05-01T13:42:33.268662Z", "start_time": "2025-05-01T13:42:33.136288Z" } }, "cell_type": "code", "source": [ "// Rename \"full_name\" column into \"name\"\n", "val dfRenamed = dfFiltered\n", " .rename { full_name }.into(\"name\")\n", " // And \"stargazers_count\" into \"starsCount\"\n", " .rename { stargazers_count }.into(\"starsCount\")\n", "dfRenamed" ], "outputs": [ { "data": { "text/html": [ " \n", " \n", " \n", " \n", " \n", " \n", "
namestarsCounttopics
JetBrains/ideavim6120[ideavim, intellij, intellij-platform...
JetBrains/MPS1241[domain-specific-language, dsl]
JetBrains/intellij-community12926[code-editor, ide, intellij, intellij...
JetBrains/intellij-scala1066[intellij-idea, intellij-plugin, scala]
JetBrains/kotlin39402[compiler, gradle-plugin, intellij-pl...
JetBrains/intellij-plugins1737[]
JetBrains/Exposed5688[dao, kotlin, orm, sql]
JetBrains/kotlin-web-site1074[kotlin]
JetBrains/idea-gitignore1181[gitignore, ignore-files, intellij, i...
JetBrains/swot1072[]
JetBrains/phpstorm-stubs1110[]
JetBrains/gradle-intellij-plugin1058[gradle, gradle-intellij-plugin, grad...
JetBrains/svg-sprite-loader1815[sprite, svg, svg-sprite, svg-stack, ...
JetBrains/resharper-unity1017[hacktoberfest, jetbrains, plugin, re...
JetBrains/kotlin-native7101[c, compiler, kotlin, llvm, objective-c]
JetBrains/create-react-kotlin-app2424[create-react-app, jetbrains-ui, kotl...
JetBrains/ring-ui2836[components, jetbrains-ui, react]
JetBrains/kotlinconf-app2628[]
JetBrains/JetBrainsMono6059[coding-font, font, ligatures, monosp...
JetBrains/intellij-platform-plugin-te...1133[intellij, intellij-idea, intellij-id...
\n", " \n", " \n", " " ], "application/kotlindataframe+json": "{\"$version\":\"2.1.1\",\"metadata\":{\"columns\":[\"name\",\"starsCount\",\"topics\"],\"types\":[{\"kind\":\"ValueColumn\",\"type\":\"kotlin.String\"},{\"kind\":\"ValueColumn\",\"type\":\"kotlin.Int\"},{\"kind\":\"ValueColumn\",\"type\":\"kotlin.String\"}],\"nrow\":24,\"ncol\":3},\"kotlin_dataframe\":[{\"name\":\"JetBrains/ideavim\",\"starsCount\":6120,\"topics\":\"[ideavim, intellij, intellij-platform, jb-official, kotlin, vim, vim-emulator]\"},{\"name\":\"JetBrains/MPS\",\"starsCount\":1241,\"topics\":\"[domain-specific-language, dsl]\"},{\"name\":\"JetBrains/intellij-community\",\"starsCount\":12926,\"topics\":\"[code-editor, ide, intellij, intellij-community, intellij-platform]\"},{\"name\":\"JetBrains/intellij-scala\",\"starsCount\":1066,\"topics\":\"[intellij-idea, intellij-plugin, scala]\"},{\"name\":\"JetBrains/kotlin\",\"starsCount\":39402,\"topics\":\"[compiler, gradle-plugin, intellij-plugin, kotlin, kotlin-library, maven-plugin, programming-language]\"},{\"name\":\"JetBrains/intellij-plugins\",\"starsCount\":1737,\"topics\":\"[]\"},{\"name\":\"JetBrains/Exposed\",\"starsCount\":5688,\"topics\":\"[dao, kotlin, orm, sql]\"},{\"name\":\"JetBrains/kotlin-web-site\",\"starsCount\":1074,\"topics\":\"[kotlin]\"},{\"name\":\"JetBrains/idea-gitignore\",\"starsCount\":1181,\"topics\":\"[gitignore, ignore-files, intellij, intellij-plugin, java]\"},{\"name\":\"JetBrains/swot\",\"starsCount\":1072,\"topics\":\"[]\"},{\"name\":\"JetBrains/phpstorm-stubs\",\"starsCount\":1110,\"topics\":\"[]\"},{\"name\":\"JetBrains/gradle-intellij-plugin\",\"starsCount\":1058,\"topics\":\"[gradle, gradle-intellij-plugin, gradle-kotlin-dsl, groovy, intellij, intellij-ides, intellij-platform, intellij-plugin, intellij-sdk, jetbrains-plugin, kotlin, plugin-verifier, publishing-dsl, setup-dsl, teamcity, travis-configuration]\"},{\"name\":\"JetBrains/svg-sprite-loader\",\"starsCount\":1815,\"topics\":\"[sprite, svg, svg-sprite, svg-stack, webpack, webpack-loader, webpack-plugin, webpack2, webpack3]\"},{\"name\":\"JetBrains/resharper-unity\",\"starsCount\":1017,\"topics\":\"[hacktoberfest, jetbrains, plugin, resharper, resharper-plugin, rider, unity, unity-editor]\"},{\"name\":\"JetBrains/kotlin-native\",\"starsCount\":7101,\"topics\":\"[c, compiler, kotlin, llvm, objective-c]\"},{\"name\":\"JetBrains/create-react-kotlin-app\",\"starsCount\":2424,\"topics\":\"[create-react-app, jetbrains-ui, kotlin, react, webpack]\"},{\"name\":\"JetBrains/ring-ui\",\"starsCount\":2836,\"topics\":\"[components, jetbrains-ui, react]\"},{\"name\":\"JetBrains/kotlinconf-app\",\"starsCount\":2628,\"topics\":\"[]\"},{\"name\":\"JetBrains/JetBrainsMono\",\"starsCount\":6059,\"topics\":\"[coding-font, font, ligatures, monospaced-font, programming-font, programming-ligatures]\"},{\"name\":\"JetBrains/intellij-platform-plugin-template\",\"starsCount\":1133,\"topics\":\"[intellij, intellij-idea, intellij-idea-plugin, intellij-platform, intellij-plugin, intellij-plugins, jetbrains-plugin]\"}]}" }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "execution_count": 7 }, { "metadata": {}, "cell_type": "markdown", "source": "## Modify Columns" }, { "metadata": {}, "cell_type": "markdown", "source": [ "Columns can be modified using the `update { }` and `convert { }` operations.\n", "Both operations select columns to modify via the Columns Selection DSL and, similar to `rename`, create an intermediate object that must be finalized to produce a new `DataFrame`.\n", "\n", "The `update` operation preserves the original column types, while `convert` allows changing the type.\n", "In both cases, column names and their positions remain unchanged.\n", "\n", "Update \"name\" and convert \"topics\":" ] }, { "metadata": { "ExecuteTime": { "end_time": "2025-05-01T13:42:33.518212Z", "start_time": "2025-05-01T13:42:33.311730Z" } }, "cell_type": "code", "source": [ "val dfUpdated = dfRenamed\n", " // Update \"name\" values with only its second part (after '/')\n", " .update { name }.with { it.split(\"/\")[1] }\n", " // Convert \"topics\" `String` values into `List` by splitting:\n", " .convert { topics }.with { it.removePrefix(\"[\").removeSuffix(\"]\").split(\", \") }\n", "dfUpdated" ], "outputs": [ { "data": { "text/html": [ " \n", " \n", " \n", " \n", " \n", " \n", "
namestarsCounttopics
ideavim6120[ideavim, intellij, intellij-platform...
MPS1241[domain-specific-language, dsl]
intellij-community12926[code-editor, ide, intellij, intellij...
intellij-scala1066[intellij-idea, intellij-plugin, scala]
kotlin39402[compiler, gradle-plugin, intellij-pl...
intellij-plugins1737[]
Exposed5688[dao, kotlin, orm, sql]
kotlin-web-site1074[kotlin]
idea-gitignore1181[gitignore, ignore-files, intellij, i...
swot1072[]
phpstorm-stubs1110[]
gradle-intellij-plugin1058[gradle, gradle-intellij-plugin, grad...
svg-sprite-loader1815[sprite, svg, svg-sprite, svg-stack, ...
resharper-unity1017[hacktoberfest, jetbrains, plugin, re...
kotlin-native7101[c, compiler, kotlin, llvm, objective-c]
create-react-kotlin-app2424[create-react-app, jetbrains-ui, kotl...
ring-ui2836[components, jetbrains-ui, react]
kotlinconf-app2628[]
JetBrainsMono6059[coding-font, font, ligatures, monosp...
intellij-platform-plugin-template1133[intellij, intellij-idea, intellij-id...
\n", " \n", " \n", " " ], "application/kotlindataframe+json": "{\"$version\":\"2.1.1\",\"metadata\":{\"columns\":[\"name\",\"starsCount\",\"topics\"],\"types\":[{\"kind\":\"ValueColumn\",\"type\":\"kotlin.String\"},{\"kind\":\"ValueColumn\",\"type\":\"kotlin.Int\"},{\"kind\":\"ValueColumn\",\"type\":\"kotlin.collections.List\"}],\"nrow\":24,\"ncol\":3},\"kotlin_dataframe\":[{\"name\":\"ideavim\",\"starsCount\":6120,\"topics\":[\"ideavim\",\"intellij\",\"intellij-platform\",\"jb-official\",\"kotlin\",\"vim\",\"vim-emulator\"]},{\"name\":\"MPS\",\"starsCount\":1241,\"topics\":[\"domain-specific-language\",\"dsl\"]},{\"name\":\"intellij-community\",\"starsCount\":12926,\"topics\":[\"code-editor\",\"ide\",\"intellij\",\"intellij-community\",\"intellij-platform\"]},{\"name\":\"intellij-scala\",\"starsCount\":1066,\"topics\":[\"intellij-idea\",\"intellij-plugin\",\"scala\"]},{\"name\":\"kotlin\",\"starsCount\":39402,\"topics\":[\"compiler\",\"gradle-plugin\",\"intellij-plugin\",\"kotlin\",\"kotlin-library\",\"maven-plugin\",\"programming-language\"]},{\"name\":\"intellij-plugins\",\"starsCount\":1737,\"topics\":[\"\"]},{\"name\":\"Exposed\",\"starsCount\":5688,\"topics\":[\"dao\",\"kotlin\",\"orm\",\"sql\"]},{\"name\":\"kotlin-web-site\",\"starsCount\":1074,\"topics\":[\"kotlin\"]},{\"name\":\"idea-gitignore\",\"starsCount\":1181,\"topics\":[\"gitignore\",\"ignore-files\",\"intellij\",\"intellij-plugin\",\"java\"]},{\"name\":\"swot\",\"starsCount\":1072,\"topics\":[\"\"]},{\"name\":\"phpstorm-stubs\",\"starsCount\":1110,\"topics\":[\"\"]},{\"name\":\"gradle-intellij-plugin\",\"starsCount\":1058,\"topics\":[\"gradle\",\"gradle-intellij-plugin\",\"gradle-kotlin-dsl\",\"groovy\",\"intellij\",\"intellij-ides\",\"intellij-platform\",\"intellij-plugin\",\"intellij-sdk\",\"jetbrains-plugin\",\"kotlin\",\"plugin-verifier\",\"publishing-dsl\",\"setup-dsl\",\"teamcity\",\"travis-configuration\"]},{\"name\":\"svg-sprite-loader\",\"starsCount\":1815,\"topics\":[\"sprite\",\"svg\",\"svg-sprite\",\"svg-stack\",\"webpack\",\"webpack-loader\",\"webpack-plugin\",\"webpack2\",\"webpack3\"]},{\"name\":\"resharper-unity\",\"starsCount\":1017,\"topics\":[\"hacktoberfest\",\"jetbrains\",\"plugin\",\"resharper\",\"resharper-plugin\",\"rider\",\"unity\",\"unity-editor\"]},{\"name\":\"kotlin-native\",\"starsCount\":7101,\"topics\":[\"c\",\"compiler\",\"kotlin\",\"llvm\",\"objective-c\"]},{\"name\":\"create-react-kotlin-app\",\"starsCount\":2424,\"topics\":[\"create-react-app\",\"jetbrains-ui\",\"kotlin\",\"react\",\"webpack\"]},{\"name\":\"ring-ui\",\"starsCount\":2836,\"topics\":[\"components\",\"jetbrains-ui\",\"react\"]},{\"name\":\"kotlinconf-app\",\"starsCount\":2628,\"topics\":[\"\"]},{\"name\":\"JetBrainsMono\",\"starsCount\":6059,\"topics\":[\"coding-font\",\"font\",\"ligatures\",\"monospaced-font\",\"programming-font\",\"programming-ligatures\"]},{\"name\":\"intellij-platform-plugin-template\",\"starsCount\":1133,\"topics\":[\"intellij\",\"intellij-idea\",\"intellij-idea-plugin\",\"intellij-platform\",\"intellij-plugin\",\"intellij-plugins\",\"jetbrains-plugin\"]}]}" }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "execution_count": 8 }, { "metadata": {}, "cell_type": "markdown", "source": "Check the new \"topics\" type out:" }, { "metadata": { "ExecuteTime": { "end_time": "2025-05-01T13:42:33.579845Z", "start_time": "2025-05-01T13:42:33.559883Z" } }, "cell_type": "code", "source": "dfUpdated.topics.type()", "outputs": [ { "data": { "text/plain": [ "kotlin.collections.List" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "execution_count": 9 }, { "metadata": {}, "cell_type": "markdown", "source": "## Adding New Columns" }, { "metadata": {}, "cell_type": "markdown", "source": [ "The `.add { }` function allows creating a `DataFrame` with a new column, where the value for each row is computed based on the existing values in that row. These values can be accessed within the row expressions.\n", "\n", "Add a new `Boolean` column \"isIntellij\":" ] }, { "metadata": { "ExecuteTime": { "end_time": "2025-05-01T13:42:33.704101Z", "start_time": "2025-05-01T13:42:33.591205Z" } }, "cell_type": "code", "source": [ "// Add a `Boolean` column indicating whether the `name` contains the \"intellij\" substring\n", "// or the topics include \"intellij\".\n", "val dfWithIsIntellij = dfUpdated.add(\"isIntellij\") {\n", " name.contains(\"intellij\") || \"intellij\" in topics\n", "}\n", "dfWithIsIntellij" ], "outputs": [ { "data": { "text/html": [ " \n", " \n", " \n", " \n", " \n", " \n", "
namestarsCounttopicsisIntellij
ideavim6120[ideavim, intellij, intellij-platform...true
MPS1241[domain-specific-language, dsl]false
intellij-community12926[code-editor, ide, intellij, intellij...true
intellij-scala1066[intellij-idea, intellij-plugin, scala]true
kotlin39402[compiler, gradle-plugin, intellij-pl...false
intellij-plugins1737[]true
Exposed5688[dao, kotlin, orm, sql]false
kotlin-web-site1074[kotlin]false
idea-gitignore1181[gitignore, ignore-files, intellij, i...true
swot1072[]false
phpstorm-stubs1110[]false
gradle-intellij-plugin1058[gradle, gradle-intellij-plugin, grad...true
svg-sprite-loader1815[sprite, svg, svg-sprite, svg-stack, ...false
resharper-unity1017[hacktoberfest, jetbrains, plugin, re...false
kotlin-native7101[c, compiler, kotlin, llvm, objective-c]false
create-react-kotlin-app2424[create-react-app, jetbrains-ui, kotl...false
ring-ui2836[components, jetbrains-ui, react]false
kotlinconf-app2628[]false
JetBrainsMono6059[coding-font, font, ligatures, monosp...false
intellij-platform-plugin-template1133[intellij, intellij-idea, intellij-id...true
\n", " \n", " \n", " " ], "application/kotlindataframe+json": "{\"$version\":\"2.1.1\",\"metadata\":{\"columns\":[\"name\",\"starsCount\",\"topics\",\"isIntellij\"],\"types\":[{\"kind\":\"ValueColumn\",\"type\":\"kotlin.String\"},{\"kind\":\"ValueColumn\",\"type\":\"kotlin.Int\"},{\"kind\":\"ValueColumn\",\"type\":\"kotlin.collections.List\"},{\"kind\":\"ValueColumn\",\"type\":\"kotlin.Boolean\"}],\"nrow\":24,\"ncol\":4},\"kotlin_dataframe\":[{\"name\":\"ideavim\",\"starsCount\":6120,\"topics\":[\"ideavim\",\"intellij\",\"intellij-platform\",\"jb-official\",\"kotlin\",\"vim\",\"vim-emulator\"],\"isIntellij\":true},{\"name\":\"MPS\",\"starsCount\":1241,\"topics\":[\"domain-specific-language\",\"dsl\"],\"isIntellij\":false},{\"name\":\"intellij-community\",\"starsCount\":12926,\"topics\":[\"code-editor\",\"ide\",\"intellij\",\"intellij-community\",\"intellij-platform\"],\"isIntellij\":true},{\"name\":\"intellij-scala\",\"starsCount\":1066,\"topics\":[\"intellij-idea\",\"intellij-plugin\",\"scala\"],\"isIntellij\":true},{\"name\":\"kotlin\",\"starsCount\":39402,\"topics\":[\"compiler\",\"gradle-plugin\",\"intellij-plugin\",\"kotlin\",\"kotlin-library\",\"maven-plugin\",\"programming-language\"],\"isIntellij\":false},{\"name\":\"intellij-plugins\",\"starsCount\":1737,\"topics\":[\"\"],\"isIntellij\":true},{\"name\":\"Exposed\",\"starsCount\":5688,\"topics\":[\"dao\",\"kotlin\",\"orm\",\"sql\"],\"isIntellij\":false},{\"name\":\"kotlin-web-site\",\"starsCount\":1074,\"topics\":[\"kotlin\"],\"isIntellij\":false},{\"name\":\"idea-gitignore\",\"starsCount\":1181,\"topics\":[\"gitignore\",\"ignore-files\",\"intellij\",\"intellij-plugin\",\"java\"],\"isIntellij\":true},{\"name\":\"swot\",\"starsCount\":1072,\"topics\":[\"\"],\"isIntellij\":false},{\"name\":\"phpstorm-stubs\",\"starsCount\":1110,\"topics\":[\"\"],\"isIntellij\":false},{\"name\":\"gradle-intellij-plugin\",\"starsCount\":1058,\"topics\":[\"gradle\",\"gradle-intellij-plugin\",\"gradle-kotlin-dsl\",\"groovy\",\"intellij\",\"intellij-ides\",\"intellij-platform\",\"intellij-plugin\",\"intellij-sdk\",\"jetbrains-plugin\",\"kotlin\",\"plugin-verifier\",\"publishing-dsl\",\"setup-dsl\",\"teamcity\",\"travis-configuration\"],\"isIntellij\":true},{\"name\":\"svg-sprite-loader\",\"starsCount\":1815,\"topics\":[\"sprite\",\"svg\",\"svg-sprite\",\"svg-stack\",\"webpack\",\"webpack-loader\",\"webpack-plugin\",\"webpack2\",\"webpack3\"],\"isIntellij\":false},{\"name\":\"resharper-unity\",\"starsCount\":1017,\"topics\":[\"hacktoberfest\",\"jetbrains\",\"plugin\",\"resharper\",\"resharper-plugin\",\"rider\",\"unity\",\"unity-editor\"],\"isIntellij\":false},{\"name\":\"kotlin-native\",\"starsCount\":7101,\"topics\":[\"c\",\"compiler\",\"kotlin\",\"llvm\",\"objective-c\"],\"isIntellij\":false},{\"name\":\"create-react-kotlin-app\",\"starsCount\":2424,\"topics\":[\"create-react-app\",\"jetbrains-ui\",\"kotlin\",\"react\",\"webpack\"],\"isIntellij\":false},{\"name\":\"ring-ui\",\"starsCount\":2836,\"topics\":[\"components\",\"jetbrains-ui\",\"react\"],\"isIntellij\":false},{\"name\":\"kotlinconf-app\",\"starsCount\":2628,\"topics\":[\"\"],\"isIntellij\":false},{\"name\":\"JetBrainsMono\",\"starsCount\":6059,\"topics\":[\"coding-font\",\"font\",\"ligatures\",\"monospaced-font\",\"programming-font\",\"programming-ligatures\"],\"isIntellij\":false},{\"name\":\"intellij-platform-plugin-template\",\"starsCount\":1133,\"topics\":[\"intellij\",\"intellij-idea\",\"intellij-idea-plugin\",\"intellij-platform\",\"intellij-plugin\",\"intellij-plugins\",\"jetbrains-plugin\"],\"isIntellij\":true}]}" }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "execution_count": 10 }, { "metadata": {}, "cell_type": "markdown", "source": "## Grouping And Aggregating" }, { "metadata": {}, "cell_type": "markdown", "source": [ "A `DataFrame` can be grouped by column keys, meaning its rows are split into groups based on the values in the key columns.\n", "The `.groupBy { }` operation selects columns and groups the `DataFrame` by their values, using them as grouping keys.\n", "\n", "The result is a `GroupBy` — a `DataFrame`-like structure that associates each key with the corresponding subset of the original `DataFrame`.\n", "\n", "Group `dfWithIsIntellij` by \"isIntellij\":" ] }, { "metadata": { "ExecuteTime": { "end_time": "2025-05-01T13:42:33.786349Z", "start_time": "2025-05-01T13:42:33.748673Z" } }, "cell_type": "code", "source": [ "val groupedByIsIntellij = dfWithIsIntellij.groupBy { isIntellij }\n", "groupedByIsIntellij" ], "outputs": [ { "data": { "text/html": [ " \n", " \n", " \n", " \n", " \n", " \n", "
isIntellijgroup
true
DataFrame [7 x 4]
namestarsCounttopicsisIntellij
ideavim6120[ideavim, intellij, intellij-platform...true
intellij-community12926[code-editor, ide, intellij, intellij...true
intellij-scala1066[intellij-idea, intellij-plugin, scala]true
intellij-plugins1737[]true
idea-gitignore1181[gitignore, ignore-files, intellij, i...true

... showing only top 5 of 7 rows

false
DataFrame [17 x 4]
namestarsCounttopicsisIntellij
MPS1241[domain-specific-language, dsl]false
kotlin39402[compiler, gradle-plugin, intellij-pl...false
Exposed5688[dao, kotlin, orm, sql]false
kotlin-web-site1074[kotlin]false
swot1072[]false

... showing only top 5 of 17 rows

\n", " \n", " \n", " " ], "application/kotlindataframe+json": "{\"$version\":\"2.1.1\",\"metadata\":{\"columns\":[\"isIntellij\",\"group\"],\"types\":[{\"kind\":\"ValueColumn\",\"type\":\"kotlin.Boolean\"},{\"kind\":\"FrameColumn\"}],\"nrow\":2,\"ncol\":2},\"kotlin_dataframe\":[{\"isIntellij\":true,\"group\":{\"data\":[{\"name\":\"ideavim\",\"starsCount\":6120,\"topics\":[\"ideavim\",\"intellij\",\"intellij-platform\",\"jb-official\",\"kotlin\",\"vim\",\"vim-emulator\"],\"isIntellij\":true},{\"name\":\"intellij-community\",\"starsCount\":12926,\"topics\":[\"code-editor\",\"ide\",\"intellij\",\"intellij-community\",\"intellij-platform\"],\"isIntellij\":true},{\"name\":\"intellij-scala\",\"starsCount\":1066,\"topics\":[\"intellij-idea\",\"intellij-plugin\",\"scala\"],\"isIntellij\":true},{\"name\":\"intellij-plugins\",\"starsCount\":1737,\"topics\":[\"\"],\"isIntellij\":true},{\"name\":\"idea-gitignore\",\"starsCount\":1181,\"topics\":[\"gitignore\",\"ignore-files\",\"intellij\",\"intellij-plugin\",\"java\"],\"isIntellij\":true},{\"name\":\"gradle-intellij-plugin\",\"starsCount\":1058,\"topics\":[\"gradle\",\"gradle-intellij-plugin\",\"gradle-kotlin-dsl\",\"groovy\",\"intellij\",\"intellij-ides\",\"intellij-platform\",\"intellij-plugin\",\"intellij-sdk\",\"jetbrains-plugin\",\"kotlin\",\"plugin-verifier\",\"publishing-dsl\",\"setup-dsl\",\"teamcity\",\"travis-configuration\"],\"isIntellij\":true},{\"name\":\"intellij-platform-plugin-template\",\"starsCount\":1133,\"topics\":[\"intellij\",\"intellij-idea\",\"intellij-idea-plugin\",\"intellij-platform\",\"intellij-plugin\",\"intellij-plugins\",\"jetbrains-plugin\"],\"isIntellij\":true}],\"metadata\":{\"kind\":\"FrameColumn\",\"columns\":[\"name\",\"starsCount\",\"topics\",\"isIntellij\"],\"types\":[{\"kind\":\"ValueColumn\",\"type\":\"kotlin.String\"},{\"kind\":\"ValueColumn\",\"type\":\"kotlin.Int\"},{\"kind\":\"ValueColumn\",\"type\":\"kotlin.collections.List\"},{\"kind\":\"ValueColumn\",\"type\":\"kotlin.Boolean\"}],\"ncol\":4,\"nrow\":7}}},{\"isIntellij\":false,\"group\":{\"data\":[{\"name\":\"MPS\",\"starsCount\":1241,\"topics\":[\"domain-specific-language\",\"dsl\"],\"isIntellij\":false},{\"name\":\"kotlin\",\"starsCount\":39402,\"topics\":[\"compiler\",\"gradle-plugin\",\"intellij-plugin\",\"kotlin\",\"kotlin-library\",\"maven-plugin\",\"programming-language\"],\"isIntellij\":false},{\"name\":\"Exposed\",\"starsCount\":5688,\"topics\":[\"dao\",\"kotlin\",\"orm\",\"sql\"],\"isIntellij\":false},{\"name\":\"kotlin-web-site\",\"starsCount\":1074,\"topics\":[\"kotlin\"],\"isIntellij\":false},{\"name\":\"swot\",\"starsCount\":1072,\"topics\":[\"\"],\"isIntellij\":false},{\"name\":\"phpstorm-stubs\",\"starsCount\":1110,\"topics\":[\"\"],\"isIntellij\":false},{\"name\":\"svg-sprite-loader\",\"starsCount\":1815,\"topics\":[\"sprite\",\"svg\",\"svg-sprite\",\"svg-stack\",\"webpack\",\"webpack-loader\",\"webpack-plugin\",\"webpack2\",\"webpack3\"],\"isIntellij\":false},{\"name\":\"resharper-unity\",\"starsCount\":1017,\"topics\":[\"hacktoberfest\",\"jetbrains\",\"plugin\",\"resharper\",\"resharper-plugin\",\"rider\",\"unity\",\"unity-editor\"],\"isIntellij\":false},{\"name\":\"kotlin-native\",\"starsCount\":7101,\"topics\":[\"c\",\"compiler\",\"kotlin\",\"llvm\",\"objective-c\"],\"isIntellij\":false},{\"name\":\"create-react-kotlin-app\",\"starsCount\":2424,\"topics\":[\"create-react-app\",\"jetbrains-ui\",\"kotlin\",\"react\",\"webpack\"],\"isIntellij\":false},{\"name\":\"ring-ui\",\"starsCount\":2836,\"topics\":[\"components\",\"jetbrains-ui\",\"react\"],\"isIntellij\":false},{\"name\":\"kotlinconf-app\",\"starsCount\":2628,\"topics\":[\"\"],\"isIntellij\":false},{\"name\":\"JetBrainsMono\",\"starsCount\":6059,\"topics\":[\"coding-font\",\"font\",\"ligatures\",\"monospaced-font\",\"programming-font\",\"programming-ligatures\"],\"isIntellij\":false},{\"name\":\"skija\",\"starsCount\":2242,\"topics\":[\"2d\",\"graphics\",\"java\",\"skia\"],\"isIntellij\":false},{\"name\":\"projector-docker\",\"starsCount\":1853,\"topics\":[\"awt\",\"docker\",\"swing\"],\"isIntellij\":false},{\"name\":\"projector-server\",\"starsCount\":1025,\"topics\":[\"awt\",\"swing\"],\"isIntellij\":false},{\"name\":\"compose-jb\",\"starsCount\":6805,\"topics\":[\"android\",\"awt\",\"compose\",\"declarative-ui\",\"desktop\",\"gui\",\"javascript\",\"kotlin\",\"multiplatform\",\"reactive\",\"swing\",\"ui\"],\"isIntellij\":false}],\"metadata\":{\"kind\":\"FrameColumn\",\"columns\":[\"name\",\"starsCount\",\"topics\",\"isIntellij\"],\"types\":[{\"kind\":\"ValueColumn\",\"type\":\"kotlin.String\"},{\"kind\":\"ValueColumn\",\"type\":\"kotlin.Int\"},{\"kind\":\"ValueColumn\",\"type\":\"kotlin.collections.List\"},{\"kind\":\"ValueColumn\",\"type\":\"kotlin.Boolean\"}],\"ncol\":4,\"nrow\":17}}}]}" }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "execution_count": 11 }, { "metadata": {}, "cell_type": "markdown", "source": [ "A `GroupBy` can be aggregated — that is, you can compute one or several summary statistics for each group.\n", "The result of the aggregation is a `DataFrame` containing the key columns along with new columns holding the computed statistics for a corresponding group.\n", "\n", "For example, `count()` computes size of group:" ] }, { "metadata": { "ExecuteTime": { "end_time": "2025-05-01T13:42:33.868726Z", "start_time": "2025-05-01T13:42:33.840038Z" } }, "cell_type": "code", "source": "groupedByIsIntellij.count()", "outputs": [ { "data": { "text/html": [ " \n", " \n", " \n", " \n", " \n", " \n", "
isIntellijcount
true7
false17
\n", " \n", " \n", " " ], "application/kotlindataframe+json": "{\"$version\":\"2.1.1\",\"metadata\":{\"columns\":[\"isIntellij\",\"count\"],\"types\":[{\"kind\":\"ValueColumn\",\"type\":\"kotlin.Boolean\"},{\"kind\":\"ValueColumn\",\"type\":\"kotlin.Int\"}],\"nrow\":2,\"ncol\":2},\"kotlin_dataframe\":[{\"isIntellij\":true,\"count\":7},{\"isIntellij\":false,\"count\":17}]}" }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "execution_count": 12 }, { "metadata": {}, "cell_type": "markdown", "source": "Compute several statistics with `.aggregate { }` that provides an expression for aggregating:" }, { "metadata": { "ExecuteTime": { "end_time": "2025-05-01T13:42:33.937900Z", "start_time": "2025-05-01T13:42:33.881232Z" } }, "cell_type": "code", "source": [ "groupedByIsIntellij.aggregate {\n", " // Compute sum and max of \"starsCount\" within each group into \"sumStars\" and \"maxStars\" columns\n", " sumOf { starsCount } into \"sumStars\"\n", " maxOf { starsCount } into \"maxStars\"\n", "}" ], "outputs": [ { "data": { "text/html": [ " \n", " \n", " \n", " \n", " \n", " \n", "
isIntellijsumStarsmaxStars
true2522112926
false8539239402
\n", " \n", " \n", " " ], "application/kotlindataframe+json": "{\"$version\":\"2.1.1\",\"metadata\":{\"columns\":[\"isIntellij\",\"sumStars\",\"maxStars\"],\"types\":[{\"kind\":\"ValueColumn\",\"type\":\"kotlin.Boolean\"},{\"kind\":\"ValueColumn\",\"type\":\"kotlin.Int\"},{\"kind\":\"ValueColumn\",\"type\":\"kotlin.Int\"}],\"nrow\":2,\"ncol\":3},\"kotlin_dataframe\":[{\"isIntellij\":true,\"sumStars\":25221,\"maxStars\":12926},{\"isIntellij\":false,\"sumStars\":85392,\"maxStars\":39402}]}" }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "execution_count": 13 }, { "metadata": {}, "cell_type": "markdown", "source": "## Sorting Rows" }, { "metadata": {}, "cell_type": "markdown", "source": [ "`.sort {}`/`.sortByDesc` sortes rows by value in selected columns, returning a DataFrame with sorted rows. `take(n)` returns a new `DataFrame` with the first `n` rows.\n", "\n", "Combine them to get Top-10 repositories by number of stars:" ] }, { "metadata": { "ExecuteTime": { "end_time": "2025-05-01T13:42:34.085822Z", "start_time": "2025-05-01T13:42:34.039730Z" } }, "cell_type": "code", "source": [ "val dfTop10 = dfWithIsIntellij\n", " // Sort by \"starsCount\" value descending\n", " .sortByDesc { starsCount }\n", " .take(10)\n", "dfTop10" ], "outputs": [ { "data": { "text/html": [ " \n", " \n", " \n", " \n", " \n", " \n", "
namestarsCounttopicsisIntellij
kotlin39402[compiler, gradle-plugin, intellij-pl...false
intellij-community12926[code-editor, ide, intellij, intellij...true
kotlin-native7101[c, compiler, kotlin, llvm, objective-c]false
compose-jb6805[android, awt, compose, declarative-u...false
ideavim6120[ideavim, intellij, intellij-platform...true
JetBrainsMono6059[coding-font, font, ligatures, monosp...false
Exposed5688[dao, kotlin, orm, sql]false
ring-ui2836[components, jetbrains-ui, react]false
kotlinconf-app2628[]false
create-react-kotlin-app2424[create-react-app, jetbrains-ui, kotl...false
\n", " \n", " \n", " " ], "application/kotlindataframe+json": "{\"$version\":\"2.1.1\",\"metadata\":{\"columns\":[\"name\",\"starsCount\",\"topics\",\"isIntellij\"],\"types\":[{\"kind\":\"ValueColumn\",\"type\":\"kotlin.String\"},{\"kind\":\"ValueColumn\",\"type\":\"kotlin.Int\"},{\"kind\":\"ValueColumn\",\"type\":\"kotlin.collections.List\"},{\"kind\":\"ValueColumn\",\"type\":\"kotlin.Boolean\"}],\"nrow\":10,\"ncol\":4},\"kotlin_dataframe\":[{\"name\":\"kotlin\",\"starsCount\":39402,\"topics\":[\"compiler\",\"gradle-plugin\",\"intellij-plugin\",\"kotlin\",\"kotlin-library\",\"maven-plugin\",\"programming-language\"],\"isIntellij\":false},{\"name\":\"intellij-community\",\"starsCount\":12926,\"topics\":[\"code-editor\",\"ide\",\"intellij\",\"intellij-community\",\"intellij-platform\"],\"isIntellij\":true},{\"name\":\"kotlin-native\",\"starsCount\":7101,\"topics\":[\"c\",\"compiler\",\"kotlin\",\"llvm\",\"objective-c\"],\"isIntellij\":false},{\"name\":\"compose-jb\",\"starsCount\":6805,\"topics\":[\"android\",\"awt\",\"compose\",\"declarative-ui\",\"desktop\",\"gui\",\"javascript\",\"kotlin\",\"multiplatform\",\"reactive\",\"swing\",\"ui\"],\"isIntellij\":false},{\"name\":\"ideavim\",\"starsCount\":6120,\"topics\":[\"ideavim\",\"intellij\",\"intellij-platform\",\"jb-official\",\"kotlin\",\"vim\",\"vim-emulator\"],\"isIntellij\":true},{\"name\":\"JetBrainsMono\",\"starsCount\":6059,\"topics\":[\"coding-font\",\"font\",\"ligatures\",\"monospaced-font\",\"programming-font\",\"programming-ligatures\"],\"isIntellij\":false},{\"name\":\"Exposed\",\"starsCount\":5688,\"topics\":[\"dao\",\"kotlin\",\"orm\",\"sql\"],\"isIntellij\":false},{\"name\":\"ring-ui\",\"starsCount\":2836,\"topics\":[\"components\",\"jetbrains-ui\",\"react\"],\"isIntellij\":false},{\"name\":\"kotlinconf-app\",\"starsCount\":2628,\"topics\":[\"\"],\"isIntellij\":false},{\"name\":\"create-react-kotlin-app\",\"starsCount\":2424,\"topics\":[\"create-react-app\",\"jetbrains-ui\",\"kotlin\",\"react\",\"webpack\"],\"isIntellij\":false}]}" }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "execution_count": 14 }, { "metadata": {}, "cell_type": "markdown", "source": "## Plotting With Kandy" }, { "metadata": {}, "cell_type": "markdown", "source": [ "Kandy is a Kotlin plotting library designed to bring Kotlin DataFrame features into chart creation, providing a convenient and typesafe way to build data visualizations.\n", "\n", "Kandy can be loaded into notebook using `%use kandy`:" ] }, { "metadata": { "ExecuteTime": { "end_time": "2025-05-01T13:42:37.589007Z", "start_time": "2025-05-01T13:42:34.130284Z" } }, "cell_type": "code", "source": "%use kandy", "outputs": [], "execution_count": 15 }, { "metadata": {}, "cell_type": "markdown", "source": "Build a simple bar chart with `.plot { }` extension for DataFrame, that allows to use extension properties inside Kandy plotting DSL (plot will be rendered as an output after cell execution):" }, { "metadata": { "ExecuteTime": { "end_time": "2025-05-01T13:42:37.812897Z", "start_time": "2025-05-01T13:42:37.592435Z" } }, "cell_type": "code", "source": [ "dfTop10.plot {\n", " bars {\n", " x(name)\n", " y(starsCount)\n", " }\n", "\n", " layout.title = \"Top 10 JetBrains repositories by stars count\"\n", "}" ], "outputs": [ { "data": { "text/html": [ " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " kotlin\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " intellij-community\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " kotlin-native\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " compose-jb\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " ideavim\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " JetBrainsMono\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " Exposed\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " ring-ui\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " kotlinconf-app\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " create-react-kotlin-app\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " 0\n", " \n", " \n", " \n", " \n", " \n", " \n", " 10,000\n", " \n", " \n", " \n", " \n", " \n", " \n", " 20,000\n", " \n", " \n", " \n", " \n", " \n", " \n", " 30,000\n", " \n", " \n", " \n", " \n", " \n", " \n", " 40,000\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " Top 10 JetBrains repositories by stars count\n", " \n", " \n", " \n", " \n", " starsCount\n", " \n", " \n", " \n", " \n", " name\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " " ], "application/plot+json": { "output_type": "lets_plot_spec", "output": { "ggtitle": { "text": "Top 10 JetBrains repositories by stars count" }, "mapping": {}, "data": { "starsCount": [ 39402.0, 12926.0, 7101.0, 6805.0, 6120.0, 6059.0, 5688.0, 2836.0, 2628.0, 2424.0 ], "name": [ "kotlin", "intellij-community", "kotlin-native", "compose-jb", "ideavim", "JetBrainsMono", "Exposed", "ring-ui", "kotlinconf-app", "create-react-kotlin-app" ] }, "kind": "plot", "scales": [ { "aesthetic": "x", "discrete": true }, { "aesthetic": "y", "limits": [ null, null ] } ], "layers": [ { "mapping": { "x": "name", "y": "starsCount" }, "stat": "identity", "sampling": "none", "inherit_aes": false, "position": "dodge", "geom": "bar" } ], "data_meta": { "series_annotations": [ { "type": "str", "column": "name" }, { "type": "int", "column": "starsCount" } ] } }, "apply_color_scheme": true, "swing_enabled": true } }, "execution_count": 16, "metadata": {}, "output_type": "execute_result" } ], "execution_count": 16 }, { "metadata": {}, "cell_type": "markdown", "source": "## Write DataFrame" }, { "metadata": {}, "cell_type": "markdown", "source": [ "A `DataFrame` supports writing to all formats that it is capable of reading.\n", "\n", "Write into Excel:" ] }, { "metadata": { "ExecuteTime": { "end_time": "2025-05-01T13:42:38.118495Z", "start_time": "2025-05-01T13:42:37.851592Z" } }, "cell_type": "code", "source": "dfWithIsIntellij.writeExcel(\"jb_repos.xlsx\")", "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "2025-05-01T13:42:37.894239Z Execution of code 'dfWithIsIntellij.wri...' ERROR Log4j2 could not find a logging implementation. Please add log4j-core to the classpath. Using SimpleLogger to log to the console...\n" ] } ], "execution_count": 17 } ], "metadata": { "kernelspec": { "display_name": "Kotlin", "language": "kotlin", "name": "kotlin" }, "language_info": { "name": "kotlin", "version": "1.9.23", "mimetype": "text/x-kotlin", "file_extension": ".kt", "pygments_lexer": "kotlin", "codemirror_mode": "text/x-kotlin", "nbconvert_exporter": "" } }, "nbformat": 4, "nbformat_minor": 0 }