giovanni.ciatto@unibo.it
Compiled on: 2024-11-21 — printable version
Fuzzy concept
Insight:
The substratum upon which software applications run
Programming languages?
Runtimes
Software platform
anything having an API enabling the writing of applications, and the runtime supporting the execution of those applications
API
application programming interface(s) a formal specification of the set of functionalities provided by a software (sub-)system for external usage, there including their input, outputs, and enviromental preconditions and effects
Runtime [system/environment]
the set of computational resources backing the execution of a software (sub-)system
all possible public interfaces / classes / structures in an OOP module
all possible commands a CLI application accept as input
all possible paths a Web service may accept HTTP request onto
any virtual machine (JVM, CRL, CPython, V8)
any operative system (Win, Mac, Linux)
any Web service
The Java Virtual Machine (JVM)
.NET’s [pronunced “dot NET”] Common Language Runtime (CRL)
Python 3
NodeJS (V8)
Each browser may be considered as a platform per se
…
standard libraries
predefined design decisions
organizational, stylistic, technical conventions
packaging conventions, import mechanisms, and software repositories
user communities
Types from the java.*
and javax.*
packages are usable from any JVM language
Many nice functionalities covering:
Many functionalities are provided by community-driven third party libraries
Everything is (indirectly) a subclass/instance of Object
Every object is potentially a lock
Default methods inherited by Object
class
toString
, equals
, hashCode
All methods are virtual by default
…
Project should be organized according to the Maven’s standard directory layout
Official stylistic conventions for most JVM languages
PascalCase
, members names in camelCase
Many technical conventions:
Iterable
interfaceIterable
s as inputCode is organized into packages
Code archives (.jar
) are Zip files containing compiled classes
Basic import mechanism: the class path
Many third-party repositories for JVM libraries
Android developers
Back-end Web developers
Desktop applications developers
Researchers in the fields of: semantic Web, multi-agent systems, etc.
…
Notably, one the richest standard libraries ever
Many nice functionalities covering:
Many functionalities are provided by community-driven third party libraries
Everything is (indirectly) a subclass/instance of object
Global Interpreter Lock (GIL)
Magic methods supporting various language features
__str__
, __eq__
, __iter__
No support for overloading
Variadic and keywords arguments
…
Project should be organized according to Kenneth Reitz’s layout
Official stylistic conventions for Python (PEP8)
PascalCase
, members names in snake_case
Many technical conventions:
__iter__
methodCode is organized into packages and modules
Code archives (.whl
) are Zip files containing Python sources
Each Python installation has an internal folder where libraries are stored
pip
simply unzips modules/packages in thereimport
statements look for packages/modules in therePypi as the official repository for Python libraries
pip
as the official tool for dependency managementData-science community
Back-end Web developers
Desktop applications developers
System administrators
…
Very limited standard library from JavaScript
Many nice functionalities covering:
Many functionalities are provided by community-driven third party libraries
Object orientation based on prototypes
Single threaded design + event loop
Asynchronous programming via continuation-passing style
Project structure is somewhat arbitrary
package.json
fileMany conventions co-exist
Many technical conventions:
Code is organized into modules
Code archives (.tar.?z
) are compressed tarball files containing JS sources
Third party libraries can be installed via npm
NPM as the official repository for JS libraries
npm
as the official tool for dependency managementFront-end Web developers
Back-end Web developers
GUI developers
…
The choice of a platform impacts developers during:
the design phase
the implementation phase
the testing phase
the release phase
Abstraction gap
the space among the problem and the prior functionalities offered by a platform. Ideally, the bigger the space the more effort is required to build the solution
Developers build solutions by leveraging the API of the platform
… as well as the API of any third-party library available for that platform
Test suites are a “project in the project”
One may test the system against as many versions as possible of the underlying platforms
One may test the system against as many OS as possible
Release
publishing some packaged software system onto a repository, hence enabling its import and exploitation
Packaging systems are platform-specific…
Repositories are platform-specific…
… release is therefore platform specific
Choice is commonly driven by design / technical decisions
However, choosing the platform is a business decision as well
Business decision: which user communities to target?
Coherency is key for success in platform selection
The abstraction gap is likely lower
More third party libraries are likey available
The potential audience is wider
Easier to find support / help in case of issues
More likely that third-party issues are timely fixed
Researchers may act as software developers
to elaborate data
to create in-silico experiments
to study software system
to create software tools improving their research
to create software tools for the community
Research institutions are not software houses
Personnel can only dedicate a fraction of their time to development
Most software artifacts are disposable (1, 2, 4)
Development efforts are discontinuous
Development teams are small
Software is commonly a means, not an objective
Commitment to software development is:
Research-oriented software development should maximise audience and impact, while minimising development and maintenance effort
Science requires reproducibility
The wider the community, the wider the impact of community-driven research software
Minimising effort can be done by improving efficiency
Choosing the right community / platform is strategical for research-oriented software
The abstraction gap is likely lower
More third party libraries are likely available
The potential audience is wider
Easier to find support / help in case of issues
More likely that third-party issues are timely fixed
Silos (in IT) are software components / systems / ecosystems having poor external interoperability (i.e. software from silos A hardly interoperates with software from silos B)
Platforms are (pretty wide) software silos
Examples:
Research communities in CS / AI may overlap with platforms communities:
What about inter-community research efforts?
Multi-platform programming is an enabler for inter-community research
let the same software tool run on multiple platforms
Create multiple artifacts, one per each supported platform, sharing the same design and functioning
design and write the software once, then port it to several platforms
Write once, build anywhere
Write first, wrap elsewhere
Assumptions:
Workflow:
Design, implement, and test most of the project via the super-language
Complete, refine, or optimise platform-specific aspects via
Build platform-specific artifacts, following platform-specific rules
Upload platform-specific artifacts on platform-specific repositories
Let N
be the amount of supported platforms
Platform-agnostic functionalities require effort which is independent from N
Platform-specific functionalities require effort proportional to N
Better to minimise platform-specific code
Relevant questions:
The abstraction gap of the common code is as wide as the one of the platform having the widest abstraction gap
Whenever a new functionality needs to be developed:
Try to realise it with common std-lib only
If not possible, try to maximise the portion of platform-agnostic code
N
times, one per target platformAssumptions:
Workflow:
Fully implement, test, and deploy the software for the main platform
For all other platforms:
re-design and re-write platform-specific API code
implement platform-specific API by calling the main platform’s code
re-write test for API code
build platform-specific packages, wrapping the main platform’s package and runtime
Upload platform-specific artifacts on platform-specific repositories
Let N
be the amount of supported platforms
Clear separation of API code from implementation code is quintessential
The effort required for writing API code is virtually the same on all platforms
Global effort is sub-linearly dependent on N
N
timesThe i
-th platform’s wrapper code will only call the main platform’s API code
Relevant questions:
JetBrains-made modern programming language
Gaining momentum since Google adopted is as official Android language
Clearly inspired by a mixture of Java, C#, Scala, and Groovy
Born in industry, for the industry
Acts as the “super language” supporting the “write once, build anywhere” approach
Reference: https://kotlinlang.org/docs/reference/mpp-dsl-reference.html
JVM
11
JavaScript (JS)
Native
Reference: https://kotlinlang.org/docs/reference/mpp-dsl-reference.html
Win by MinGW, Linux
iOS, macOS
Android
Building a multi-platform Kotlin project is only supported via Gradle
Gradle is a build automation + dependency management system
Gradle simultaneously enables & constrains the multi-platform workflow
Kotlin enforces strong segregation of the platform-agnostic and platform-specific parts
common main code: platform-agnostic code which only depends on:
common test code: platform-agnostic test code which only depends on:
kotlin.test
or Kotest)T
(e.g. jvm
, js
, etc.)
T
-specific main code: main Kotlin code targeting the T
platform, depending on:
T
-specific standard libraryT
-specific third-party librariesT
-specific test code: test Kotlin code targeting the T
platform, depending on:
T
-specific main codeT
-specific test libraries<root>/
│
├── build.gradle.kts # multi-platform build description
├── settings.gradle.kts # root project settings
│
└── src/
├── commonMain/
│ └── kotlin/ # platform-agnostic Kotlin code here font-weight: normal;
├── commonTest/
│ └── kotlin/ # platform-agnostic test code written in Kotlin here
│
├── jvmMain/
│ ├── java/ # JVM-specific Java code here
│ ├── kotlin/ # JVM-specific Kotlin code here
│ └── resources/ # JVM-specific resources here
├── jvmTest
│ ├── java/ # JVM-specific test code written in Java here
│ ├── kotlin/ # JVM-specific test code written in Kotlin here
│ └── resources/ # JVM-specific test resources here
│
├── jsMain/
│ └── kotlin/ # JS-specific Kotlin code here
├── jsTest/
│ └── kotlin/ # JS-specific Kotlin code here
│
├── <TARGET>Main/
│ └── kotlin/ # <TARGET>-specific Kotlin code here
└── <TARGET>Test/
└── kotlin/ # <TARGET>-specific test code written in Kotlin here
Defines several aspects of the project:
which version of the Kotlin compiler to adopt
plugins {
kotlin("multiplatform") version "1.9.10" // defines plugin and compiler version
}
which repositories should Gradle use when looking for dependencies
repositories {
mavenCentral() // use MCR for downloading dependencies (recommneded)
// other custom repositories here (discouraged)
}
Defines several aspects of the project:
which platforms to target (reference here)
kotlin {
// declares JVM as target
jvm {
withJava() // jvm-specific targets may include java sources
}
// declares JavaScript as target
js {
useCommonJs() // use CommonJS for JS depenencies management
// or useEsModules()
binaries.executable() // enables tasks for Node packages generation
// the target will consist of a Node project (with NodeJS's stdlib)
nodejs {
runTask { /* configure project running in Node here */ }
testRuns { /* configure Node's testing frameworks */ }
}
// alternatively, or additionally to nodejs:
browser { /* ... */ }
}
// other targets here
}
other admissible targets:
android
kotlin {
sourceSets {
val commonMain by getting {
dependencies {
api(kotlin("stdlib-common"))
implementation("group.of", "multiplatform-library", "X.Y.Z") // or api
}
}
val commonTest by getting {
dependencies {
implementation(kotlin("test-common"))
}
}
val jvmMain by getting {
dependencies {
api(kotlin("stdlib-jdk8"))
implementation("group.of", "jvm-library", "X.Y.Z") // or api
}
}
val jvmTest by getting {
dependencies {
implementation(kotlin("test-junit"))
}
}
val jsMain by getting {
dependencies {
api(kotlin("stdlib-js"))
implementation(npm("npm-module", "X.Y.Z")) // lookup on https://www.npmjs.com
}
}
val jsTest by getting {
dependencies {
implementation(kotlin("test-js"))
}
}
}
}
configure Kotlin compiler options
kotlin {
sourceSets.all {
languageSettings.apply {
// provides source compatibility with the specified version of Kotlin.
languageVersion = "1.8" // possible values: "1.4", "1.5", "1.6", "1.7", "1.8", "1.9"
// allows using declarations only from the specified version of Kotlin bundled libraries.
apiVersion = "1.8" // possible values: "1.3", "1.4", "1.5", "1.6", "1.7", "1.8", "1.9"
// enables the specified language feature
enableLanguageFeature("InlineClasses") // language feature name
// allow using the specified opt-in
optIn("kotlin.ExperimentalUnsignedTypes") // annotation FQ-name
// enables/disable progressive mode
progressiveMode = true // false by default
}
}
}
details about:
plugins {
kotlin("multiplatform") version "1.9.10"
}
repositories {
mavenCentral()
}
kotlin {
jvm {
withJava()
}
js {
nodejs {
runTask { /* ... */ }
testRuns { /* ... */ }
}
// alternatively, or additionally to nodejs:
browser { /* ... */ }
}
sourceSets {
val commonMain by getting {
dependencies {
api(kotlin("stdlib-common"))
implementation("group.of", "multiplatform-library", "X.Y.Z") // or api
}
}
val commonTest by getting {
dependencies {
implementation(kotlin("test-common"))
}
}
val jvmMain by getting {
dependencies {
api(kotlin("stdlib-jdk8"))
implementation("group.of", "jvm-library", "X.Y.Z") // or api
}
}
val jvmTest by getting {
dependencies {
implementation(kotlin("test-junit"))
}
}
val jsMain by getting {
dependencies {
api(kotlin("stdlib-js"))
implementation(npm("npm-module", "X.Y.Z")) // lookup on https://www.npmjs.com
}
}
val jsTest by getting {
dependencies {
implementation(kotlin("test-js"))
}
}
all {
languageVersion = "1.8"
apiVersion = "1.8"
enableLanguageFeature("InlineClasses")
optIn("kotlin.ExperimentalUnsignedTypes")
progressiveMode = true // false by default
}
}
}
Let T
denote the target name (e.g. jvm
, js
, etc.)
<T>MainClasses
compiles the main code for platform T
jvmMainClasses
, jsMainClasses
<T>TestClasses
compiles the test code for platform T
jvmTestClasses
, jsTestClasses
<T>Jar
compiles the main code for platform T
and produces a JAR out of it
jvmJar
, jsJar
<T>MainClasses
, but NOT <T>TestClasses
<T>Test
executes tests for platform T
jvmTest
, jsTest
<T>MainClasses
, AND on <T>TestClasses
compileProductionExecutableKotlinJs
compiles the JS main code into a Node project
js
target to be enabledbinaries.executable()
configuration to be enabledassemble
creates all JARs (hence compiling for main code for all platforms)
test
executes tests for all platforms
check
like test
but it may also include other checks (e.g. style) if configured
build
check
+ assemble
Ordinary Kotlin sources
When in common
:
Common
)expect
keyword@JvmStatic
, @JsName
, etc.Let T
denote some target platform
When in T
-specific source sets
T
-specific std-libT
-specific Kotlin librariesT
-specific librariesactual
keywordEach platform T
may allow for specific keywords
external
modifierexpect
/actual
mechanism for specialising common APIDeclaring an expect
ed function / type declaration in common code…
… enforces the existence of a corresponding actual
definition for all platforms
Draw a platform-agnostic design for your domain entities
expect
keyword to declare platform-agnostic factoriesAssess the abstraction gap, for each target platform
For each target platform:
actual
keyword to implement platform-specific factoriesCSV (comma-separated values) files, e.g.:
# HEADER_1, HEADER_2, HEADER_3, ...
# character '#' denotes the beginning of a single-line comment
# first line conventionally denotes columns names (headers)
field1, filed2, field3, ...
# character ',' acts as field separator
"field with spaces", "another field, with comma", "yet another field", ...
# character '"' acts as field delimiter
# other characters may be used to denote comments, separators, or delimiters
JVM and JS std-lib do not provide direct support for CSV
Requirements:
We’ll follow a domain-driven approach:
Domain entities (can be realised as common code):
Table
: in-memory representation of a CSV fileRow
: generic row in a CSV fileHeader
: special row containing the names of the columnsRecord
: special row containing the values of a line in a CSV fileMain functionalities (require platform specific code):
Table
programmaticallyTable
Table
Table
into a CSV fileRow
interface// Represents a single row in a CSV file.
interface Row : Iterable<String> {
// Gets the value of the field at the given index.
operator fun get(index: Int): String
// Gets the number of fields in this row.
val size: Int
}
Header
interface// Represents the header of a CSV file.
// A header is a special row containing the names of the columns.
interface Header : Row {
// Gets the names of the columns.
val columns: List<String>
// Checks whether the given column name is present in this header.
operator fun contains(column: String): Boolean
// Gets the index of the given column name.
fun indexOf(column: String): Int
}
Record
interface// Represents a record in a CSV file.
interface Record : Row {
// Gets the header of this record (useful to know column names).
val header: Header
// Gets the values of the fields in this record.
val values: List<String>
// Checks whether the given value is present in this record.
operator fun contains(value: String): Boolean
// Gets the value of the field in the given column.
operator fun get(column: String): String
}
Table
interface// Represents a table (i.e. an in-memory representation of a CSV file).
interface Table : Iterable<Row> {
// Gets the header of this table (useful to know column names).
val header: Header
// Gets the records in this table.
val records: List<Record>
// Gets the row at the given index.
operator fun get(row: Int): Row
// Gets the number of rows in this table.
val size: Int
}
AbstractRow
class// Base implementation for the Row interface.
internal abstract class AbstractRow(protected open val values: List<String>) : Row {
override val size: Int
get() = values.size
override fun get(index: Int): String = values[index]
override fun equals(other: Any?): Boolean {
if (this === other) return true
if (other == null || this::class != other::class) return false
return values == (other as AbstractRow).values
}
override fun hashCode(): Int = values.hashCode()
// Returns a string representation of this row as "<type>(field1, field2, ...)".
protected fun toString(type: String?): String {
var prefix = ""
var suffix = ""
if (type != null) {
prefix = "$type("
suffix = ")"
}
return values.joinToString(", ", prefix, suffix) { "\"$it\"" }
}
// Returns a string representation of this row as "Row(field1, field2, ...)".
override fun toString(): String = toString("Row")
// Makes it possible to iterate over the fields of this row, via for-each loops.
override fun iterator(): Iterator<String> = values.iterator()
}
DefaultHeader
class// Default implementation for the Header interface.
internal class DefaultHeader(columns: Iterable<String>) : Header, AbstractRow(columns.toList()) {
// Cache of column indexes, for faster lookup.
private val indexesByName = columns.mapIndexed { index, name -> name to index }.toMap()
override val columns: List<String>
get() = values
override fun contains(column: String): Boolean = column in indexesByName.keys
override fun indexOf(column: String): Int = indexesByName[column] ?: -1
override fun iterator(): Iterator<String> = values.iterator()
override fun toString(): String = toString("Header")
}
DefaultRecord
classinternal class DefaultRecord(override val header: Header, values: Iterable<String>) : Record, AbstractRow(values.toList()) {
init {
require(header.size == super.values.size) {
"Inconsistent amount of values (${super.values.size}) w.r.t. to header size (${header.size})"
}
}
override fun contains(value: String): Boolean = values.contains(value)
override val values: List<String>
get() = super.values
override fun get(column: String): String =
header.indexOf(column).takeIf { it in 0..< size }?.let { values[it] }
?: throw NoSuchElementException("No such column: $column")
override fun toString(): String = toString("Record")
}
DefaultTable
classinternal class DefaultTable(override val header: Header, records: Iterable<Record>) : Table {
// Lazy, defensive copy of the records.
override val records: List<Record> by lazy { records.toList() }
override fun get(row: Int): Row = if (row == 0) header else records[row - 1]
override val size: Int
get() = records.size + 1
override fun iterator(): Iterator<Row> = (sequenceOf(header) + records.asSequence()).iterator()
override fun equals(other: Any?): Boolean {
if (this === other) return true
if (other == null || other !is DefaultTable) return false
if (header != other.header) return false
if (records != other.records) return false
return true
}
override fun hashCode(): Int {
var result = header.hashCode()
result = 31 * result + records.hashCode()
return result
}
override fun toString(): String = this.joinToString(", ", "Table(", ")")
}
To enforce separation among API and implementation code, it’s better:
Convention in Kotlin is to create factory methods as package-level fun
ctions
io/github/gciatto/csv/Csv.kt
fileFactory methods may be named after the concept they create: <concept>Of(args...)
headerOf(columns...)
, recordOf(header, values...)
, etc.Class diagram:
Csv.kt
file// Headers creation from columns names
fun headerOf(columns: Iterable<String>): Header = DefaultHeader(columns)
fun headerOf(vararg columns: String): Header = headerOf(columns.asIterable())
// Creates anonymous headers, with columns named after their index
fun anonymousHeader(size: Int): Header = headerOf((0 ..< size).map { it.toString() })
// Records creation from header and values
fun recordOf(header: Header, columns: Iterable<String>): Record = DefaultRecord(header, columns)
fun recordOf(header: Header, vararg columns: String): Record = recordOf(header, columns.asIterable())
// Tables creation from header and records
fun tableOf(header: Header, records: Iterable<Record>): Table = DefaultTable(header, records)
fun tableOf(header: Header, vararg records: Record): Table = tableOf(header, records.asIterable())
// Tables creation from rows (anonymous header if none is provided)
fun tableOf(rows: Iterable<Row>): Table {
val records = mutableListOf<Record>()
var header: Header? = null
for (row in rows) {
when (row) {
is Header -> header = row
is Record -> records.add(row)
}
}
require(header != null || records.isNotEmpty())
return tableOf(header ?: anonymousHeader(records[0].size), records)
}
Iterable
and vararg
argumentsDummy instances object:
object Tables {
val irisShortHeader = headerOf("sepal_length", "sepal_width", "petal_length", "petal_width", "class")
val irisLongHeader = headerOf("sepal length in cm", "sepal width, in cm", "petal length", "petal width", "class")
fun iris(header: Header): Table = tableOf(
header,
recordOf(header, "5.1", "3.5", "1.4", "0.2", "Iris-setosa"),
recordOf(header, "4.9", "3.0", "1.4", "0.2", "Iris-setosa"),
recordOf(header, "4.7", "3.2", "1.3", "0.2", "Iris-setosa")
)
}
Some basic tests in file TestCSV.kt
, e.g. (bad way of writing test methods, don’t do this at home)
@Test
fun recordBasics() {
val record = Tables.iris(Tables.irisShortHeader).records[0]
assertEquals("5.1", record[0])
assertEquals("5.1", record["sepal_length"])
assertEquals("0.2", record[3])
assertEquals("0.2", record["petal_width"])
assertEquals("Iris-setosa", record[4])
assertEquals("Iris-setosa", record["class"])
assertFailsWith<IndexOutOfBoundsException> { record[5] }
assertFailsWith<NoSuchElementException> { record["missing"] }
}
Run tests via Gradle task test
(also try to run tests for specific platforms, e.g. jvmTest
or jsTest
)
We designed a set of domain entities
Row
, Header
, Record
, Table
We implemented the domain entities as common code
We wrote some unit tests for the domain entities
Let’s focus now on more complex functionalities
Table
Table
into a CSV fileI/O is platform-specific, hence we need platform-specific code
I/O functionalities are supported by fairly different API in JVM and JS
java.io
package vs. JS’ fs
moduleDo we need to rewrite the same business logic twice? (once for JVM, once for JS)
Let’s try to decompose the problem as follows:
Table
: file Table
Table
as CSV file: Table
Remarks:
Table
” part is platform-agnostic
Configuration
: set of characters to be used to parse / represent CSV files
Formatter
: converts Rows
into strings, according to some Configuration
Parser
: converts some source into a Table
, according to some Configuration
Configuration
classdata class Configuration(val separator: Char, val delimiter: Char, val comment: Char) {
// Checks whether the given string is a comment (i.e. starts with the comment character).
fun isComment(string: String): Boolean = // ...
// Gets the content of the comment line (i.e. removes the initial comment character).
fun getComment(string: String): String? = // ...
// Checks whether the given string is a record (i.e. it contains the delimiter character)
fun isRecord(string: String): Boolean = // ...
// Retrieves the fields in the given record (i.e. splits the string at the separator character)
fun getFields(string: String): List<String> = // ...
// Checks whether the given string is a header (i.e. simultaneously a record and a comment)
fun isHeader(string: String): Boolean = // ...
// Retrieves the column names in the given header
fun getColumnNames(string: String): List<String> = // ...
}
Formatter
interfaceinterface Formatter {
// The source of this formatter (i.e. the rows to be formatted).
val source: Iterable<Row>
// The configuration of this formatter (i.e. the characters to be used).
val configuration: Configuration
// Formats the source of this formatter into a sequence of strings (one per each row in the source)
fun format(): Iterable<String>
}
Parser
interfaceinterface Parser {
// The source to be parsed by this parser (must be interpretable as string)
val source: Any
// The configuration of this parser (i.e. the characters to be used).
val configuration: Configuration
// Parses the source of this parser into a sequence of rows (one per each row in the source)
fun parse(): Iterable<Row>
}
DefaultFormatter
classclass DefaultFormatter(override val source: Iterable<Row>, override val configuration: Configuration) : Formatter {
// Lazily converts each row from the source into a string, according to the configuration.
override fun format(): Iterable<String> = source.asSequence().map(this::formatRow).asIterable()
// Converts the given row into a string, according to the configuration.
private fun formatRow(row: Row): String = when (row) {
is Header -> formatAsHeader(row)
else -> formatAsRecord(row)
}
// Formats the given row as a header (putting the comment character at the beginning).
private fun formatAsHeader(row: Row): String = "${configuration.comment} ${formatAsRecord(row)}"
// Formats the given row as a record (using the separator and delimiter characters accordingly).
private fun formatAsRecord(row: Row): String =
row.joinToString("${configuration.separator} ") {
val delimiter = configuration.delimiter
"$delimiter$it$delimiter"
}
}
AbstractParser
classclass AbstractParser(override val source: Any, override val configuration: Configuration) : Parser {
// Empty methods to be overridden by sub-classes to initialize/finalise parsing.
protected open fun beforeParsing() { /* does nothing by default */ }
protected open fun afterParsing() { /* does nothing by default */ }
// Template method that parses the source into a sequence of strings (one per line).
protected abstract fun sourceAsLines(): Sequence<String>
// Parses the source into a sequence of rows (skipping comments, looking for at most 1 header).
override fun parse(): Iterable<Row> = sequence {
beforeParsing()
var header: Header? = null
var i = 0
for (line in sourceAsLines()) {
if (line.isBlank()) {
continue
} else if (configuration.isHeader(line)) {
if (header == null) {
header = headerOf(configuration.getColumnNames(line))
yield(header)
}
} else if (configuration.isComment(line)) {
continue
} else if (configuration.isRecord(line)) {
val fields = configuration.getFields(line)
if (header == null) {
header = anonymousHeader(fields.size)
yield(header)
}
try {
yield(recordOf(header, fields))
} catch (e: IllegalArgumentException) {
throw IllegalStateException("Invalid CSV at line $i: $line", e)
}
} else {
error("Invalid CSV line at $i: $line")
}
i++
}
afterParsing()
}.asIterable()
}
StringParser
classclass StringParser(override val source: String, configuration: Configuration)
: AbstractParser(source, configuration) {
// Splits the source string into lines.
override fun sourceAsLines(): Sequence<String> = source.lineSequence()
}
Additions to the Csv.kt
file:
const val DEFAULT_SEPARATOR = ','
const val DEFAULT_DELIMITER = '"'
const val DEFAULT_COMMENT = '#'
// Converts the current container of rows into a CSV string, using the given characters.
fun Iterable<Row>.formatAsCSV(
separator: Char = DEFAULT_SEPARATOR,
delimiter: Char = DEFAULT_DELIMITER,
comment: Char = DEFAULT_COMMENT
): String = DefaultFormatter(this, Configuration(separator, delimiter, comment)).format().joinToString("\n")
// Parses the current CSV string into a table, using the given characters.
fun String.parseAsCSV(
separator: Char = DEFAULT_SEPARATOR,
delimiter: Char = DEFAULT_DELIMITER,
comment: Char = DEFAULT_COMMENT
): Table = StringParser(this, Configuration(separator, delimiter, comment)).parse().let(::tableOf)
object CsvStrings {
val iris: String = """
|# sepal_length, sepal_width, petal_length, petal_width, class
|5.1,3.5,1.4,0.2,Iris-setosa
|4.9,3.0,1.4,0.2,Iris-setosa
|4.7,3.2,1.3,0.2,Iris-setosa
""".trimMargin()
val irisWellFormatted: String = """
|# "sepal_length", "sepal_width", "petal_length", "petal_width", "class"
|"5.1", "3.5", "1.4", "0.2", "Iris-setosa"
|"4.9", "3.0", "1.4", "0.2", "Iris-setosa"
|"4.7", "3.2", "1.3", "0.2", "Iris-setosa"
""".trimMargin()
// other dummy constants here
}
Tests involving parsing be like:
@Test
fun parsingFromCleanString() {
val parsed: Table = CsvStrings.iris.parseAsCSV()
assertEquals(
expected = Tables.iris(Tables.irisShortHeader),
actual = parsed
)
}
Tests involving formatting be like:
@Test
fun formattingToString() {
val iris: Table = Tables.iris(Tables.irisShortHeader)
assertEquals(
expected = CsvStrings.irisWellFormatted,
actual = iris.formatAsCSV()
)
}
Further additions to the Csv.kt
file:
// Reads and parses a CSV file from the given path, using the given characters.
expect fun parseCsvFile(
path: String,
separator: Char = DEFAULT_SEPARATOR,
delimiter: Char = DEFAULT_DELIMITER,
comment: Char = DEFAULT_COMMENT
): Table
expect
keyword, and the the lack of function bodyString
as a platform-agnostic representation of paths
I/O (over textual files) is mainly supported by means of the following classes:
Buffered readers support reading a file line-by-line
On Kotlin/JVM, Java’s std-lib is available as Kotlin’s std-lib
We may simply create a new sub-type of AbstractParser
File
s as sourcesBufferedReader
behind the scenes to read files line-by-lineIn the jvmMain
source set
let’s define the following JVM-specific class:
class FileParser(
override val source: File, // source is now forced to be a File
configuration: Configuration
) : AbstractParser(source, configuration) {
// Lately initialised reader, corresponding to source
private lateinit var reader: BufferedReader
// Opens the source file, hence initialising the reader, before each parsing
override fun beforeParsing() { reader = source.bufferedReader() }
// Closes the reader, after each parsing
override fun afterParsing() { reader.close() }
// Lazily reads the source file line-by-line
override fun sourceAsLines(): Sequence<String> = reader.lines().asSequence()
}
let’s create the Csv.jvm.kt
file containing:
actual fun parseCsvFile(
path: String,
separator: Char,
delimiter: Char,
comment: Char
): Table = FileParser(File(path), Configuration(separator, delimiter, comment)).parse().let(::tableOf)
parseCsvFile
is implemented on the JVMactual
keyword, and the presence of a function bodyFileParser
in the function body
FileParser
is internal class for filling the abstraction gap on the JVMI/O (over textual files) is mainly supported by means of the following things:
These function supports reading / writing a file in one shot
On Kotlin/JS, Node’s std-lib is not directly available
external
declarations)To-do list for Kotlin/JS:
external
declarations for the Node’s std-lib functions to be usedStringParser
(after reading the whole file)In the jsMain
source set
let’s create the NodeFs.kt
file (containing external
declarations for the fs
module):
@file:JsModule("fs")
@file:JsNonModule
package io.github.gciatto.csv
// Kotlin mapping for: https://nodejs.org/api/fs.html#fsreadfilesyncpath-options
external fun readFileSync(path: String, options: dynamic = definedExternally): String
// Kotlin mapping for: https://nodejs.org/api/fs.html#fswritefilesyncfile-data-options
external fun writeFileSync(file: String, data: String, options: dynamic = definedExternally)
@JsModule
annotation instructs the compiler about where to look up for the fs
moduleexternal
declarations are Kotlin signatures of JS functiondynamic
is a special type that can be used to represent any JS object
external
stuffdefinedExternally
is stating that a parameter is optional (default value is defined in JS)let’s create the Csv.js.kt
file containing:
actual fun parseCsvFile(
path: String,
separator: Char,
delimiter: Char,
comment: Char
): Table = readFileSync(path, js("{encoding: 'utf8'}")).parseAsCSV(separator, delimiter, comment)
parseCsvFile
is implemented on JSactual
keyword, and the presence of a function bodyreadFileSync
to read a file as a string in one shotTable
(namely, parseAsCSV
)js("...")
magic function
readFileSync
We may test our CSV library in common-code
The test suite may:
The hard part is step 1: two platform-specific steps
file Utils.kt
in commonTest
:
// Creates a temporary file with the given name, extension, and content, and returns its path.
expect fun createTempFile(name: String, extension: String, content: String): String
notice the expect
ed function
file Utils.jvm.kt
in jvmTest
:
import java.io.File
actual fun createTempFile(name: String, extension: String, content: String): String
val file = File.createTempFile(name, extension)
file.writeText(content)
return file.absolutePath
}
File.createTempFile
file Utils.js.kt
in jsTest
:
private val Math: dynamic by lazy { js("Math") }
@JsModule("os") @JsNonModule
external fun tmpdir(): String
actual fun createTempFile(name: String, extension: String, content: String): String {
val tag = Math.random().toString().replace(".", "")
val path = "$tmpDirectory/$name-$tag.$extension"
writeFileSync(path, content)
return path
}
file CsvFiles.kt
in commonMain
:
object CsvFiles {
// Path of the temporary file containing the string CsvStrings.iris
// (file lazily created upon first usage).
val iris: String by lazy { createTempFile("iris.csv", CsvStrings.iris) }
// Path of the temporary file containing the string CsvStrings.irisWellFormatted
// (file lazily created upon first usage).
val irisWellFormatted: String by lazy {
createTempFile("irisWellFormatted.csv", CsvStrings.irisWellFormatted)
}
// other paths here, corresponding to other constants in CsvStrings
}
tests for parsing be like:
@Test
fun testParseIris() {
val parsedFromString = CsvStrings.iris.parseAsCSV()
val readFromFile = parseCsvFile(CsvFiles.iris)
assertEquals(parsedFromString, readFromFile)
}
Kotlin multi-platform projects can be assembled as JARs
The jvmMain
source set is compiled into a JVM-compliant JARs
*Jar
or assemble
tasksThe jsMain
source set is compiled into either
.klib
), enabling imporing the project tas dependency in Kotlin/JS projects
*Jar
or assemble
taskscompileProductionExecutableKotlinJs
taskTask for JVM-only compilation of re-usable packages: jvmJar
Effect:
$PROJ_DIR/build/libs/$PROJ_NAME-jvm-$PROJ_VERSION
The JAR does not contain dependencies
Ad-hoc Gradle plugins/code is needed for creating fat Jar
Task for JS-only compilation of re-usable packages: compileProductionExecutableKotlinJs
Effect:
$ROOT_PROJ_DIR/build/js/packages/$ROOT_PROJ_NAME-$PROJ_NAME
:
package.json
fileSupport for packing as NPM package via the npm-publish
Gradle plugin
Kotlin code can be called from the target platforms’ main languages
Understading the mapping among Kotlin and other languages is key
How are Kotlin’s syntactical categories mapped to other platforms/languages?
class MyClass {}
interface MyType {}
public class MyClass {}
public interface MyType {}
Int
int
/ Integer
, Short
short
/ Short
, etc.val x: Int = 0
val y: Int? = null
int x = 0;
Integer y = null;
Any
Object
, kotlin.collections.List
java.util.List
, etc.val x: Any = "ciao"
val y: kotlin.collections.List<Int> = listOf(1, 2, 3)
val z: kotlin.collections.MutableList<String> = mutableListOf("a", "b", "c")
Object x = "ciao";
java.util.List<Integer> y = java.util.Arrays.asList(1, 2, 3);
java.util.List<String> z = java.util.List.of("a", "b", "c");
interface MyType {
val x: Int
var y: String
}
public interface MyType {
int getX();
String getY(); void setY(String y);
}
JvmField
annotation is adoptedimport kotlin.jvm.JvmField
class MyType {
@JvmField
val x: Int = 0
@JvmField
var y: String = ""
}
public class MyType {
public final int x = 0;
public String y = "";
}
X.kt
are mapped to static methods of class XKt
// file MyFile.kt
fun f() {}
public static class MyFileKt {
public static void f() {}
}
JvmName
annotation is exploited@file:JvmName("MyModule")
import kotlin.jvm.JvmName
fun f() {}
public class MyModule {
public static void f() {}
}
object X
is mapped to Java class X
with
INSTANCE
to accessobject MySingleton {}
public static class MySingleton {
private MySingleton() {}
public static final MySingleton INSTANCE = new MySingleton();
}
X
’s companion object is mapped to public static final field named Companion
on class X
class MyType {
private constructor()
companion object {
fun of(): MyType = MyType()
}
}
// usage:
val x: MyType = MyType.of()
public class MyType {
private MyType() {}
public static final class Companion {
public MyType of() { return new MyType(); }
}
public static final Companion Companion = new Companion();
}
// usage:
MyType x = MyType.Companion.of();
X
’s companion object’s member M
tagged with @JvmStatic
is mapped to static member M
on class X
import kotlin.jvm.JvmStatic
class MyType {
private constructor()
companion object {
@JvmStatic
fun of(): MyType = MyType()
}
}
// usage:
val x: MyType = MyType.of()
public class MyType {
private MyType() {}
public static MyType of() { return new MyType(); }
}
// usage:
MyType x = MyType.of();
fun f(vararg xs: Int) {
val ys: Array<Int> = xs
}
void f(int... xs) {
Integer[] ys = xs;
}
class MyType { }
fun MyType.myMethod() {}
// usage:
val x = MyType()
x.myMethod() // postfix
public class MyType {}
public void myMethod(MyType self) {}
// usage:
MyType x = new MyType();
myMethod(x); // prefix
import kotlin.sequences.*
val x = (0 ..< 10).asSequence() // 0, 1, 2, ..., 9
.filter { it % 2 == 0 } // 0, 2, 4, ..., 8
.map { it + 1 } // 1, 3, 5, ..., 9
.sum() // 25
import static kotlin.sequences.SequencesKt.*;
int x = sumOfInt(
map(
filter(
asSequence(
new IntRange(0, 9).iterator()
),
it -> it % 2 == 0
),
it -> it + 1
)
);
fun f(a: Int = 1, b: Int = 2, c: Int = 3) = // ...
// usage:
f(b = 5)
void f(int a, int b, int c) { /* ... */ }
// usage:
f(1, 5, 3);
@JvmOverloads
to generate overloaded methods for Java
import kotlin.jvm.JvmOverloads
@JvmOverloads
fun f(a: Int = 1, b: Int = 2, c: Int = 3) = // ...
void f(int a, int b, int c) { /* ... */ }
void f(int a, int b) { f(a, b, 3); }
void f(int a) { f(a, 2, 3); }
void f() { f(1, 2, 3); }
// missing overloads:
// void f(int a, int c) { f(a, 2, c); }
// void f(int b, int c) { f(1, b, c); }
// void f(int c) { f(1, 2, c); }
// void f(int b) { f(1, b, 3); }
Compile our CSV lib and import it as a dependency in a novel Java library
Alternatively, add Java sources to the jvmTest
source set, and use the library
Listen to the teacher presenting key points
Disclaimer: the generated JS code is not really meant to be read by humans
@JsExport
annotation
kotlin.js.ExperimentalJsExport
opt-in@file:Suppress("NON_EXPORTABLE_TYPE")
package my.package
import kotlin.js.JsExport
import kotlin.js.JsName
@JsExport
class MyType {
val nonExportableType: Long
}
@JsExport
fun myFunction() = // ...
var module = require("project-name");
var MyType = module.my.package.MyType;
var myFunction = module.my.package.myFunction;
@JsName
annotation can be used to control the name of a member
package my.package
@JsName("sumNumbers")
fun sum(vararg numbers: Int): Int = // ...
@JsName("sumIterableOfNumbers")
fun sum(numbers: Iterable<Int>): Int = // ...
@JsName("sumSequenceOfNumbers")
fun sum(numbers: Sequence<Int>): Int = // ...
var module = require("project-name");
var sumNumbers = module.my.package.sumNumbers;
var sumIterableOfNumbers = module.my.package.sumIterableOfNumbers;
var sumSequenceOfNumbers = module.my.package.sumSequenceOfNumbers;
class MyClass(private val argument: String) {
@JsName("method")
fun method(): String = argument + "!"
}
function MyClass(argument) {
this.randomName_1 = argument
}
MyClass.prototype.method = function () {
return this.randomName_1 + "!"
}
Kotlin numeric types, except for kotlin.Long
, are mapped to JavaScript Number
kotlin.Char
is mapped to JS Number
representing character code.
Kotlin preserves overflow semantics for kotlin.Int
, kotlin.Byte
, kotlin.Short
, kotlin.Char
and kotlin.Long
kotlin.Long
is not mapped to any JS object, as there is no 64-bit integer number type in JS. It is emulated by a Kotlin class
kotlin.String
is mapped to JS String
kotlin.Any
is mapped to JS Object
(new Object()
, {}
, and so on)
kotlin.Array
is mapped to JS Array
Kotlin collections (List
, Set
, Map
, and so on) are not mapped to any specific JS type
kotlin.Throwable
is mapped to JS Error
practical consequence: no way to distinguish numbers by type
val x: Int = 23
val y: Any = x
println(y as Float) // fails on JVM, works on JS
Kotlin’s dynamic
overrides the type system, and it is translated “1-to-1”
val string1: String = "some string"
string1.missingMethod() // compilation error
val string2: dynamic = "some string"
string2.missingMethod() // compilation ok, runtime error
kotlin
Companion objects are treated similarly to Kotlin/JVM
Extension methods are treated similarly to Kotlin/JVM
Variadic functions are compiled to JS functions accepting an array
fun f(vararg xs: String) = // ...
// usage:
f("a")
f("a", "b")
f("a", "b", "c")
// usage:
f(["a"])
f(["a", "b"])
f(["a", "b", "c"])
Conceptual workflow:
O
(e.g. Win, Mac, Linux):
T
:
V
of the platform T
(e.g. LTS releases + latest)
<T>MainClasses
)<T>TestClasses
)<T>Test
)master
branch)
T
kotlin-multiplatform
:
R
of platform T
Even more complicated, because release on Maven Central is complex
Setting up a CI/CD pipeline of this kind requires a lot of work
Recommendations:
Takeaway: each platform has some preferred main repository where users expect to find packages onto
JVM
Kotlin / Multiplatform
JS
Android
Mac
/ iOS
Windows
Linux
Python
.Net
In the end of the day, the Python interpreter is C++ software
Most commonly, efficient Python code is written in C++ and then wrapped into Python
Put it simply, the JVM too is C++ software
Can Python code be written in Java and then wrapped into Python?
Apparently yes, via several ad-hoc bridging technologies:
Why exactly JPype?
Ensure you have a JVM installed on your system
JAVA_HOME
environment variable set to the JVM installation directoryEnsure your compiled Java code is available as a .jar
file
/path/to/my.jar
Install the JPype package via pip install JPype1
You first need JPype to start a JVM instance in your Python process
JAVA_HOME
environment variableimport jpype
# start the JVM
jpype.startJVM(classpath=["/path/to/my.jar"])
Once the JVM is started, one can import Java classes and call their methods as if they were Python objects
import jpype.imports # this is necessary to import Java classes
from java.lang import System # import the java.lang.System class
System.out.println("Hello World!")
Overview on the official documentation
Java classes are presented wherever possible similar to Python classes
the only major difference is that Java classes and objects are closed and cannot be modified
Overview on the official documentation
Java exceptions extend from Python exceptions
Java exceptions can be dealt with in the same way as Python native exceptions
try
-except
blocksJException
serves as the base class for all Java exceptions
Overview on the official documentation
most Python primitives directly map into Java primitives
however, Python does not have the same primitive types…
… hence, explicit casts may be needed in some cases
each primitive Java type is exposed in JPype (jpype.JBoolean
, .JByte
, .JChar
, .JShort
, .JInt
, .JLong
, .JFloat
, .JDouble
).
Overview on the official documentation
Java strings are similar to Python strings
they are both immutable and produce a new string when altered
most operations can use Java strings in place of Python strings
when comparing or using strings as dictionary keys, all JString
objects should be converted to Python
Overview on the official documentation
Java arrays are mapped to Python lists
more precisely, they operate like Python lists, but they are fixed in size
reading a slice from a Java array returns a view of the array, not a copy
passing a slide of a Python list to Java will create a copy of the sub-list
Overview on the official documentation
Java collections are overloaded with Python syntax where possible
Java’s Iterable
s are mapped to Python iterables by overriding the __iter__
method
Java’s Collection
s are mapped to Python containers by overriding __len__
Java’s Map
s support Python’s dictionaries syntax by overriding __getitem__
and __setitem__
Java’s List
s support Python’s lists syntax by overriding __getitem__
and __setitem__
Overview on the official documentation
Java interfaces can be implemented in Python, via JPype’s decorators
Java’s open / abstract classes cannot be extended in Python
Python lambda expressions can be cast’d to Java’s functional interfaces
Overview on the official documentation
none, there is no way to convert
explicit (E), JPype can convert the desired type, but only explicitly via casting
implicit (I), JPype will convert as needed
exact (X), like implicit, but takes priority in overload selection
Consider the following example of Python code with JPype:
import jpype.imports
from java.lang import System
System.out.println(1)
System.out.println(2.0)
System.out.println('A')
Which overload of System.out.println
is called among the many admissible ones?
1
is convertible to Java’s int
, long
, and short
int
is the exact match2.0
is convertible to Java’s float
and double
double
is the exact match'A'
is convertible to Java’s String
and char
String
is the exact matchConsider the following example of Python code with JPype:
import jpype
csv = jpype.JPackage("io.github.gciatto.csv.Csv")
csv.headerOf(["filed", "another field"])
This would raise the following error:
TypeError: Ambiguous overloads found for io.github.gciatto.csv.Csv.headerOf(list) between:
public static final io.github.gciatto.csv.Header io.github.gciatto.csv.Csv.headerOf(java.lang.Iterable)
public static final io.github.gciatto.csv.Header io.github.gciatto.csv.Csv.headerOf(java.lang.String[])
list
is convertible to both Java’s Iterable
and String[]
To solve this issue, one can explicitly cast the Python list
to the desired Java type:
import jpype
import jpype.imports
from java.lang import Iterable as JIterable
csv = jpype.JClass("io.github.gciatto.csv.Csv")
csv.headerOf(JIterable@["field", "another field"])
# returns Header("field", "another field")
One may customise the behaviour of Java types in Python by providing custom implementations for them
@JImplementationFor
decoratorIn that case the special method __jclass_init__
is called on the custom implementation, just once, to configure the class
In type hierarchies, implementations provided for superclasses are inherited by subclasses
Consider for instance the following customisations, allowing to use Java collections with Python syntax
from typing import Iterable, Sequence
@jpype.JImplementationFor("java.lang.Iterable")
class _JIterable:
def __jclass_init__(self):
Iterable.register(self) # makes this class a subtype of Iterable, to speed up isinstance checks
def __iter__(self):
return self.iterator()
@jpype.JImplementationFor("java.util.Collection")
class _JCollection:
def __len__(self):
return self.size() # supports "len(coll)" syntax
def __delitem__(self, i):
return self.remove(i) # supports "del coll[i]" syntax
def __contains__(self, i):
return self.contains(i) # supports "i in coll" syntax
# __iter__ is inherited from _JIterable
# because in Java: Collection extends Iterable
@jpype.JImplementationFor('java.util.List')
class _JList(object):
def __jclass_init__(self):
Sequence.register(self) # makes this class a subtype of Sequence, to speed up isinstance checks
def __getitem__(self, ndx):
return self.get(ndx) # supports "list[i]" syntax
def append(self, obj):
return self.add(obj) # supports "list.append(obj)" syntax
# __len__, __delitem__, __contains__, __iter__ are inherited from _JCollection
this is taken directly from JPype’s codebase
The code wrapped via JPype is not Pythonic by default
It is important to make the wrapped code as Pythonic as possible
io.github.gciatto.csv.Csv
jcsv.Csv
snake_case
instead of camelCase
__len__
for java.util.Collection
__getitem__
for java.util.List
All such refinements can be done in JPype via customisations of the Java types
For all public types in the wrapped Java library:
jcsv
package (pt. 1)The jcsv
package is a Pythonic wrapper for our JVM-based io.github.gciatto.csv
library
Java’s type definition are brought to Python in jcsv/__init__.py
:
import jpype
import jpype.imports
from java.lang import Iterable as JIterable
_csv = jpype.JPackage("io.github.gciatto.csv")
Table = _csv.Table
Row = _csv.Row
Record = _csv.Record
Header = _csv.Header
Formatter = _csv.Formatter
Parser = _csv.Parser
Configuration = _csv.Configuration
Csv = _csv.Csv
CsvJvm = _csv.CsvJvm
making it possible to write the following code on the user side:
from jcsv import Table, Record, Header
jcsv
package (pt. 2)Parsing and formatting operations are mapped straightforwardly to Python functions:
# jcsv/__init__.py
def parse_csv_string(string, separator = Csv.DEFAULT_SEPARATOR, delimiter = Csv.DEFAULT_DELIMITER, comment = Csv.DEFAULT_COMMENT):
return Csv.parseAsCSV(string, separator, delimiter, comment)
def parse_csv_file(path, separator = Csv.DEFAULT_SEPARATOR, delimiter = Csv.DEFAULT_DELIMITER, comment = Csv.DEFAULT_COMMENT):
return CsvJvm.parseCsvFile(str(path), separator, delimiter, comment)
def format_as_csv(rows, separator = Csv.DEFAULT_SEPARATOR, delimiter = Csv.DEFAULT_DELIMITER, comment = Csv.DEFAULT_COMMENT):
return Csv.formatAsCSV(JIterable@rows, separator, delimiter, comment)
jcsv
package (pt. 3)Ad-hoc factory method is provided for building Header
instances:
# jcsv/__init__.py
from jcsv.python import iterable_or_varargs
def header(*args):
if len(args) == 1 and isinstance(args[0], int):
return Csv.anonymousHeader(args[0])
return iterable_or_varargs(args, lambda xs: Csv.headerOf(JIterable@map(str, xs)))
making it possible to write the following code on the user side:
import jcsv
header1 = jcsv.header("column1", "column2", "column3")
header2 = jcsv.header(3) # anonymous header with 3 columns
columns = (f"column{i}" for i in range(1, 4)) # generator expression
header3 = jcsv.header(columns) # same as header1, but passing an interable
Function iterable_or_varargs
aims at simulating multiple overloads:
# jcsv/python.py
from typing import Iterable
def iterable_or_varargs(args, f):
assert isinstance(args, Iterable)
if len(args) == 1:
item = args[0]
if isinstance(item, Iterable):
return f(item)
else:
return f([item])
else:
return f(args)
jcsv
package (pt. 4)Ad-hoc factory method is provided for building Record
instances:
# jcsv/__init__.py
def record(header, *args):
return iterable_or_varargs(args, lambda xs: Csv.recordOf(header, JIterable@map(str, xs)))
Ad-hoc factory method is provided for building Table
instances:
# jcsv/__init__.py
def __ensure_header(h):
return h if isinstance(h, Header) else header(h)
def __ensure_record(r, h):
return r if isinstance(r, Record) else record(h, r)
def table(header, *args):
header = __ensure_header(header)
args = [__ensure_record(row, header) for row in args]
return iterable_or_varargs(args, lambda xs: Csv.tableOf(header, JIterable@xs))
jcsv
package (pt. 5)The Row
class is customised to make it more Pythonic:
# jcsv/__init__.py
@jpype.JImplementationFor("io.github.gciatto.csv.Row")
class _Row:
def __len__(self):
return self.getSize()
def __getitem__(self, item):
if isinstance(item, int) and item < 0:
item = len(self) + item
try:
return self.get(item)
except _java.IndexOutOfBoundsException as e:
raise IndexError(f"index {item} out of range") from e
@property
def size(self):
return len(self)
len(row)
instead of row.getSize()
row[i]
instead of row.get(i)
row[-i]
instead of row.get(row.getSize() - i - 1)
IndexError
be raised instead of IndexOutOfBoundsException
row.size
instead of row.getSize()
jcsv
package (pt. 6)The Header
shall inherit all customisation for Row
, plus the following ones:
@jpype.JImplementationFor("io.github.gciatto.csv.Header")
class _Header:
@property
def columns(self):
return [str(c) for c in self.getColumns()]
def __contains__(self, item):
return self.contains(item)
def index_of(self, column):
return self.indexOf(column)
header.columns
instead of header.getColumns()
column in header
instead of header.contains(column)
header.index_of(column)
instead of header.indexOf(column)
jcsv
package (pt. 7)The Record
shall inherit all customisation for Row
, plus the following ones:
@jpype.JImplementationFor("io.github.gciatto.csv.Record")
class _Record:
@property
def header(self):
return self.getHeader()
@property
def values(self):
return [str(v) for v in self.getValues()]
def __contains__(self, item):
return self.contains(item)
record.header
instead of record.getHeader()
record.values
instead of record.getValues()
value in record
instead of record.contains(value)
jcsv
package (pt. 8)The Table
class is customised too, to make it more Pythonic:
@jpype.JImplementationFor("io.github.gciatto.csv.Table")
class _Table:
@property
def header(self):
return self.getHeader()
def __len__(self):
return self.getSize()
def __getitem__(self, item):
if isinstance(item, int) and item < 0:
item = len(self) + item
try:
return self.get(item)
except _java.IndexOutOfBoundsException as e:
raise IndexError(f"index {item} out of range") from e
@property
def records(self):
return self.getRecords()
@property
def size(self):
return len(self)
table.header
instead of table.getHeader()
len(table)
instead of table.getSize()
table[i]
instead of table.get(i)
table[-i]
instead of table.get(table.getSize() - i - 1)
record in table
instead of table.contains(record)
table.records
instead of table.getRecords()
.jar
s in JPype projects (pt. 1)csv-python/
├── build.gradle.kts # this is where the generation of csv.jar is automated
├── jcsv
│ ├── __init__.py
│ ├── jvm
│ │ ├── __init__.py # this is where JPype is loaded
│ │ └── csv.jar # this the Fat-JAR of the JVM-based library
│ └── python.py
├── requirements.txt
└── test
├── __init__.py
├── test_parsing.py
└── test_python_api.py
We need to ensure that the JVM-based library is available on the system where jcsv
is installed
The build.gradle.kts
file automates the generation of the csv.jar
file
jcsv/jvm
directoryThe jcsv/jvm/__init__.py
file loads JPype and the csv.jar
file
.jar
s in JPype projects (pt. 2)Snippet from the build.gradle.kts
:
tasks.create<Copy>("createCoreJar") {
group = "Python"
val shadowJar by project(":csv-core").tasks.getting(Jar::class)
dependsOn(shadowJar)
from(shadowJar.archiveFile) {
rename(".*?\\.jar", "csv.jar")
}
into(projectDir.resolve("jcsv/jvm"))
}
Content of the jcsv/jvm/__init__.py
file:
import jpype
from pathlib import Path
# the directory where csv.jar is placed
CLASSPATH = Path(__file__).parent
# the list of all .jar files in CLASSPATH
JARS = [str(j.resolve()) for j in CLASSPATH.glob('*.jar')]
jpype.startJVM(classpath=JARS)
Important line in jcsv/__init__.py
:
import jcsv.jvm
this is forcing the startup of the JVM with the correct classpath whenever someone is using the jcsv
module
We need to ensure that some JVM is available on the system where jcsv
is installed
Notice that the JVM is available as a Python dependency too:
This means that the JVM can be automatically downloaded and installed via pip
:
pip install jdk4py
… or added as a dependency to the requirements.txt
file:
JPype1==1.4.1
jdk4py==17.0.7.0
so, one may simply need to configure JPype to use that JVM:
# jcsv/jvm/__init__.py
import jpype, sys
from jdk4py import JAVA_HOME
def jvm_lib_file_names():
if sys.platform == "win32":
return {"jvm.dll"}
elif sys.platform == "darwin":
return {"libjli.dylib"}
else:
return {"libjvm.so"}
def jvmlib():
for name in __jvm_lib_file_names():
for path in JAVA_HOME.glob(f"**/{name}"):
if path.exists:
return str(path)
return None
jpype.startJVM(jvmpath=jvmlib())
Unit tests are essential to ensure the correctness of the Pythonic API
Consider for instance tests in:
test/test_parsing.py
test/test_python_api.py
It is important to test all the costumisations and factory methods