What are software platforms?

• Fuzzy concept

  • often mentioned in courses / literature, yet not precisely defined

• Insight:

The substratum upon which software applications run

What are software platforms?

Operating systems (?)

• Most software runs on top of an operating system (OS)

• So yes, all operating systems are platforms

• Is there more?

  • e.g. JabRef: Java application running on top of JVM
    • the JVM is an application for the OS
  • e.g. Draw.io: Web application running on top of browsers
    • the browser is an application for the OS

What are software platforms?

Operating systems + ???

• Programming languages?

  • not that simple
    • e.g. Scala apps can call Java code

• Runtimes

  • i.e. the set of libraries and conventions backing programming languages

What are APIs and runtimes?

API $\equiv$ application programming interface(s) $\stackrel{\Delta}{=}$ a formal specification of the set of functionalities provided by a software (sub-)system for external usage, including their inputs, outputs, and environmental preconditions and effects

• client-server metaphor is implicit

Runtime [system/environment] $\approx$ the set of computational resources backing the execution of a software (sub-)system

• we say that “runtimes support API”

What are APIs and runtimes?

Examples of APIs

• all possible public interfaces / classes / structures in an OOP module

  • and their public/protected methods / fields / properties / constructors
    • and their formal arguments, return types, throwable exceptions

• all possible commands a CLI application accepts as input

  • and their admissible sub-commands, options, and arguments
    • and the corresponding outputs, exit values, and side effects

• all possible routes a Web service may accept HTTP requests on

  • and their admissible HTTP methods
    • and their admissible query / path / body / header parameters
      • and the corresponding status codes, and response bodies

What are APIs and runtimes?

Examples of runtimes

• any virtual machine or interpreter (JVM, CLR, CPython, V8)

  • and their standard libraries
  • and their type system and internal conventions
    • e.g. primitive/reference types in the JVM, value/reference types in the CLR, global lock in CPython

• any operating system (Win, Mac, Linux)

  • and their system calls, daemons, package managers, default commands, etc.
  • and their program memory, access control, file system models

• any Web service

  • and the protocols they leverage
  • and their URL structuring model
  • the data schema of their input/output objects
  • the authentication / authorization mechanisms they support

Example: the JVM platform

Standard and community libraries: JVM

• JDK APIs are shared across JVM languages: Java, Kotlin, Scala, etc.
• Community libraries add domain-specific abstractions (e.g. Commons CSV), resolved by Maven / Gradle

Predefined design decisions: JVM

public class DesignDecisions {
    static class Counter {
        private int value;

        public synchronized void increment() {
            value++; // the receiver's monitor protects this update
        }

        @Override
        public String toString() {
            return "Counter(" + value + ")";
        }
    }

    public static void main(String[] args) {
        Counter counter = new Counter();
        counter.increment();
        Object object = counter; // every class extends Object
        System.out.println(object); // virtual call: Counter(1)
        int primitive = 1; // not an Object
        Object boxed = primitive; // autoboxed to Integer

Organizational, stylistic, technical conventions: JVM

project/
├── pom.xml (or build.gradle.kts)
└── src/
    ├── main/java/it/unibo/spe/
    │   └── CsvRow.java
    └── test/java/it/unibo/spe/
        └── CsvRowTest.java

package it.unibo.spe;

import java.util.Collection;
import java.util.Iterator;
import java.util.List;

public class CsvRow implements Iterable<String> {
    private final List<String> values;

    public CsvRow(Collection<String> values) {
        this.values = List.copyOf(values);
    }

    public List<String> getValues() { // JavaBean getter
        return values;
    }

    @Override
    public Iterator<String> iterator() {
        return values.iterator();
    }

    public static void main(String... args) { // String[]
        var row = new CsvRow(List.of("Ada", "42"));
        for (String value : row) System.out.println(value);
        assert row.getValues().equals(List.of("Ada", "42"));
    }
}

Packaging, imports, repositories: JVM

Resolve dependencies first; import classes in source code

User communities: JVM

The same JVM library can reach several communities through their preferred JVM language

Example: the .NET platform

Standard and community libraries: .NET

• The base class library provides common APIs for C#, F#, VB.NET, etc.
• Community packages add higher-level abstractions, distributed through NuGet

Predefined design decisions: .NET / C#

        Point first = new Point { X = 1 };
        Point copy = first; // struct: copy the value
        copy.X = 2;
        var counter = new Counter();
        var alias = counter; // class: copy the reference
        alias.Value = 2;
        object boxed = first; // box a copy of the struct
        Console.WriteLine($"{first.X}, {copy.X}"); // 1, 2
        Console.WriteLine(counter); // Counter(2)

public struct Point
{
    public int X { get; set; }
}

public class Counter
{
    public int Value { get; set; }
    public override string ToString() => $"Counter({Value})";
}

Organizational, stylistic, technical conventions: .NET

PlatformExamples/
├── PlatformExamples.sln
└── CsvDemo/
    ├── CsvDemo.csproj
    ├── Program.cs
    ├── Point.cs
    ├── Counter.cs
    ├── Models/
    │   └── CsvRow.cs
    ├── bin/  (generated assemblies)
    └── obj/  (generated build files)

using System.Collections;
using System.Collections.Generic;

namespace CsvDemo.Models;

public class CsvRow : IEnumerable<string>
{
    private readonly string[] _values;

    public CsvRow(params string[] values) // variadic arguments
    {
        _values = (string[])values.Clone();
    }

    public int Count => _values.Length; // property, not GetCount()

    public IEnumerator<string> GetEnumerator() =>
        ((IEnumerable<string>)_values).GetEnumerator();

    IEnumerator IEnumerable.GetEnumerator() => GetEnumerator();
}

Packaging, imports, repositories: .NET

NuGet resolves packages; assembly references make types available to the compiler

User communities: .NET / C#

Frameworks have their own target platforms: e.g. WPF is Windows-only; Unity uses its own runtime tooling

Example: the Python platform

Standard and community libraries: Python

• The standard library already covers CSV / JSON, complex numbers, etc.
• Community packages add arrays, tables, ML models and Web frameworks

Predefined design decisions: Python

class Row:
    def __init__(self, *values):
        self.values = values

    def __str__(self):
        return ",".join(self.values)

    def __len__(self):
        return len(self.values)


row = Row("Ada", "42")
print(str(row))  # Ada,42: dispatches to __str__
print(len(row))  # 2: dispatches to __len__
print(isinstance(42, object))  # True, even for integers

Organizational, stylistic, technical conventions: Python

project/
├── pyproject.toml
├── src/csv_demo/
│   ├── __init__.py
│   └── csv_row.py
└── tests/
    └── test_csv_row.py

class CsvRow:  # PascalCase type name
    def __init__(self, *values, **metadata):
        self.values = values       # tuple
        self.metadata = metadata   # dict

    def __iter__(self):
        return iter(self.values)


def join_values(row):  # snake_case; duck typing
    return ",".join(row)  # any iterable of strings


row = CsvRow("Ada", "42", source="demo")
print(join_values(row))  # Ada,42
print(join_values(["Ada", "42"]))  # same protocol

Packaging, imports, repositories: Python

pip installs a distribution; import loads a module

User communities: Python

Scientific libraries reach users who expect Python objects and protocols, even when implemented in C / C++

Example: the NodeJS platform

Standard and community libraries: NodeJS

• JavaScript built-ins + Node APIs form the platform’s standard facilities
• Community packages add higher-level abstractions; browser DOM APIs are not part of Node

Predefined design decisions: NodeJS

const rowPrototype = {
  toString() { return this.values.join(','); }
};
const row = Object.create(rowPrototype);
row.values = ['Ada', '42'];
console.log(row.toString()); // inherited: Ada,42

const order = [];
async function readRow() {
  order.push('start');
  await Promise.resolve(row); // continuation runs later
  order.push('resumed');
}
const pending = readRow();
order.push('caller'); // await did not block the caller
await pending;
console.log(order); // ['start', 'caller', 'resumed']

Organizational, stylistic, technical conventions: NodeJS

project/
├── package.json
├── src/
│   └── csv-row.mjs
└── test/
    └── csv-row.test.mjs

export class CsvRow { // PascalCase type name
  constructor(...values) { // rest arguments: Array
    this.values = values;
  }

  [Symbol.iterator]() { // iterable protocol
    return this.values[Symbol.iterator]();
  }
}

function joinValues(row) { // camelCase; duck typing
  return [...row].join(','); // any iterable
}

const row = new CsvRow('Ada', '42');
console.log(joinValues(row)); // Ada,42
console.log(joinValues(['Ada', '42'])); // same protocol

Packaging, imports, repositories: NodeJS

npm installs packages; import / require loads their exported modules

User communities: NodeJS

Front-end code runs in the browser; Node often runs its build tools or server-side rendering

Platforms from software developers’ perspective

The choice of a platform impacts developers during:

• the design phase

• the implementation phase

• the testing phase

• the release phase

How the platform affects the design phase

• One may choose the platform which minimizes the abstraction gap w.r.t. the problem at hand

Abstraction gap $\approx$ the space between the problem and the pre-existing functionalities offered by a platform. Intuitively, the bigger the space, the more effort is required to build the solution

How the platform affects the implementation phase

• Developers write solutions by leveraging the API of the platform

• … as well as the API of any third-party library available for that platform

How the platform affects the testing phase

• Test suites are a “project in the project”

  • so remarks are similar to the implementation phase

• One may test the system against as many versions as possible of the underlying platforms

• One may test the system against as many OSs as possible

  • virtual platforms may behave differently depending on the OS

• Both are typically automated via CI build matrices (cf. CI lecture)

How the platform affects the release phase

Release $\approx$ publishing some packaged software system onto a repository, hence enabling its import and exploitation

• Packaging systems are platform-specific…

• Repositories are platform-specific…

• … release is therefore platform specific (cf. version selection and CI)

What about research-oriented software? (pt. 1)

Researchers may act as software developers

1. to elaborate data

2. to create in-silico experiments

3. to study software systems

4. to create software tools improving their research

5. to create software tools for the community

   • this is how many FOSS tools have been created

What about research-oriented software? (pt. 2)

Research institutions are not software houses

• Personnel can only dedicate a fraction of their time to development

• Most software artifacts are disposable (data elaboration, in-silico experiments, personal tools)

• Development efforts are discontinuous

• Development teams are small

• Software is commonly a means, not an objective

• Commitment to software development is:

  • commonly on the individual
  • sometimes on the project
  • rarely on the institution

What about research-oriented software? (pt. 3)

Research-oriented software development should maximise audience and impact, while minimising development and maintenance effort

• Science requires reproducibility

  • a wide(r) user base is a facilitator for reproducibility

• The wider the community, the wider the impact of community-driven research software

  • more potential citations

• Minimising effort can be done by improving efficiency

  • by means of software engineering

General benefits of coherence (in general, and for researchers)

Choosing the right community / platform is strategic, especially for research-oriented software

• The abstraction gap is likely lower

  • less development effort

• More third party libraries are likely available

  • less development effort

• The potential audience is wider

  • more value, more impact, more reproducibility

• Easier to find support / help in case of issues

  • potentially, less development effort

• More likely that third-party issues are fixed in a timely manner

  • potentially, less development effort

About technological silos

Silos (in IT) are software components / systems / ecosystems having poor external interoperability (i.e. software from silo A hardly interoperates with software from silo B)

• Platforms are (pretty wide) software silos

  • making software from any two platforms interoperate is non-trivial
  • making software from the same platform interoperate is easier

• Examples:

  • intra-platform: Kotlin program calling Java library
  • inter-platform: Python program calling a JVM library (e.g. via JPype, see later)

Open research communities vs. technological silos

• Research communities in CS / AI may overlap with platform (or language) communities:

  • e.g. neural networks researchers $\rightarrow$ Python
  • e.g. data science $\rightarrow$ Python | R
  • e.g. symbolic AI $\rightarrow$ JVM | Prolog
  • e.g. multi-agent systems $\rightarrow$ JVM
  • e.g. Semantic Web $\rightarrow$ JVM | Python

• What about inter-community research efforts?

  • they may need interoperability among different silos
  • without which, research is slowed down

Multi-platform programming is an enabler for inter-community research

Approaches for multi-platform programming

1. Write once, build anywhere

   • software is developed using some sort of “super-language”
   • code from the “super-language” is automatically compiled for all platforms
   • e.g. Kotlin Multiplatform, Haxe, Scala.js + Scala Native
   • not to be confused with Java’s “write once, run anywhere”, where one artifact runs on one (virtual) platform, available on many OSs

2. Write first, wrap elsewhere

   • software is developed for some principal platform
   • implementations for all other platforms are wrappers of the first one

Write once, build anywhere (concept)

Write once, build anywhere (explanation)

• Assumptions:

  • one “super-language” exists, having:
    1. a code-generator targeting multiple platforms
       • e.g. compiler, transpiler, etc.
    2. the same standard library implemented for all those platforms

• Workflow:

  1. Design, implement, and test most of the project via the super-language
     • this is the platform-agnostic (a.k.a. “common”) part of the project
  2. Complete, refine, or optimise platform-specific aspects via
     • platform-specific code
     • platform-specific third-party libraries
  3. Build platform-specific artifacts, following platform-specific rules
  4. Upload platform-specific artifacts on platform-specific repositories

Write once, build anywhere (analysis)

Let N be the number of supported platforms

• Platform-agnostic functionalities require effort which is independent of N

• Platform-specific functionalities require effort proportional to N

• Better to minimise platform-specific code

  • by maximising common code
  • this is true both for main code and for test code

• Relevant questions:

  • what to realise as common code? what as platform-specific code?
  • how to set the boundary of the common (resp. platform-specific) code?

Takeaway

The abstraction gap of the common code is at least as wide as the one of the platform having the widest abstraction gap (the common std-lib is roughly the intersection of what all platforms offer)

Write once, build anywhere (strategy)

Whenever a new functionality needs to be developed:

1. Try to realise it with common std-lib only

2. If not possible, try to maximise the portion of platform-agnostic code

   • make it possible to plug platform-specific aspects

3. For each functionality which cannot be realised as purely platform-agnostic:

   1. design a platform-agnostic interface
   2. implement the interface N times, one per target platform

Write first, wrap elsewhere (concept)

Write first, wrap elsewhere (explanation)

• Assumptions:

  • one “main” platform exists such that
    • code from the “main” platform can be called from other target platforms
  • the software has been designed in a platform-agnostic way

• Workflow:

  1. Fully implement, test, and deploy the software for the main platform

  2. For all other platforms:

     1. re-design and re-write platform-specific API code
     2. implement platform-specific API by calling the main platform’s code
     3. re-write tests for API code
     4. build platform-specific packages, wrapping the main platform’s package and runtime

  3. Upload platform-specific artifacts on platform-specific repositories

Write first, wrap elsewhere (analysis)

Let N be the number of supported platforms

• Clear separation of API code from implementation code is quintessential

• The effort required for writing API code is virtually the same on all platforms

• Global effort is still linear in N, but with a much smaller constant

  • implementing API and wrapper code is a manual task
  • still less effort than implementing the same design N times

• The i-th platform’s wrapper code will only call the main platform’s API code

• Relevant questions:

  • how to deal with platform-specific aspects?
  • how to create platform-agnostic design?

Running example

Multi-platform CSV parsing lib

• CSV (comma-separated values) files, e.g.:

  # HEADER_1, HEADER_2, HEADER_3, ...
  # character '#' denotes the beginning of a single-line comment
  # first line conventionally denotes column names (headers)
  
  field1, field2, field3, ...
  # character ',' acts as field separator
  
  "field with spaces", "another field, with comma", "yet another field", ...
  # character '"' acts as field delimiter
  
  # other characters may be used to denote comments, separators, or delimiters
  

• JVM and JS std-lib do not provide direct support for CSV

• Requirements:

  • library for parsing / manipulating CSV files
  • parsing = loading from file
  • manipulating = creating / editing / saving CSV files, programmatically
  • support for both JVM and JS

About the example

• We’ll follow a domain-driven approach:

  1. focus on designing / implementing the domain entities
  2. then, we’ll add functionalities for manipulating them
  3. finally, we’ll write unit and integration tests

• Domain entities (can be realised as common code):

  • Table: in-memory representation of a CSV file
  • Row: generic row in a CSV file
  • Header: special row containing the names of the columns
  • Record: special row containing the values of a line in a CSV file

• Main functionalities (some of which require platform-specific code):

  1. creating a Table programmatically
  2. reading columns, rows, and values from a Table
  3. parsing a CSV file into a Table
  4. saving a Table into a CSV file

• Full code: unibo-spe/multiplatform-example (module csv-core)

Domain entities

top to bottom direction

package "kotlin" {
    interface Iterable<T>
}

package "io github gciatto csv" {

    interface Row {
        + get(index: Int): String
        + size: Int
    }

    Row -up-|> Iterable: //T// = String

    interface Header {
        + columns: List<String>
        + contains(column: String): Boolean
        + indexOf(column: String): Int
    }

    Row <|-- Header

    interface Record {
        + header: Header
        + values: List<String>
        + contains(value: String): Boolean
        + get(column: String): String
    }

    Row <|-- Record

    Record "*" *-right- "1" Header

    interface Table {
        + header: Header
        + records: List<Record>
        + get(row: Int): Row
        + size: Int
    }
    
    Table "*" *-u- "1" Header
    Table "*" o-u- "*" Record
    Table -u----|> Iterable: //T// = Row
}

The Row interface

/** Represents a single row in a CSV file. */
@JsExport
interface Row : Iterable<String> {
    /** Gets the value of the field at the given index. */
    @JsName("getByIndex")
    operator fun get(index: Int): String

    /** Gets the number of fields in this row. */
    @JsName("size")
    val size: Int
}

• Please ignore the @JsExport and @JsName annotations for now, they are explained later

The Header interface

/** Represents the header of a CSV file, i.e. a special row containing the names of the columns. */
@JsExport
interface Header : Row {
    /** Gets the names of the columns. */
    @JsName("columns")
    val columns: List<String>

    /** Checks whether the given column name is present in this header. */
    @JsName("contains")
    operator fun contains(column: String): Boolean

    /** Gets the index of the given column name (-1 if missing). */
    @JsName("indexOf")
    fun indexOf(column: String): Int
}

The Record interface

/** Represents a record in a CSV file. */
@JsExport
interface Record : Row {
    /** Gets the header of this record (useful to know column names). */
    @JsName("header")
    val header: Header

    /** Gets the values of the fields in this record. */
    @JsName("values")
    val values: List<String>

    /** Checks whether the given value is present in this record. */
    @JsName("contains")
    operator fun contains(value: String): Boolean

    /** Gets the value of the field in the given column. */
    @JsName("getByName")
    operator fun get(column: String): String
}

The Table interface

/** Represents a table, i.e. an in-memory representation of a CSV file. */
@JsExport
interface Table : Iterable<Row> {
    /** Gets the header of this table (useful to know column names). */
    @JsName("header")
    val header: Header

    /** Gets the records in this table. */
    @JsName("records")
    val records: List<Record>

    /** Gets the row at the given index (0 is the header). */
    @JsName("get")
    operator fun get(row: Int): Row

    /** Gets the number of rows in this table (header included). */
    @JsName("size")
    val size: Int
}

Common implementations of domain entities

<p>top to bottom direction</p>
<p>package &ldquo;io github gciatto csv <strong>impl</strong>&rdquo; {
interface Row</p>
<pre><code>abstract class AbstractRow {
    - values: List&lt;String&gt;
    # constructor(values: List&lt;String&gt;)
    # toString(type: String?): String
}

Row &lt;|-d- AbstractRow

interface Header

class DefaultHeader {
    - indexesByName: Map&lt;String, Int&gt;
    + constructor(columns: Iterable&lt;String&gt;)
}

Row &lt;|-d- Header
Header &lt;|-d- DefaultHeader
AbstractRow &lt;|-d- DefaultHeader

interface Record

class DefaultRecord {
    + constructor(header: Header, values: Iterable&lt;String&gt;)
}

Row &lt;|-d- Record
Record &lt;|-- DefaultRecord
AbstractRow &lt;|-d- DefaultRecord

class DefaultTable {
    + constructor(header: Header, records: Iterable&lt;Record&gt;)
}

interface Table

Table &lt;|-- DefaultTable
</code></pre>
<p>}</p>

The AbstractRow class

internal abstract class AbstractRow(protected open val values: List<String>) : Row {
    override val size: Int
        get() = values.size

    override fun get(index: Int): String = values[index]
    override fun equals(other: Any?): Boolean {
        if (this === other) return true
        if (other == null || this::class != other::class) return false

        other as AbstractRow

        return values == other.values
    }

    override fun hashCode(): Int = values.hashCode()

    protected fun toString(type: String?): String {
        var prefix = ""
        var suffix = ""
        if (type != null) {
            prefix = "$type("
            suffix = ")"
        }
        return values.joinToString(", ", prefix, suffix) { "\"$it\"" }
    }

    override fun toString(): String = toString("Row")

    override fun iterator(): Iterator<String> = values.iterator()
}

• Base implementation for the Row interface: toString(type) renders rows as <type>("field1", "field2", ...)

The DefaultHeader class

internal class DefaultHeader(columns: Iterable<String>) : Header, AbstractRow(columns.toList()) {
    private val indexesByName by lazy {
        values.mapIndexed { index, name -> name to index }.toMap()
    }

    override val columns: List<String>
        get() = values

    override fun contains(column: String): Boolean = column in indexesByName.keys

    override fun indexOf(column: String): Int = indexesByName[column] ?: -1

    override fun iterator(): Iterator<String> = values.iterator()

    override fun toString(): String = toString("Header")
}

• indexesByName is a lazily-computed cache of column indexes, for faster lookup

The DefaultRecord class

internal class DefaultRecord(
    override val header: Header,
    values: Iterable<String>
) : Record, AbstractRow(values.toList()) {

    init {
        require(header.size == super.values.size) {
            "Inconsistent amount of values (${super.values.size}) w.r.t. header size (${header.size})"
        }
    }

    override fun contains(value: String): Boolean = values.contains(value)

    override val values: List<String>
        get() = super.values

    override fun get(column: String): String =
        header.indexOf(column).takeIf { it in 0..< size }?.let { values[it] }
            ?: throw NoSuchElementException("No such column: $column")

    override fun toString(): String = toString("Record")
}

The DefaultTable class

internal class DefaultTable(override val header: Header, records: Iterable<Record>) : Table {
    // Eager, defensive copy of the records
    override val records: List<Record> = records.toList()

    override fun get(row: Int): Row = if (row == 0) header else records[row - 1]

    override val size: Int
        get() = records.size + 1

    override fun iterator(): Iterator<Row> = (sequenceOf(header) + records.asSequence()).iterator()
    override fun equals(other: Any?): Boolean {
        if (this === other) return true
        if (other !is DefaultTable) return false
        if (header != other.header) return false
        if (records != other.records) return false

        return true
    }

    override fun hashCode(): Int {
        var result = header.hashCode()
        result = 31 * result + records.hashCode()
        return result
    }

    override fun toString(): String = joinToString(", ", "Table(", ")")
}

Need for factory methods

• To enforce separation between API and implementation code, it’s better:

  • to keep interfaces public, and classes internal
  • to provide factory methods for creating instances of the interfaces

• Convention in Kotlin is to create factory methods as package-level functions

  • e.g. contained into the io/github/gciatto/csv/Csv.kt file

• Factory methods may be named after the concept they create: <concept>Of(args...)

  • e.g. headerOf(columns...), recordOf(header, values...), etc.

• Class diagram:

    @startuml
    skinparam monochrome false
    class Csv{
        + headerOf(columns): Header
        + anonymousHeader(size: Int): Header
        ..
        + recordOf(header, values): Record
        ..
        + tableOf(header, records): Table
        + tableOf(rows): Table
    }
    @enduml
    

The Csv.kt file (pt. 1)

Factories for headers (possibly anonymous, i.e. with columns named after their index), records, and tables:

fun headerOf(columns: Iterable<String>): Header = DefaultHeader(columns)

@JsExport
fun headerOf(vararg columns: String): Header = headerOf(columns.asIterable())
fun headerOf(columns: Sequence<String>): Header = headerOf(columns.asIterable())

@JsExport
fun anonymousHeader(size: Int): Header = headerOf((0 ..< size).map { it.toString() })

fun recordOf(header: Header, columns: Iterable<String>): Record = DefaultRecord(header, columns)

@JsExport
fun recordOf(header: Header, vararg columns: String): Record = recordOf(header, columns.asIterable())
fun recordOf(header: Header, columns: Sequence<String>): Record = recordOf(header, columns.asIterable())

fun tableOf(header: Header, records: Iterable<Record>): Table = DefaultTable(header, records)

@JsExport
@JsName("table")
fun tableOf(header: Header, vararg records: Record): Table = tableOf(header, records.asIterable())
fun tableOf(header: Header, records: Sequence<Record>): Table = tableOf(header, records.asIterable())

• Notice that each factory method is overloaded
  • to support Iterable, Sequence, and vararg arguments
  • this is convenient for Kotlin programmers who will use our library

The Csv.kt file (pt. 2)

Table creation from rows (anonymous header if none is provided):

fun tableOf(rows: Iterable<Row>): Table {
    val records = mutableListOf<Record>()
    var header: Header? = null
    for (row in rows) {
        when (row) {
            is Header -> header = row
            is Record -> records.add(row)
        }
    }
    require(header != null || records.isNotEmpty())
    return tableOf(header ?: anonymousHeader(records[0].size), records)
}

@JsExport
@JsName("tableOf")
fun tableOf(vararg rows: Row): Table = tableOf(rows.asIterable())

fun tableOf(rows: Sequence<Row>): Table = tableOf(rows.asIterable())

Time for unit testing (pt. 1)

Dummy instances object, in Tables.kt:

object Tables {
    val irisShortHeader = headerOf("sepal_length", "sepal_width", "petal_length", "petal_width", "class")

    val irisLongHeader = headerOf(
        "sepal length in cm",
        "sepal width, in cm",
        "petal length",
        "petal width",
        "class"
    )

    fun iris(header: Header): Table = tableOf(
        header,
        recordOf(header, "5.1", "3.5", "1.4", "0.2", "Iris-setosa"),
        recordOf(header, "4.9", "3.0", "1.4", "0.2", "Iris-setosa"),
        recordOf(header, "4.7", "3.2", "1.3", "0.2", "Iris-setosa")
    )
}

Time for unit testing (pt. 2)

Some basic tests in TestCSV.kt (bad way of writing test methods, don’t do this at home):

    @Test
    fun recordBasics() {
        val record = Tables.iris(Tables.irisShortHeader).records[0]
        assertEquals("5.1", record[0])
        assertEquals("5.1", record["sepal_length"])
        assertEquals("3.5", record[1])
        assertEquals("3.5", record["sepal_width"])
        assertEquals("1.4", record[2])
        assertEquals("1.4", record["petal_length"])
        assertEquals("0.2", record[3])
        assertEquals("0.2", record["petal_width"])
        assertEquals("Iris-setosa", record[4])
        assertEquals("Iris-setosa", record["class"])
        assertFailsWith<IndexOutOfBoundsException> { record[5] }
        assertFailsWith<NoSuchElementException> { record["missing"] }
    }

• Run tests via Gradle task allTests (also try to run tests for specific platforms, e.g. jvmTest or jsTest)

Summary of what we did so far

1. We designed a set of domain entities

   • Row, Header, Record, Table
   • we provided a set of factory methods for creating instances of those entities

2. We implemented the domain entities as common code

3. We wrote some unit tests for the domain entities

   • again as common code

4. Let’s focus now on more complex functionalities

   • e.g. parsing a CSV file into a Table
   • e.g. saving a Table into a CSV file
   • all such features require I/O functionalities

I/O is platform-specific, hence we need platform-specific code

How much platform-specific code is needed?

• I/O functionalities are supported by fairly different APIs in the JVM and JS

  • e.g. JVM’s java.io package vs. JS’ fs module
  • sadly, Kotlin std-lib does not provide a common API for I/O

• Do we need to rewrite the same business logic twice? (once for JVM, once for JS)

  • yes, but we can minimise the amount of code to be rewritten

• Let’s try to decompose the problem as follows:

  • parsing CSV into Table: file $\rightarrow$ string(s) $\rightarrow$ Table
  • saving Table as CSV file: Table $\rightarrow$ string(s) $\rightarrow$ file

• Remarks:

  1. only the “file $\leftrightarrow$ string(s)” part is platform-specific
  2. conversely, the “string(s) $\leftrightarrow$ Table” part is platform-agnostic
     • let’s realise this one first, as common code

Three more entities to be modelled

• Configuration: set of characters to be used to parse / represent CSV files

  • e.g. separator, delimiter, comment, etc.

• Formatter: converts Rows into strings, according to some Configuration

• Parser: converts some source into a Table, according to some Configuration

  • source $\approx$ anything that can be interpreted as a string to be parsed (e.g. file, string, etc.)

@startuml
skinparam monochrome false

package "io github gciatto csv" {

    class Configuration {
        + separator: Char
        + delimiter: Char
        + comment: Char
        --
        + isComment(string: String): Boolean
        + isRecord(string: String): Boolean
        + isHeader(string: String): Boolean
        ..
        + getComment(string: String): String?
        + getFields(string: String): List<String>
        + getColumnNames(string: String): List<String>
    }

    interface Formatter {
        + source: Iterable<Row>
        + configuration: Configuration
        + format(): Iterable<String>
    }

    interface Parser {
        + source: Any
        + configuration: Configuration
        + parse(): Iterable<Row>
    }

    Formatter "1" o-- "1" Configuration
    Parser "1" o-- "1" Configuration

}
@enduml

The Configuration class

data class Configuration(val separator: Char, val delimiter: Char, val comment: Char) {
    // Checks whether the given string is a comment (i.e. starts with the comment character).
    fun isComment(string: String): Boolean = TODO()

    // Gets the content of the comment line (i.e. removes the initial comment character).
    fun getComment(string: String): String? = TODO()

    // Checks whether the given string is a record (i.e. it contains at least one field)
    fun isRecord(string: String): Boolean = TODO()

    // Retrieves the fields in the given record (i.e. splits the string at the separator character)
    fun getFields(string: String): List<String> = TODO()

    // Checks whether the given string is a header (i.e. a comment whose content is a record)
    fun isHeader(string: String): Boolean = TODO()

    // Retrieves the column names in the given header
    fun getColumnNames(string: String): List<String> = TODO()
}

• implementation of methods is long and boring, but straightforward
  • it relies on regular expressions, which are supported by Kotlin’s common std-lib
  • details in Configuration.kt

The Formatter interface

/** Converts rows into strings, according to some [Configuration]. */
@JsExport
interface Formatter {
    /** The source of this formatter (i.e. the rows to be formatted). */
    @JsName("source")
    val source: Iterable<Row>

    /** The configuration of this formatter (i.e. the characters to be used). */
    @JsName("configuration")
    val configuration: Configuration

    /** Formats the source into a sequence of strings (one for each row in the source). */
    @JsName("format")
    fun format(): Iterable<String>
}

The Parser interface

/** Converts some source into rows, according to some [Configuration]. */
@JsExport
interface Parser {
    /** The source to be parsed by this parser (must be interpretable as string). */
    @JsName("source")
    val source: Any

    /** The configuration of this parser (i.e. the characters to be used). */
    @JsName("configuration")
    val configuration: Configuration

    /** Parses the source into a sequence of rows (one for each row in the source). */
    @JsName("parse")
    fun parse(): Iterable<Row>
}

Common implementation of novel entities

@startuml
skinparam monochrome false

package "io github gciatto csv impl" {

    class Configuration
    interface Formatter
    interface Parser

    DefaultFormatter "1" o-- "1" Configuration
    AbstractParser "1" o-- "1" Configuration

    class DefaultFormatter {
        + constructor(source: Iterable<Row>, configuration: Configuration)
        - formatRow(row: Row): String
        - formatAsHeader(row: Row): String
        - formatAsRecord(row: Row): String
    }

    Formatter <|-- DefaultFormatter

    abstract class AbstractParser {
        # constructor(source: Any, configuration: Configuration)
        # beforeParsing()
        # afterParsing()
        # {abstract} sourceAsLines(): Sequence<String>
    }

    Parser <|-- AbstractParser

    class StringParser {
        + source: String
        + constructor(source: String, configuration: Configuration)
        + sourceAsLines(): Sequence<String>
    }

    AbstractParser <|-- StringParser
}
@enduml

The DefaultFormatter class

internal class DefaultFormatter(
    override val source: Iterable<Row>,
    override val configuration: Configuration
) : Formatter {
    override fun format(): Iterable<String> = source.asSequence().map(this::formatRow).asIterable()

    private fun formatAsHeader(header: Header): String = "${configuration.comment} ${formatAsRecord(header)}"

    private fun formatAsRecord(row: Row): String =
        row.joinToString("${configuration.separator} ") {
            val delimiter = configuration.delimiter
            "$delimiter$it$delimiter"
        }

    private fun formatRow(row: Row): String = when (row) {
        is Header -> formatAsHeader(row)
        else -> formatAsRecord(row)
    }
}

• format lazily converts each row into a string: headers are prefixed by the comment character, fields are wrapped by the delimiter character

The AbstractParser class (pt. 1)

internal abstract class AbstractParser(
    override val source: Any,
    override val configuration: Configuration
) : Parser {

    // Hooks to be overridden by sub-classes to initialise / finalise parsing (e.g. open / close files)
    protected open fun beforeParsing() = Unit
    protected open fun afterParsing() = Unit

    // Template method that splits the source into a sequence of strings (one per line)
    protected abstract fun sourceAsLines(): Sequence<String>

• beforeParsing / afterParsing are hooks, sourceAsLines is the abstract step of a template method

The AbstractParser class (pt. 2)

    override fun parse(): Iterable<Row> = sequence {
        beforeParsing()
        try {
            var header: Header? = null
            for ((i, line) in sourceAsLines().withIndex()) {
                if (line.isBlank()) {
                    continue
                } else if (configuration.isHeader(line)) {
                    if (header == null) {
                        header = headerOf(configuration.getColumnNames(line))
                        yield(header)
                    }
                } else if (configuration.isComment(line)) {
                    continue
                } else if (configuration.isRecord(line)) {
                    val fields = configuration.getFields(line)
                    if (header == null) {
                        header = anonymousHeader(fields.size)
                        yield(header)
                    }
                    try {
                        yield(recordOf(header, fields))
                    } catch (e: IllegalArgumentException) {
                        throw IllegalStateException("Invalid CSV at line ${i + 1}: $line", e)
                    }
                } else {
                    error("Invalid CSV at line ${i + 1}: $line")
                }
            }
        } finally {
            afterParsing() // executed even in case of errors
        }
    }.asIterable()

• sequence { ... yield(...) } produces rows lazily, one at a time
• finally ensures afterParsing() is called upon errors too (not if the consumer stops iterating early)

The StringParser class

internal class StringParser(override val source: String, configuration: Configuration) : AbstractParser(source, configuration) {
    override fun sourceAsLines(): Sequence<String> = source.lineSequence()
}

• it simply splits the source string into lines

Providing functionalities via extension methods

Additions to the Csv.kt file:

const val DEFAULT_SEPARATOR = ','
const val DEFAULT_DELIMITER = '"'
const val DEFAULT_COMMENT = '#'

@JsExport
fun Iterable<Row>.formatAsCSV(
    separator: Char = DEFAULT_SEPARATOR,
    delimiter: Char = DEFAULT_DELIMITER,
    comment: Char = DEFAULT_COMMENT
): String = DefaultFormatter(this, Configuration(separator, delimiter, comment)).format().joinToString("\n")

@JsExport
fun String.parseAsCSV(
    separator: Char = DEFAULT_SEPARATOR,
    delimiter: Char = DEFAULT_DELIMITER,
    comment: Char = DEFAULT_COMMENT
): Table = StringParser(this, Configuration(separator, delimiter, comment)).parse().let(::tableOf)

• formatAsCSV converts any container of rows into a CSV string, parseAsCSV parses a CSV string into a table

Unit tests and usage examples (pt. 1)

Dummy instances for testing (other constants in CsvStrings.kt):

object CsvStrings {
    val iris: String = """
    |# sepal_length, sepal_width, petal_length, petal_width, class
    |5.1,3.5,1.4,0.2,Iris-setosa
    |4.9,3.0,1.4,0.2,Iris-setosa
    |4.7,3.2,1.3,0.2,Iris-setosa
    """.trimMargin()

    val irisWellFormatted: String = """
    |# "sepal_length", "sepal_width", "petal_length", "petal_width", "class"
    |"5.1", "3.5", "1.4", "0.2", "Iris-setosa"
    |"4.9", "3.0", "1.4", "0.2", "Iris-setosa"
    |"4.7", "3.2", "1.3", "0.2", "Iris-setosa"
    """.trimMargin()

Unit tests and usage examples (pt. 2)

    @Test
    fun parsingFromCleanString() {
        val iris: Table = CsvStrings.iris.parseAsCSV()
        assertEquals(
            expected = Tables.iris(Tables.irisShortHeader),
            actual = iris
        )
    }

    @Test
    fun formattingToString() {
        val iris: Table = Tables.iris(Tables.irisShortHeader)
        assertEquals(
            expected = CsvStrings.irisWellFormatted,
            actual = iris.formatAsCSV()
        )
    }

Time to go platform-specific

• Further additions to the Csv.kt file (reading and parsing a CSV file from the given path):

expect fun parseCsvFile(
    path: String,
    separator: Char = DEFAULT_SEPARATOR,
    delimiter: Char = DEFAULT_DELIMITER,
    comment: Char = DEFAULT_COMMENT
): Table

• notice the expect keyword, and the lack of function body
• we’re just declaring the signature of a platform-specific function
• there is no type for representing paths in Kotlin’s common std-lib
  • hence we’re using String as a platform-agnostic representation of paths
    • this is a suboptimal choice

Time to go platform-specific on the JVM (pt. 1)

• I/O (over textual files) is mainly supported by means of the following classes:

  • java.io.File
  • java.io.BufferedReader
  • java.io.BufferedWriter

• Buffered readers support reading a file line-by-line

• On Kotlin/JVM, Java’s std-lib is available as Kotlin’s std-lib

  • hence, we may use Java’s std-lib directly
  • the abstraction gap is close to 0

• We may simply create a new sub-type of AbstractParser

  • using Files as sources
  • creating a BufferedReader behind the scenes to read files line-by-line

Time to go platform-specific on the JVM (pt. 2)

In the jvmMain source set

A JVM-specific parser, FileParser.kt:

internal class FileParser(override val source: File, configuration: Configuration) : AbstractParser(source, configuration) {
    private lateinit var reader: BufferedReader

    override fun beforeParsing() {
        reader = source.bufferedReader()
    }

    override fun afterParsing() {
        reader.close()
    }

    override fun sourceAsLines(): Sequence<String> = reader.lines().asSequence()
}

• the source is now forced to be a File
• the lateinit reader is opened before parsing, and closed afterwards
• the file is read lazily, line-by-line

Time to go platform-specific on the JVM (pt. 3)

The Csv.jvm.kt file:

@file:JvmName("CsvJvm")

package io.github.gciatto.csv

import java.io.File
import kotlin.jvm.JvmName

actual fun parseCsvFile(
    path: String,
    separator: Char,
    delimiter: Char,
    comment: Char
): Table = FileParser(File(path), Configuration(separator, delimiter, comment)).parse().let(::tableOf)

• this is how parseCsvFile is implemented on the JVM
• notice the actual keyword, and the presence of a function body
• notice the usage of FileParser in the function body
  • FileParser is an internal class filling the abstraction gap on the JVM
• @file:JvmName is explained later

Time to go platform-specific on the JS (pt. 1)

• I/O (over textual files) is mainly supported by means of the following things:

  • the fs module
  • the fs.readFileSync function
  • the fs.writeFileSync function

• These functions support reading / writing a file in one shot

  • i.e. they return / generate the whole content of the file as a string
  • quite inefficient if the file is big

• On Kotlin/JS, Node’s std-lib is not directly available

  • one must instruct the compiler about
    1. where to look up std-lib functions (module name)
    2. how to map JS functions to Kotlin functions (external declarations)
  • the abstraction gap is non-negligible

• To-do list for Kotlin/JS:

  1. create external declarations for the Node’s std-lib functions to be used
  2. implement JS-specific code via the StringParser (after reading the whole file)

Time to go platform-specific on the JS (pt. 2)

In the jsMain source set

First, let’s create the NodeFs.kt file (containing external declarations for the fs module):

@file:JsModule("fs")

package io.github.gciatto.csv

external fun readFileSync(path: String, options: dynamic = definedExternally): String

external fun writeFileSync(file: String, data: String, options: dynamic = definedExternally)

• Kotlin mappings for readFileSync and writeFileSync
• the @file:JsModule("fs") annotation instructs the compiler that the declarations of this file come from the fs module
• external declarations are Kotlin signatures of JS functions
• dynamic is a special type that can be used to represent any JS object
  • it overrides Kotlin’s type system, hence it should be used with care
  • it prevents developers from declaring too much external stuff
  • good to use when the JS API is not known in advance or uses union types
• definedExternally is stating that a parameter is optional (default value is defined in JS)

Time to go platform-specific on the JS (pt. 3)

Then, let’s create the Csv.js.kt file containing:

@JsExport
actual fun parseCsvFile(
    path: String,
    separator: Char,
    delimiter: Char,
    comment: Char
): Table = readFileSync(path, js("{encoding: 'utf8'}")).parseAsCSV(separator, delimiter, comment)

• this is how the parseCsvFile is implemented on JS
• notice the actual keyword, and the presence of a function body
• notice the usage of readFileSync to read a file as a string in one shot
• notice the exploitation of common code for parsing the string into a Table (namely, parseAsCSV)
• @JsExport makes the function callable from JS code (see later)
• notice the js("...") magic function
  • it allows JS code to be written directly in Kotlin
  • the provided string should contain bare JS code, which the compiler will output as-is
  • it’s a way to fill the abstraction gap on JS
  • we use it to create a JS object on the fly, to provide optional parameters to readFileSync

How much platform-specific code did we write?

• Recall the plan: only the “file $\leftrightarrow$ string(s)” part is platform-specific

Platform-agnostic testing (strategy)

• We may test our CSV library in common-code

  • so as to avoid repeating testing code

• The test suite may:

  1. create temporary files with known paths, containing known CSV data
  2. read those paths, and parse them as CSV
  3. assert that the parsed tables are as expected

• The hard part is step 1: two platform-specific steps

  1. discovering the temp directory of the current system
  2. writing a file

Multi-platform testing (preliminaries, pt. 1)

• file Utils.kt in commonTest (creates a temporary file with the given name and content, and returns its path):

internal fun String.splitExtension(): Pair<String, String> {
    val splitted = split(".")
    return if (splitted.size > 1) {
        splitted.subList(0, splitted.lastIndex).joinToString(".") to splitted.last()
    } else {
        this to ""
    }
}

expect fun createTempFile(name: String, content: String): String

• notice the expected function

• file Utils.jvm.kt in jvmTest:

actual fun createTempFile(name: String, content: String): String {
    val (n, e) = name.splitExtension()
    val file = File.createTempFile(n, ".$e")
    file.writeText(content)
    return file.absolutePath
}

• cf. documentation of method File.createTempFile

Multi-platform testing (preliminaries, pt. 2)

• file NodeOs.kt in jsMain (external declaration for the os module):

@file:JsModule("os")

package io.github.gciatto.csv

external fun tmpdir(): String

• file Utils.js.kt in jsTest:

import kotlin.random.Random

private val tmpDirectory by lazy {
    tmpdir()
}

actual fun createTempFile(name: String, content: String): String {
    val (n, e) = name.splitExtension()
    val tag = Random.nextLong().toULong()
    val path = "$tmpDirectory/$n-$tag.$e"
    writeFileSync(path, content)
    return path
}

• cf. documentation of os.tmpdir
• kotlin.random.Random is part of the common std-lib, so no JS-specific code is needed for random numbers

Multi-platform testing (actual tests)

• file CsvFiles.kt in commonTest (paths of temporary files, lazily created upon first usage, containing the strings in CsvStrings):

object CsvFiles {
    val iris: String by lazy {
        createTempFile("iris.csv", CsvStrings.iris)
    }

    val irisWellFormatted: String by lazy {
        createTempFile("irisWellFormatted.csv", CsvStrings.irisWellFormatted)
    }

• tests for parsing (in TestCsvFiles.kt) be like:

    @Test
    fun testParseIris() {
        val parsedFromString = CsvStrings.iris.parseAsCSV()
        val readFromFile = parseCsvFile(CsvFiles.iris)
        assertEquals(parsedFromString, readFromFile)
    }

Output of a multi-platform build

• Kotlin multi-platform libraries are published as a set of artifacts (one per target, plus Gradle module metadata)

  • enabling importing the project as a dependency in other multi-platform projects

• The jvmMain source set is compiled into a JVM-compliant JAR

  • enabling importing the project as a dependency in JVM projects
  • enabling the creation of runnable JARs
  • via the various *Jar or assemble tasks

• The jsMain source set is compiled into either

  • a Kotlin library (.klib), enabling importing the project as a dependency in Kotlin/JS projects
    • via the various *Jar or assemble tasks
  • a NodeJS project
    • via the jsProductionExecutableCompileSync (or jsProductionLibraryCompileSync) task

Output of JVM compilation

• Task for JVM-only compilation of re-usable packages: jvmJar

• Effect:

  1. compile the project as JVM bytecode
  2. pack the bytecode into a JAR file
  3. move the JAR file into $PROJ_DIR/build/libs/$PROJ_NAME-jvm-$PROJ_VERSION.jar

• The JAR does not contain dependencies

• Ad-hoc Gradle plugins or code are needed to create a fat JAR

  • e.g. the running example uses the Shadow plugin, cf. task shadowJar

Output of JS compilation

• Task for JS-only compilation of re-usable packages: jsProductionLibraryCompileSync (with binaries.library())

  • or jsProductionExecutableCompileSync (with binaries.executable())

• Effect:

  1. Generate Node project into $ROOT_PROJ_DIR/build/js/packages/$ROOT_PROJ_NAME-$PROJ_NAME:
     • JS code (.mjs files, when using ES modules)
     • package.json file
  2. Dead code elimination removing unused code
     • i.e. code which is neither exported nor (indirectly) called by main

• Support for packing as npm package via the npm-publish Gradle plugin

Calling Kotlin from other platforms

• Kotlin code can be called from the target platforms’ main languages

  • e.g. Java, JavaScript, etc.

• Understanding the mapping between Kotlin and other languages is key

  • it impacts the usability of Kotlin libraries for ordinary platform users

How are Kotlin’s syntactical categories mapped to other platforms/languages?

• the answer varies on a per-platform basis

Kotlin–Java Mapping (pt. 1)

• Kotlin class $\equiv$ Java class

class MyClass {}
interface MyType {}

public class MyClass {}
public interface MyType {}

• No syntactical difference between primitive and reference types
  • Int $\leftrightarrow$ int / Integer, Short $\leftrightarrow$ short / Short, etc.

val x: Int = 0
val y: Int? = null

int x = 0;
Integer y = null;

• Some Kotlin types are replaced by Java types at compile time
  • e.g. Any $\rightarrow$ Object, kotlin.collections.List $\rightarrow$ java.util.List, etc.

val x: Any = "ciao"
val y: kotlin.collections.List<Int> = listOf(1, 2, 3)
val z: kotlin.collections.MutableList<String> = mutableListOf("a", "b", "c")

Object x = "ciao";
java.util.List<Integer> y = java.util.Arrays.asList(1, 2, 3);
java.util.List<String> z = java.util.List.of("a", "b", "c");

• Other Kotlin types are mapped to homonymous Java types

Kotlin–Java Mapping (pt. 2)

• Kotlin properties are mapped to getter / setter methods

interface MyType {
    val x: Int
    var y: String
}

public interface MyType {
    int getX();
    String getY(); void setY(String y);
}

• … unless the JvmField annotation is adopted

import kotlin.jvm.JvmField
class MyType {
    @JvmField
    val x: Int = 0
    @JvmField
    var y: String = ""
}

public class MyType {
    public final int x = 0;
    public String y = "";
}

• Kotlin’s top-level functions in file X.kt are mapped to static methods of class XKt

// file MyFile.kt
fun f() {}

public final class MyFileKt {
    public static void f() {}
}

• … unless the JvmName annotation is exploited (e.g. @file:JvmName("Csv") in our Csv.kt)

@file:JvmName("MyModule")
import kotlin.jvm.JvmName
fun f() {}

public final class MyModule {
    public static void f() {}
}

Kotlin–Java Mapping (pt. 3)

• Kotlin’s object X is mapped to Java class X with
  • private constructor
  • public static final field named INSTANCE to access

object MySingleton {}

public final class MySingleton {
    private MySingleton() {}
    public static final MySingleton INSTANCE = new MySingleton();
}

• Class X’s companion object is mapped to public static final field named Companion on class X

class MyType private constructor() {
    companion object {
        fun of(): MyType = MyType()
    }
}

// usage:
val x: MyType = MyType.of()

public class MyType {
    private MyType() {}
    public static final class Companion {
        public MyType of() { return new MyType(); }
    }
    public static final Companion Companion = new Companion();
}

// usage:
MyType x = MyType.Companion.of();

Kotlin–Java Mapping (pt. 4)

• Class X’s companion object’s member M tagged with @JvmStatic is mapped to static member M on class X

import kotlin.jvm.JvmStatic
class MyType private constructor() {
    companion object {
        @JvmStatic
        fun of(): MyType = MyType()
    }
}

// usage:
val x: MyType = MyType.of()

public class MyType {
    private MyType() {}
    public static MyType of() { return new MyType(); }
}

// usage:
MyType x = MyType.of();

• Kotlin’s variadic functions are mapped to Java’s variadic methods

fun f(vararg xs: Int) {
    val ys: IntArray = xs
}

void f(int... xs) {
    int[] ys = xs;
}

• Kotlin’s extension methods are mapped to static Java methods with one more argument (the receiver)

// file MyFile.kt
class MyType { }
fun MyType.myMethod() {}

// usage:
val x = MyType()
x.myMethod() // postfix

public final class MyFileKt {
    public static void myMethod(MyType $this$myMethod) {}
}

// usage:
MyType x = new MyType();
MyFileKt.myMethod(x); // prefix

Kotlin–Java Mapping (pt. 5)

• practical consequence: usability of fluent chains, in Java, is suboptimal
  • e.g. sequence operations are implemented as extension methods

import kotlin.sequences.*

val x = (0 ..< 10).asSequence() // 0, 1, 2, ..., 9
    .filter { it % 2 == 0 } // 0, 2, 4, ..., 8
    .map { it + 1 } // 1, 3, 5, ..., 9
    .sum() // 25

import static kotlin.sequences.SequencesKt.*;

int x = sumOfInt(
            map(
                filter(
                    asSequence(
                            new IntRange(0, 9).iterator()
                    ),
                    it -> it % 2 == 0
                ),
                it -> it + 1
            )
        );

• optional arguments exist in Kotlin but they do not exist in Java

fun f(a: Int = 1, b: Int = 2, c: Int = 3) { /* ... */ }

// usage:
f(b = 5)

void f(int a, int b, int c) { /* ... */ }

// usage:
f(1, 5, 3);

Kotlin–Java Mapping (pt. 6)

• practical consequence:
  • optional arguments are handy in Kotlin, but…
  • … they are painful to use in Java
  • mitigation strategy: use @JvmOverloads to generate overloaded methods for Java
    • by design, the annotation only generates overloads dropping trailing parameters

import kotlin.jvm.JvmOverloads

@JvmOverloads
fun f(a: Int = 1, b: Int = 2, c: Int = 3) { /* ... */ }

void f(int a, int b, int c) { /* ... */ }
void f(int a, int b) { f(a, b, 3); }
void f(int a) { f(a, 2, 3); }
void f() { f(1, 2, 3); }

// overloads that are not generated (some would even clash, having the same JVM signature):
// void f(int a, int c) { f(a, 2, c); }
// void f(int b, int c) { f(1, b, c); }
// void f(int c) { f(1, 2, c); }
// void f(int b) { f(1, b, 3); }

Kotlin–Java Mapping (pt. 7)

• Kotlin has no checked exceptions: use @Throws to let Java callers catch them

  @Throws(IOException::class)
  fun readFile(path: String): String { /* ... */ }  // Java: String readFile(String path) throws IOException
  

• Kotlin’s nullability is exposed to Java via annotations (@NotNull, @Nullable)

  • Java types are seen in Kotlin as platform types (e.g. String!), whose nullability is unknown

• internal members are public in the bytecode, but their names are mangled

  • e.g. internal fun f() in module csv-core becomes f$csv_core(), which is discouraging to call from Java

• Reference: calling Kotlin from Java

Kotlin–Java Mapping Example

• Compile our CSV lib and import it as a dependency in a novel Java library

• Alternatively, add Java sources to the jvmTest source set, and use the library

• Listen to the teacher presenting key points

Kotlin–JavaScript Mapping (pt. 1)

Disclaimer: the generated JS code is not really meant to be read by humans

• DCE will eliminate unused code
  • “unused” $\equiv$ “neither exported, nor (indirectly) used by exported code or main”
  • code is exported by means of the @JsExport annotation
    • to be used on API types and functions
    • requires the kotlin.js.ExperimentalJsExport opt-in
  • not all syntactical constructs or types are currently exportable
    • members that cannot be exported can be excluded via @JsExport.Ignore
    • as a last resort, the warning can be suppressed (as the running example does)

package my.pkg

import kotlin.js.JsExport

@JsExport
class MyType {
    @JsExport.Ignore // Sequence is not exportable
    val numbers: Sequence<Int> = sequenceOf(1, 2, 3)
}

@JsExport
fun myFunction() { /* ... */ }

// ES modules: packages are flattened
import { MyType, myFunction } from "project-name";

// CommonJS / UMD: packages are preserved
const pkg = require("project-name").my.pkg;
const { MyType, myFunction } = pkg;

Kotlin–JavaScript Mapping (pt. 2)

• Kotlin supports overloading, JS does not
  • class / interface member names are mangled to avoid clashes
  • the @JsName annotation can be used to control the name of a member
    • to be used on API types and functions

package my.pkg

@JsName("sumNumbers")
fun sum(vararg numbers: Int): Int = numbers.sum()

@JsName("sumIterableOfNumbers")
fun sum(numbers: Iterable<Int>): Int = numbers.sum()

@JsName("sumSequenceOfNumbers")
fun sum(numbers: Sequence<Int>): Int = numbers.sum()

import {
    sumNumbers,
    sumIterableOfNumbers,
    sumSequenceOfNumbers
} from "project-name";

• Kotlin class $\equiv$ JS prototype (with the default ES5 target)

class MyClass(private val argument: String) {
    @JsName("method")
    fun method(): String = argument + "!"
}

function MyClass(argument) {
    this.randomName_1 = argument
}
MyClass.prototype.method = function () {
    return this.randomName_1 + "!"
}

Kotlin–JavaScript Mapping (pt. 3)

• Primitive type mappings are non-trivial (cf. documentation)

  • Kotlin numeric types, except for kotlin.Long, are mapped to JavaScript Number

  • kotlin.Char is mapped to a JS Number representing the character code.

  • Kotlin preserves overflow semantics for kotlin.Int, kotlin.Byte, kotlin.Short, kotlin.Char and kotlin.Long

  • kotlin.Long is emulated by a Kotlin class by default

    • it can be mapped to JS BigInt (experimental), with the es2020 target or the -Xes-long-as-bigint compiler option

  • kotlin.String is mapped to JS String

  • kotlin.Any is mapped to JS Object (new Object(), {}, and so on)

  • kotlin.Array is mapped to JS Array

  • Kotlin collections are mapped to KtList, KtSet, KtMap (and their mutable variants)

    • which expose JS views via, e.g., asJsReadonlyArrayView()

  • kotlin.Throwable is mapped to JS Error

Kotlin–JavaScript Mapping (pt. 4)

• practical consequence: no way to distinguish numbers by type

  val x: Int = 23
  val y: Any = x
  println(y as Float) // fails on JVM, works on JS
  

• Kotlin’s dynamic overrides the type system, and it is translated “1-to-1”

val string1: String = "some string"
string1.missingMethod() // compilation error

val string2: dynamic = "some string"
string2.missingMethod() // compilation ok, runtime error

• Kotlin’s common std-lib is implemented in JS
  • it is bundled into the compiled output (e.g. kotlin-kotlin-stdlib.mjs), and dead code is eliminated

Kotlin–JavaScript Mapping (pt. 5)

• Companion objects are treated similarly to Kotlin/JVM

• Extension methods are treated similarly to Kotlin/JVM

• Variadic functions are compiled to JS functions accepting an array

  • which are not really variadic in JS

fun f(vararg xs: String) { /* ... */ }

// usage:
f("a")
f("a", "b")
f("a", "b", "c")

// usage:
f(["a"])
f(["a", "b"])
f(["a", "b", "c"])