DataLake SDK

A managed Apache Iceberg data catalog for large-scale analytics on object storage with ACID transactions and schema flexibility.

Import

import { $ } from 'sdk.do'

Operations

create

Create a new Iceberg data catalog.

const result = await $.DataLake.create({
  // parameters
})

delete

Permanently remove the data lake and all its tables.

const result = await $.DataLake.delete({
  // parameters
})

createTable

Create a new Iceberg table in the catalog.

const result = await $.DataLake.createTable({
  // parameters
})

dropTable

Remove an Iceberg table from the catalog.

const result = await $.DataLake.dropTable({
  // parameters
})

query

Execute SQL queries using R2 SQL engine.

const result = await $.DataLake.query({
  // parameters
})

insert

Insert data into an Iceberg table.

const result = await $.DataLake.insert({
  // parameters
})

update

Update existing records in an Iceberg table.

const result = await $.DataLake.update({
  // parameters
})

delete

Delete records from an Iceberg table.

const result = await $.DataLake.delete({
  // parameters
})

snapshot

Create a new snapshot for time-travel queries.

const result = await $.DataLake.snapshot({
  // parameters
})

timeTravel

Query data as it existed at a specific point in time.

const result = await $.DataLake.timeTravel({
  // parameters
})

evolveSchema

Add, rename, or delete columns without rewriting data.

const result = await $.DataLake.evolveSchema({
  // parameters
})

listTables

List all tables in the catalog.

const result = await $.DataLake.listTables({
  // parameters
})

getSchema

Retrieve the schema for a specific table.

const result = await $.DataLake.getSchema({
  // parameters
})

Events

created

Triggered when a new Iceberg data catalog is created.

import { on, send } from 'sdk.do'

// Subscribe to event
on.DataLake.created(async (data) => {
  console.log('Event received:', data)
})

// Emit event
await send.DataLake.created({ /* data */ })

deleted

Triggered when data lake is removed.

import { on, send } from 'sdk.do'

// Subscribe to event
on.DataLake.deleted(async (data) => {
  console.log('Event received:', data)
})

// Emit event
await send.DataLake.deleted({ /* data */ })

Table.created

Triggered when a new table is created in the catalog.

import { on, send } from 'sdk.do'

// Subscribe to event
on.DataLake.Table.created(async (data) => {
  console.log('Event received:', data)
})

// Emit event
await send.DataLake.Table.created({ /* data */ })

Table.dropped

Triggered when a table is removed from the catalog.

import { on, send } from 'sdk.do'

// Subscribe to event
on.DataLake.Table.dropped(async (data) => {
  console.log('Event received:', data)
})

// Emit event
await send.DataLake.Table.dropped({ /* data */ })

queried

Triggered when SQL query is executed via R2 SQL.

import { on, send } from 'sdk.do'

// Subscribe to event
on.DataLake.queried(async (data) => {
  console.log('Event received:', data)
})

// Emit event
await send.DataLake.queried({ /* data */ })

Data.inserted

Triggered when data is inserted into a table.

import { on, send } from 'sdk.do'

// Subscribe to event
on.DataLake.Data.inserted(async (data) => {
  console.log('Event received:', data)
})

// Emit event
await send.DataLake.Data.inserted({ /* data */ })

Data.updated

Triggered when records are updated in a table.

import { on, send } from 'sdk.do'

// Subscribe to event
on.DataLake.Data.updated(async (data) => {
  console.log('Event received:', data)
})

// Emit event
await send.DataLake.Data.updated({ /* data */ })

Data.deleted

Triggered when records are deleted from a table.

import { on, send } from 'sdk.do'

// Subscribe to event
on.DataLake.Data.deleted(async (data) => {
  console.log('Event received:', data)
})

// Emit event
await send.DataLake.Data.deleted({ /* data */ })

Snapshot.created

Triggered when a new snapshot is created.

import { on, send } from 'sdk.do'

// Subscribe to event
on.DataLake.Snapshot.created(async (data) => {
  console.log('Event received:', data)
})

// Emit event
await send.DataLake.Snapshot.created({ /* data */ })

Query.timeTraveled

Triggered when historical data is queried.

import { on, send } from 'sdk.do'

// Subscribe to event
on.DataLake.Query.timeTraveled(async (data) => {
  console.log('Event received:', data)
})

// Emit event
await send.DataLake.Query.timeTraveled({ /* data */ })

Schema.evolved

Triggered when table schema is modified.

import { on, send } from 'sdk.do'

// Subscribe to event
on.DataLake.Schema.evolved(async (data) => {
  console.log('Event received:', data)
})

// Emit event
await send.DataLake.Schema.evolved({ /* data */ })

DataLake SDK

On this page