sdTF - Structured data transfer format

Overview

sdTF (Structured Data Transfer Format) is a file format for the efficient transfer and loading of trees of data items as used by parametric 3D modeling software like Grasshopper. In addition data items can be decorated by attributes. Read more about the specification of sdTF and its principles on GitHub.

ShapeDiver makes use of sdTF for storing Grasshopper tree data as well as their attributes and transferring them via the Geometry Backend API. The ShapeDiver plugin for Grasshopper contains components for outputting tree data as sdTF and inputting tree data from sdTF. We offer SDKs that make it simple to use sdTF in your applications.

Applications

• Export and import Grasshopper tree data including attributes and efficiently transfer it over APIs.

• Plug ShapeDiver and Grasshopper into other applications, remotely.

• Chain up Grasshopper models running on ShapeDiver.

Features

sdTF has been designed to allow fast and individual access to the tree’s metadata and data items by means of HTTP Range requests. The structure of the tree and a description of the included data items This is fully supported in our TypeScript SDK.

sdTF is not limited to Grasshopper, it has been designed to support arbitrary trees of data.

Tree items can be decorated with attributes, which can be primitives as well as data items. This allows geometric objects to be decorated with other geometric objects.

SDKs

TypeScript SDK

We provide an SDK for JavaScript, TypeScript, node.js via npm. Please check out how to use it and the examples provided: https://www.npmjs.com/package/@shapediver/sdk.sdtf-v1.

The SDK runs in backend applications using node.js, and in modern browsers. It’s easy to use and integrates with https://github.com/mcneel/rhino3dm/blob/main/docs/javascript/RHINO3DM.JS.md.

.NET SDK

The sdTF SDK for .NET is undergoing some changes and is currently only available on request, it will be published on NuGet in Q4 2022.

Getting started

Example model:

All_Inputs_Outputs.ghx

Defining sdTF outputs

The “ShapeDiver Output” component outputs an sdTF file as well as a glTF file. The glTF is used to transfer display geometry corresponding to the data in the glTF.

This output component has a flexible number of inputs. Use as much as you like. The resulting sdTF file will contain one “chunk” of data for each component input. Only use floating parameter components to feed data into the output component, it will show an error otherwise.

Right-click the ShapeDiver Output component to save an sdTF or glTF file locally. This calls the exact same method that will be run on our servers.

You can load sdTF files using the Load sdTF item of the ShapeDiver menu in Grasshopper.

On the Geometry Backend API, the sdTF and the glTF show up in the content array of the output:

Read more about outputs here: Outputs on the API.

On the API the output also contains a property called chunks. This property contains information about the chunks of data (the data trees) that you can expect to find in any sdTF coming out of this output. Remember that there will be one chunk for each input of the ShapeDiver Output component.

Examples of what to do with the sdTF:

• Feed the href to the sdTF into the TypeScript SDKs readFromUrl function.

• Download the sdTF if you want to feed it into another Grasshopper model running on ShapeDiver (more on that below).

Defining sdTF inputs (parameters)

Use floating parameter components that do not have any inputs, and give them a name starting with the prefix SD_, or put them into a group whose name starts with SD_. In both cases, the prefix SD_ will not show up on the Geometry Backend API.

An example of a Curve and a Mesh parameter on the API:

Parameters that support sdTF can be identified by their type starting with a lowercase “s”, e.g. sCurve and sMesh. We call these parameters sdTF parameters. Read more about parameters here: Parameters (inputs)

In order to feed sdTF data into these inputs, do this:

1. Request an sdTF upload using the TypeScript SDK’s function sdtf.requestUpload (you can request several sdTF uploads at once). Corresponding API call: https://sdeuc1.eu-central-1.shapediver.com/api/v2/docs/#/sdTF/post_api_v2_session__sessionId__sdtf_upload.
   The response property asset.sdtf will contain both an upload URL and a unique id for the sdTF.

2. Upload the sdTF to the URL which you received. You can use the SDK’s function utils.upload to do this.

3. Use the id which you received for the sdTF as the value of the parameter when sending a computation (utils.submitAndWaitForCustomization) or export (utils.submitAndWaitForExport) request.

Uploaded sdTF files stay where they are, you can reuse them later by remembering their id.

Chunk selection logic

As mentioned above, sdTF files may contain multiple chunks (trees of data). The following logic is used to select a specific chunk for an sdTF parameter.

Simple case

This case applies if you are using the id of an uploaded sdTF file as the value of an sdTF parameter, as explained above.

1. In case the sdTF contains just a single chunk, use this chunk if the chunk data type is compatible with the input.

2. Look for a chunk whose directly assigned name, or whose attribute called name, matches the id of the sdTF parameter.

3. Look for a chunk whose directly assigned name, or whose attribute called name, matches the name of the sdTF parameter.

In case none of these steps succeeds, no data is set for the sdTF parameter.

Advanced case

You can override the chunk selection logic and explicitly define which chunk should be used for an sdTF parameter. If you want to do this, instead of setting the value of the sdTF parameter to the id of the uploaded sdTF, you need to set the value of the sdTF parameter to a JSON string. The JSON object follows this schema:

{
  "asset": {
    "id": "Unique_id_of_the_uploaded_sdTF_file",
    "chunk": {
      "id": "Id_of_the_chunk_which_should_be_used_or",
      "name": "Name_of_the_chunk_which_should_be_used"
    }
  }
}

The property asset.id needs to be specified in any case. This logic applies:

1. In case the sdTF contains just a single chunk, use this chunk if the chunk data type is compatible with the input.

2. If asset.chunk.id is defined, look for a chunk whose directly assigned name matches the given asset.chunk.id. Note: The chunk’s attributes are not considered in this case.

3. If asset.chunk.name is defined, look for a chunk whose attribute called name matches the given asset.chunk.name. Note: The chunk’s directly assigned name is not considered in this case.

4. If both asset.chunk.id and asset.chunk.name are not defined, fall back to the simple case, that is:

   1. Look for a chunk whose directly assigned name, or whose attribute called name, matches the id of the sdTF parameter.

   2. Look for a chunk whose directly assigned name, or whose attribute called name, matches the name of the sdTF parameter.

Key takeaways

• sdTF files need to be uploaded first, their id is used as parameter value.

• Each sdTF may contain multiple chunks. You can pack data for all your sdTF parameters into a single sdTF, or use multiple ones.

• Say you have previously uploaded a big sdTF containing chunks for multiple parameters. Now you have some new data for one of the parameters. You can create a new sdTF containing just the updated chunk, and reuse the id of the previously uploaded sdTF for the other parameters.