Use Case 1: Provable Data Availability

Synopsis

Use Textile’s Basin to prove data availability in both the immediate term and in the long-term.

The Challenge

For many ML/AI and compute networks, there is a lot of data that comes from compute jobs in the form of proofs that need to be globally available for a short period of time (hours or days). Proof data may include model inference proofs, model checkpoint data (params and hyperparams), etc. This data needs to be available for a short period of time following job completion to handle disputes, and then longer-term to allow anyone on the network to verify or resolve a dispute by downloading the complete proof data.

Our Solution

Basin provides object storage with verifiable data pipelines and decentralized architecture, allowing compute networks to guarantee data is available for compute, dispute resolution and more:

• Generate data streams that include timestamps included in a final proof.
• Create globally unique and addressable job streams.
• Enable read-availability for hours or days, then archive data to decentralized storage for the long-term (e.g. Filecoin)
• In the future, “combine” or otherwise reference smaller proofs of partial computation with others to generate an overall “job” or training proof.

Benefits

• A decentralized solution with inventive mechanisms to ensure data is available according to your needs
• Data provenance and metadata to enable time based querying for dispute resolution

Use Case 2: Shared Model Data

Synopsis

Textile Vaults enable various data sources to combine into an open, verifiable and collaborative data environment.

The Challenge

Users often submit training jobs that require large, open data such as WikiText and Flickr data, and smaller ones such as mnist. Or, they include their own data that needs to be made available to compute nodes. Ideally, all this data can be made available over a common protocol to facilitate collaboration and data liquidity.