Install

Select a programming language:

bash

$pip install cartonml-nightly

Load and run a model

If you want to run an existing carton model (or "a carton" for short), you can just pass in a file path or URL.

import asyncio
import cartonml as carton
import numpy as np

async def main():
    # Note this might take a while the first time you use Carton.
    # Make sure to enable logging as described below
    model = await carton.load("https://carton.pub/google-research/bert-base-uncased")
    out = await model.infer({
        "input": np.array(["Today is a good [MASK]."])
    })
    print(out)
    # {
    #     'scores': array([[12.977381]]),
    #     'tokens': array([['day']], dtype='<U3')
    # }

asyncio.run(main())

See the "Loading a model" docs for more details.

Pack a model

To create a carton, you need to pack a model from a supported framework. The below example packs a TorchScript model.

import asyncio
import cartonml as carton

async def main():
    packed_model_path = await carton.pack(
        "/path/to/model.pt",
        runner_name="torchscript",
        # `required_framework_version` is a semver version range.
        # The below value means any 2.0.x version is okay.
        required_framework_version="=2.0"
    )

asyncio.run(main())

The packing procedure can be slightly different depending on the framework the original model is in. See the "Pack a model" docs for more details.

Load an unpacked model

Carton also supports loading an unpacked model via the load_unpacked method. This is conceptually the same as pack followed by load, but is implemented more efficiently internally. It supports all the options that load and pack support.

import asyncio
import cartonml as carton

async def main():
    model = await carton.load_unpacked(
        "/path/to/model.pt",
        runner_name="torchscript",
        # `required_framework_version` is a semver version range.
        # The below value means any 2.0.x version is okay.
        required_framework_version="=2.0"
    )

asyncio.run(main())

Fetch model metadata

It's possible to fetch model metadata without loading a model. This function only fetches the data needed to provide the requested metadata. Therefore it's quite efficient even with large models.

import asyncio
import cartonml as carton

async def main():
    info = await carton.get_model_info("https://carton.pub/cartonml/basic_example")

    print(info.model_name)
    # 'Test Model'

    print(info.short_description)
    # 'A short description that should be less than or equal to 100 characters.'

asyncio.run(main())

See the metadata docs for a list of available fields.

Logging

Carton routes all of its log messages to Python logging. Don't forget to configure logging before calling any functions in Carton:

import logging

async def main():
    # Configure logging format
    FORMAT = '[%(asctime)s %(levelname)s %(name)s] %(filename)s:%(lineno)d %(message)s'
    logging.basicConfig(format=FORMAT)

    # If you want trace messages to show up, set the log level to <= 5
    # logging.getLogger().setLevel(5)
    logging.getLogger().setLevel(logging.INFO)

Up next...

Create your own model

Browse existing ones