Skip to main content

Handlers

Generator Handler​

Snowcell provides a powerful streaming capability that allows users to receive live updates from serverless functions, which is especially useful for long-running tasks like generating content from AI models or batch data processing. This real-time functionality ensures that intermediate results can be streamed to clients as they are produced. Snowcell supports two types of streaming: synchronous generator functions and asynchronous generator functions.

Example: Synchronous Generator Handler​

The following example demonstrates how to implement a synchronous generator handler. This handler sends updates at each stage of processing and streams the output in real time:

import snowcell

def generator_handler(job):
    task_data = job["input_data"]
    for part in range(5):
        # Perform a step-by-step operation, streaming results as they become available.
        yield f"Processing part {part} of task: {task_data}"

snowcell.serverless.start(
    {
        "handler": generator_handler,  # Required: Defines the generator function to be executed.
        "return_aggregate_stream": True  # Optional: Enables outputs to be available at multiple endpoints.
    }
)

In this example, the handler processes the job input in stages, sending incremental updates as each part of the task is processed. The function yields a message after each step, allowing users to track progress in real time.

By default, partial results from a generator function are streamed through the /stream endpoint as they are generated. If you also want to make these results available through the /run or /runsync endpoints after the job completes, you can enable the return_aggregate_stream option. When set to True, all intermediate outputs are collected and returned as an aggregated result once the job finishes.

Asynchronous Handler​

Snowcell supports asynchronous handlers, which are ideal for handling tasks that require non-blocking operations. These handlers allow you to efficiently manage I/O-bound processes like API requests, large file processing, or long-running computations without locking up the execution environment.

Writing Asynchronous Handlers​

Asynchronous handlers are written using Python's async and await syntax, providing the ability to handle tasks concurrently and improve performance. The following example shows how you can implement an asynchronous handler to process jobs and return multiple outputs progressively, making it suitable for tasks such as processing real-time data or incrementally delivering results from complex models.

Example: Asynchronous Handler​

import snowcell
import asyncio

async def async_handler(job):
    input_data = job["input_data"]

    # Simulate asynchronous processing
    for step in range(5):
        await asyncio.sleep(2)  # Simulate an async task like API interaction or long computation
        yield f"Async result for step {step} of job: {input_data}"

# Configure and start the serverless function
snowcell_sdk.serverless.start(
    {
        "handler": async_handler,  # Required: Async function that handles the job
        "return_aggregate_stream": True,  # Optional: Enable aggregated results for regular endpoints
    }
)

In this example, the asynchronous handler processes each step of the job independently, yielding results over time. This is ideal for scenarios where tasks can be split into smaller, asynchronous steps (e.g., querying external APIs, streaming data, or long-running computations).

Advantages of Asynchronous Handlers​

-Increased Efficiency: Async handlers allow for non-blocking operations, meaning multiple tasks can run concurrently without waiting for others to finish. This boosts overall efficiency, especially when dealing with I/O-heavy operations like network requests or file reads.

-Scalability: Asynchronous handlers are highly scalable, making them perfect for applications with high-frequency requests or large volumes of data. By not blocking on each operation, the system can handle more tasks in parallel.

-Flexibility: With the ability to yield results progressively, async handlers are a great fit for long-running jobs or streaming scenarios where partial results are needed before the entire task completes.

Best Practices for Writing Asynchronous Handlers​

  • Use Async and Await Correctly: Ensure that long-running or I/O-bound operations (e.g., API calls, file reads) are properly handled using await. This prevents blocking and keeps the function responsive.
  • Utilize yield for Streaming: For tasks that produce multiple outputs over time, take advantage of yield to stream partial results. This keeps users updated with progress, especially in real-time applications.
  • Test for Async Scenarios: Asynchronous code introduces unique challenges like race conditions or handling exceptions in await operations. Thoroughly test your async handlers to cover potential edge cases and errors.

Using asynchronous handlers in Snowcell not only improves performance but also enhances responsiveness for tasks requiring real-time processing or interaction with multiple external systems. By structuring your function in this way, you can efficiently scale to handle many concurrent jobs without sacrificing performance.