Package com.snowflake.ingest.streaming

Interface SnowflakeStreamingIngestClient

All Superinterfaces:
AutoCloseable
public interface SnowflakeStreamingIngestClient extends AutoCloseable
A class that is the starting point for using the Streaming Ingest client APIs, a single client maps to exactly one account/database/schema/pipe in Snowflake; however, multiple clients can point to the same account/database/schema/pipe. Each client will contain information for Snowflake authentication and authorization, and it will be used to create one or more `SnowflakeStreamingIngestChannel`

  • Method Details

    • close

      void close()
      Close the client, which will flush first and then release all the resources

      TODO (SNOW-1916563): decide whether we want to match the V1 behavior

      Specified by:
      close in interface AutoCloseable

    • close

      CompletableFuture<Void> close(boolean waitForFlush, @Nonnull Duration timeoutDuration)
      Close the client. If waitForFlush=true this function will make sure all the data in all channels managed by this client is flushed to the Snowflake server side or return a timeout error if the timeout is reached.
      Parameters:
      waitForFlush - whether to wait for all channels to be flushed
      timeoutDuration - timeout duration

    • initiateFlush

      void initiateFlush()
      Initiates a flush by the Client which causes all outstanding buffered data to be flushed to Snowflake. Note that data can still be accepted by the Client - this is an asynchronous call and will return after flush is initiated for all Channels opened by this Client

    • openChannel

      OpenChannelResult openChannel(String channelName)
      Open a channel with a given channel name. The channel is opened on the account/database/schema/pipe defined by the current client. The channel will reuse the latest persisted offset token if the channel is reopened.
      Parameters:
      channelName - the open channel request
      Returns:
      a OpenChannelResult object containing the channel and the channel status

    • openChannel

      OpenChannelResult openChannel(String channelName, String offsetToken)
      Open a channel with a given channel name. The channel is opened on the account/database/schema/pipe defined by the current client
      Parameters:
      channelName - the open channel request
      offsetToken - the optional offset token to set on the channel to be opened. If not provided and this reopens an existing channel, the latest persisted offset token will be reused.
      Returns:
      a OpenChannelResult object containing the channel and the channel status

    • dropChannel

      void dropChannel(String channelName)
      Drop the specified channel on the server using the channel name

      Note that this call will blindly drop the latest version of the channel and any pending data will be lost. It will also delete Offset Token and other state from Snowflake servers. So only use it if you are completely done ingesting data for this channel. If you open a channel with the same name in the future, it will behave like a new channel.

      Also see SnowflakeStreamingIngestChannel.close() to close channels. That approach will drop the local version of the channel and if the channel has been concurrently reopened by another client, that version of the channel won't be affected.

      Parameters:
      channelName - the channel name to drop

    • getLatestCommittedOffsetTokens

      Map<String,String> getLatestCommittedOffsetTokens(List<String> channelNames)
      Return the latest committed/persisted offset token for the given list of channels in Snowflake
      Parameters:
      channelNames - the list of channel names
      Returns:
      the latest committed offset token for the channel

    • getChannelStatus

      ChannelStatusBatch getChannelStatus(List<String> channelNames)
      Return the channel status for the given list of channels from Snowflake
      Parameters:
      channelNames - the list of channel names
      Returns:
      the channel status for the channel

    • isClosed

      boolean isClosed()
      Check if the client is closed
      Returns:
      a boolean which indicates whether the client is closed

    • waitForFlush

      CompletableFuture<Void> waitForFlush(Duration timeoutDuration)
      Asynchronously waits for all buffered data in all channels managed by this client to be flushed to the Snowflake server side. This method triggers a flush of all pending data across all channels and waits for the flush operations to complete.

      Behavior:

      • If timeoutDuration is null, the operation will wait indefinitely
      • The method coordinates flush operations across all channels managed by this client
      • If the client is already closed when this method is called, an exception is thrown immediately
      Parameters:
      timeoutDuration - The maximum time to wait for all channels to flush. If null, the operation will wait indefinitely.
      Returns:
      A CompletableFuture<Void> that completes successfully when all channels flush successfully within the timeout period. The future completes exceptionally with:
      • TimeoutException if the timeout was reached before all channels could flush
      • SFException with error code "ClientClosed" if the client is closed
      • Other SFException types if flush operations encounter errors
      Throws:
      IllegalArgumentException - if timeoutDuration is negative
      SFException - with error code "ClientClosed" if the client is already closed when called

    • getDBName

      String getDBName()
      Get the database name
      Returns:
      name of the database

    • getSchemaName

      String getSchemaName()
      Get the schema name
      Returns:
      name of the schema

    • getPipeName

      String getPipeName()
      Get the pipe name
      Returns:
      name of the pipe

    • getClientName

      String getClientName()
      Get the client name
      Returns:
      the client name