Skip to main content
Skip to main content
Edit this page

kafka_consumers

Querying in ClickHouse Cloud

The data in this system table is held locally on each node in ClickHouse Cloud. Obtaining a complete view of all data, therefore, requires the clusterAllReplicas function. See here for further details.

Contains information about Kafka consumers. Applicable for Kafka table engine (native ClickHouse integration)

Columns:

  • database (String) - database of the table with Kafka Engine.
  • table (String) - name of the table with Kafka Engine.
  • consumer_id (String) - Kafka consumer identifier. Note, that a table can have many consumers. Specified by kafka_num_consumers parameter.
  • assignments.topic (Array(String)) - Kafka topic.
  • assignments.partition_id (Array(Int32)) - Kafka partition id. Note, that only one consumer can be assigned to a partition.
  • assignments.current_offset (Array(Int64)) - current offset.
  • exceptions.time, (Array(DateTime)) - timestamp when the 10 most recent exceptions were generated.
  • exceptions.text, (Array(String)) - text of 10 most recent exceptions.
  • last_poll_time, (DateTime) - timestamp of the most recent poll.
  • num_messages_read, (UInt64) - number of messages read by the consumer.
  • last_commit_time, (DateTime) - timestamp of the most recent poll.
  • num_commits, (UInt64) - total number of commits for the consumer.
  • last_rebalance_time, (DateTime) - timestamp of the most recent Kafka rebalance
  • num_rebalance_revocations, (UInt64) - number of times the consumer was revoked its partitions
  • num_rebalance_assignments, (UInt64) - number of times the consumer was assigned to Kafka cluster
  • is_currently_used, (UInt8) - consumer is in use
  • last_used, (UInt64) - last time this consumer was in use, unix time in microseconds
  • rdkafka_stat (String) - library internal statistic. See https://github.com/ClickHouse/librdkafka/blob/master/STATISTICS.md . Set statistics_interval_ms to 0 disable, default is 3000 (once in three seconds).

Example: