Execution timeout when loading csv file to ClickHouseContainer

Question

When attempting to upload a CSV file (~100 lines and < 100KB) to ClickHouseContainer running in junit, the client is timing out with the following error:

An exception or error caused a run to abort: Code: 159. Execution timed out, server ClickHouseNode [uri=any://localhost:51733/default]@675749754

I have the following code to initialize a ClickHouseContainer

val image: ClickHouseContainer =
  new ClickHouseContainer("clickhouse/clickhouse-server:latest")
    .withUrlParam("compress_algorithm", "zstd")
    .withUsername("default")
    .withPassword("")
    .withExposedPorts(8123, 8443, 9000, 9440)
    .waitingFor(
      Wait
        .forHttp("/ping")
        .forPort(8123)
        .forStatusCode(200)
        .withStartupTimeout(Duration.ofSeconds(600)));
image.start()

Once the container starts, I am using ClickHouseClient to load the table from a CSV file.

val server = ClickHouseNode.builder
  .host(image.getHost)
  .port(image.getMappedPort(ClickHouseProtocol.HTTP.getDefaultPort))
  .database("default")
  .credentials(ClickHouseCredentials.fromUserAndPassword("default", ""))
  .build

// Create a CSV file reference
val file = ClickHouseFile.of(csvFile, ClickHouseCompression.NONE, ClickHouseFormat.CSV)

// Create a client
val client = ClickHouseClient.newInstance(server.getProtocol)

// Specify settings, load data to the specified table and wait for completion
client
  .write(server)
  .set("input_format_csv_skip_first_lines", "0")
  .set("format_csv_delimiter", ",")
  .table(tableName)
  .data(file)
  .executeAndWait()

I even tried increasing the socket timeout on the client, but this did not help and instead I was getting connect reset by peer.

val client = ClickHouseClient
  .builder()
  .option(ClickHouseClientOption.SOCKET_TIMEOUT, 10000)
  .option(ClickHouseClientOption.MAX_THREADS_PER_CLIENT, 100)
  .option(ClickHouseClientOption.MAX_EXECUTION_TIME, 1000)
  .build()

The CSV file is too small, hence I don't expect that inserting the rows to clickhouse should take so long.

Jianfei Hu · Accepted Answer · 2024-01-17 01:39:02Z

0

I can't tell the cause by the information above. But a few things might help with troubleshooting

Inspect system.query_log and find failed queries associated exception, https://clickhouse.com/docs/en/operations/system-tables/query_log. You can filter by timestamp/query string.
Increase logger level and inspect logs. You should at least see your queries printed out in the log and error message if insertions fails.
If you don't see your queries anywhere, consider to rule out container and application environment network, run ClickHouse directly and access via localhost:8123.

answered Jan 17, 2024 at 1:39

Jianfei Hu

5044 silver badges11 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Execution timeout when loading csv file to ClickHouseContainer

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related