Sentry 后端云原生中間件實(shí)踐 ClickHouse PaaS ，為 Snuba 事件分析引擎提供動力

2023-02-13 12:07:38來源：黑客下午茶

ClickHouse PaaS 云原生多租戶平臺(Altinity.Cloud)

官網(wǎng)：https://altinity.cloud

PaaS 架構(gòu)概覽

(資料圖)

設(shè)計(jì)一個擁有云原生編排能力、支持多云環(huán)境部署、自動化運(yùn)維、彈性擴(kuò)縮容、故障自愈等特性，同時提供租戶隔離、權(quán)限管理、操作審計(jì)等企業(yè)級能力的高性能、低成本的分布式中間件服務(wù)是真挺難的。

SaaS 模式交付給用戶

Sentry Snuba 事件大數(shù)據(jù)分析引擎架構(gòu)概覽

Snuba 是一個在 Clickhouse 基礎(chǔ)上提供豐富數(shù)據(jù)模型、快速攝取消費(fèi)者和查詢優(yōu)化器的服務(wù)。以搜索和提供關(guān)于 Sentry 事件數(shù)據(jù)的聚合引擎。

數(shù)據(jù)完全存儲在 Clickhouse 表和物化視圖中，它通過輸入流(目前只有 Kafka 主題)攝入，可以通過時間點(diǎn)查詢或流查詢(訂閱)進(jìn)行查詢。

文檔：https://getsentry.github.io/snuba/architecture/overview.html

Kubernetes ClickHouse Operator什么是 Kubernetes Operator？

Kubernetes Operator 是一種封裝、部署和管理 Kubernetes 應(yīng)用的方法。我們使用 Kubernetes API（應(yīng)用編程接口）和 kubectl 工具在 Kubernetes 上部署并管理 Kubernetes 應(yīng)用。

??https://kubernetes.io/zh-cn/docs/concepts/extend-kubernetes/operator/??Altinity Operator for ClickHouse

Altinity：ClickHouse Operator 業(yè)界領(lǐng)先開源提供商。

Altinity：https://altinity.com/GitHub：https://github.com/Altinity/clickhouse-operatorYoutube：https://www.youtube.com/@Altinity

當(dāng)然這種多租戶隔離的 ClickHouse 中間件 PaaS 云平臺，公司或云廠商幾乎是不開源的。

RadonDB ClickHouse??https://github.com/radondb/radondb-clickhouse-operator????https://github.com/radondb/radondb-clickhouse-kubernetes??

云廠商(青云)基于 altinity-clickhouse-operator 定制的。對于快速部署生產(chǎn)集群做了些優(yōu)化。

Helm + Operator 快速上云 ClickHouse 集群云原生實(shí)驗(yàn)環(huán)境VKE K8S Cluster，Vultr托管集群(v1.23.14)Kubesphere v3.3.1 集群可視化管理，全棧的 Kubernetes 容器云 PaaS 解決方案。Longhorn 1.14，Kubernetes 的云原生分布式塊存儲。部署 clickhouse-operator

這里我們使用 RadonDB 定制的 Operator。

values.operator.yaml定制如下兩個參數(shù)：

# operator 監(jiān)控集群所有 namespace 的 clickhouse 部署watchAllNamespaces: true# 啟用 operator 指標(biāo)監(jiān)控enablePrometheusMonitor: true

helm 部署 operator：

cd vip-k8s-paas/10-cloud-native-clickhouse# 部署在 kube-systemhelm install clickhouse-operator ./clickhouse-operator -f values.operator.yaml -n kube-systemkubectl -n kube-system get po | grep clickhouse-operator# clickhouse-operator-6457c6dcdd-szgpd       1/1     Running   0          3m33skubectl -n kube-system get svc | grep clickhouse-operator# clickhouse-operator-metrics   ClusterIP      10.110.129.244     8888/TCP    4m18skubectl api-resources | grep clickhouse# clickhouseinstallations            chi          clickhouse.radondb.com/v1              true         ClickHouseInstallation# clickhouseinstallationtemplates    chit         clickhouse.radondb.com/v1              true         ClickHouseInstallationTemplate# clickhouseoperatorconfigurations   chopconf     clickhouse.radondb.com/v1              true         ClickHouseOperatorConfiguration

部署 clickhouse-cluster

這里我們使用 RadonDB 定制的 clickhouse-cluster helm charts?？焖俨渴?2 shards + 2 replicas + 3 zk nodes 的集群。

values.cluster.yaml定制：

clickhouse:    clusterName: snuba-clickhouse-nodes    shardscount: 2    replicascount: 2...zookeeper:  install: true  replicas: 3

helm 部署 clickhouse-cluster：

kubectl create ns cloud-clickhousehelm install clickhouse ./clickhouse-cluster -f values.cluster.yaml -n cloud-clickhousekubectl get po -n cloud-clickhouse# chi-clickhouse-snuba-ck-nodes-0-0-0   3/3     Running   5 (6m13s ago)   16m# chi-clickhouse-snuba-ck-nodes-0-1-0   3/3     Running   1 (5m33s ago)   6m23s# chi-clickhouse-snuba-ck-nodes-1-0-0   3/3     Running   1 (4m58s ago)   5m44s# chi-clickhouse-snuba-ck-nodes-1-1-0   3/3     Running   1 (4m28s ago)   5m10s# zk-clickhouse-0                       1/1     Running   0               17m# zk-clickhouse-1                       1/1     Running   0               17m# zk-clickhouse-2                       1/1     Running   0               17m

借助 Operator 快速擴(kuò)展 clickhouse 分片集群使用如下命令，將shardsCount改為3：

kubectl edit chi/clickhouse -n cloud-clickhouse

查看 pods：

kubectl get po -n cloud-clickhouse# NAME                                  READY   STATUS    RESTARTS       AGE# chi-clickhouse-snuba-ck-nodes-0-0-0   3/3     Running   5 (24m ago)    34m# chi-clickhouse-snuba-ck-nodes-0-1-0   3/3     Running   1 (23m ago)    24m# chi-clickhouse-snuba-ck-nodes-1-0-0   3/3     Running   1 (22m ago)    23m# chi-clickhouse-snuba-ck-nodes-1-1-0   3/3     Running   1 (22m ago)    23m# chi-clickhouse-snuba-ck-nodes-2-0-0   3/3     Running   1 (108s ago)   2m33s# chi-clickhouse-snuba-ck-nodes-2-1-0   3/3     Running   1 (72s ago)    119s# zk-clickhouse-0                       1/1     Running   0              35m# zk-clickhouse-1                       1/1     Running   0              35m# zk-clickhouse-2                       1/1     Running   0              35m

發(fā)現(xiàn)多出chi-clickhouse-snuba-ck-nodes-2-0-0與chi-clickhouse-snuba-ck-nodes-2-1-0。分片與副本已自動由Operator新建。

小試牛刀ReplicatedMergeTree+Distributed+Zookeeper 構(gòu)建多分片多副本集群連接 clickhouse

我們進(jìn)入 Pod, 使用原生命令行客戶端clickhouse-client連接。

kubectl exec -it chi-clickhouse-snuba-ck-nodes-0-0-0 -n cloud-clickhouse -- bashkubectl exec -it chi-clickhouse-snuba-ck-nodes-0-1-0 -n cloud-clickhouse -- bashkubectl exec -it chi-clickhouse-snuba-ck-nodes-1-0-0 -n cloud-clickhouse -- bashkubectl exec -it chi-clickhouse-snuba-ck-nodes-1-1-0 -n cloud-clickhouse -- bashkubectl exec -it chi-clickhouse-snuba-ck-nodes-2-0-0 -n cloud-clickhouse -- bashkubectl exec -it chi-clickhouse-snuba-ck-nodes-2-1-0 -n cloud-clickhouse -- bash

我們直接通過終端分別進(jìn)入這 6 個 pod。然后進(jìn)行測試：

clickhouse-client --multiline -u username -h ip --password passowrd# clickhouse-client -m

創(chuàng)建分布式數(shù)據(jù)庫查看system.clusters

select * from system.clusters;

2.創(chuàng)建名為test的數(shù)據(jù)庫

create database test on cluster "snuba-ck-nodes";# 刪除：drop database test on cluster "snuba-ck-nodes";

在各個節(jié)點(diǎn)查看，都已存在test數(shù)據(jù)庫。

show databases;

創(chuàng)建本地表(ReplicatedMergeTree)建表語句如下：

在集群中各個節(jié)點(diǎn)test數(shù)據(jù)庫中創(chuàng)建t_local本地表，采用ReplicatedMergeTree表引擎，接受兩個參數(shù)：

zoo_path— zookeeper 中表的路徑，針對表同一個分片的不同副本，定義相同路徑。

"/clickhouse/tables/{shard}/test/t_local"

replica_name— zookeeper 中表的副本名稱

CREATE TABLE test.t_local on cluster "snuba-ck-nodes"(    EventDate DateTime,    CounterID UInt32,    UserID UInt32)ENGINE = ReplicatedMergeTree("/clickhouse/tables/{shard}/test/t_local", "{replica}")PARTITION BY toYYYYMM(EventDate)ORDER BY (CounterID, EventDate, intHash32(UserID))SAMPLE BY intHash32(UserID);

宏(macros)占位符：

建表語句參數(shù)包含的宏替換占位符(如：{replica})。會被替換為配置文件里 macros 部分的值。

查看集群中 clickhouse 分片&副本節(jié)點(diǎn)configmap：

kubectl get configmap -n cloud-clickhouse | grep clickhouseNAME                                             DATA   AGEchi-clickhouse-common-configd                    6      20hchi-clickhouse-common-usersd                     6      20hchi-clickhouse-deploy-confd-snuba-ck-nodes-0-0   2      20hchi-clickhouse-deploy-confd-snuba-ck-nodes-0-1   2      20hchi-clickhouse-deploy-confd-snuba-ck-nodes-1-0   2      20hchi-clickhouse-deploy-confd-snuba-ck-nodes-1-1   2      20hchi-clickhouse-deploy-confd-snuba-ck-nodes-2-0   2      19hchi-clickhouse-deploy-confd-snuba-ck-nodes-2-1   2      19h

查看節(jié)點(diǎn)配置值：

kubectl describe configmap chi-clickhouse-deploy-confd-snuba-ck-nodes-0-0  -n cloud-clickhouse

創(chuàng)建對應(yīng)的分布式表(Distributed)

CREATE TABLE test.t_dist on cluster "snuba-ck-nodes"(    EventDate DateTime,    CounterID UInt32,    UserID UInt32)ENGINE = Distributed("snuba-ck-nodes", test, t_local, rand());# drop table test.t_dist on cluster "snuba-ck-nodes";

這里，Distributed 引擎的所用的四個參數(shù)：

cluster - 服務(wù)為配置中的集群名(snuba-ck-nodes)database - 遠(yuǎn)程數(shù)據(jù)庫名(test)table - 遠(yuǎn)程數(shù)據(jù)表名(t_local)sharding_key - (可選) 分片key(CounterID/rand())

查看相關(guān)表，如：

use test;show tables;# t_dist# t_local

通過分布式表插入幾條數(shù)據(jù)：

# 插入INSERT INTO test.t_dist VALUES ("2022-12-16 00:00:00", 1, 1),("2023-01-01 00:00:00",2, 2),("2023-02-01 00:00:00",3, 3);

任一節(jié)點(diǎn)查詢數(shù)據(jù)：

select * from test.t_dist;

實(shí)戰(zhàn)，為 Snuba 引擎提供 ClickHouse PaaS拆解與分析 Sentry Helm Charts

在我們遷移到 Kubernetes Operator 之前，我們先拆解與分析下 sentry-charts 中自帶的 clickhouse & zookeeper charts。

非官方 Sentry Helm Charts：

??https://github.com/sentry-kubernetes/charts??

他的Chart.yaml如下：

apiVersion: v2appVersion: 22.11.0dependencies:- condition: sourcemaps.enabled  name: memcached  repository: https://charts.bitnami.com/bitnami  version: 6.1.5- condition: redis.enabled  name: redis  repository: https://charts.bitnami.com/bitnami  version: 16.12.1- condition: kafka.enabled  name: kafka  repository: https://charts.bitnami.com/bitnami  version: 16.3.2- condition: clickhouse.enabled  name: clickhouse  repository: https://sentry-kubernetes.github.io/charts  version: 3.2.0- condition: zookeeper.enabled  name: zookeeper  repository: https://charts.bitnami.com/bitnami  version: 9.0.0- alias: rabbitmq  condition: rabbitmq.enabled  name: rabbitmq  repository: https://charts.bitnami.com/bitnami  version: 8.32.2- condition: postgresql.enabled  name: postgresql  repository: https://charts.bitnami.com/bitnami  version: 10.16.2- condition: nginx.enabled  name: nginx  repository: https://charts.bitnami.com/bitnami  version: 12.0.4description: A Helm chart for Kubernetesmaintainers:- name: sentry-kubernetesname: sentrytype: applicationversion: 17.9.0

這個 sentry-charts 將所有中間件 helm charts 耦合依賴在一起部署，不適合 sentry 微服務(wù) & 中間件集群擴(kuò)展。更高級的做法是每個中間件擁有定制的 Kubernetes Operator(如：clickhouse-operator) & 獨(dú)立的 K8S 集群，形成中間件 PaaS 平臺對外提供服務(wù)。

這里我們拆分中間件 charts 到獨(dú)立的 namespace 或單獨(dú)的集群運(yùn)維。設(shè)計(jì)為：

ZooKeeper 命名空間：cloud-zookeeper-paasClickHouse 命名空間：cloud-clickhouse-paas獨(dú)立部署 ZooKeeper Helm Chart

這里 zookeeper chart 采用的是 bitnami/zookeeper，他的倉庫地址如下：

??https://github.com/bitnami/charts/tree/master/bitnami/zookeeper????https://github.com/bitnami/containers/tree/main/bitnami/zookeeper??ZooKeeper Operator 會在后續(xù)文章專項(xiàng)討論。創(chuàng)建命名空間：

kubectl create ns cloud-zookeeper-paas

簡單定制下values.yaml：

# 暴露下 prometheus 監(jiān)控所需的服務(wù)metrics:  containerPort: 9141  enabled: true........service:  annotations: {}  clusterIP: ""  disableBaseClientPort: false  externalTrafficPolicy: Cluster  extraPorts: []  headless:    annotations: {}    publishNotReadyAddresses: true  loadBalancerIP: ""  loadBalancerSourceRanges: []  nodePorts:    client: ""    tls: ""  ports:    client: 2181    election: 3888    follower: 2888    tls: 3181  sessionAffinity: None  type: ClusterIP

注意：在使用支持外部負(fù)載均衡器的云提供商的服務(wù)時，需設(shè)置 Sevice 的 type 的值為 "LoadBalancer"，將為 Service 提供負(fù)載均衡器。來自外部負(fù)載均衡器的流量將直接重定向到后端 Pod 上，不過實(shí)際它們是如何工作的，這要依賴于云提供商。

helm 部署：

helm install zookeeper ./zookeeper -f values.yaml -n cloud-zookeeper-paas

集群內(nèi)，可使用zookeeper.cloud-zookeeper-paas.svc.cluster.local:2181對外提供服務(wù)。

zkCli 連接 ZooKeeper：

export POD_NAME=$(kubectl get pods --namespace cloud-zookeeper-paas -l "app.kubernetes.io/name=zookeeper,app.kubernetes.io/instance=zookeeper,app.kubernetes.io/compnotallow=zookeeper" -o jsnotallow="{.items[0].metadata.name}")kubectl -n cloud-zookeeper-paas exec -it $POD_NAME -- zkCli.sh# test[zk: localhost:2181(CONNECTED) 0] ls /[zookeeper][zk: localhost:2181(CONNECTED) 1] ls /zookeeper[config, quota][zk: localhost:2181(CONNECTED) 2] quit# 外部訪問# kubectl port-forward --namespace cloud-zookeeper-paas svc/zookeeper 2181: & zkCli.sh 127.0.0.1:2181

查看zoo.cfg

kubectl -n cloud-zookeeper-paas exec -it $POD_NAME -- cat /opt/bitnami/zookeeper/conf/zoo.cfg

# The number of milliseconds of each ticktickTime=2000# The number of ticks that the initial# synchronization phase can takeinitLimit=10# The number of ticks that can pass between# sending a request and getting an acknowledgementsyncLimit=5# the directory where the snapshot is stored.# do not use /tmp for storage, /tmp here is just# example sakes.dataDir=/bitnami/zookeeper/data# the port at which the clients will connectclientPort=2181# the maximum number of client connections.# increase this if you need to handle more clientsmaxClientCnxns=60## Be sure to read the maintenance section of the# administrator guide before turning on autopurge.## https://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance## The number of snapshots to retain in dataDirautopurge.snapRetainCount=3# Purge task interval in hours# Set to "0" to disable auto purge featureautopurge.purgeInterval=0## Metrics Providers## https://prometheus.io Metrics ExportermetricsProvider.className=org.apache.zookeeper.metrics.prometheus.PrometheusMetricsProvider#metricsProvider.httpHost=0.0.0.0metricsProvider.httpPort=9141metricsProvider.exportJvmInfo=truepreAllocSize=65536snapCount=100000maxCnxns=0recnotallow=falsequorumListenOnAllIPs=false4lw.commands.whitelist=srvr, mntr, ruokmaxSessinotallow=40000admin.serverPort=8080admin.enableServer=trueserver.1=zookeeper-0.zookeeper-headless.cloud-zookeeper-paas.svc.cluster.local:2888:3888;2181server.2=zookeeper-1.zookeeper-headless.cloud-zookeeper-paas.svc.cluster.local:2888:3888;2181server.3=zookeeper-2.zookeeper-headless.cloud-zookeeper-paas.svc.cluster.local:2888:3888;2181

獨(dú)立部署 ClickHouse Helm Chart

這里 clickhouse chart 采用的是 sentry-kubernetes/charts 自己維護(hù)的一個版本：

sentry snuba 目前對于 clickhouse 21.x 等以上版本支持的并不友好，這里的鏡像版本是yandex/clickhouse-server:20.8.19.4。??https://github.com/sentry-kubernetes/charts/tree/develop/clickhouse??ClickHouse Operator + ClickHouse Keeper 會在后續(xù)文章專項(xiàng)討論。

這個自帶的 clickhouse-charts 存在些問題，Service 部分需簡單修改下允許配置 "type:LoadBalancer" or "type:NodePort"。

創(chuàng)建命名空間：

kubectl create ns cloud-clickhouse-paas

簡單定制下values.yaml：

注意上面zoo.cfg的 3 個 zookeeper 實(shí)例的地址：

server.1=zookeeper-0.zookeeper-headless.cloud-zookeeper-paas.svc.cluster.local:2888:3888;2181server.2=zookeeper-1.zookeeper-headless.cloud-zookeeper-paas.svc.cluster.local:2888:3888;2181server.3=zookeeper-2.zookeeper-headless.cloud-zookeeper-paas.svc.cluster.local:2888:3888;2181

# 修改 zookeeper_serversclickhouse:  configmap:    zookeeper_servers:      config:      - hostTemplate: "zookeeper-0.zookeeper-headless.cloud-zookeeper-paas.svc.cluster.local"        index: clickhouse        port: "2181"      - hostTemplate: "zookeeper-1.zookeeper-headless.cloud-zookeeper-paas.svc.cluster.local"        index: clickhouse        port: "2181"      - hostTemplate: "zookeeper-2.zookeeper-headless.cloud-zookeeper-paas.svc.cluster.local"        index: clickhouse        port: "2181"      enabled: true      operation_timeout_ms: "10000"      session_timeout_ms: "30000"# 暴露下 prometheus 監(jiān)控所需的服務(wù)metrics:  enabled: true

當(dāng)然這里也可以不用 Headless Service，因?yàn)槭峭粋€集群的不同 namespace 的內(nèi)部訪問，所以也可簡單填入 ClusterIP 類型 Sevice：

# 修改 zookeeper_serversclickhouse:  configmap:    zookeeper_servers:      config:      - hostTemplate: "zookeeper.cloud-zookeeper-paas.svc.cluster.local"        index: clickhouse        port: "2181"      enabled: true      operation_timeout_ms: "10000"      session_timeout_ms: "30000"# 暴露下 prometheus 監(jiān)控所需的服務(wù)metrics:  enabled: true

helm 部署：

helm install clickhouse ./clickhouse -f values.yaml -n cloud-clickhouse-paas

連接 clickhouse

kubectl -n cloud-clickhouse-paas exec -it clickhouse-0 -- clickhouse-client --multiline --host="clickhouse-1.clickhouse-headless.cloud-clickhouse-paas"

驗(yàn)證集群

show databases;select * from system.clusters;select * from system.zookeeper where path = "/clickhouse";

當(dāng)前 ClickHouse 集群的 ConfigMap

kubectl get configmap -n cloud-clickhouse-paas | grep clickhouse

clickhouse-config    1      28hclickhouse-metrica   1      28hclickhouse-users     1      28h

clickhouse-config(config.xml)

    /var/lib/clickhouse/    /var/lib/clickhouse/tmp/    /var/lib/clickhouse/user_files/    /var/lib/clickhouse/format_schemas/    /etc/clickhouse-server/metrica.d/metrica.xml    users.xml    clickhouse    0.0.0.0    8123    9000    9009    4096    3    100    8589934592    5368709120    UTC    022    false                3600    3600    60    1            system        query_log
        toYYYYMM(event_date)        7500                system        query_thread_log
        toYYYYMM(event_date)        7500                /clickhouse/task_queue/ddl                trace        /var/log/clickhouse-server/clickhouse-server.log        /var/log/clickhouse-server/clickhouse-server.err.log        1000M        10

clickhouse-metrica(metrica.xml)

                        zookeeper-0.zookeeper-headless.cloud-zookeeper-paas.svc.cluster.local            2181                            zookeeper-1.zookeeper-headless.cloud-zookeeper-paas.svc.cluster.local            2181                            zookeeper-2.zookeeper-headless.cloud-zookeeper-paas.svc.cluster.local            2181                30000        10000                                                                                true                    clickhouse-0.clickhouse-headless.cloud-clickhouse-paas.svc.cluster.local                    9000                    default                    true                                                                            true                    clickhouse-1.clickhouse-headless.cloud-clickhouse-paas.svc.cluster.local                    9000                    default                    true                                                                            true                    clickhouse-2.clickhouse-headless.cloud-clickhouse-paas.svc.cluster.local                    9000                    default                    true

clickhouse-users(users.xml)

Sentry Helm Charts 定制接入 ClickHouse PaaS, 單集群多節(jié)點(diǎn)

我們簡單修改values.yml

禁用 sentry-charts 中的 clickHouse & zookeeper

clickhouse:  enabled: falsezookeeper:  enabled: false

修改externalClickhouse

externalClickhouse:  database: default  host: "clickhouse.cloud-clickhouse-paas.svc.cluster.local"  httpPort: 8123  password: ""  singleNode: false  clusterName: "clickhouse"  tcpPort: 9000  username: default

注意：

這里只是簡單的集群內(nèi)部接入1個多節(jié)點(diǎn)分片集群，而 Snuba 系統(tǒng)的設(shè)計(jì)是允許你接入多個 ClickHouse多節(jié)點(diǎn)多分片多副本集群，將多個 Schema 分散到不同的集群，從而實(shí)現(xiàn)超大規(guī)模吞吐。因?yàn)槭峭粋€集群的不同 namespace 的內(nèi)部訪問，所以這里簡單填入類型為 ClusterIP Sevice 即可。注意這里singleNode要設(shè)置成false。因?yàn)槲覀兪嵌喙?jié)點(diǎn)，同時我們需要提供clusterName：源碼分析：這將用于確定：以及確定來使用不同的 ClickHouse Table Engines 等。當(dāng)然，ClickHouse 本身是一個單獨(dú)的技術(shù)方向，這里就不展開討論了。

將運(yùn)行哪些遷移（僅本地或本地和分布式表）

查詢中的差異 - 例如是否選擇了 _local 或 _dist 表

部署

helm install sentry ./sentry -f values.yaml -n sentry

驗(yàn)證 _local 與 _dist 表以及 system.zookeeper

kubectl -n cloud-clickhouse-paas exec -it clickhouse-0 -- clickhouse-client --multiline --host="clickhouse-1.clickhouse-headless.cloud-clickhouse-paas"show databases;show tables;select * from system.zookeeper where path = "/clickhouse";

高級部分 & 超大規(guī)模吞吐接入 ClickHouse多集群/多節(jié)點(diǎn)/多分片/多副本的中間件 PaaS

獨(dú)立部署多套 VKE LoadBlancer+ VKE K8S Cluster + ZooKeeper-Operator + ClickHouse-Operator，分散 Schema 到不同的集群以及多節(jié)點(diǎn)分片。

分析 Snuba 系統(tǒng)設(shè)計(jì)查看測試用例源碼，了解系統(tǒng)設(shè)計(jì)與高階配置

關(guān)于針對 ClickHouse 集群各個分片、副本之間的讀寫負(fù)載均衡、連接池等問題。Snuba 在系統(tǒng)設(shè)計(jì)、代碼層面部分就已經(jīng)做了充分的考慮以及優(yōu)化。

關(guān)于 ClickHouse Operator 獨(dú)立的多個云原生編排集群以及 Snuba 系統(tǒng)設(shè)計(jì)等高級部分會在 VIP 專欄直播課單獨(dú)講解。

關(guān)鍵詞：命名空間系統(tǒng)設(shè)計(jì) 提供服務(wù) 們是如何采用的是