血液中心网站建设规范,网络设计开发网站,通信工程企业网站建设,小型教育网站开发Exploring storage and computing separation for ClickHouse - JuiceFS Blog
ClickHouse 存算分离改造#xff1a;小红书自研云原生数据仓库实践
唯品会翻牌ClickHouse后#xff0c;实现百亿级数据自助分析_语言 开发_dbaplus社群_InfoQ精选文章 在思考如何实现存算…Exploring storage and computing separation for ClickHouse - JuiceFS Blog
ClickHouse 存算分离改造小红书自研云原生数据仓库实践
唯品会翻牌ClickHouse后实现百亿级数据自助分析_语言 开发_dbaplus社群_InfoQ精选文章 在思考如何实现存算分离感觉可以像JuiceFS利用多盘存储隔离资源。 多盘配置 path/var/lib/clickhouse//pathstorage_configurationdisksdisk_name_1path/mnt/A123456/data//path/disk_name_1/diskspoliciespolicy_name_1volumesvolume_name_0diskdisk_name_1/disk/volume_name_0/volumes/policy_name_1/policies
/storage_configuration
但是还有个位置zk的多副本配置了怎么弄还有就是如何读data。 查看存储策略
select policy_name,volume_name,disks from system.storage_policies ┌─policy_name───┬─volume_name───┬─disks───────────┐
│ default │ default │ [default] │
│ policy_name_1 │ volume_name_0 │ [disk_name_1] │
└───────────────┴───────────────┴─────────────────┘
生成mergeTree表写数据
CREATE TABLE myFirstReplacingMT
(key Int64,someCol String,eventTime DateTime
)
ENGINE ReplacingMergeTree
ORDER BY key SETTINGS storage_policy policy_name_1;INSERT INTO myFirstReplacingMT Values (1, first, 2020-01-01 01:01:01);
INSERT INTO myFirstReplacingMT Values (1, second, 2020-01-01 00:00:00);
查看位置
SELECTname,data_paths,metadata_path,storage_policy
FROM system.tables
WHERE name LIKE myFir% 从文件导入
换一个简单的表 test_batch
CREATE TABLE test_batch (a Int64,b Int64)
ENGINE ReplacingMergeTree() ORDER BY a
由clickhouse-local生成数据 sudo echo -e 1,2\n2,3 | clickhouse-local --input-format CSV -S a Int64,b Int64 -N tmp_table -q CREATE TABLE test_batch (a Int64,b Int64) ENGINE ReplacingMergeTree() ORDER BY a;INSERT INTO TABLE test_batch SELECT a,b FROM tmp_table; --logger.console --path /tmp/test/testlocal/ ls testlocal/data/_local/test_batch/all_1_1_0/
checksums.txt columns.txt count.txt data.bin data.cmrk3 default_compression_codec.txt metadata_version.txt primary.cidx serialization.json 拷贝到server上查看一下存储位置
SELECTname,data_paths,metadata_path,storage_policy
FROM system.tables
WHERE name LIKE test_batch% 拷贝到data_paths下detached文件夹 sudo cp -r ./testlocal/data/_local/test_batch/all_1_1_0/ /mnt/xxx/data/store/xxx/detached/ 然后在service上 ALTER TABLE test_batch ATTACH PART all_1_1_0; 导入进去了看看分区我这里已经重复操作了3次也就是导入了3次 SELECTpartition,name,active
FROM system.parts
WHERE table test_batchQuery id: 111┌─partition─┬─name──────┬─active─┐
│ tuple() │ all_1_1_0 │ 1 │
│ tuple() │ all_2_2_0 │ 1 │
│ tuple() │ all_3_3_0 │ 1 │
└───────────┴───────────┴────────┘ Reference
真是秀ClickHouse批量导入还可以这样玩?-腾讯云开发者社区-腾讯云 (tencent.com)
ClickHouse多盘存储配置-腾讯云开发者社区-腾讯云 (tencent.com)
