cdh管理hadoop集群:副本增删
·
1, 副本增加:1-> 2
需求:历史数据只有一个副本,需要修改为两个副本
- 登陆7180页面, 配置hdfs的replications个数
- 手动设置历史文件的副本数 (
增大)
#1,查看某文件的副本数
[hdfs@cdh1 ~]$ hdfs dfs -stat '%o,%r' /a.txt #blocksize, replication
134217728,1
#2, 修改文件副本数,-R -W参数可选
hdfs dfs -setrep 2 /a.txt
2, 副本减少:2-> 1
- 登陆7180页面, 配置hdfs的replications个数
- 手动设置历史文件的副本数 (
减少)
#1,查看某文件的副本数
[hdfs@cdh1 ~]$ hdfs dfs -stat '%o,%r' /a.txt #blocksize, replication
134217728,2
#2, 修改文件副本数,-R -W参数可选
hdfs dfs -setrep 1 /a.txt
#3,查看namenode,datanode日志,观察过程
[hdfs@cdh1 ~]$ tail /var/log/hadoop-hdfs/hadoop-cmf-hdfs-NAMENODE-cdh1.log.out
2019-09-12 11:32:27,246 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: Decreasing replication from 2 to 1 for /a.txt
2019-09-12 11:32:27,246 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 4 Total time for transactions(ms): 8 Number of transactions batched in Syncs: 0 Number of syncs: 4 SyncTimes(ms): 120
2019-09-12 11:32:28,066 INFO BlockStateChange: BLOCK* BlockManager: ask 192.168.56.151:50010 to delete [blk_1073741910_1086]
[hdfs@cdh1 ~]$ tail /var/log/hadoop-hdfs/hadoop-cmf-hdfs-DATANODE-cdh1.log.out
2019-09-12 11:32:29,123 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService: Scheduling blk_1073741910_1086 file /dfs/dn/current/BP-1987302714-192.168.56.151-1568201717933/current/finalized/subdir0/subdir0/blk_1073741910 for deletion
2019-09-12 11:32:29,126 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService: Deleted BP-1987302714-192.168.56.151-1568201717933 blk_1073741910_1086 file /dfs/dn/current/BP-1987302714-192.168.56.151-1568201717933/current/finalized/subdir0/subdir0/blk_1073741910
更多推荐
所有评论(0)