1, 副本增加:1-> 2

需求:历史数据只有一个副本,需要修改为两个副本

  • 登陆7180页面, 配置hdfs的replications个数
  • 手动设置历史文件的副本数 (增大 )
#1,查看某文件的副本数
[hdfs@cdh1 ~]$ hdfs dfs -stat '%o,%r' /a.txt  #blocksize, replication
134217728,1

#2, 修改文件副本数,-R  -W参数可选
hdfs dfs -setrep 2  /a.txt  

2, 副本减少:2-> 1

  • 登陆7180页面, 配置hdfs的replications个数
  • 手动设置历史文件的副本数 (减少 )
#1,查看某文件的副本数
[hdfs@cdh1 ~]$ hdfs dfs -stat '%o,%r' /a.txt  #blocksize, replication
134217728,2

#2, 修改文件副本数,-R  -W参数可选
hdfs dfs -setrep 1  /a.txt  

#3,查看namenode,datanode日志,观察过程
[hdfs@cdh1 ~]$ tail  /var/log/hadoop-hdfs/hadoop-cmf-hdfs-NAMENODE-cdh1.log.out 
2019-09-12 11:32:27,246 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: Decreasing replication from 2 to 1 for /a.txt
2019-09-12 11:32:27,246 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 4 Total time for transactions(ms): 8 Number of transactions batched in Syncs: 0 Number of syncs: 4 SyncTimes(ms): 120
2019-09-12 11:32:28,066 INFO BlockStateChange: BLOCK* BlockManager: ask 192.168.56.151:50010 to delete [blk_1073741910_1086]

[hdfs@cdh1 ~]$ tail  /var/log/hadoop-hdfs/hadoop-cmf-hdfs-DATANODE-cdh1.log.out
2019-09-12 11:32:29,123 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService: Scheduling blk_1073741910_1086 file /dfs/dn/current/BP-1987302714-192.168.56.151-1568201717933/current/finalized/subdir0/subdir0/blk_1073741910 for deletion
2019-09-12 11:32:29,126 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService: Deleted BP-1987302714-192.168.56.151-1568201717933 blk_1073741910_1086 file /dfs/dn/current/BP-1987302714-192.168.56.151-1568201717933/current/finalized/subdir0/subdir0/blk_1073741910

更多推荐