HDFS受元数据上限限制,NameNode锁粒度太大,尽管有联邦可以理论上解决该问题,但是联邦在使用的过程中,依然会有各种各样诡异的问题,
   社区新出了一种方案,采用对象存储的方式存储数据即Ozone(HDDS),将Namenode做了拆分,这里肯定有人会问为什么不去改动namenode,目前
   Hadoop源代码已经达到了惊人的几百万行代码,代码组件的耦合性非常高,即使是Hadoop PMC&Committer也很难下手.
      在HDFS中,主要的角色分为NameNode,DataNode.同样在HDDS(Ozone)中,保留了DataNode角色,把NameNode拆分为OM&SCM.即Ozone Manager,
   Storage Container Manager. OM负责集群外部写请求的转发于SCM或者读请求转发于DataNode,SCM负责存储DataNode中存储的对象管理以及
   DataNode心跳等.

    <repository>
      <id>${distMgmtSnapshotsId}</id>
      <name>${distMgmtSnapshotsName}</name>
      <url>${distMgmtSnapshotsUrl}</url>
    </repository>
    <repository>
      <id>repository.jboss.org</id>
      <url>http://repository.jboss.org/nexus/content/groups/public/</url>
      <snapshots>
        <enabled>false</enabled>
      </snapshots>
    </repository>

<property>
   <name>ozone.enabled</name>
   <value>true</value>
</property>

<!--存放元数据-->
<property>
   <name>ozone.metadata.dirs</name>
   <value>/opt/data/meta</value>
</property>

<!--OM地址-->
<property>
   <name>ozone.om.address</name>
   <value>bq-ozone-01:9862</value>
</property>

<property>
   <name>ozone.scm.names</name>
   <value>bq-ozone-01</value>
</property>

<!--SCM地址-->
<property>
   <name>ozone.scm.client.address</name>
   <value>bq-ozone-01:9860</value>
</property>

<!--datanode id标识存放位置-->
<property>
   <name>ozone.scm.datanode.id</name>
   <value>/opt/data/meta/node/datanode.id</value>
</property>

[root@bq-ozone-01 profile.d]# ozone sh volume create volume1
[root@bq-ozone-01 profile.d]# ozone sh volume list
{
  "metadata" : { },
  "name" : "volume1",
  "admin" : "root",
  "owner" : "root",
  "creationTime" : "2020-03-29T15:14:37.538Z",
  "acls" : [ {
    "type" : "USER",
    "name" : "root",
    "aclScope" : "ACCESS",
    "aclList" : [ "ALL" ]
  }, {
    "type" : "GROUP",
    "name" : "root",
    "aclScope" : "ACCESS",
    "aclList" : [ "ALL" ]
  } ],
  "quota" : 1152921504606846976
}

[root@bq-ozone-01 profile.d]# ozone sh bucket create volume1/bucket1
[root@bq-ozone-01 profile.d]# ozone sh bucket list /volume1
{
  "metadata" : { },
  "volumeName" : "volume1",
  "name" : "bucket1",
  "storageType" : "DISK",
  "versioning" : false,
  "creationTime" : "2020-03-29T15:15:54.665Z",
  "encryptionKeyName" : null
}

[root@bq-ozone-01 profile.d]# ozone sh key  put volume1/bucket1/key1/hosts /etc/hosts
[root@bq-ozone-01 profile.d]# ozone sh key info volume1/bucket1/key1/hosts
{
  "volumeName" : "volume1",
  "bucketName" : "bucket1",
  "name" : "key1/hosts",
  "dataSize" : 239,
  "creationTime" : "2020-03-29T15:16:36.432Z",
  "modificationTime" : "2020-03-29T15:16:38.257Z",
  "replicationType" : "RATIS",
  "replicationFactor" : 3,
  "ozoneKeyLocations" : [ {
    "containerID" : 1,
    "localID" : 103907000085381120,
    "length" : 239,
    "offset" : 0
  } ],
  "metadata" : { },
  "fileEncryptionInfo" : null
}

[root@bq-ozone-01 ~]# ozone sh key get /volume1/bucket1/key1/hosts myfile
[root@bq-ozone-01 ~]# ls
 myfile
[root@bq-ozone-01 ~]# cat myfile 
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.42.131 bq-ozone-01
192.168.42.132 bq-ozone-02
192.168.42.133 bq-ozone-03

 ERROR org.apache.hadoop.hdds.scm.block.BlockManagerImpl: 
 Unable to allocate a block for the size: 268435456, type: RATIS, factor: THREE
 

以上记录了笔者安装测试Ozone(HDDS)过程,后期会更新关于Ozone其他文章.希望对读者起到帮助作用.