Kafka data structures in Zookeeper

轉自:https://cwiki.apache.org/confluence/display/KAFKA/Kafka+data+structures+in+Zookeeper

0.8:

1. Topic registration info:
/brokers/topics/[topic] :

Schema:
"fields" :
    [ {"name""version""type""int""doc""version id"},
      {"name""partitions",
       "type": {"type""map",
                "values": {"type""array""items""int""doc""a list of replica ids"},
                "doc""a map from partition id to replica list"},
      }
    ]
}
Example:
{
  "version"1,
  "partitions": {"0": [013] } }
}

2. Partition state info:
/brokers/topics/[topic]/partitions/[partitionId]/state

Schema:
"fields":
    [ {"name""version""type""int""doc""version id"},
      {"name""isr",
       "type": {"type""array",
                "items""int",
                "doc""an array of the id of replicas in isr"}
      },
      {"name""leader""type""int""doc""id of the leader replica"},
      {"name""controller_epoch""type""int""doc""epoch of the controller that last updated the leader and isr info"},
      {"name""leader_epoch""type""int""doc""epoch of the leader"}
    ]
}
 
Example:
{
  "version"1,
  "isr": [0,1],
  "leader"0,
  "controller_epoch"1,
  "leader_epoch"0
}

3. Broker registration info:
/brokers/ids/[brokerId]

Schema:
"fields":
    [ {"name""version""type""int""doc""version id"},
      {"name""host""type""string""doc""ip address or host name of the broker"},
      {"name""port""type""int""doc""port of the broker"},
      {"name""jmx_port""type""int""doc""port for jmx"}
    ]
}
 
Example:
{
  "version"1,
  "host""192.168.1.148",
  "port"9092,
  "jmx_port"9999
}

4. Controller epoch: 

/controller_epoch -> int (epoch)

5. Controller registration:

/controller -> int (broker id of the controller)

6. Consumer registration:

/consumers/[groupId]/ids/[consumerId]

Schema:
"fields":
    [ {"name""version""type""int""doc""version id"},
      {"name""pattern""type""string""doc""can be of static, white_list or black_list"},
      {"name""subscription""type" : {"type""map""values": {"type""int"},
                                         "doc""a map from a topic or a wildcard pattern to the number of streams"}      }    ]
}
 
Example:
static subscription:
{
  "version"1,
  "pattern""static",
  "subscription": {"topic1"1"topic2"2}
}
 
 
A whitelist subscription:
{
  "version"1,
  "pattern""white_list",
  "subscription": {"abc"1}}
 
A blacklist subscription:
{
  "version"1,
  "pattern""black_list",
  "subscription": {"abc"1}}

7. Consumer owner:

/consumers/[groupId]/owners/[topic]/[partitionId] -> string (consumerId)

8. Consumer offset:

/consumers/[groupId]/offsets/[topic]/[partitionId] -> long (offset)

9. Re-assign partitions

/admin/reassign_partitions

{
   "fields":[
      {
         "name":"version",
         "type":"int",
         "doc":"version id"
      },
      {
         "name":"partitions",
         "type":{
            "type":"array",
            "items":{
               "fields":[
                  {
                     "name":"topic",
                     "type":"string",
                     "doc":"topic of the partition to be reassigned"
                  },
                  {
                     "name":"partition",
                     "type":"int",
                     "doc":"the partition to be reassigned"
                  },
                  {
                     "name":"replicas",
                     "type":"array",
                     "items":"int",
                     "doc":"a list of replica ids"
                  }
               ],
            }
            "doc":"an array of partitions to be reassigned to new replicas"
         }
      }
   ]
}
 
Example:
{
  "version"1,
  "partitions":
     [
        {
            "topic""Foo",
            "partition"1,
            "replicas": [013]
        }
     ]            
}

10. Preferred replication election

/admin/preferred_replica_election

{
   "fields":[
      {
         "name":"version",
         "type":"int",
         "doc":"version id"
      },
      {
         "name":"partitions",
         "type":{
            "type":"array",
            "items":{
               "fields":[
                  {
                     "name":"topic",
                     "type":"string",
                     "doc":"topic of the partition for which preferred replica election should be triggered"
                  },
                  {
                     "name":"partition",
                     "type":"int",
                     "doc":"the partition for which preferred replica election should be triggered"
                  }
               ],
            }
            "doc":"an array of partitions for which preferred replica election should be triggered"
         }
      }
   ]
}
 
Example:
 
{
  "version"1,
  "partitions":
     [
        {
            "topic""Foo",
            "partition"1         
        },
        {
            "topic""Bar",
            "partition"0         
        }
     ]            
}

11. Delete topics
/admin/delete_topics

Schema:
"fields":
    [ {"name""version""type""int""doc""version id"},
      {"name""topics",
       "type": { "type""array""items""string""doc""an array of topics to be deleted"}
      } ]
}
 
Example:
{
  "version"1,
  "topics": ["foo""bar"]
}

0.8.1

Topic Configuration

/config/topics/[topic_name]

Example

{
  "version"1,
  "config": {
    "config.a""x",
    "config.b""y",
    ...
  }
}

/config/changes/[config_change_x] -> "topic_name"

Contains the name of the topic that changed.

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章