bing
Flat 10% & upto 50% off + 10% Cashback + Free additional Courses. Hurry up
×
UPTO
50%
OFF!
Intellipaat
Intellipaat

How to monitor Kafka?

Yammer Metrics is used for reporting purposes between the brokers and the clients.

DescriptionValue
No of under replicated partitions0
It means that the server is on active controlOnly one server is 1
Leader election rate (LER)Server failure not 0
Unclean LER0
Partition countsMostly even near servers
Leader replica countsMostly even near servers
ISR shrinks rate

Normally the ISR and expansion is 0.

The partitions will shrink if the server powers down. Again, that partition will be expanded once the replicas of the servers are up.

ISR expansion rateSame as above
Max lags in messages btw follower and leader replicasLag must be proportional  to the size of a request made by the producer
Lag in messages per follower replicaLag must be proportional  to the size of a request made by the producer
If such condition then requests wait near the producernon-zero if ack=-1 is used
It is the span for which the request will be waitingDuring the producer request if act =-1
The processors will be constantWhen time is greater than 0.3 and between 0 and 1
Here the request handler threads will be constantWhen time is greater than 0.3 and between 0 and 1
Quota metrics per client-idThrottle-time is the time for which the client-id is throttled i.e it is 0, and the byte-rate is the rate at which the data is produced or consumed in bytes/sec

New producer Monitoring

Description
These are the threads that were blocked and are waiting to  add their records by the buffer memoryWaiting-threads
The largest buffer than can be used by the clientBuffer-total-bytes
buffer-available-bytes
It indicates the overall usable buffer memory
This is  the  time  for which the  fixer waits for the space assignedbufferpool-wait-time
Bytes count  for each partition  and for each request made by the partitionbatch-size-avg
Maximum bytes for each partition and on each requestbatch-size-max
It is the rate of compression in average countcompression-rate-avg
Time spend by record in averagerecord-queue-time-avg
Highest time  spend by the recordrecord-queue-time-max
The  rate at which the record is retriedrecord-retry-rate
 It is the rate at which the record error occursrecord-error-rate
It is the largest size any record can be ofrecord-size-max
It the average size is any recordrecord-size-avg
It is the age  of the present metadatametadata-age
It is the rate at which the connection can be closedconnection-close-rate
 It is the rate at which the connection can be createdconnection-creation-rate
 The rate of the network operationsnetwork-io-rate
The rate at which the bytes are outgoingoutgoing-byte-rate
The rate at which  the requests are sentrequest-rate
It is the average size of all the requests  that is sentrequest-size-avg
It is the largest size any request is sentrequest-size-max
It is the rate at which the bytes enterincoming-byte-rate
The rate at which the responses are obtainedresponse-rate
 It is the rate of selection of input,output performanceselect-rate
 It is the average time for which the input output waitsio-wait-ratio
 It is the average time for which the input, output call in nsio-time-ns-avg
It is the time for which the input, outputio-ratio
thread spends
It is the number of present active connectionsconnection-count
It is the number of bytes send with respect to the timeoutgoing-byte-rate
 It is the rate of requests sent in each second for a noderequest-rate
 It is the average of the size of the requestsrequest-size-avg
 The largest size any request can be of request-size-max
 It is the rate in which the responses are obtainedincoming-byte-rate
 It is the average of the request latencyrequest-latency-avg
It is the maximum of the samerequest-latency-max
It is the rate at which the answers to the requests are obtainedresponse-rate
It is the rate at which the records are sent to the Topicrecord-send-rate
It is the rate at which the bytes are sent to the Topicbyte-rate
For a topic it is the rate at which the records are compressedcompression-rate
It is the rate at which the records are tried again to be sent to the Topicrecord-retry-rate
It is the rate at which error occurs when records are being sent to the Topicrecord-error-rate
It is the maximum time in which the request can be throttled by the serverproduce-throttle-time-max
It is the average amount of time any request can be throttled by the server.produce-throttle-time-avg

 

Always keep in mind that for a consumer to be in good position, keep the max lag less than the threshold and fetching rate should be always larger than 0.

Previous Next

Download Interview Questions asked by top MNCs in 2019?

"0 Responses on Monitoring"

    Leave a Message

    100% Secure Payments. All major credit & debit cards accepted Or Pay by Paypal.
    top

    Sales Offer

    Sign Up or Login to view the Free Monitoring.