Posts

Showing posts with the label apache-kafka

What are the advantages and disadvantages of Kafka over Apache Pulsar

What are the advantages and disadvantages of Kafka over Apache Pulsar Yahoo developed Pulsar, pub-sub messaging system and made it open source. Its now Apache's incubating project. Since Kafka is also used for same purpose. Want to know, major plus and minus points of Kafka over Pulsar. 4 Answers 4 I played a bit with both lately, and here is what I gathered. Neutral: Pro pulsar: Pro Kafka: Also read: I would appreciate an explanation from the downvoters. – nha Dec 8 '17 at 10:01 pro Pulsar: 6 items, pro Kafka: 1 item. I had not downwoted, but this answer does seem biased. – c69 Jun 27 at 3:44 ...

How can I add two orderers to the same channel?

How can I add two orderers to the same channel? I'm trying to build a network with two orders using Kafka. In the first network example there is a script named./script.sh that creates a channel with an associated orderer that runs this command: peer channel create -o orderer.example.com:7050 -c $CHANNEL_NAME -f ./channel-artifacts/channel.tx --tls true --cafile /opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem >&log.txt I already started implementing Kafka. But my question is How can I change this command to create a channel with two orders? Or is there a better way to do this? 1 Answer 1 You do not add orderers to channel. An orderer belongs to an orderer organization. You can define multiple consortium that an orderer organization can serve. The definition of consortium is pro...

kafka brokers bind to many networks, does it work?

kafka brokers bind to many networks, does it work? what if a cluster is bind to more than one subnet with different ip addresses for every broker, a dns name for every ip addr. ? This is inside the overall company network (dns included), no exposure to public network. No routing among subnets. Is it possible to configure the cluster in order to not make a mess and respect configured networking? I mean, I configure listeners with the names of the internal subnet (the one with kafka brokers and zookeeper servers) and external listeners with the names of every subnet the broker is bind to. Supposed configuration (part of): advertised.listeners=EXTERNAL://broker1.subnet1.localdomain:9993,EXTERNAL://broker2.subnet1.localdomain:9993,EXTERNAL://broker1.subnet2.localdomain:9993,EXTERNAL://broker2.subnet2.localdomain:9993,EXTERNAL://broker1.subnet3.localdomain:9993,EXTERNAL://broker2.subnet3.localdomain:9993,INTERNAL://broker1.internalsubnet.localdomain:9990 inter.broker.listener.name=INT...

Confluent's Kafka REST Proxy vs Kafka Client

Confluent's Kafka REST Proxy vs Kafka Client I am curious about the advantages and disadvantages of Confluent's Kafka REST Proxy and the producer/consumer implemented with the kafka official client library. i know that Confluent's Kafka REST Proxy is used for administrative tasks and for languages ​​not supported by the kafka client. So, what are the advantages of the kafka client? Which Kafka Client? Java? Rdkafka? – cricket_007 2 days ago @cricket_007 Java client. i don't know Rdkafka – 김태우 yesterday Well, the REST Proxy uses the Java client internally, so I don't don't really understand your question. The main benefit would be scalability ov...

Unknown configuration 'errors.deadletterqueue.topic.name'

Unknown configuration 'errors.deadletterqueue.topic.name' I am trying to configure Sink Kafka Connect for Google Cloud PubSub Service. Using following command to configure Kafka Connect: curl -X POST -H 'Content-Type: application/json' -H 'Accept: application/json' -d '{ "name": "pubsub_test", "config": { "connector.class": "com.google.pubsub.kafka.sink.CloudPubSubSinkConnector", "tasks.max": "1", "topics": "kafka_test_topic", "cps.topic": "cps_test_topic", "cps.project": "cps_test_project" } }' http://localhost:8083/connectors In status, I have a following message: {"name":"pubsub_test","connector": {"state":"RUNNING","worker_id":"connect:8083"}, "tasks":[{"state":"FAILED","trace":"org.apache.kafka.common.config....