kafka生產者發送數據時保證同一id的數據都發送到同一個分區中(同一topic時)
import com.alibaba.fastjson.JSON;
import com.emg.receiver.ecar.bean.Result;
import com.emg.receiver.ecar.bean.Track;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RestController;
import reactor.core.publisher.Mono;
@RestController
@Slf4j
public class EcarCtrl {
@Autowired
private org.springframework.kafka.core.KafkaTemplate<Integer, String> KafkaTemplate;
@Value("${spring.kafka.template.ecar-topic}")
private String TOPIC;
@PostMapping(path = "/gatherGps", consumes = "application/json")
private Mono<Result> echo(@RequestBody String gpsJson) {
Track track = JSON.parseObject(gpsJson, Track.class);
Long terminalId = track.getTerminalId();
String str = terminalId.toString();
Integer tid = Integer.valueOf(str);
Integer partition = tid % 9;
//將同一id的數據都發送到一個分區中,需要將此數據的id作爲key發送,具體發送到那個分區中,就用 (id % 分區總數)取模就行了
/*
9是一共9個分區
tid%9就是說發送到那個分區中,
tid就是一個key ,保證相同key的數據都發送到一個分區中
gpsJosn就是真正數據了
*/
KafkaTemplate.send(TOPIC, partition, tid, gpsJson);
return Mono.just(Result.builder().success(true).message("數據接收成功").build());
}
}
錯誤
我的情況是:kafka集羣在101-103機器上,然後將kafka生產者的代碼打成jar包放到105機器上運行,就會報org.apache.kafka.common.errors.TimeoutException: 這種錯誤。然後我將101-103的IP地址映射到105機器上(/etc/hosts),就正常運行了。具體參考:https://blog.csdn.net/maoyuanming0806/article/details/80553632