ds調用zeppelin的接口超時問題解決

背景:

zeppelin版本0.8.2

ds 3.14

通過ds調用zeppelin的note時,是整個note執行,報超時。(單個Paragraph沒有問題)

源碼分析:

默認的http是kong.unirest.Unirest,單例

默認超時時間是1分鐘

解決:

 /**
     * Unirest。Config參數前綴字符串
     */
    public static final String  UNIREST_CONFIG_PREFIX = "uni_";
    /**
     * 設置Unirest。Config參數相關值
     */
    public void setUnirestConfig() {
    	try {
    		final String parameters = this.zeppelinParameters.getParameters();
        	Map<String, Object> zeppelinParamsMap = new HashMap<>();
            if (parameters != null) {
                ObjectMapper mapper = new ObjectMapper();
                zeppelinParamsMap = mapper.readValue(parameters, Map.class);
            }
            if(MapUtils.isEmpty(zeppelinParamsMap)) {
            	logger.info("zeppelinParamsMap isEmpty ");
            	return;
            }
            // zeppelinUnirestConfigMap
            Map<String, Object> zeppelinUnirestConfigMap = new HashMap<>();
            for(Entry<String, Object> entry:zeppelinParamsMap.entrySet()) {
            	String key = entry.getKey();
            	if(StringUtils.isEmpty(key) || !key.startsWith(UNIREST_CONFIG_PREFIX)) {
            		continue;
            	}
            	key = key.replace(UNIREST_CONFIG_PREFIX, "");
            	zeppelinUnirestConfigMap.put(key, entry.getValue());
            }
            if(MapUtils.isEmpty(zeppelinUnirestConfigMap)) {
            	logger.info("zeppelinUnirestConfigMap isEmpty ");
            	return;
            }
            logger.info("zeppelinUnirestConfigMap {} ",zeppelinUnirestConfigMap);
            Config config = Unirest.config();
            for(Entry<String, Object> entry:zeppelinUnirestConfigMap.entrySet()) {
            	  String key = entry.getKey();
            	  Object val = entry.getValue();
            	  //logger.info("methodUtils.invokeMethod methodName {} val= {}",methodName,val);
            	  //MethodUtils.invokeMethod(config, methodName, val);
            	  logger.info("field.set name {} val= {}",config,val);
            	  Field field = config.getClass().getDeclaredField(key);
	      	      field.setAccessible(true);
	      	      field.set(config,val);
            }
    	}catch (Exception e) {
    		 logger.error("zeppelin setUnirestConfig failed with error", e);
		}
    	
    }

 

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章