本文已参与「开源摘星计划」,欢迎正在阅读的你加入。活动链接:https://github.com/weopenprojects/WeOpen-Star


提示:以下是本篇文章正文内容,下面源码分析可供参考,如有出错请指正!

一、seatunnel-connectors-flink-http

1.总览

seatunnel-connectors-flink-http

2.源码分析

这里是SeaTunnel项目http连接的数据源端模块,可以读入基于http协议封装的数据接口。pom.xml引用依赖如下:
seatunnel-connectors-flink-http

Http类的getData()函数如下:

    public DataSet<Row> getData(FlinkEnvironment env) {
    	//从文件路径读请求参数
    	//文件路径格式, as a URI (e.g., "file:///some/local/file" or "hdfs://host:port/file/path")
        String syncValues = getSyncValues(env.getBatchEnvironment(), syncPath);
        LOG.info("sync values->{}", syncValues);
        //将json格式字符串的请求参数转换HashMap
        Map syncMap = jsonToMap(syncValues);
        if (!syncMap.isEmpty()) {
            requestMap.putAll(syncMap);
        }
        HttpClientResult response = new HttpClientResult();
        try {
            Map headerMap = jsonToMap(header);
            if (POST.equals(method)) {
                //发送Post请求
                response = HttpClientUtils.doPost(url, headerMap, requestMap);
            } else {
                //发送Get请求
                response = HttpClientUtils.doGet(url, headerMap, requestMap);
            }
        } catch (Exception e) {
            LOG.error("http call error!", e);
            throw new RuntimeException(e);
        }

        LOG.info("http respond code->{}", response.getCode());
        LOG.info("http respond body->{}", response.getContent());
		//返回数据集
        return env.getBatchTableEnvironment().toDataSet(
            env.getBatchTableEnvironment().fromValues(
                DataTypes.ROW(DataTypes.FIELD("rawMsg", DataTypes.STRING())),
                response.getContent()
            ),
            Row.class
        );
    }

更多推荐